Regardless of what any of us say or think on here, YOU feel very strongly about all of this. And regardless of the facts or what he says, it doesn't change how you FEEL.
"Pedophile" is a very strong word and I wouldn't recommend marrying anyone you could associate that word with. Whether it pertains to past, present, or future actions!
I would try to find someone who you can only say positive things about and nothing so negative about. Sometimes life hands you signs and you just have to follow your instincts and intuition on things, and trust that you are following the right course in life. If you are here asking for advice, it sounds like there is an alarm going off in your head. Listen to it.
Lots of guys over 18 sleep with girls under 18, so if it is something that is important to YOU, maybe you should add it to things you look or ask a guy before you get serious with them. And don't settle for someone who doesn't meet your criteria. Whether this particular criteria is such that someone might say is too strict doesn't matter, it is your life, your partner, and your right to be happy and find someone whose morals and actions in life match yours. That someone might have a criteria on their list that doesn't mean anything to you just the same. And don't feel bad about the decision to leave him if you do, be confident in your decisions always because your instincts are YOURS and in that way they are never wrong.