First off, dont put words in his mouth, dont act out the scenario of telling him. I know I said "he'd probably do this blah, blah, blah" but really what you're doing is making up excuses. You seem really afraid to tell him how you feel. If he thinks you are attacking his family make it clear that you are not. If he tells you that you are being silly, be firm and tell him that you are serious about how you feel.
I always do that I'll say to myself well "he or she will do this or that..." and usually I am totally off and people are pleased that I finally told them how I feel. Seriously, if you cant talk to him you probably should just leave and go back home because no relationship will work w/o basic communication. If he doesnt respect how you feel about things say good bye.