If he really loves you then he would'nt keep running back and forth between the two of you. What you need to do is give him an ultimatum, he's telling you one thing and telling her another. If he really loves you and you say he spends most of his time with you then, you have to make him choose don't let him have his cake and eat it too. Always make yourself first, never second he will respect you more for that, and if he does'nt then somebody else will.