Have you checked out the possibility that he's married or otherwise attached? I get the feeling that's he goes Mia with slow responses. Sure people are busy, especially doctors. Either that or he is treating what he has with you are very casual.