You're not required to date everyone. He wasn't right for you. That's all you need to know.
However, as for the topic itself it's the reverse for me. When I feel good I think I look better. That's likely because I have had some life threatening health issues. I know I look awful when I'm battling a critical illness. For example, twice my weight went under 100 pounds. I looked sickly. Because I WAS sickly. I certainly didn't feel up to getting dolled up. Some might argue if I had put in the effort to fix my hair and put on makeup and put on a stylish outfit I would have felt better. But when you're battling potential sepsis you aren't going to feel or get better by putting on a fancy dress or some eyeliner.
Now that my health is better I guess I place less importance on clothing and makeup because I know those things are (to me, and I emphasize to me) are superficial. Clean, neat, presentable and appropriate to the situation or environment. I'd rather be wearing joggers and feeling relatively well than be in a pricey or stylish outfit but feeling lousy. Sure, it doesn't have to be an either/or but my priorities have been adjusted.
The last few men I dated seldom dressed up. They were clean but they usually wore shorts or jeans and a T-shirt. My toxic ex was slovenly but he has issues, to say the least. But he was an outlier.