Just to clarify, we had coffee during work hours, we didn't hang out outside work. While we were talking during our coffee meet, he invited me to join him and his team for drinks after work. But that didn't happen yet.
But yeah, it's totally possible that he changed his mind in the meantime for whatever reason.