It sounds like you're going through a mix of emotions right now, which is totally understandable. It's natural to feel scared and worried about how things will go, especially with all the uncertainties. Just take things one step at a time and try to communicate openly with Jerry when you feel ready.
As for what others might think, it's your life and your decision, so focus on what feels right for you. Surround yourself with supportive people who will be there for you no matter what.