If I wait to start speaking, won’t that make me more hesitant to start speaking the language in the future?

You can think about it this way: when you just start learning a language, you have never spoken it before. You haven’t spoken it in your whole life. Let’s say that you don’t know any Swedish and you have never studied it. Would you say: “I’m used to not speaking Swedish”? Well, of course not! Before you start speaking a language, you simply haven’t got used to speaking it yet. It doesn’t mean that your whole life you have been getting used to not speaking it. Leaving speaking for later doesn’t mean that you are going to “get used to not speaking it”. You already are as “used to not speaking it” as you can ever be, and waiting is not going to change anything. On the other hand, waiting until you have reached a certain level in the language means that the interactions you have with native speakers will be more positive. You will have fewer negative interactions, and be less hesitant to speak it.