Speaking is a skill like any other, and we improve when we do more of it. Therefore, talking in the language as much as you can, right from the start, is the most efficient way to become fluent. And you don't need to acquire a large vocabulary first to do so, either.
In our mother tongues, we use the vocabulary we have to express what we need, even if better, more accurate words may be available but unknown to us. Similary, it's important to learn to make the most of the words you have in Japanese, right from the beginning. This is not about producing perfect sentences, but about acquiring the ability to string words together into meaningful units in a flash. This requires a lot of split-second planning -- planning sounds, intonation, word order, word choice, conjugation, etc. -- that only practice can allow.
While there is nothing we can say that will make you fluent all of a sudden, it's time you changed your language habits and put yourself in an optimal position to acquire the ease you want in speaking Japanese.
I recommend you set aside a certain period of time everyday, preferably several times a day, dedicated to speaking Japanese. No one to talk to? Talk to yourself. Imagine scenarios, create dialogues, make up conversations, etc, but all orally and orally only. You need to set up a regimen of internal monologue where you can take time to create the sentences you need to express yourself. You must move from passive listening and writing to oral production on the fly. If putting a sentence together takes time, that's fine, you HAVE time. Once you have the sentence or phrase you need, keep repeating it until you can say it with relative ease and without painful pauses. It's not about learning that specific phrase, it's about feeling sentences that flow, giving your brain an idea of the kind of flow you want. Then play with the sentence. Change a word. Change the emphasis. Change the tense. Play with it. It may not be grammatically correct, but it doesn't matter -- you need to create and feel the flow of spoken, meaningful utterances. Of course, if you can use sentences you've heard as a starting point, either from real life experiences or from TV, podcasts, etc., all the better, but you shouldn't let that deter you from speaking.
If you can have a quiet, private place where talking out loud is okay, all the better. You should even use gesture, or express yourself with your whole body because these kinesthetic clues help the brain. Otherwise, you can mutter to yourself, whether it's in the car, in the shower, walking in the park, biking, etc. Think about your daily life. Express the things you need to explain your day, your worries, your ideas.
Unless you live in a setting where you are constantly required to express yourself in the language, internal monologue is the key to fluent speech production.
