I advise you get some friends to check your stream as viewers to see if everything is working as intended. Otherwise, it's an interesting experience where you will learn some new skills and hopefully get more subs.
If you watch other streams, popular streams, most of them have a schedule that they list on there channel page below the stream. If you make a schedule, stream when you say you're going to stream, then you'll start to build a community around your channel.
If you quit streaming out of the blue and don't stream for a while, then chances are you won't have those same people coming back when you stream again.
You're not going to go live and suddenly get 100+ people watching you. You have to give people a reason to want to watch you play a game. Interact with the people watching and make the most out of your broadcasts.
Getting a couple people who actively come to your stream each time and making them moderators will help them want to keep coming back. If you have a couple people, then you'll start to get more and more eventually.
It all depends on the game you play as well. Whats popular, what are other top streamers doing to get the views. Watch and learn from others and try to do it better.