In my view, the only thing that limits them is food supply. In spring when the queens come out of hibernation you can see heaps of them flying around, they cannot possibly all create a successful nest. Which we also know because each surving nest can create hundreds or in some cases thousands of queens, and if they all created nests we would be feet deep in wasps.
My own thoughts are that these queen wasps create nests, but competition for food means that only so many survive. As many as the food resources of the land will bear. Or, many of them may survive, but do poorly.
I do recall in Leeston, we had a site being bothered by wasps. It was a simple matter to track them back to the nest and kill it, end of problem. But that was Leeston, not a place where there are a heckuva lot of wasps. What I think happens in places where they are dense, is they cover everything and pretty much strip all food resources. Like, I can remember a place where there were so many wasps that we couldn't eat our lunch outside for fear of getting a wasp in the mouth. Had to leap into the cab and slam the doors, then kill all the wasps that came in with us, and only then eat our lunch in peace.
In such places all food resources are stripped and then population growth is limited. But if some nests are found and destroyed, that frees up food resources for other colonies, that then increase accordingly.
Which is why in wasp dense areas, my own experience is that finding and killing individual nests does not often achieve much. Just thankful for Vespex.