Would it be accurate to say there is no such thing as small government under capitalism, or is that silly?

From what I can gather, when capitalist democracies cut government departments, they just end up outsourced to private companies who reap whatever infrastructure was there and replace it with their own, cheap and barely functioning version for profit.

So basically the government service is still there, it's just privatised and crappier.