In this paper, we quantitatively analyze to what extent a benevolent government should issue debt in a model where households are subject to idiosyncratic productivity shocks, insurance markets are missing and borrowing is restricted. In this environment, issuing government bonds facilitates saving for self-insurance. Despite this, we find that in a calibrated version of the model that is consistent with the skewed wealth and earnings distribution observable in the U.S., the government should buy private bonds, and not issue public debt in the long run. The reason is that in the U.S., a large fraction of the population has almost no wealth or is even in debt. The wealth-poor, however, do not profit from an increase in the interest rate following an increase in public debt. Instead, they gain from higher wages that result from a reduction in debt. We show that even when the short run costs of higher capital taxation are taken into account, it still pays off to reduce government debt on overall. Moreover, we find that endogenizing household’s borrowing constraints by assuming limited commitment leads to even higher asset levels being optimal in the long run.