In the Argentine hyperinflations of 1989 and 1990, quasi-fiscal deficits were a major part of the problem. The Central Bank´s quasi-fiscal activities are financed directly by money printing but in some cases the monetary authority tries to sterilize the effect on the money supply by issuing debt or by increasing reserve requirements (it is not uncommon to pay interest on reserves when this happens). Thus, a new source of quasi-fiscal deficit arises, i.e. the interest payments on the Bank´s liabilities. When nominal interest rates are high and debt reaches unsustainable levels, the interest payments can take a life of their own leading to hyperinflation. The traditional explanation is that the Central Bank has to finance the quasi-fiscal deficit through the use of the inflation tax but as inflation increases money demand drops and there is a limit to how much revenue can be collected which is determined by a Laffer curve. Trying to finance a quasi-fiscal deficit beyond that limit (or any fiscal deficit for that matter) leads to hyperinflation. In this paper we demonstrate that very high inflation can arise even if money demand is perfectly inelastic with respect to inflation and the real value of interest payments is relatively low. The key insight is that if expected inflation is a function of the current state of the economy the Central Bank has an additional incentive to alter the future state which results in higher inflation today.