In 1969, Shoup postulated that the presence of interrelated taxes in a tax system would reinforce the tax penalty system ("self-reinforcing penalty system of taxes"). In this paper, we have tried to formally develop this idea. We find that in order for tax reinforcement to be maintained, it is necessary for interrelated taxes to be administered by a single tax administration, or if they are administered by different tax administrations, the level of collaboration between them has to be sufficiently high. If so, tax evasion in interrelated taxes might be considered as an alternative explanation for the gap between the levels of tax evasion that can be guessed in practice and the much higher levels predicted by the classical tax evasion theory (Allingham and Sandmo, 1972; Yitzhaki, 1974). Otherwise, the result anticipated by Shoup may even be reversed. Moreover, as long as collaboration is imperfect, the classical results of the comparative statics might change, since in some cases, although global tax compliance increases when faced with a variation in a tax parameter, it can decrease in a single tax.