We consider a class of linear programs on graphs with total variation regularization and a budgetary constraint. For these programs, we give a characterization of basic solutions in terms of rooted spanning forests with orientation on the underlying graph. This leads to an interpretation of the simplex method in terms of simple graph operations on these underlying forests. We exploit this structure to produce an accelerated simplex method and empirically show that such improvements can lead to an order of magnitude improvement in time when compared to state-of-the-art solvers.