The strategy improvement algorithm for mean payoff games and parity games is a local improvement algorithm, just like the simplex algorithm for linear programs. Their similarity has turned out very useful: many lower bounds on running time for the simplex method have been created from lower bounds for strategy improvement. However, earlier connections between these algorithms required constructing an intermediate Markov decision process, which is not always possible. We prove a formal, direct connection between the two algorithms, showing that many variants of strategy improvement for parity and mean payoff games are truly an instance of the simplex algorithm, under mild nondegeneracy assumptions. As a result of this, we derive some combinatorial properties of the structure of strategy sets of various related games on graphs. In particular, we show a connection to lopsided sets.