We study games with utilitarian preferences: the sum of individual utility functions is a generalized ordinal potential for the game. It turns out that generically, any finite game with a potential, ordinal potential, or generalized ordinal potential is better reply equivalent to a game with utilitarian preferences. It follows that generically, finite games with a generalized ordinal potential are better reply equivalent to potential games. For infinite games we show that a continuous game has a continuous ordinal potential, iff there is a better reply equivalent continuous game with utilitarian preferences. For such games we show that best reply improvement paths can be used to approximate equilibria arbitrarily closely.