In the past, graph-cover decoding, here called blockwise graph-cover decoding (B-GCD), has been introduced as a theoretical tool to better understand message-passing iterative decoding, especially to better understand the connections between max-product algorithm decoding and linear programming decoding. In this paper we introduce symbolwise graph-cover decoding (S-GCD), a theoretical tool that helps to better understand the connections between sum-product algorithm decoding and Bethe free energy minimization. S-GCD motivates a deeper study of the properties of the Bethe free energy and of the pseudo-dual of the Bethe free energy, a study which allows one to re-derive and shed new light on many known results for sum-product algorithm decoding. In particular, it gives a new interpretation of a result by Yedidia, Freeman, and Weiss (namely the result that fixed points of the sum-product algorithm correspond to stationary points of the Bethe free energy), of the fundamental polytope, of stopping sets, of EXIT charts, of the Maxwell construction/decoder, and of the asymptotic growth rate of the average Hamming weight distribution of certain (protograph) code ensembles.