We study the evolution of galaxy rest-frame ultraviolet (UV) colors in the epoch 4 < z < 8. We use new wide-field near-infrared data in GOODS-S from the CANDELS, HUDF09 and ERS programs to select galaxies via photometric redshift measurements. Our sample consists of 2812 candidate galaxies at z > 3.5, including 113 at z = 7 to 8. We fit the observed spectral energy distribution to a suite of synthetic stellar population models, and measure the value of the UV spectral slope (beta) from the best-fit model spectrum. The median value of beta evolves significantly from -1.82 (+0.00,-0.04) at z = 4, to -2.37 (+0.26,-0.06) at z = 7. Additionally, we find that faint galaxies at z = 7 have beta = -2.68 (+0.39,-0.24) (~ -2.4 after correcting for observational bias); this is redder than previous claims in the literature, and does not require "exotic" stellar populations to explain their colors. This evolution can be explained by an increase in dust extinction, with the timescale consistent with low-mass AGB stars forming the bulk of the dust. We find no significant (< 2-sigma) correlation between beta and M_UV when measuring M_UV at a consistent rest-frame wavelength of 1500 A. This is particularly true at bright magnitudes, though our results do show evidence for a weak correlation at faint magnitudes when galaxies in the HUDF are considered separately, hinting that dynamic range in sample luminosities may play a role. We do find a strong correlation between beta and the stellar mass at all redshifts, in that more massive galaxies exhibit redder colors. The most massive galaxies in our sample have red colors at each redshift, implying that dust can build up quickly in massive galaxies, and that feedback is likely removing dust from low-mass galaxies at z > 7. Thus the stellar-mass - metallicity relation, previously observed up to z ~ 3, may extend out to z = 7 - 8.

Accepted to the Astrophysical Journal. 19 pages, 8 figures and 5 tables. The complete data for Table 3 will be available in the online version of the Astrophysical Journal, and a version is included in the source files for this submission