This is the first in a series of papers that present a new classical statistical treatment of the system of a charged harmonic oscillator (HO) immersed in an omnipresent stochastic zero-point (ZP) electromagnetic radiation field. This paper establishes the Gaussian statistical properties of this ZP field using Bourret's postulate that all statistical moments of the stochastic field plane waves at a given space-time point should agree with their corresponding quantized field vacuum expectations. This postulate is more than adequate to derive the Planck spectrum classically via Boyer's and Theimer's methods, but it requires that the stochastic amplitude of each linearly polarized plane wave in the field contain two independent Gaussian random variables, not just a random phase as has sometimes been assumed. In the succeeding papers in the series, the total motion of a charged HO is described by a fully renormalized dipole-approximation Abraham-Lorentz equation. This leads without further approximation to the following major results concerning this stochastic electrodynamics (SED) of the HO: i) The ensemble-average Liouville equation for the oscillator-ZP field system in the presence of an arbitrary applied classical radiation field is exactly equivalent to the usual time-dependent Schrodinger equation supplemented by an explicit radiation reaction vector potential similar to that of the Crisp-Jaynes-Stroud theory; ii) this SED Schrodinger equation for the HO is incomplete, insmuch as there exists a companion equation that restricts initial conditions such that the corresponding Wigner phase-space distribution is always positive; iii) the wave function of the SED Schrodinger equation has thea priori significance of position probability amplitude; iv) first-order transition rates predicted for the HO by this theory agree with those predicted by quantum electrodynamics for resonance absorption and spontaneous emission, which occurs with no triggering necessary; and v) if SED is taken seriously, then the concepts of quantized energies and photons must be abandoned.