Continuous and homogeneous thin gold films are fabricated on Si–SiO2 substrates using an additional seed layer based on Cu oxide or oxidized Cu. The optical properties of these gold films with thicknesses from 3 to 50 nm are studied with the help of variable angle spectroscopic ellipsometry. The optical constants and the dielectric functions of thin Au films in the wavelength range of 240–1700 nm are extracted by fitting measured ellipsometric data using the Fresnel isotropic and anisotropic models. By applying the Drude approximation to the infrared range, values of the plasma frequency ωp and relaxation time τ are obtained as a function of film thickness. It is found that plasma frequency ωp of the gold films is mostly constant in the thickness range of 3–50 nm with an average being close to the bulk value ωp ≈ 8.45 eV. The relaxation time decreases dramatically from the bulk values of τ ≈ 14 fs to τ ≈ 2 fs for gold films of 3 nm in an agreement with the confinement effect. It is established that the thinnest gold films (≈3 nm) are described better by an anisotropic layer and the underlying reasons for such optical anisotropy are discussed.