Many approaches to geoacoustic inversion are based implicitly on the assumptions that data errors are Gaussian-distributed and spatially uncorrelated (i.e., have a diagonal covariance matrix). However, the latter assumption is often not valid due to theory errors, and can lead to reduced accuracy for geoacoustic parameter estimates and underestimation of parameter uncertainties. This paper examines the effects of data error (residual) covariance in matched-field geoacoustic inversion. An inversion approach is developed based on a nonparametric method of estimating the full covariance matrix (including off-diagonal terms) from the data residuals and explicitly including this covariance in the misfit function. Qualitative and quantitative statistical tests for Gaussianity and for correlations in complex residuals are considered to validate the inversion results. The approach is illustrated for Bayesian geoacoustic inversion of broadband, vertical-array acoustic data measured in the Mediterranean Sea.