PurposeMRSI has shown great promise in the detection and monitoring of neurologic pathologies such as tumor. A necessary component of data processing includes the quantitation of each metabolite, typically done through fitting a model of the spectrum to the data. For high‐resolution volumetric MRSI of the brain, which may have ~10,000 spectra, significant processing time is required for spectral analysis and generation of metabolite maps.MethodsA novel unsupervised deep learning architecture that combines a convolutional neural network with a priori models of the spectrum is presented. This architecture, a convolutional encoder–model decoder (CEMD), combines the strengths of adaptive and unbiased convolutional networks with models of magnetic resonance and is readily interpretable.ResultsThe CEMD architecture performs accurate spectral fitting for volumetric MRSI in patients with glioblastoma, provides whole‐brain fitting in 1 min on a standard computer, and handles a variety of spectral artifacts.ConclusionA new architecture combining physics domain knowledge with convolutional neural networks has been developed and is able to perform rapid spectral fitting of whole‐brain data. Rapid processing is a critical step toward routine clinical practice.