High‐quality Quantitative Susceptibility Mapping (QSM) with Nonlinear Dipole Inversion (NDI) is developed with pre‐determined regularization while matching the image quality of state‐of‐the‐art reconstruction techniques and avoiding over‐smoothing that these techniques often suffer from. NDI is flexible enough to allow for reconstruction from an arbitrary number of head orientations and outperforms COSMOS even when using as few as 1‐direction data. This is made possible by a nonlinear forward‐model that uses the magnitude as an effective prior, for which we derived a simple gradient descent update rule. We synergistically combine this physics‐model with a Variational Network (VN) to leverage the power of deep learning in the VaNDI algorithm. This technique adopts the simple gradient descent rule from NDI and learns the network parameters during training, hence requires no additional parameter tuning. Further, we evaluate NDI at 7 T using highly accelerated Wave‐CAIPI acquisitions at 0.5 mm isotropic resolution and demonstrate high‐quality QSM from as few as 2‐direction data.