First-principles methods provide unmatched accuracy and robustness for the computational simulation of materials. However, the computational cost of first-principles methods scales unfavorably with the problem size N, with commonly used density functional theory (DFT) and wavefunction theory (WFT) methods typically exhibiting at least O(N^3) scaling. Consequently, first-principles methods are routinely restricted to system sizes of a few to a few hundred atoms. Beyond this, large-scale material simulations or the creation of extensive reference datasets, e.g. for training machine learning models, often become computationally prohibitive. As a result, cost-efficient alternatives that offer the same high accuracy and broad applicability as first-principles methods are highly sought after. Consequently, this thesis focuses on the development of low-cost electronic structure methods based on cost-efficient first-principles baselines, enhanced by (semi-)empirical and physics-inspired corrections. Potential cost-reducing measures were rigorously analyzed to assess their computational advantages relative to potential losses in accuracy or precision from systematic and stochastic errors. Based on this analysis, guidelines were established for designing robust and reliable baselines and corresponding corrections. A key outcome of this work is the development and implementation of the PBE/min+s/LPC method in the FHI-aims code. PBE/min+s/LPC allows for the cost-efficient calculation of structural properties of inorganic solids. The baseline, PBE/min+s, uses a compact and computationally inexpensive min+s basis set. To address the systematic overestimation of bond lengths caused by basis set incompleteness, a linear pairwise correction (LPC) was developed. LPC's minimally invasive form as well as requiring only one fitting parameter per elemental species, ensures the method's generalizability. PBE/min+s/LPC demonstrates reliable results for structural properties, enabling large-scale geometry optimizations and molecular dynamics simulations. Additionally, PBE/min+s/LPC has been successfully employed to relax the structure of a complex supramolecular architecture on Ag(111), a task for which conventional first-principles methods are prohibitively expensive. In a second project, the applicability of the Bayesian error estimation functional (BEEF-vdW) was extended to electronic structure codes without efficient fast fourier transform algorithms by employing cost-efficient atom-pairwise and many-body dispersion corrections. The results indicate that atom-pairwise methods, such as the Tkatchenko-Scheffler (TS) van der Waals correction and, particularly, the exchange-dipole model (XDM) method, provide a well-rounded description of molecules, solids and chemisorption processes. In a third project, a Bayesian linear regression approach was used to obtain robust and transferable parameterizations as well as uncertainty estimates for spin-component-scaled second-order Møller–Plesset perturbation theory (SCS-MP2), termed BSCS-MP2. The obtained BSCS-MP2 uncertainty estimates are shown to effectively reflect changes in data quality and model complexity as well as being robust for out-of-sample inference.