Bayesian regularization is a central tool in modern‐day statistical and machine learning methods. Many applications involve high‐dimensional sparse signal recovery problems. The goal of our paper is to provide a review of the literature on penalty‐based regularization approaches, from Tikhonov (Ridge, Lasso) to horseshoe regularization.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods Statistical Models > Linear Models Statistical Models > Bayesian Models