Motivated by a central banker with a symmetric but non-quadratic loss function, we show in this note that the approximations of two plausible loss functions of this type will include a quartic term. For skewed distributions, we establish that such a loss function implies a systematic inflation bias even when the bank targets the natural rate. Moreover, we show that the weights in an optimal combination of forecasts will differ from that under quadratic loss. We illustrate these differences using simulated data and data from the Livingston Surveys of Professional Forecasters.