Fair forecast comparisons are exceedingly rare in the literature of solar forecasting. Since many published works have been operating under the condition “the proposed methods outperform the benchmarks,” it is unlikely that the actual advancement in solar forecasting science is indeed successive in that the latter works supersede the former. It follows that one must hold fast to skepticism on those model superiority claims until they can be truly justified. In order to quantify the real progress, one needs to not only employ formal verification methods but also make inquiries on predictability, which, even in the field of statistics, is a controversial topic. Although predictability on its own is hard to define, let alone to quantify, one logically attractive proxy is to examine the performance of a short-range forecasting method relative to that of an optimal long-range forecasting method. This strategy is reasonable on the account that a climatological forecast in the short-range horizon marks the worst-case scenario, by which the relative improvement due to an alternative forecasting method can be gauged: situations with high predictability correspond to large relative improvements, and that with low predictability correspond to low relative improvements. If this argument can be considered admissible, the remaining task is to examine what properties ought the short-range forecasting method possess. This paper proposes, in this regard, a new measure of predictability for solar irradiance.