Which significance test performs the best in climate simulations?

Article English OPEN
Decremer, Damien ; Chung, Chul E. ; Ekman, Annica M. L. ; Brandefelt, Jenny (2014)
  • Publisher: Co-Action Publishing
  • Journal: Tellus A (issn: 1600-0870)
  • Related identifiers: doi: 10.3402/tellusa.v66.23139
  • Subject: climate model; statistics; significance testing; internal variability | climate; climate modeling

Climate change simulated with climate models needs a significance testing to establish the robustness of simulated climate change relative to model internal variability. Student’s t-test has been the most popular significance testing technique despite more sophisticated techniques developed to address autocorrelation. We apply Student’s t-test and four advanced techniques in establishing the significance of the average over 20 continuous-year simulations, and validate the performance of each technique using much longer (375–1000 yr) model simulations. We find that all the techniques tend to perform better in precipitation than in surface air temperature. A sizable performance gain using some of the advanced techniques is realised in the model Ts output portion with strong positive lag-1 yr autocorrelation (> + 0.6), but this gain disappears in precipitation. Furthermore, strong positive lag-1 yr autocorrelation is found to be very uncommon in climate model outputs. Thus, there is no reason to replace Student’s t-test by the advanced techniques in most cases.Keywords: autocorrelation, temporal correlation, internal variability, climate noise, significance test, Student’s t-test(Published: 25 January 2014)Citation: Tellus A 2014, 66, 23139, http://dx.doi.org/10.3402/tellusa.v66.23139
  • References (29)
    29 references, page 1 of 3

    Akaike, H. 1974. A new look at the statistical model identification. IEEE Trans. Automat. Contr. 19, 716 723.

    Box, G. E. P. and Jenkins, G. M. 1976. Time Series Analysis: Forecasting and Control. Holden-Day, the University of Michigan, Ann Arbor, MI, USA, 575 pp.

    Chung, C. E., Ramanathan, V. and Kiehl, J. T. 2002. Effects of the South Asian absorbing haze on the northeast monsoon and surface air heat exchange. J. Clim. 15, 2462 2476.

    Collins, W. D., Rasch, P. J., Boville, B. A., Hack, J. J., McCaa, J. R. and co-authors. 2006. The formulation and atmospheric simulation of the Community Atmosphere Model Version 3 (CAM3). J. Clim. 19, 2144 2161.

    Efron, B. 1982. The Jackknife, the Bootstrap, and Other Resampling Plans. Society for Industrial Mathematics, Philadelphia, PA, USA, 92 pp.

    Efron, B. and Tibshirani, R. J. 1993. An Introduction to the Bootstrap. Chapman and Hall, New York, NY, USA, 436 pp.

    Elmore, K. L., Baldwin, M. E. and Schultz, D. M. 2006. Field significance revisited: spatial bias errors in forecasts as applied to the Eta Model. Mon. Weather Rev. 134, 519 531.

    Fraedrich, K. and Blender, R. 2003. Scaling of atmosphere and ocean temperature correlations in observations and climate models. Phys. Rev. Lett. 90, 108501.

    Gent, P. R., Danabasoglu, G., Donner, L. J., Holland, M. M., Hunke, E. C. and co-authors. 2011. The Community Climate System Model Version 4. J. Clim. 24, 4973 4991.

    Jin, C. S., Ho, C. H., Kim, J. H., Lee, D. K., Cha, D. H. and co-authors. 2013. Critical role of northern off-equatorial sea surface temperature forcing associated with central pacific el Nin˜ o in more frequent tropical cyclone movements toward East Asia. J. Clim. 26, 2534 2545.

  • Metrics
    No metrics available
Share - Bookmark