DeRisk SE Asia

Evaluating Seasonal Forecasts

Seasonal forecasts provide a long-range outlook of changes in the Earth system over periods of a few weeks or months, as a result of predictable changes in some of the slow-varying components of the system -Copernicus

Using Copernicus Climate Change Service information - 2023

Seasonal Forecasts

While there are a number of leading forecast producing centres around the world they all have a varying degree of skill depending on the location and time of the year. Here we will evaluate some of the worlds leading seasonal forecasts to enable users to see if there is any consensus any/or how each model have performed in the past for this time of year.

Area:

Element:

Forecast Type:

Outlook:

Multi-Model forecast

ECMWF forecast

UKMO forecast

DWD forecast

METEO forecast

CMCC forecast

NCEP forecast

JMA forecast

Hindcast Skill

For each model the skill is calculated by comparing the modelled hindcasts with the observed values for the same region and season using the the Australian Gridded Climate Data. For the current set of models the Skill is calculated using the hindcast values from 1993-2016.

For the Chance of Above the Median forecasts the skill is determined by using a Hit Score (Percent correct) where a forecasts can be either Right or Wrong (categorical)
For the Anomaly Mean forecasts the skill is calculated using the Linear Error in Probability Space (LEPS) which is a scoring system that awards forecasts for being closer to the observed values and penalises the score for being further away.
Revised “LEPS” Scores for Assessing Climate Model Simulations and Long-Range Forecasts - J. M. Potts; C. K. Folland; I. T. Jolliffe; D. Sexton J. Climate (1996) 9 (1): 34-53.

* Please note that there are many different skills testing algorithms which can be used in this framework

Multi-Model skill

ECMWF skill

UKMO skill

DWD skill

METEO skill

CMCC skill

NCEP skill

JMA skill

Multi-Model Forecasts

As some models perform better (or worse) in some regions and at certain times of the year it is not always appropriate to simply combine all the models (either by averaging or combining all ensembles).

By combining all the available models (shown above) we have selected the output with the highest hindcast skill for each region (grid point) and time of year and combined them into a multi-model forecast (prototype product).