Evaluating Seasonal Forecasts

Seasonal forecasts provide a long-range outlook of changes in the Earth system over periods of a few weeks or months, as a result of predictable changes in some of the slow-varying components of the system -Copernicus

Using Copernicus Climate Change Service information - 2023

Seasonal Forecasts

While there are a number of leading forecast producing centres around the world they all have a varying degree of skill depending on the location and time of the year. Here we will evaluate some of the worlds leading seasonal forecasts to enable users to see if there is any consensus any/or how each model have performed in the past for this time of year.

Multi-Model forecast
Image
ECMWF forecast
Image
UKMO forecast
Image
DWD forecast
Image
METEO forecast
Image
CMCC forecast
Image
NCEP forecast
Image
JMA forecast
Image

Hindcast Skill

For each model the skill is calculated by comparing the modelled hindcasts with the observed values for the same region and season using Gridded Climate Data. For the current set of models the Skill is calculated using the hindcast values from 1993-2016.

  • For the Chance of Above the Median forecasts the skill is determined by using a Hit Score (Percent correct) where a forecasts can be either Right or Wrong (categorical)
  • For the Anomaly Mean and Tercile forecasts the skill is calculated using the Mean Absolute Error (MAE) / Root Mean Square Error (RMSE).

* Please note that there are many different skills testing algorithms which can be used in this framework

Multi-Model skill
Image
ECMWF skill
Image
UKMO skill
Image
DWD skill
Image
METEO skill
Image
CMCC skill
Image
NCEP skill
Image
JMA skill
Image

Multi-Model Forecasts

As some models perform better (or worse) in some regions and at certain times of the year it is not always appropriate to simply combine all the models (either by averaging or combining all ensembles).

By combining all the available models (shown above) we have selected the output with the highest hindcast skill for each region (grid point) and time of year and combined them into a multi-model forecast (prototype product).