Inter Annual Variability of ERA5 Track Forecasts
Mike Fiorino
29 May 2024
18 January 2025
NB: draft version...redid the tracking for ERA5 to analyze in greater detail...stay tuned
The Question...in WPAC...
This plot from my brief at the 2024 INDOPACOM TCC provoked a question about year-to-year jumps in the mean ERA5 72-h (tau72) Position Error (PE) in WPAC for the years 1979-2021:
 |
| Figure 1. Mean 72-h (tau72) PE (nmi) for JTWC v ERA5 1979-2021 |
There are a number of reasons but the first and most important was a bug in my verification code. Thank you Fernando Prates of ECMWF! The well-know variability in JTWC forecasts that follows the 2-y rotation cycle of JTWC directors is also apparent 1980-1998. Since that period, the Director is a civilian...
Another possible reason is year-to-year variability in seasonal forecast difficulty commonly defined by CLImatology-PERsistence error. The basic idea is that seasons with a greater number of non-climatological storms (higher CLIPER PE) are more difficult for NWP and the human to forecast.
The corrected plot below that includes CLIPER PE tells a different story...
 |
| Figure 2. 72-h mean PE for CLIPER (red) v ERA5 (gold). The black line is the smoothed using a Haming filter. |
The smoothed curves tend to follow each other, but note that the minimum in ERA5 is around 2009 and that CLIPER increases faster in the 2010s and 2020s. The biggest spike is in 1985...
and in the atLANTic...
The same plot in the LANT:
 |
| Figure 3. as Fig. 2 but for the LANT basin. The CLIPER model was changed in 2016. |
First note that the CLIPER and ERA5 trends are roughly aligned with ERA5 maintaining a downward trend 2010-2022. The CLIPER models are not the same at JTWC v NHC. I need to get the global 5-d CLIPER model of Sim Aberson to generate a more consistent climatological baseline.
However, there are three spikes in 1980, 1986 and 2013 the are investigated.
Another source of inter-annual variability (IAV) in PE is the actual IAV or some seasons have more TCs than others...
POD...the other metric...WPAC
Probability of Detection (POD) is an under appreciated metric of forecast skill/performance. Basically, it is the percentage of time a model/human forecast is made for a verifying position. One of the most annoying aspects of 'the models' is that they can miss a forecast time, but I has the official forecaster can not making model-v-human comparisons a problem. My forecast is verified, but the model might not and it always seems to happen for the hard storms.
The basic verification rule is that the storm must be a TC at both the initial and verifying time or tau. A season may have 30 TCs and all 30 TCs will be in the tau0 POD stats, but not generally - only 2/3s will have verifying positions at tau72 or the 72-h (3-d) seasonal mean PE. Further, it only takes one big 72-h PE to distort or 'ruin' the mean. Thus, to understand how/where a 72-h mean PE comes from, the 72-h POD and the 72-h PE by storm should be examined.
Here is the 0- and 72-h POD in WPAC:
 |
| Figure 4. tau 0- (top line ~ 97-99%) and 72-h (line below the 0 line ~ 90-95%) ERA5 POD. The grey bar is the count for tau0 (0 h) and the black bar tau72 (72 h) |
Let's compare to actual variability from https://tcact.wxmap2.com/2023.nhem/tc.act.climo.ts.2023.nhem.htm
 |
| Figure 5. sACEd (scaled ACE days = ACE/(35*35kts). WPAC 1979-2022. Green bars are month total, red 30-y climatology. The blue line is the 5-y running mean. |
Visually, the black bars in Fig. 4 align fairly well with the blue line in Fig. 5, but it cannot be perfect because the ERA5 forecast TCs come from a tracker. Neither the ERA5 model itself or the tracker work like nature. The lower ERA5 tau0 POD 1979-2000 comes partly from variability in the observing system and the ability of ERA5 to analyze a TC in the surface wind fields. It is impressive though that in the 2000s ERA5 detects/forecasts TCs at nearly 95%!
POD...the other metric...LANT
The same POD plot in the LANT:
 |
| Figure 6. as in Fig. 4 but for the LANT |
First, the number of cases is smaller in the LANT compared to WPAC, but the tau72 POD is much lower from 1979-2000 and does not reach parity with WPAC until about 2005.
The big difference must be related to the type and number of LANT TCs, but the lower 2010s tau72 PE is a head scratcher. Consider the 1986 season. Here is the 1986 LANT POD:
 |
| Figure 7. 1986 LANT POD/POO (% overwarning of % of cases with a forecast but no verifying TC position). |
The rapid falloff from a decent 93% at tau0 to 30% at tau72 needs to be addressed...
No comments:
Post a Comment