Evaluating Machine Learning–based probabilistic lightning forecasts using the HRRR: A comparison to three forecast baselines

These NN probability forecasts (NNPFs) were produced for HRRR forecasts in 2021–22 using NNs trained with 0000 UTC HRRR forecasts from 2019 to 2020 paired with ≥1 observed cloud-to-ground (CG1) flash as the target variable. CG1-NNPF skill was evaluated against three baselines: smoothed HRRR-based surrogate lightning forecasts, SPC probabilistic thunderstorm outlooks, and calibrated ensemble thunderstorm guidance from the High-Resolution Ensemble Forecast (HREF) system.

The hourly maximum updraft speed (UP) diagnostic was the most skillful HRRR surrogate for predicting CG1, outperforming diagnostics such as hourly maximum lightning threat and updraft helicity. The UP forecasts were compared to the NNPFs by using the most skillful combination of UP threshold and smoothing length scale at each forecast hour. The 4- and 1-h CG1-NNPFs exceeded the skill of the UP forecasts in 2021 for nearly all forecast hours, with reduced skill differences overnight compared to the daytime.