ICON parameterizes moist convection using a bulk mass flux convection scheme . For cloud ice detrainment from convection, a mean particle diameter of Di,conv = 200 µm is assumed, corresponding to a mean mass of mi,conv = 10⁻⁹ kg. A smaller mean mass would increase the number of detrained ice particles in the upper troposphere, leading to shorter phase relaxation times in convective anvils and reduced ice supersaturation. The assumed size also affects the effective radius of anvil clouds explicitly represented in the model.

For homogeneous ice nucleation, the parameterization by is used. It accounts for the presence of pre-existing ice particles and is applied using grid-scale vertical velocity and ice supersaturation. However, it neglects subgrid-scale variability, which may lead to an underestimation of nucleation events. The impact on cloud ice number concentration is less straightforward. While nucleation events in nature occur on much smaller spatial scales, the model assumes that nucleated particles are evenly distributed across the grid box once the event is triggered.

Heterogeneous nucleation is represented using the INAS (Ice Nucleating Active Sites) approach of , which includes parameterizations for deposition and immersion freezing on mineral dust and soot. Since prognostic aerosol fields are not available in ICON, but only in ICON-ART, a constant dust number concentration of Ndust,0 = 1 × 10³ m⁻³ is assumed in the upper troposphere above p0 = 200 hPa. Below that pressure height the profile increases following 1Ndust(p)=Ndust,0max⁡min⁡exp⁡γdustpp0,200,1 with γdust = 1 × 10⁻³. The dust surface area S‾dust is calculated based on a lognormal particle size distribution with a mean diameter of 1 µm and a standard deviation of 2.5. The number of nucleated ice particles is then diagnosed as: 2Ni*=Ndust1-exp⁡-S‾dustnST,Si. Here, nS is the INAS density in m⁻², parameterized according to Eq. (7) for deposition and Eq. (5) for immersion freezing in .

In numerical models, newly formed ice particles are typically diagnosed each timestep using ΔNi = Ni*-Nipre, where Nipre is the number of pre-existing ice particles. However, this can overestimate heterogeneous nucleation since Nipre is reduced by sedimentation or aggregation, while Ndust remains constant. This effectively creates an unlimited reservoir of ice-nucleating particles. To avoid this artifact, a budget variable is introduced as described in KS15. A relaxation timescale of 2 h is applied to simulate the recovery of nucleating particle availability due to atmospheric mixing.

Radiosonde data from a given station are mostly horizontally fixed and provide dense vertical coverage. To generate matched ICON–radiosonde data pairs, the ICON grid cell center closest to each radiosonde station was first identified. Subsequently, radiosonde observations were linearly interpolated to the ICON levels, as the model provides mean values across vertical layers with considerably lower resolution than the radiosonde data. No horizontal interpolation was applied. However, the impact is expected to be minimal, as typical horizontal scales of ISSRs are on the order of 140 km .

For temporal matching, the start time of the accent was used as a reference, and we select the corresponding ICON simulation whose initial time is closest to the observation time minus the required lead time. Since the simulation provides hourly forecasts, this approach ensures temporal matching to the nearest hour. The exact lead time is explicitly stated in all evaluations and never below the required spin-up time of 6 h.

Over the 14-month verification period, this approach yielded approximately 820 000 spatio-temporal matching points from more than 63 000 radiosonde profiles. Figure b shows example radiosonde profiles of temperature and RH_ice from one station, compared with ICON ensemble values.

IAGOS data represent aircraft-based observations and thus capture horizontal trajectories spanning several hours. Matched ICON–IAGOS data pairs were generated by identifying all ICON grid cell centers that were nearest to at least one point along each flight path. Each selected ICON cell was then paired with its closest flight data point, and the model data were vertically interpolated to match the altitude of that observation. For temporal matching, the minimum lead time was fixed at 6 h to account for the required ICON spin-up. Since flights span several hours, different ICON simulations were used, each selected based on the initial time closest to the observation time minus the 6 h lead time. As ICON simulations are initialized in 6 h intervals, this approach may result in a maximum temporal mismatch of ±3 h.

Over the 4-month verification period, this procedure yielded approximately 200 000 spatio-temporal matching points from 625 flights. Figure e shows a sample time series of temperature and RH_ice from an intercontinental flight, together with the corresponding ICON ensemble values.

To assess the discriminative capability of the EPS, we employ the discrimination diagram, which visualizes two conditional distributions of the forecast probabilities: one conditioned on the event being observed in the measurement data, and the other conditioned on the event not being observed.

These distributions are represented as normalized histograms of the EPS forecast probabilities. A clear separation between the two distributions indicates strong discriminability, reflecting the ensemble's ability to assign higher probabilities to observed events and lower probabilities to observed non-events. This method provides a threshold-independent diagnostic of classification performance in a probabilistic forecasting framework. An example sketch of a discrimination diagram is provided in Fig. c(i).

The ROC curve is a powerful threshold-dependent verification tool to evaluate the performance of a binary classification model. Such a model typically predicts not just a binary label directly, but rather a scalar score (in our context, this score is the predicted event probability). A score is turned into an event prediction if it is above a certain threshold. The threshold itself becomes part of the model; by varying the threshold, we effectively obtain a multitude of models, each with its own POD and FPR. The ROC shows the POD versus the FPR for all of these models at once. The top-left corner corresponds to a perfect classification model. An example sketch of a ROC curve is provided in Fig. c(ii).

Figure b displays the relative frequency distributions of the observed RH_ice compared to the corresponding model-based distributions from the operational ICON 1-Mom (top) and the new ICON 2-Mom (bottom) configurations. Pronounced differences emerge in the tail of the density distribution, which reflects ice supersaturation. ICON 1-Mom exhibits a sharp peak near 100 %, followed by a rapid decline, with maximum RH_ice values reaching only ≈ 103 %. In contrast, ICON 2-Mom more accurately captures the tail structure, slightly overshooting at low supersaturation but successfully reproducing the upper range, including RH_ice values up to 135 %. A few higher values were excluded from the plot due to axis truncation, ensuring comparability without distortion from rare outliers.

We examined the 2D histograms of RH_ice of spatio-temporally matched points between Vaisala RS41 radiosonde data and ICON forecasts (Fig. c). While ICON 2-Mom reproduces the observed supersaturation range reasonably well – unlike ICON 1-Mom – noticeable scatter remains around the one-to-one line. However, perfect agreement between modeled and observed RH_ice values is not strictly required in our context. Crucially, the model must reliably distinguish between ISSR events and non-events, as both have significant operational implications for flight planning and routing. To assess this capability, we proceed below with a verification based on categorical performance metrics.

In the remainder of this study, we consider events of the type RHice>threshold, with threshold ∈ {100%,105%,110%,120%}.

Figure a compares the FBI between ICON 1-Mom and ICON 2-Mom for these events. For the ISSR event (blue curves), the FBI is slightly above 1 for ICON 2-Mom, indicating a modest overprediction, whereas ICON 1-Mom exhibits lower values around 0.75, reflecting underprediction. In both configurations, the FBI remains relatively constant across the examined altitude range. At higher RH_ice thresholds, the FBI for ICON 2-Mom is slightly below 1 at lower heights but rises to a maximum of approximately 1.5 near 12 km for the event RH_ice > 120 %. In contrast, ICON 1-Mom yields an FBI of zero across the entire height range, indicating a failure to detect high supersaturation events. These results demonstrate that the two-moment scheme not only predicts ice supersaturation more frequently than the one-moment scheme – which consistently underestimates event occurrence – but also tends to slightly overestimate observed event frequency.

The POD for ISSR events (RH_ice > 100 %) increases from approximately 0.4 for ICON 1-Mom to around 0.6 for ICON 2-Mom, remaining nearly constant across the altitude range in both configurations. For events defined by higher RH_ice thresholds, ICON 2-Mom retains some detection capability, with POD values gradually decreasing to about 0.15–0.2 for RH_ice > 120 %. In contrast, consistent with the FBI results, ICON 1-Mom fails to detect RH_ice values above 105 %, yielding POD values near zero throughout the vertical domain.

The FPR remains relatively low across all cases, peaking slightly above 0.1 for ICON 2-Mom at RH_ice > 100 % (Fig. (c)), indicating a limited tendency toward false alarms.

Both schemes yield similar precision values between 0.5 and 0.55 for ISSR events across the entire altitude range (Fig. d). For higher RH_ice thresholds, the precision of ICON 2-Mom declines progressively, reaching values as low as 0.2 for RH_ice > 120 %. In contrast, ICON 1-Mom produces very few or no positive predictions in these regimes, rendering precision largely undefined; accordingly, it is omitted for these cases.

In the context of flight planning, accurate prediction of non-ISSR conditions is equally critical, as false negatives in this category can lead to unnecessary re-routing and, consequently, avoidable increases in CO₂ emissions. When treating the complementary events (RH_ice ≤ threshold) as “positive” events, the model exhibits high precision, with average values exceeding 0.9 across all threshold levels. Combined with the low false positive rate observed for RH_ice > threshold events, this high precision underscores the reliability of ICON 2-Mom in correctly identifying non-ISSR conditions.

The MCC shown in Fig. e summarizes overall classification performance. For ISSR/non-ISSR classification (blue curves), ICON 2-Mom achieves an average MCC of 0.47, indicating moderate predictive skill. In comparison, ICON 1-Mom yields consistently lower values between 0.38 and 0.39. At higher RH_ice thresholds, the MCC of ICON 2-Mom declines progressively, reaching a minimum of approximately 0.16. In contrast, MCC values for ICON 1-Mom approach zero or become undefined where the numerator vanishes, reflecting a lack of predictive capability in these regimes.

In summary, for ISSR events, ICON 2-Mom achieves a moderate MCC of nearly 0.5 and a POD that is approximately 50 % higher than that of the operational ICON 1-Mom, while maintaining a relatively low FPR below 0.1 across most altitudes. Despite this improvement, a POD of 0.6 indicates that a substantial fraction of events remains undetected. To address this, we continue to investigate potential gains from the ensemble setup introduced in Sect. .

To evaluate the ensemble's ability to distinguish between ISSR and non-ISSR conditions, we consider the discrimination diagram introduced in Sect. . Figure a shows the conditional distributions of forecast probabilities for observed and non-observed events (events are defined as RH_ice > 100 %, and higher thresholds). For ISSR events, the “not observed” distribution peaks sharply at zero and declines rapidly, indicating strong agreement among ensemble members when no supersaturation is present. In contrast, the “observed” distribution is relatively uniform, suggesting that the ensemble assigns a broad range of probabilities to actual events. As the threshold for supersaturation increases, the “observed” distribution becomes more left-skewed and increasingly overlaps with the “not observed” distribution, indicating a decline in discriminative skill for more extreme events.

To conclude, for the ISSR event, the diagram shows little overlap between the two conditional forecast probability distributions below and above ∼0.3, suggesting that a threshold-based conversion of forecast probabilities aimed at classifying ISSR versus non-ISSR may be appropriate.

As ICON 2-Mom EPS consists of ten ensemble members, ISSR forecast probabilities can be 0, 0.1, 0.2, …, 1. Thus, these values represent the relevant potential thresholds to turn the event forecast probability into an event prediction – yielding classification models as introduced in Sect. . We refer to these classification models as decision models, with the k-out-of-10 decision model (or simply decision model k) defining the threshold as k/10: if at least k of the 10 ensemble members predict the event, the model outputs a positive prediction: k-out-of-10 decision model:pconv=1,ifp≥k10,0,otherwise. To evaluate the performance of these pseudo-deterministic decision models, we use the ROC curve (Sect. ). For the ISSR event, the ROC curve (Fig. b) shows strong discriminative skill for thresholds of 0.2 and 0.3 (decision models 2 and 3), with POD > 0.8 and FPR < 0.17. A comparison between the scores of the EPS-based decision models and the deterministic ICON 2-Mom model (inset of Fig. b) shows a substantial improvement in the POD for ISSR events, from approximately 0.6 in the deterministic case to 0.8–0.9 when using ensemble-based decision models. This gain in POD is accompanied by a moderate increase in the FPR, rising from ∼ 0.1 to values between 0.13 and 0.23, depending on the chosen threshold. These results highlight the added value of ensemble forecasts in enhancing event detection or classification.

While the ROC curve provides a comprehensive view of classification performance across thresholds, it treats both classes equally and may obscure performance nuances in the presence of class imbalance. Therefore, we also evaluate the precision–recall (PR) curve, which focuses specifically on the model's performance on the positive class. Similarly to the construction of the ROC curve, the PR curve plots the recall (equivalent to POD) against precision. In Fig. a, each EPS-based decision model is represented as a point on the PR curve. The closer the points are to the top right corner, the higher the recall and precision. Although recall remains high even for intermediate thresholds, overall precision is only moderate and deteriorates further for more extreme supersaturation events. This reflects the increasing difficulty of making accurate positive predictions as the event definition becomes more stringent.

We also conduct a complementary analysis by treating the complementary conditions (e.g., non-ISSR) as the positive events, as this perspective is equally relevant for flight routing applications. The PR curve approaches the top right corner, reflecting both high POD and precision, and a zoomed-in view providing details of this region is shown in Appendix  Fig. a.

To conclude this subsection, we shift to a more holistic model assessment using the MCC, as introduced in Sect. . The MCC provides a balanced measure of classification skill across both event and non-event categories, making it particularly valuable in the context of imbalanced datasets.

For the ISSR/non-ISSR classification, EPS-based decision models 1–7 consistently outperform their deterministic counterparts (i.e., individual ensemble members), with decision models 3 and 4 achieving the highest MCC values of approximately 0.55. In contrast, the deterministic models yield MCC scores around 0.47 (see Fig. c). These results reinforce the advantage of ensemble-based decision strategies in capturing both sides of the classification task more effectively.

Table  summarizes the maximum MCC values achieved for each ice supersaturation threshold, along with the indices of the corresponding EPS-based decision models. For reference, MCC values of the deterministic (single-member) models are shown in brackets. The final columns report the associated POD and FPR values, enabling direct comparison with ROC-based performance. In most cases, the decision models with highest MCC also show favorable POD–FPR combinations, underscoring their robustness across metrics. For the remainder of this study, we focus on ROC-based evaluation using its associated scores, POD and FPR, as a representative framework for assessing decision model performance.

The RH_ice density of the IAGOS data, limited to the Northern Hemisphere for better comparison with our radiosonde verification, confirms the characteristic bimodal shape of the RH_ice density (see inset of Fig. ). Compared to the ICON data, the first peak in the IAGOS density appears shifted to the right, suggesting fewer near-zero RH_ice values in the IAGOS dataset than in the ICON data. The peak around RH_ice ≈ 100 % is shifted to the left and is less pronounced in the IAGOS data. It also does not reach the same high RH_ice values as ICON.

Nevertheless, up to RH_ice > 120 %, the shape of the ROC curves (see Fig. ) derived from the IAGOS data closely resembles those derived from the radiosonde data (compare Fig. b). These findings strengthen our verification insights across different, independent observation systems.

Although for many flights 12 h forecasts are sufficient, we now consider lead time increments from 12 h up to a maximum of 48 h, which is the standard time horizon of weather forecasts for flight planning. Figure  shows that – as the lead time increases – the ROC curves shift slightly to the right, indicating higher FPR. In contrast, no downward shift of the ROC curves is observed for high POD values of around 0.8 for the first 36 h and the POD only starts to decrease after 36 h. Overall, the degradation is not that severe, and at least up to 36 h, potential scores remain roughly in the range of POD > 0.8 and FPR < 0.2.

We further incorporate ensemble spread information in order to get more reliable scores in more specific situations. The ensemble spread should be an indication of the confidence in a forecast and is typically measured by the standard deviation (SD). Therefore, in the context of ISSR forecasts, we further differentiate the ROC curve based on the underlying SD at each grid point, particularly to achieve a lower FPR.

The inset of Fig. a shows a histogram of the standard deviation of RHice/100 %; more than 50 % of the ensemble forecasts have a SD below 0.1, with a peak near zero, and only a small proportion have SD values greater than 0.2. The colored bins in the histogram serve as a legend for the ROC curves in the main Fig. a: the EPS forecasts are partitioned with respect to their SD and the corresponding ROC curves are shown in the same color. The trend is consistent with our expectation; the lower the SD, the closer the corresponding ROC curve is to the upper left corner, and vice verse, the higher the SD, the closer the ROC curve is to the diagonal, indicating that the model has low skill in these cases. In particular, a significantly improved ROC shape is obtained in more than half of the cases – with POD of 0.9–1 and FPR ≤ 0.1 via decision models 1–2. In case the SD is greater than 0.1, the ROC curves tend more and more to the diagonal and at least five or six members should indicate ice supersaturation to achieve an FPR of ≤ 0.1. In these cases – depending on the specific SD – only a lower POD of 0.3–0.8 can be obtained.

Given the significant variation in ROC curve shapes across different SD regimes, we analyze the SD values of different RH_ice regimes, particularly when RH_ice is around or above 100 %. In Fig. b, summary statistics of SD are shown for increasing 10 % bins of RH_ice. Following an increase in SD values, they decrease before 100 % and reach another local minimum in the RH_ice regime of 100 %–110 % with a median around 0.1. The relative mean squared error (RMSE) shows a similar qualitative behavior up to RH_ice < 120 %. For higher RH_ice regimes, the RMSE increases to its maximum over the whole range of values.

In Fig. c, the full RH_ice histograms of the observations and the ensemble forecasts are shown, as well as both conditioned on SD ≤ 0.1; in the case of the observations this is done by assigning the SD-value of the corresponding spatio-temporally matched EPS point. For low SD-values (SD ≤ 0.1), the corresponding conditional RH_ice histograms show a large peak for low humidity values in the same range as the full unconditioned histograms. Another peak is observed for RH_ice values around 100 %, which is approximately one order of magnitude lower than that of the unconditioned histograms. This difference persists in the supersaturation regime of the histograms, where the maximum RH_ice values reached in the conditional case are around 130 %, based on the 820 000 verification points (where all counts below 100 were cut in this plot). When comparing the conditional histograms of the model and the observations, the observation histogram exhibits a slightly lower peak around 100 %, similar to the difference observed in the full histograms. In conclusion, even when the model exhibits high confidence, as reflected by a low SD, the histogram still displays intermediate supersaturation. This suggests that certain ISSRs can be well predicted.

The increased predictability in the regime around RH_ice ≈ 100 % can be explained by a more stable microphysical behavior in this near-thermodynamic equilibrium state, which is captured by the model. In this regime, mature cirrus clouds are dominant compared to young or short-lived cirrus clouds which often form in regions of high ice supersaturation, driven by upward motion from gravity waves or deep convection. These young clouds experience rapid crystal growth due to significant mesoscale temperature fluctuations caused by gravity waves, which create high spatio-temporal variability in supersaturation. The fluctuating vertical motions and ice crystal concentrations make forecasting cloud evolution difficult. As a result, young and short-lived cirrus clouds introduce significant uncertainty in predicting supersaturation, as the microphysical processes are highly dynamic and rapidly changing. In contrast, mature cirrus clouds, approaching thermodynamic equilibrium (RH_ice ≈ 100 %), display weak supersaturation conditions, typically linked to slow, steady-state ascent. Under these conditions, ice crystals grow and gradually deplete ambient water vapor, creating a balanced system that enhances the predictability of ice crystal evolution and overall cloud dynamics.

In clear-sky regions, where clouds and associated microphysical processes are absent, the predictability of RH_ice is governed primarily by large-scale thermodynamic and dynamical processes. Supersaturation can persist in these regions due to the lack of ice nuclei. Observations show that clear-sky supersaturation is often associated with weak vertical motions and low temperatures in the upper troposphere, particularly in mid- and high-latitude regions . However, mesoscale temperature fluctuations caused by gravity waves can still occur, challenging predictability, particularly for models that do not resolve mesoscale temperature or humidity fluctuations. Overall, while the absence of cloud feedbacks simplifies the microphysical environment, potential variability in temperature, humidity, and vertical motion still introduces uncertainty, i.e., the predictability of RH_ice in clear skies depends on the given specific large- and mesoscale thermodynamic and dynamical processes.