Three sources of climate model projection uncertainty are commonly distinguished :

scenario uncertainty, given different anthropogenic emission scenarios of greenhouse gases and aerosols (typical scenarios range from strong mitigation of climate change to unmitigated growth of emissions);

As mentioned above, international climate model development efforts have not resulted in reduced model uncertainty over time e.g.. To address this longstanding issue, a variety of approaches have been suggested to evaluate climate models and to weight their projections, particularly through systematic comparisons of the modelled climate statistics and relationships against those found in Earth observations. Current methods can be broadly separated into two major groups: (a) statistical climate model evaluation approaches and (b) emergent constraints.

The challenge to constrain future projections on the basis of observations is a difficult one. Any attempt to establish robust relationships between the (observable) past and simulated future (unobservable) will be hampered by the non-stationary nature of the climate system. Any information content that can be gained from observations will, naturally and intuitively, diminish as the climate changes. In addition, once relationships of this kind have been put forward, the various methods discussed in Sect. 1.2 typically lead to different suggested constraints for median climate change responses and confidence intervals e.g.. This raises the next central question: which of the methods should we trust (most)? By any means, this is not a small question considering the significant possible impacts associated with future changes in climate.

We identify three broad issues which make progress on this central question particularly difficult and which we suggest can be addressed by incorporating machine learning ideas into observational constraint frameworks. Further limitations are discussed in Sect. IV of . The three we wish to highlight here are as follows:

The indirect nature of the link between the past performance measures and the future response to be constrained. While, nowadays, most emergent constraints are suggested together with a plausible theoretical link between the observable measure and the future response , the connection is always indirect . In many cases, this might, indeed, lead to a scientifically robust relationship; however, this robustness is, in practice, difficult to evaluate objectively. Clearly, the situation is not very different in model evaluation methods which, for example, aim to correlate the historical model-consistent standard deviation in precipitation with its future response. The indirect nature of these links means that one can attempt to manipulate x (the “observed”) in models to better match the observational record. If this leads to the desired improvement in y (i.e. the simulated response) then that would be a targeted way to improve climate models. However, there is clearly no guarantee that apparent improvements in modelling historical x will translate into constrained future responses .

The central idea behind CFA for observational constraints is the training of a function f relating multiple large-scale environmental variables X to a target variable y over time t: 1y(t)≈f(X(t);θ).

Ultimately, we wish to constrain climate model uncertainty in projected changes in y given already observed relationships between X and y. A first major difference compared to emergent constraints is that the functions are trained only on historical data (observations or, for consistency, climate model simulations under historical forcing conditions). The parameters θ, which characterize the function f, can later be considered to be measures of the importance of the controlling-factor relationships found. In this data-driven framework, f (and, thus, θ) can be learned individually from sets of both observational (providing observational functions fobs,m) and climate model data (providing model-derived functions fCMIP,k).

The workflow of the CFA framework is illustrated in Fig. . Expert knowledge is pivotal when selecting the factors X (yellow box) that are thought to “control” y (violet box). However, in contrast to emergent constraints where similar arguments apply to select physically plausible constraints, the physical mechanisms suggested to link the predictors to the predictand can be far more granular in CFA. Distinct thermodynamic and dynamic phenomena driving variability in the predictand can be distinguished, e.g. linking cloud occurrence to a combination of large-scale patterns of sea surface temperatures, relative humidity, and atmospheric-stability measures . Returning to Fig. , machine learning (central grey box) is used to derive the strength of the relationships between the factors and y. The generalization skill of these functions trained on the historical data is easily validated based on independent test data. A good first test case is, again, already observed data or historical simulations (e.g. left-out years not used during training and cross-validation), especially of extreme historical events such as the 2015–2016 El Niño event . Of course, these test data are not used during training and cross-validation or the hyperparameter tuning see longer discussions on these issues in. In Fig. , an example is shown for a hypothetical observational test case for the year 2012 if data from that year were not used for training. We re-iterate that separate functions can be learned and then validated in such a fashion for both observational data (fobs,m) and simulations conducted with various climate models (typically, historical simulations run with different climate models, indexed by k, leading to functions fCMIP,k).

To clarify, emergent constraints in combination with machine learning frameworks have been suggested as well e.g.. However, CFAs are different in two ways: firstly, emergent constraint functions learn from emergent behaviour across climate change responses of an entire model ensemble by correlating variables characterizing the models' past behaviour (e.g. a measure of internal variability) to the model-consistent future responses in a quantity of interest (e.g. the equilibrium climate sensitivity). CFAs instead learn from internal variability and use these relationships in a climate-invariant context to constrain the future response without the latter being involved in the fitting process. Secondly, because the sample size for the relationships learned is no longer limited by the number of models in the ensemble (as is the case for emergent constraints – typically in the range of around 10–60 CMIP models), the general setting is more suitable for the application of machine learning, which strongly depends on the availability of a sufficient number of training samples. The review examples below used monthly mean data. In principle, even much higher temporal resolutions could be used, e.g. up to daily extremes, which might open up new routes for constraining changes in specific extreme weather events .

The next important step is to validate – across a representative climate model ensemble – that the functions learned based on historical data also perform well under climate change scenarios, i.e. if fCMIP,k can also skilfully predict the model-consistent climate change response (indicated by Δ) if provided with model-consistent changes in the controlling factors: 2ΔyCMIP,k(t)≈fCMIP,k(ΔXCMIP,k(t)). Note that, for most predictand and controlling-factor variables, this will pose an extrapolation step in relation to previously unobserved value ranges. As discussed in the Introduction, this extrapolation step under, for example, strong CO₂ forcing poses particular challenges for non-linear ML techniques that one might want to apply to any given CFA. Similarly, it might limit the scope of applying CFA to non-linear observational constraint problems. We see various pathways to address these challenges in CFAs, some of which have not yet been explored in the CFA literature. We will discuss these in Sect. 3.

If the projections are reproduced well across the ensemble of climate models, this implies that the learned relationships are approximately climate-invariant, thus opening up a new link between historically observable relationships and the future climate response, at least to the degree that is currently represented in state-of-the-art climate models. This is exciting because it provides a more direct approach to constrain model uncertainty than emergent constraints are able to provide. In the end, one can simply obtain an observational calibration of each model's response by combining the observed function(s) fobs,m with each individual model response in the controlling factors: 3ΔyCMIP, constrained,k,m(t)=fobs,m(ΔXCMIP,k(t)). Finally, since the machine learning predictions will not be perfect, the resulting distribution of observationally constrained climate model responses will need to be combined further with the method-intrinsic prediction error (see Fig.  and the explanation in its caption) to obtain a final observationally constrained distribution for Δy. Note that we also indexed the function fobs with the index “m” here. The index indicates that both and trained a number of different observational functions to create the observationally constrained distribution for each model to sample and represent observational uncertainty in the relationships learned as well. For simplicity, we have dropped this index in Fig. 2.

Before we discuss the two specific applications of the machine-learning-based CFA framework, it is important to point out two built-in assumptions with regard to the nature of the resulting observational constraints:

By compartmentalizing the prediction of y into two contributors in the form of parameters θ and controlling factors X, the constraint will be based on the observed θobs. However, current versions of CFA do not address uncertainty in the controlling-factor responses across the climate model ensemble, which essentially remains untouched.

Changes in cloud properties (amount, optical depth, altitude) are the leading uncertainty factor in global warming projections under increasing atmospheric CO₂ . A driving force behind this uncertainty is the still relatively coarse spatial resolution of global models, meaning that processes involved in cloud formation have to be parameterized instead of being explicitly resolved. Improvements to parameterizations relying on machine learning ideas have been suggested elsewhere e.g. and will not be discussed further here. Instead, as a first example, we will focus on CFA as an alternative viewpoint to constrain uncertainty in global cloud feedback mechanisms. As such, CFA attempts to find constraining relationships at larger spatial scales, similarly to – but, as outlined above, in important points differently to – emergent constraints. CFAs have already been used extensively to constrain uncertainty related to specific cloud feedback types, though this has primarily been done with low-dimensional multiple linear regression approaches including <10 controlling factors. A few CFA studies used non-linear machine learning methods as well, but to understand historical cloud variations rather than to derive observational constraints on future projections e.g..

Previous observational constraint studies with lower-dimensional multiple linear regression, mostly focused on regionally confined major low-cloud decks e.g. because changes in their cumulative shortwave reflectivity contribute a large fraction to the overall uncertainty in global cloud feedback . Building on this work, developed a statistical learning analysis using ridge regression as a linear form of machine learning. This new approach to CFA allowed them to improve on previous CFA constraints and to expand the scope beyond the low-cloud decks to the global scale for both shortwave (clouds are reflective, thus cooling climate) and longwave (clouds can trap terrestrial radiation, thus warming climate) cloud radiative effects. Here, we will briefly review these results as an example of how CFA can be developed to constrain model uncertainty more effectively by including machine learning ideas. A sketch of the framework is shown in Fig. .

As in previous lower-dimensional CFA for clouds, focused on a relatively short, well-observed period during the satellite era. In their set-up, this translates into a regression approach in which cloud-radiative anomalies at grid point r, dC(r,t), are approximated as a linear function of anomalies in a set of M meteorological cloud-controlling factors dXi(r,t): 4dC(r,t)≈∑i=1M∂C(r,t)∂Xi(r,t)⋅dXi(r,t)=∑i=1Mθi(r)⋅dXi(r,t), where the parameters θi(r) represent the learned sensitivities of C(r,t) to the controlling factors. Here, C(r,t) could, in principle, be different types of measures to characterize cloud contributions to shorter-term variations (here, monthly) and long-term changes (including the climate change response) in Earth's energy budget. separated shortwave from longwave cloud radiative effects, and further common decompositions include high-cloud and low-cloud contributions, as well as changes in cloud fractions, cloud top pressure, and cloud optical depth . As a key difference in relation to previous studies, which focused on grid-point-wise relationships, e.g. between surface temperature at point r and C(r,t), regressed cloud radiative anomalies at grid point r as a function of the controlling factors within a 105° × 55° (long × lat) gridded domain centred on r (Fig. b, c), rendering the regression high-dimensional. The contribution of each controlling factor to dC(r,t) is then obtained by the scalar product of the spatial vectors θi(r) and dXi(r,t).

An important choice is the set of controlling factors. Heuristics that motivate various predictors for low-cloud decks can be found in , and those for high clouds can be found in . In , the authors used five different patterns of cloud-controlling factors, which were used to train the predictions on historical data. However, for an effective constraint on the cloud feedback under abrupt-4 × CO₂ forcing across CMIP5 and CMIP6 models, they only considered two factors that drive the main part of the climate change response (rather than variability), at least when averaged globally. These were patterns of surface temperature (the most important factor) and of the estimated inversion strength (EIS, an important modulating factor, though a different stability measure was used over land). Overall, the study demonstrated that the use of machine learning ideas opens the door to consider a larger spatial context, which improves the CFA function in terms of its predictions and, eventually, also improves the overall observational constraint (Fig. 3d). This further allowed for the extension of CFA frameworks of cloud feedback mechanisms from specific low-cloud analyses to the global scale and to new cloud types (in particular, high clouds; see ).

The linearity assumption appears to work well to the first order for global cloud feedback, but this is not guaranteed for many other uncertain Earth system feedbacks. A first counter-example can be found in , who adapted the framework presented in to constrain uncertainty in changes in specific humidity across the stratosphere. This stratospheric-water-vapour feedback is, indeed, highly uncertain in CMIP models, with model responses ranging from virtually no response to more than a tripling of concentrations relative to present-day values in 4 × CO₂ simulations. This, in turn, makes significant contributions to uncertainties in projections of global warming , the tropospheric-circulation response , and the recovery of the ozone layer .

To address this uncertainty, defined a CFA using ridge regression , in which they predicted monthly mean water vapour concentrations in the tropical lower stratosphere (qstrat) as a function of temperature variations in the upper troposphere and lower stratosphere (UTLS). Their analysis was directly motivated by the strong mechanistic link between tropical UTLS temperature and water vapour entry rates; see, for example, . Their final controlling-factor function was defined as follows: 5log⁡(qstrat(t))=f(θ,T;t,τmax)=∑ilat∑jlong∑kp∑ττmaxθijk,τdTijkt-τ+θ0, which takes into standard-scaled account temperature anomalies dT across a whole longitude–latitude–altitude cube of the tropical to mid-latitude UTLS region over τmax monthly time lags. Using this function, both internal variability in qstrat (for observations and CMIP models) and the long-term climate change response (CMIP models) could be predicted well.

However, under abrupt-4 × CO₂ forcing, the function notably only held true after log-transforming the predictand before training, which apparently led to a quasi-linearization of the relationships to be learned (Fig. ). The need for such a transformation is not unexpected due to the known approximately exponential relationship between temperature and saturation water vapour concentrations and simply underlines that similar CFAs could be designed for many other uncertain Earth system feedbacks, even if non-linear, if appropriate physics-informed transformations can be applied.

As already implied by the stratospheric-water-vapour example, not all relationships we wish to constrain will be linear. For example, while not typically considered in the emergent constraint literature, the aerosol effective radiative forcing (ERF) is defined with reference to an un-observed pre-industrial atmospheric state and so faces many of the same challenges described above (see also Fig. ). Since the relationships between aerosol emissions and cloud properties and between cloud properties and radiative forcing are known to be non-linear , extrapolating from observed to unobserved climate states, while necessary, is fraught with danger.

Besides the obvious risk that, if we naively attempted to fit non-linear functions to such relationships, we could easily over-fit our data, Fig.  shows the opposite risk in that assuming the non-linearities to be small based on the observed data (inset) could lead us to under-fitting the response over larger ranges. If at all possible, we should look to collect observations in these outlying regions, perhaps looking at particularly clean atmospheric conditions in the case of aerosol .

Looking beyond emergent constraints and towards the CFA framework discussed in Sect. 2, we further highlight four strategies to address the extrapolation challenge in non-linear contexts. In our opinion, these strategies have not yet been exploited sufficiently in the existing literature and could be promising pathways for future work:

Linearizations and quasi-linearizations. In the stratospheric-water-vapour example, we demonstrated how linearizing relationships can help tackle non-linear observational-constraint challenges. In particular, prior physical knowledge – such as the approximately exponential relationship between temperature and specific humidity – can be used to transform the regression problem towards a more linear behaviour, thus facilitating extrapolation.

Confounding occurs when an extraneous variable influences both the dependent variable and an independent variable, leading to a spurious association. This is particularly challenging in climate science, where numerous interacting processes can lead to complex relationships between variables. For instance, in the context of Fig. , the apparent influence of an observed variable on an unobserved variable may actually be mediated or obscured by another uncontrolled variable, such as temperature. This confounding can severely compromise the identification and validation of emergent constraints or controlling-factor relationships. Machine learning methods, though powerful in detecting patterns, are not inherently equipped to distinguish causal relationships from mere correlations unless specifically designed to do so. A possibility to address this challenge through causal discovery methods will be discussed in Sect. 4.2.

Clearly, any observational constraint approach that requires climate models to validate the mathematical model used to constrain the future response is potentially affected by blind spots in the ensemble. For example, blind spots could be potentially missing physical mechanisms across all models as implied in, for example, for Southern Hemisphere sea surface temperature changes. This limitation, however, applies in similar ways to all types of approaches discussed here, including classic statistical climate model evaluation, emergent constraints, and CFA. For CFA, this affects the evaluation of the climate-invariance property of the relationships found if they are to be evaluated well beyond historical climate forcing levels.

Still, a well-chosen set of proxy variables as predictors for CFA can, to some extent, help to buffer against such effects. In the stratospheric-water-vapour example, the authors focused on the CO₂-driven climate feedback. As it stands, such an approach brackets out other potential mechanisms for future changes in stratospheric water vapour through chemical mechanisms related to methane or to changes in the background stratospheric aerosol loading . However, the monthly mean temperature variations around the tropopause will naturally integrate multiple mechanisms contributing to water vapour variability, some of which the authors did not explicitly think of during their framework design. Notably, the same variations will never truly reflect the most intuitive mechanism of the immediate dehydration of air parcels during their ascent from the troposphere into the stratosphere. The latter would require a Lagrangian perspective and much higher temporal and spatial resolutions in the data the CFA is applied to. At the same time, other processes potentially contributing to water vapour variations, such as convective overshooting, radiation–circulation interactions, or cirrus clouds , will likely already have an effect in the present day and would thus be part of the observationally derived parameters in the constraint functions (i.e. lowering or increasing the observationally derived sensitivities).

Having said that, what always remains uncertain in CFA is whether the distribution of controlling-factor changes in the ensemble of climate models truly encapsulates their future true response to CO₂ forcing. If not, constraining functions learned from past data might provide a different constraint on the future feedback if combined with a set of controlling-factor responses hypothesized to better represent suggested blind-spot mechanisms. In any case, such tests could be valuable to explore the implications of potential climate model blind spots for the robustness of observational constraints. Specific simulations with a supposedly more mechanistically complete model or simulations subject to larger ranges of values for uncertain climate model parameters (see also perturbed physics simulations discussed in Sect. 4.3) could be useful starting points in this regard. Tests along these lines could provide valuable insights with respect to the sensitivity of CFA observational constraints to varying the assumptions inherent in state-of-the-art climate models.

In many cases, we already have a reasonable approximation of the functional form of a physical response but would like to capture uncertain elements, such as free parameters or closures, in a consistent and transparent way. In the stratospheric-water-vapour example above, this was the known non-linear relationship between temperature and saturation water vapour. In Bayesian terms, we already have an informative prior. As such, using a Bayesian approach can be a powerful way of encoding this information and updating it with observations to provide predictions with well-calibrated uncertainties.

One recent example of this utilizes the functional form of a simple energy balance model (FaIR, in this case; ) as a prior for a Gaussian process (GP) emulation of the temperature response to a given forcing . By constructing the statistical (machine learning) model to respect the physical form of the response, it is able to better predict future warming. Importantly, for this discussion, this approach performs significantly better than an unconstrained GP when making out-of-sample predictions (extrapolating). For example, by training both GPs only on outputs from a global climate model (GCM) representing the historical period, the physical GP is able to accurately predict future warming under SSP2-4.5, while the plain GP quickly reverts to its mean function. This behaviour is not confined to GPs; any highly parameterized regression technique (such as a neural network) would produce spurious results without the strong regularization that the physical form provides. Similarly, physical constraints imposed on machine learning cost functions, as is the case in physics-informed machine learning , could be powerful tools to be used in this context.

Causal discovery and inference techniques allow us to robustly detect potential constraints and to address the challenge of confounding variables, respectively . Methods such as causal discovery or the use of instrumental variables could help in distinguishing true climate signals from confounding noise. Furthermore, enhancing the datasets with more comprehensive metadata that capture potential confounders and applying robust statistical techniques to explicitly model these confounders can aid in mitigating their effects. Such approaches would strengthen the reliability of machine-learning-driven analyses, ensuring that the emergent constraints or CFAs reflect more accurate and physically plausible relationships that hold under various climate change scenarios. An interesting analogy is that with CFA from Sect. 2; significant confounding, which might change the detected historical relationships under climate change, should also lead to a corresponding decrease in the predictive skill of the climate change response under, for example, 4 × CO₂ forcing. As such, poorly performing CFA extrapolations might be a good indicator of poorly designed causal (proxy) relationships among the controlling factors and the predictand.

Perturbed physics ensembles (PPEs) present a significant opportunity in the realm of CFA by allowing researchers to systematically explore the sensitivity of climate models to changes in physical parameterizations. By adjusting various parameters within a climate model, PPEs generate a range of plausible climate outcomes, which can then be analysed to understand how specific processes impact model outputs. This systematic variation of parameters helps isolate the influence of individual factors, thereby providing deeper insights into the workings of climate models than is possible by simply comparing a small ensemble of qualitatively different models.

The utility of PPEs extends beyond the internal processes of models to potentially enhance our understanding of real-world observations. By identifying which parameters and model configurations yield the best alignment with observed climate data, researchers can infer which physical processes might be driving observed changes in the climate system. This transfer of learning from models to learning from observations is crucial for improving the robustness and credibility of climate projections. Moreover, the knowledge gained through PPEs can guide the development of more refined machine learning algorithms that are capable of incorporating complex, non-linear interactions discovered in observations. Thus, combined with the causal discovery approaches outlined above, PPEs not only enrich our understanding of climate models but can also serve as a resource for informing robust (physical) CFAs.