While the growth rate of atmospheric CO

In this study, we propose combining the two approaches to produce global NEE estimates, with the goal of capitalizing on each approach's strengths and mitigating their limitations. We do this by constraining the data-driven FLUXCOM model with regional estimates of NEE derived from an ensemble of atmospheric inversions from the Global Carbon Budget 2021. To do this, we need to overcome a series of scientific and technical challenges when combining information about diverse physical variables, which are influenced by different processes at different spatial and temporal scales. We design a modeling structure that optimizes NEE by considering both the model's performance at the in situ level, based on eddy-covariance measurements, and at the level of large regions, based on atmospheric inversion estimates of NEE and their uncertainty. This resulting “dual-constraint” data-driven flux model improves on information based on single constraints (either top down or bottom up), producing robust locally resolved and globally consistent NEE spatio-temporal fields.

Compared to reference estimates of the global land sink from the literature, e.g., Global Carbon Budgets, our double-constraint inferred global NEE shows a considerably smaller bias in global and tropical NEE compared to the underlying bottom-up data-driven model estimates (i.e., single constraint). The mean seasonality of our double-constraint inferred global NEE is also more consistent with the Global Carbon Budget and atmospheric inversions. At the same time, our model allows for more robustly spatially resolved NEE. The improved performance of the double-constraint model across spatial and temporal scales demonstrates the potential for adding a top-down constraint to a bottom-up data-driven flux model.

The annual growth of carbon dioxide in the atmosphere has been directly measured with high accuracy since 1957

Two different approaches to constrain sources and sinks of carbon dioxide can be distinguished: top-down and bottom-up estimates

The strengths and limitations of each approach are largely inherited from the “point of view” of the system. Top-down approaches, using regional and global observations and mesoscale to global-scale atmospheric transport, view the integral of fluxes from large areas. They produce reliable estimates of the magnitude and variability of latitudinal distribution of NEE, finding solutions that are in line with the global atmospheric growth rate

Bottom-up approaches include a diversity of measurements, from small-scale direct observations at the leaf, plant, plot, and ecosystem scale to remote-sensing observations of relevant proxies (e.g., biomass, greenness)

Previous studies have suggested that observations of atmospheric CO

The carbon fluxes included in estimates from top-down and bottom-up approaches are not exactly the same, as described in detail in

This study uses the same in situ data as used in the FLUXCOM system in

Atmospheric inversions use observed CO

Atmospheric inversions from the Global Climate Project 2022

For the top-down constraint (referred to as “atmospheric”), we use the estimates of the land–atmosphere exchange from five models from the ensemble of atmospheric inversions from the Global Climate Budget

The inversion estimates as provided by

While pixel-level NEE estimated by atmospheric inversions are known to be under-constrained

Note that the ensemble of inversions used here is not the source of the land sink estimate of the GCB2022, which is calculated as the residual land sink derived from other major independent terms in the global carbon budget (emissions from fossil fuels and industry (

This study is based on two models, illustrated in Fig.

The EC model considers only the first term of the objective function (red lines). The EC-ATM model consists of the bottom-up data-driven flux model (red lines) plus an additional constraint derived from atmospheric inversions (orange lines). In the first training pass, the neural network takes meteorological observations from eddy-covariance towers, along with remotely sensed (RS) data to create an inference of NEE, which is compared with the observed NEE in the first term of the objective function. In the second training pass, the neural network takes meteorological and RS variables at pre-selected pixels for each region. The inferred NEE at these pixels is fed into the regional bridge models to create inferences of regional NEE, which are compared with the regional integrals from an ensemble of atmospheric inversions. For global inference the neural network takes global meteorological data from ERA5 along with RS data to estimate NEE for all land pixels (green lines).

In this section, we first present a description of the predictors used for NEE upscale (referred to as driver data (Sect.

We use the driver variables from the FLUXCOM-RS+METEO with ERA5 forcing ensemble as in

The bottom-up data-driven flux model takes as input observations of meteorological and remotely sensed drivers at a location, available from either eddy-covariance towers or satellite platforms, and outputs an inference of the NEE for that location. This bottom-up model consists of a feed-forward neural network or a set of fully connected network layers, which we train using the standard gradient-based backpropagation algorithm

The bottom-up model (EC model) is trained using an objective function with one term which compares the model's inference of NEE and the observed NEE at the eddy-covariance site. The EC model is run and trained only on data from eddy-covariance towers and co-located pixels. The EC model is identical to an ensemble member of the FLUXCOM system

To improve regional and global upscaling performance, this study builds a second model (EC-ATM model), starting with the same bottom-up model, to which we add a second term to the objective function comparing the NEE output, inferred at regional scale using the regional models, with the integrals of regional NEE from an ensemble of atmospheric inversions (Fig.

When calculating the atmospheric term of the objective function, running the bottom-up model for every land pixel and fully calculating the global integral of NEE is too computationally intensive to train a data-driven model in a reasonable time frame. This study solves this problem by using pre-computed statistical models to calculate fast approximations of the regional integral based on a limited number of pixels for each region. We build this set of bridge models from the FLUXCOM RS+METEO NEE results

The bridge models are created using least absolute shrinkage and selection operator (Lasso) regression. Lasso regression extends ordinary least squares (OLS) regression by adding a term to the objective function (Eq.

The dependent variable for the Lasso regression is the regional integral of the monthly ensemble mean NEE derived from the global NEE data from the FLUXCOM intercomparison, specifically the RS+METEO setup

The full set of independent variables, or pixels in the region, are reduced to improve the stability of the regression and to test the reliability of the technique by way of randomized trials. The reduced set of candidate pixels,

The robustness of the sparse linear model was tested by 1000 runs with random stratified sampling within the discovered classes. The output shows stable spatial locations of contributing pixels within each region. A representative heat map of contributing pixels (Fig.

The stability of these spatial regimes indicate that there is a statistical link at the spatial resolution of the analysis between the contributing pixels and the regional integral of the EC model. The contributing pixels cover the range of the PFTs in the region (Fig.

Each data-driven flux model is trained using a 10-fold cross-validation scheme, splitting the eddy-covariance observations by site into 10 equal subsets or “folds”, holding out one fold per training cycle for validation, creating 10 model members. The composition of the folds is the same as those in

At each training step, the EC model neural network

The EC-ATM model has two constraints. The first,

To create the second constraint,

This regional monthly estimate,

This

The two terms of the objective function,

The total loss of the EC-ATM model is then calculated as follows:

After training any specific model, we carefully checked the validity of our assumptions, and the appropriateness of using bridge models. A known limitation of this method is the instability caused by large changes in the learned spatial pattern of NEE during training. These changes can lead to a decoupling between the model response and the NEE data from FLUXCOM RS+METEO V1 underlying the bridge models. This means the

We assume that the spread across the inversion estimates of regional NEE at each training set allows for improved NEE constraints by providing a measure of their uncertainty. In order to evaluate how the EC-ATM NEE estimates depend on this uncertainty constraint, we performed a sensitivity analysis where we trained several models with the atmospheric constraint coming from either one inversion (zero spread), two randomly selected inversions, or three randomly selected inversions, in addition to the standard setup with five inversions. The goal of this analysis is to evaluate how NEE from the EC-ATM model trained with these limited subsets differs from NEE calculated using the full ensemble of inversions. This allows us to better understand how our use of an ensemble of inversions in training, and the uncertainty normalization strategy influence the use of information in the model.

We first compare the global NEE estimates from the EC and the EC-ATM models (Fig.

Panel

The GCB22 results are not available at the RECCAP2 regional level, so to assess the performance of the models at regional scale we use the ensemble of atmospheric inversions for comparison. We acknowledge that the estimates are not fully independent, since they are based on the same data used to train our model, but we expect our training approach (see Sect.

Figure

In annual regional results (Table

Results of annual NEE aggregated by regions. Pearson's correlation coefficient (

We evaluate the performance of the EC and EC-ATM models in capturing the overall temporal variability in NEE by estimating the normalized Nash–Sutcliffe model efficiency (nNSE) metric for regional monthly NEE values over all years (2001–2017) (Fig.

Normalized Nash–Sutcliffe model efficiency over all regions ordered by EC-ATM performance. An nNSE of 1.0 represents perfect skill where the EC-ATM or EC perfectly reproduce the monthly regional integrals of the atmospheric inversion ensemble. An nNSE of 0 represents no skill. An nNSE of 0.5 is where the model predicts the reference better than repeating the annual mean of the atmospheric inversion ensemble.

Mean seasonal cycle of ensemble mean of monthly NEE (Pg C per month) for a representative tropical region (Brazil, BRA), extratropical region (Europe, EU), and the globe for the years 2001–2017. The solid line shows the ensemble mean, and the shaded region is the mean

The mean seasonal cycle (MSC) of the global NEE estimated by the EC-ATM model shows a clear adjustment towards the atmospheric inversion ensemble mean and away from the estimates from the EC model and FLUXCOM RS+METEO results, consistent with the rationale to use a double-constraint approach (Fig.

Results of monthly NEE aggregated by region: Pearson's

The EC-ATM model shows an improvement in the global RMSE of monthly NEE from 1.54 Pg C per month for the EC mean to 0.13 Pg C per month for the EC-ATM model mean (Fig.

In Appendix Fig.

These results indicate that additional atmospheric constraints are indeed reflected in the EC-ATM model at seasonal and continental scales, i.e., the scales where atmospheric data are most informative.

Figure

The spatial patterns of mean annual NEE estimated by the EC-ATM model are considerably different from those estimated by the EC model (Fig.

Mean

It should be noted that there is a known large disagreement between atmospheric inversions in the tropical regions and the location of sources and sinks varies strongly across atmospheric inversion models

Similar to the results of global IAV, the EC-ATM and EC model estimates display much weaker year-to-year variability in annual NEE compared to the mean of inversions (Fig.

Monthly mean fluxes for 2001–2017 for 4 selected months. The left column shows the EC model results, the middle column shows EC-ATM model results, and the right column shows the difference between them (EC-ATM

We further evaluate the spatial distribution of NEE for 4 selected months representative of different seasons: January, April, July, and October (Fig.

The spatial distribution of mean annual and monthly NEE estimated by the EC-ATM model is largely consistent with that estimated by the EC model, while reducing the tropical sink. EC-ATM shows some irregularities in the spatial patterns where there is insufficient information in either the eddy-covariance data or atmospheric inversions to robustly localize the NEE.

Spatial patterns of the per-pixel temporal correlation (Pearson's

We then compare the monthly pixel-level correlation between NEE estimated by the EC and EC-ATM models and by the atmospheric inversions (Fig.

The comparison of EC and EC-ATM modeled NEE at the eddy-covariance sites (Fig.

Comparison of inference of daily NEE from the EC-ATM and EC models with corresponding tower observations, across the whole set of available eddy-covariance observations. The

The RMSE of inferred NEE at the eddy-covariance tower level for the EC and EC-ATM models is very similar across tower sites globally, and by the PFT that is most represented in the pixel, or majority PFT. A breakdown of tower performance by the pixel-majority PFT is available in the Appendix

Comparison of EC and EC-ATM model output at two Brazilian eddy-covariance sites. The

At individual tower sites the EC-ATM model can learn different responses than the EC model. At tower BR-Ji2 (see Fig.

In this study, we aimed to evaluate the hypothesis that combining top-down and bottom-up estimates of land–atmosphere carbon fluxes could contribute to improve the estimates of regional and global land carbon sinks

At the global annual level the EC-ATM results show much closer correspondence with the GCPB22 residual land sink (Fig.

The impact of the complementary or non-complementary function of the combined constraints in the final model can be seen in the differences between tropical and extratropical regions. In general, the EC, EC-ATM, and FLUXCOM results are quite similar in extratropical region, especially in the northern extratropics where the eddy-covariance network is dense and the eddy-covariance data collection method is more robust, and the atmospheric inversions have lower uncertainty (see USA and EU data in Tables

We note that this approach only partly resolved some of the weaknesses of data-driven models. The EC-ATM shows very limited improvement relative to the EC model and the FLUXCOM RS+METEO V1

Another reason for the IAV underestimation in both EC and EC-ATM models might be missing information in the training set, for example due to the fact that semi-arid tropical regions, where the NEE is strongly impacted by climate variance and that account for a very large portion of IAV in the global land sink

While model structure could contribute to this issues, experiments (not shown) with varying architecture (number of neurons, number of layers, layer connectivity), activation functions, and loss terms of the EC and EC-ATM models were not successful in reducing the bias in the magnitude of NEE IAV. We also conducted sensitivity experiments on the objective function both structurally and in its formulation. Structurally, the number of regions included in the atmospheric term at each step and the number of months included at each step were varied. The best results, as indicated in the methods, was considering a full year for each region at each training step.

The sparsity of observations concerns not only the FLUXNET network but also the network of atmospheric monitoring sites used by atmospheric inversions. Given the small number of tropical observations considered in top-down constraints and the inherited uncertainty from priors and transport models, which are not directly managed by the system in this study

The system presented in this study is still dependent on the priors and transport models in the underlying atmospheric inversions and is still subject to the underlying uncertainty of these data, particularly in the tropics where both bottom-up and top-down systems lack sufficient observations. During training, the EC-ATM model tries to account for uncertainties in two ways: the error between the inference and the inversion data is normalized in the loss atmospheric loss term by the spread of the inversion estimates by time and region, which reflects uncertainty across inversion models (Eq.

We perform a “one-against-many” sensitivity analysis, where we trained several models with the atmospheric constraint coming from either one inversion (zero spread), two randomly selected inversions, or three randomly selected inversions. This analysis, shown in the Appendix (Figs.

Our results also show the potential for a confounding effect from the training process. The EC-ATM model is a learned statistical response between the drivers and the training data. There are mismatches between the EC-ATM inference of NEE and the atmospheric data used for the top-down constraint. The atmospheric inversion NEE data, despite being adjusted for fossil fuel, fire, and riverine fluxes, still implicitly includes disturbance and trade fluxes, along with other flux components that are not seen by eddy-covariance measurements and are not accounted for in our model. This means that in reducing the loss terms (Eq.

Our study aimed to demonstrate that adding an atmospheric top-down constraint can positively impact the evolution of a bottom-up data-driven flux model during training, leading to meaningful improvement in local to global NEE estimates. The study demonstrated the positive impact of regional atmospheric information on the training of a well-established data-driven flux model

In this study, we combined regional integrals of NEE from atmospheric inversions with in situ NEE from eddy-covariance measurements. We note, however, that these could be multiple data streams at different scales (temporal, spatial) or in different formats (grid, point). Incorporating these different data streams would require different model formulations, potentially including neural network architecture and objective functions, as well as data-driven or physics-based bridge models to create the link from the data-driven flux model to these new data.

This multi-scale approach proposed here may allow us to leverage large volumes of additional data for constraining a data-driven flux model. By substituting a physical model for the statistical bridge model used here, a double-constraint data-driven flux model could generate an inference of NEE across diverse temporal and spatial scales. Because, unlike the statistical bridge models, these additional data could vary with the local meteorology, covering a range of biomes, the data-driven flux model would see a more diverse training set. This could improve the performance of the data-driven flux model by learning from a more representative distribution of the driver variables across the land surface. In the future, this logic could be used for a variety of datasets, for example by pairing the archive of eddy-covariance observations with tall-tower observations of the mole fraction of CO

RECCAP2 region ID, names, and abbreviations.

RECCAP2 regions.

Driver variables used for the data-driven EC and EC-ATM models and the calculation of the drivers from the base variables above. The global dataset uses only the MODIS and ERA5 data, while the data used at the eddy-covariance sites also uses meteorological observations from the tower instruments. See

All variables are daily values for 2000–2017 at 0.5° spatial resolution.

Hyperparameters for reported EC and EC-ATM model runs.

Model architecture showing that the model is a feed-forward neural network or a set of fully connected network layers. The fully connected layers consist of nodes or “neurons”, which are exposed to the output of all neurons in the previous layer. Nonlinearity is introduced by passing each node output through a nonlinear activation function. Our network is a set of three fully connected layers with the ReLU activation function

Robustness of contribution pixel selection. A heat map of pixel inclusion in the sparse linear model using Lasso regression is shown. Values represent the log-scaled number of pixel inclusions in the non-zero set of parameters across 500 regressions using a randomized subset of the data. Pixels that are most often included provide a more important constraint on the calculation of a regionally summed NEE, minimizing Eq. (

The representation of PFTs across all contributing pixels in all regions. All PFTs are the majority type per pixel. This image shows the relative number of times a certain PFT is included in the optimal set of contributing pixels that construct a regional integral of NEE when selecting from all global land pixels. The black outlines show the proportion of that majority PFT type globally. A per-region analysis of PFT inclusion is available in Appendix

Regional composition of PFT in contributing pixels.

Estimates of NEE from land from recent studies.

MSC of the ensemble mean of all regions (in Pg C per month). The solid line is the ensemble mean, and the shaded region is the mean

Scatterplots of eddy-covariance NEE (

The MSC results for the “one-against-many” training runs. A separate EC-ATM model was trained for each individual inversion system to test the impact of the full ensemble and loss normalization in the full study. The solid lines are the EC-ATM models trained using the named inversion system, and the dotted lines are MSC of the inversion system regional and global integrals. For all panels, the

The MSC results for the “one-against-many” training runs, with EC-ATM models optimized against two inversion systems. The solid lines are the EC-ATM models trained using the named pair of inversion systems, and the dotted lines are MSC of the named pair's mean regional and global integrals. For all panels, the

The MSC results for the “one-against-many” training runs, with an EC-ATM model optimized against three inversion systems. The solid line is the EC-ATM model trained using the named set of inversion systems, and the dotted line is the MSC of the named set's mean regional and global integrals. For all panels, the

Inversion data are available at

SU, AB, MR, and FG designed the study. SU performed the analysis and drafted the manuscript. AB, MR, and WP provided analysis and support. BK provided expertise with the machine learning framework. All authors revised and edited the text.

The contact author has declared that none of the authors has any competing interests.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

We would like to thank Martin Jung, Jakob A. Nelson, Sophia Walther, and the FLUXCOM team for their structural support, feedback, and discussion. The Authors would like to thank the producers of the Inversion data included in this study: Ingrid Luijkx and Wouter Peters (CTE), Frederic Chevallier and the Copernicus Atmosphere Monitoring Service (CAMS), Christian Roedenbeck (Jena Carboscope sEXTocNEET), Yosuke Niwa (NISMON-CO2), and Liang Feng and Paul Palmer (UoE).

This work used eddy-covariance data acquired by the FLUXNET community and in particular by the following networks: AmeriFlux (U.S. Department of Energy, Biological and Environmental Research, Terrestrial Carbon Program (DE-FG02-04ER63917 and DE-FG02-04ER63911)), AfriFlux, AsiaFlux, CarboAfrica, CarboEuropeIP, CarboItaly, CarboMont, ChinaFlux, Fluxnet-Canada (supported by CFCAS, NSERC, BIOCAP, Environment Canada, and NRCan), GreenGrass, KoFlux, LBA, NECC, OzFlux, TCOS-Siberia, and USCCC. We acknowledge the financial support provided to the eddy-covariance data harmonization by CarboEuropeIP, FAO-GTOS-TCO, iLEAPS, Max Planck Institute for Biogeochemistry, National Science Foundation, University of Tuscia, Université Laval, Environment Canada, and the US Department of Energy and the database development and technical support from the Berkeley Water Center, Lawrence Berkeley National Laboratory, Microsoft Research eScience, Oak Ridge National Laboratory, the University of California – Berkeley, and the University of Virginia.

This research has been supported by the European Research Council (ERC) Synergy Grant “Understanding and modeling the Earth System with Machine Learning (USMILE)” under the Horizon 2020 research and innovation programme (grant agreement no. 855187) The article processing charges for this open-access publication were covered by the Max Planck Society.

This paper was edited by Leiming Zhang and reviewed by two anonymous referees.