Technical note: Evaluation of the Copernicus Atmosphere Monitoring Service Cy48R1 upgrade of June 2023

The Copernicus Atmosphere Monitoring Service (CAMS) provides daily analyses and forecasts of the composition of the atmosphere, including the reactive gases such as O₃, CO, NO₂, HCHO and SO₂; aerosol species; and greenhouse gases. The global CAMS analysis system (IFS-COMPO) is based on the ECMWF Integrated Forecasting System (IFS) for numerical weather prediction (NWP) and assimilates a large number of composition satellite products on top of the meteorological observations ingested in IFS. The CAMS system receives regular upgrades, following the upgrades of IFS. The last upgrade, Cy48R1, operational since 27 June 2023, was major with a large number of code changes, both for IFS-COMPO and for NWP. The main IFS-COMPO innovations include the introduction of full stratospheric chemistry; a major update of the emissions; a major update of the aerosol model, including the representation of secondary organic aerosol; several updates of the dust life cycle and optics; updates to the inorganic chemistry in the troposphere; and the assimilation of Visible Infrared Imaging Radiometer Suite (VIIRS) aerosol optical depth (AOD) and TROPOspheric Monitoring Instrument (TROPOMI) CO. The CAMS Cy48R1 upgrade was validated using a large number of independent measurement datasets, including surface in situ, surface remote sensing, routine aircraft, and balloon and satellite observations. In this paper we present the validation results for Cy48R1 by comparing them with the skill of the previous operational system (Cy47R3), with the independent observations as reference, for the period October 2022 to June 2023, during which daily forecasts from both cycles are available. Major improvements in skill are found for the ozone profile in the lower–middle stratosphere and for stratospheric NO₂ due to the inclusion of full stratospheric chemistry. Stratospheric trace gases compare well with the Atmospheric Chemistry Experiment Fourier Transform Spectrometer (ACE-FTS) observations between 10 and 200 hPa, with larger deviations between 1 and 10 hPa. The impact of the updated emissions is especially visible over East Asia and is beneficial for the trace gases O₃, NO₂ and SO₂. The CO column assimilation is now anchored by the Infrared Atmospheric Sounding Interferometer (IASI) instead of the Measurements Of Pollution in The Troposphere (MOPITT) instrument, which is beneficial for most of the CO comparisons, and the assimilation of TROPOMI CO data improves the model CO field in the troposphere. In general the aerosol optical depth has improved globally, but the dust evaluation shows more mixed results. The results of the 47 comparisons are summarised in a scorecard, which shows that 83 % of the evaluation datasets show a neutral or improved performance of Cy48R1 compared to the previous operational CAMS system, while 17 % indicate a (slight) degradation. This demonstrates the overall success of this upgrade.

1 Introduction

The Copernicus Atmosphere Monitoring Service (CAMS, http://atmosphere.copernicus.eu, last access: 7 June 2024) is the service component of the European Earth Observation programme Copernicus focusing on atmospheric composition (Peuch et al., 2022). The CAMS system was developed during a sequence of European research projects (Hollingsworth et al., 2008) and became an operational service in 2015. The CAMS global near-real-time (NRT) service provides daily analyses and 5 d forecasts of reactive trace gases and aerosol concentrations, as well as delayed-mode greenhouse gas analyses and forecasts. CAMS makes use of the measurements of the fleet of Copernicus Earth observation satellites, the Sentinel missions, as well as other relevant satellite and surface observations. Apart from the near-real-time products, CAMS produces global reanalyses for reactive gases and aerosols (Inness et al., 2019; Flemming et al., 2017) and greenhouse gases (Agustí-Panareda et al., 2023). CAMS furthermore produces daily forecasts and analyses of air quality in Europe based on an ensemble of air quality models (Peuch et al., 2022; Marécal et al., 2015).

The global CAMS system is part of the Integrated Forecasting System (IFS) of ECMWF, a system used to produce medium-range numerical weather predictions (NWPs) (ECMWF, 2024h). The modelling of reactive gases, aerosols and greenhouse gases is fully integrated in IFS (ECMWF, 2023). The assimilation of satellite data for atmospheric composition is part of the IFS 4D-Var data assimilation suite, and CAMS assimilates the complete set of meteorological observations together with the composition data. CAMS forecasts are therefore combined weather–composition forecasts. The extended IFS system developed by CAMS for trace gases and aerosols is referred to as IFS-COMPO. A second CAMS system, IFS-GHG, is developed for CO₂ and CH₄ analyses, (high-resolution) forecasts, and reanalyses. This paper focuses on the evaluation of the IFS-COMPO NRT products.

The CAMS effort (Peuch et al., 2022) includes dedicated scientific activities to continuously develop and improve the modelling and satellite assimilation aspects of IFS-COMPO, including the chemistry code (Flemming et al., 2015; Huijnen et al., 2016, 2019) and the development of the aerosol scheme (Rémy et al., 2019, 2022). The data assimilation of reactive gases is described in Inness et al. (2015). Recent extensions relevant to Cy48R1 include the modelling of stratospheric chemistry (Huijnen et al., 2016; Chabrillat et al., 2023) and the assimilation of TROPOspheric Monitoring Instrument (TROPOMI) CO (Inness et al., 2022b) and Visible Infrared Imaging Radiometer Suite (VIIRS) aerosol optical depth (AOD) (Garrigues et al., 2023). Since Cy48R1, the IFS upgrade documentation has also included a chapter with a detailed discussion of the IFS-COMPO-related modelling and data assimilation changes (ECMWF, 2023).

The CAMS service makes extensive use of independent measurement datasets of proven quality, available for different parts of the world, to evaluate the quality of the forecast, analysis and reanalysis products (Eskes et al., 2015; Katragkou et al., 2015; Lefever et al., 2015; Cuevas et al., 2015; Wagner et al., 2021; Agustí-Panareda et al., 2023). A total on the order of 65 measurement collections are used, obtained from observational networks like the Network for the Detection of Atmospheric Composition Change (NDACC), the World Meteorological Organization Global Atmosphere Watch (WMO-GAW) programme, the Aerosol Robotic Network (AERONET), the In-service Aircraft for a Global Observing System (IAGOS), the Integrated Carbon Observation System (ICOS) and the International Arctic Systems for Observing the Atmosphere (IASOA) and from space organisations like the National Aeronautics and Space Administration (NASA), European Space Agency (ESA), the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), and others. The CAMS global near-real-time service component is evaluated in a series of validation reports that are produced every 3 months (Benedictow et al., 2023; ECMWF, 2024d). The CAMS reanalysis validation reports for the aerosols and reactive gases (Bennouna et al., 2023) and greenhouse gases (GHGs) (Ramonet et al., 2021) are updated on a yearly basis. Note that the production of the GHG reanalysis was interrupted (Agustí-Panareda et al., 2023) for a few years due to a degraded quality of the input satellite data. New system upgrades are evaluated before becoming operational, with CAMS Cy47R3 discussed in Eskes et al. (2021) and Cy48R1 in Eskes et al. (2023b).

The ECMWF NWP and CAMS operational systems are upgraded at the same time and follow the same model cycles. A CAMS upgrade implies, therefore, a simultaneous upgrade of the NWP part of IFS in IFS-COMPO and IFS-GHG. The last upgrade (and topic of this paper) is the upgrade from Cy47R3 to Cy48R1, implemented on 27 June 2023. Upgrades normally occur at least once a year, but Cy48R1 includes almost 2 years of developments. The delay is linked to the move of the ECMWF high-performance computing facilities from Reading, UK, to Bologna, Italy. As a result, Cy48R1 represents a major upgrade both for NWP and for CAMS. Detailed information about the ECMWF IFS upgrades can be found in ECMWF (2024h).

The CAMS pre-operational analyses and forecasts of CO₂ and CH₄ use an independent configuration of the IFS. The upgrade of the greenhouse gas system to Cy48R1 took place on 27 February 2024, and a separate upgrade verification report was written (Langerock et al., 2024). The greenhouse gas products will not be discussed in this paper.

Before each upgrade, the new analysis and forecast configuration is operated as a so-called experimental suite or e-suite in parallel to the operational near-real-time service (the o-suite, e.g. Cy47R3 in this paper) for about half a year. For the 27 June 2023 upgrade to Cy48R1, the e-suite run is available from 1 October 2022 to the moment of the upgrade, 27 June 2023. This implies that there is only a limited 9 months of data in the autumn, winter and spring available to the evaluation, with a focus on wintertime. The choice to generate 9 months of the e-suite data is made based on practical timing and computer resource considerations. The length of the e-suite run has been discussed in the CAMS team, and for the next upgrade to Cy49R1, planned for the end of 2024, the e-suite will hopefully cover a full year. CAMS also produces control runs without the assimilation of atmospheric composition data. The control runs allow us to distinguish changes due to the model from changes due to the assimilation of the atmospheric composition observations.

In this paper, the validation results are presented from comparisons of the performance of the Cy48R1 e-suite runs and e-suite control run with the operational run (o-suite and o-suite control run), all compared with independent observations. Prior to the upgrade, a preliminary evaluation of the new cycle was presented in a CAMS report (Eskes et al., 2023b), but this evaluation covered a relatively short period of about 5 months of CAMS results that were available at the time. This paper extends the validation by several months by including e-suite and o-suite analyses and forecasts up to the day of the upgrade, leading to more refined conclusions.

In Sect. 2 we summarise the changes implemented in Cy48R1, and in Sect. 3 we provide an overview of the validation datasets used. The validation results are shown in Sect. 4 in the form of a series of plots comparing e-suite, o-suite and independent observations.

2 Overview of the CAMS global system

The operational CAMS global system provides analyses of the atmospheric composition of aerosols and reactive gases worldwide by blending satellite data and atmospheric simulation through a process called data assimilation. An overview of all satellite datasets assimilated in the CAMS global system can be found in Peuch et al. (2022) and Eskes et al. (2023b). Further details on the different production runs and their data usage can be found on the CAMS global product website (ECMWF, 2024e). The history of CAMS system upgrades, the data products, the satellite data assimilated (and monitored) and relevant references are available on the CAMS data documentation website (ECMWF, 2024b), while the results of the operational satellite data monitoring are available on the CAMS data-monitoring website (ECMWF, 2024c). CAMS is continually extending its activity by testing and using new emerging datasets such as trace gas retrievals from the geostationary instruments, the Geostationary Environmental Monitoring Spectrometer (GEMS) and Tropospheric Emissions: Monitoring of Pollution (TEMPO); using improved retrievals from past and present missions; and preparing for future missions such as Sentinel-4 and Sentinel-5.

Copernicus products are made available for free. The CAMS products can be accessed through the Atmosphere Data Store (ADS) at https://ads.atmosphere.copernicus.eu/ (last access: 7 June 2024). The available o-suite output includes analysis fields available every 3 h, for the two 12 h analysis windows per day: twice-daily 5 d forecasts starting from the analysis at 00:00 and 12:00 UTC, with 2D fields available hourly and 3D fields available every 3 h. The output includes a large number of trace gases, aerosol composition and optical properties of the aerosols, as detailed on the ADS. Most of the comparisons documented below make use of the first 24 h of the forecast runs starting 00:00 UTC, which allows a direct comparison with the control run. For some comparisons the analysis results were used, as indicated.

A control run is produced for both the e-suite (e-suite control run) and the o-suite (o-suite control run). These runs are useful for distinguishing the impact of data assimilation changes from the impact of modelling changes. The control run applies the same settings as the respective o-suite and e-suite, but the data assimilation is not switched on for atmospheric composition. The meteorology in the control run is initialised with the meteorological fields from the o-suite (or e-suite). The control run archive contains 5 d forecasts starting from the 00:00 UTC analysis.

The global forecasting system is continually being evaluated to ensure the output meets the expected requirements. Comprehensive evaluation and quality assurance (EQA) reports are provided for the CAMS system on a quarterly basis (ECMWF, 2024d). CAMS uses a multitude of independent datasets to routinely monitor its global forecasts. It works with various data providers, acquiring the observations with appropriate timelines and generating graphics that show the differences between the forecasts and the independent observations (ECMWF, 2024g). The two main websites are the global evaluation server for near-real-time analysis and forecast products (ECMWF, 2024f) and the AeroVal evaluation server to evaluate the reanalysis products (ECMWF, 2024a).

In the next subsections, the model, data assimilation and emissions upgrades of the CAMS e-suite (Cy48R1) are summarised, and the observations used are introduced.

2.1 The CAMS e-suite (Cy48R1)

This upgrade encompasses several significant scientific advancements. Detailed documentation of the CAMS Cy48R1 upgrade for aerosols and reactive gases is available in ECMWF (2023). More information regarding the aerosol/chemistry changes and the meteorological changes is provided in ECMWF (2024i) and CAMS (2024), respectively. In the following subsections the model, assimilation and emissions updates are summarised.

2.1.1 Model updates

The CAMS IFS cycle 48R1 is based on the ECMWF's IFS cycle 48R1. The model consists of gas-phase chemistry modules for the troposphere (based on the CB05 scheme; see Williams et al., 2022, and references therein) and the stratosphere (based on the Belgian Assimilation System for Chemical ObsErvations (BASCOE) scheme; see Errera et al., 2019, and Huijnen et al., 2016, and references therein) that includes 123 active tracers. The distinction between the tropospheric and stratospheric chemistry schemes occurs at the tropopause, which is determined based on the temperature lapse rate of the model. The aerosol scheme AER (originally based on the Laboratoire de Météorologie Dynamique general circulation model (LMDZT GCM); Reddy, 2005) is a bulk-bin scheme that consists of 16 active species, which are coupled with the chemistry schemes in various ways (ECMWF, 2023).

Before Cy48R1, only ozone was modelled in the stratosphere using a linear parameterisation. The stratospheric chemistry module activated in Cy48R1 (Huijnen et al., 2016) is a re-implementation of the BASCOE chemical module, and it involves 64 species engaging in 157 gas-phase, 9 heterogeneous and 53 photolytic reactions (Errera et al., 2019). It encompasses ozone-depleting substances, greenhouse gases and other species vital for comprehensive stratospheric ozone photochemistry (Chabrillat et al., 2023). Additionally, basic sulfur chemistry is included to represent gas-phase sulfuric acid formation and enable coupling with the sulfate aerosol module, featuring OCS, SO₂, SO₃ and H₂SO₄ reactions. Note that the 64 tracers listed as active in stratospheric chemistry include HCN and CH₃CN, as well as the tracers to represent sulfur chemistry, which are additions specific to CAMS, on top of the default BASCOE chemistry with 58 species described in Errera et al. (2019).

In the troposphere, inorganic chemistry was updated; hydrogen cyanide (HCN) and acetonitrile (CH₃CN) are now included as long-lived tracers, serving as indicators of biomass burning activity. For the degradation of organic compounds, the basic isoprene oxidation scheme was replaced with a more explicit approach based on Stavrakou et al. (2010) and further modified according to Lamarque et al. (2012) and Myriokefalitakis et al. (2020). This updated scheme contains reaction products including glyoxal (CHOCHO), glycolaldehyde, isoprene peroxide, hydroxyacetone and two hydroxyaldehydes (Williams et al., 2022). The scheme was further expanded to include an explicit parameterisation of the aromatics tracers xylene and toluene, acting as precursors for secondary organic aerosol (SOA).

Recent developments of the CAMS aerosol modelling are described in Rémy et al. (2022). The Cy48R1 upgrade introduced notable changes in aerosol. The SOA species are now represented with dedicated tracers, distinguishing biogenic and anthropogenic origins, and coupled with the tropospheric chemistry for their production. The e-folding time that converts hydrophobic components of organic matter (OM) and black carbon (BC) into hydrophilic forms has been decreased to 2.8 h from 1.16 d. The assumed number size distribution for dust, which used to compute the offline dust aerosol optical properties (mass extinction, single-scattering albedo and asymmetry parameter), has been updated based on values from Ryder et al. (2018) derived from aircraft measurements over the tropical eastern Atlantic. The refractive index has also been updated. The dust source function, which is used to modulate dust emissions, has been recomputed based on a 3-year analysis of the MIDAS product (Gkikas et al., 2021), leading to monthly variations instead of fixed yearly values. A regional redistribution of total dust emissions into the three dust bins has been implemented, based on long simulations of dust mineralogy, which directs relatively high emissions to finer dust bins (1 and 2) compared to Cy47R3. The dust mass emissions and burden are significantly higher in Cy48R1 (see details in ECMWF, 2023), which leads to an increase in dust optical depth globally by about 30 %. Note however that the mass extinction coefficient (MEC), which is calculated by dividing aerosol optical depth by aerosol mass, is reduced for dust in Cy48R1 (Fig. S1 in the Supplement). Thus, although dust optical depth increases, dust mass extinction efficiency decreases. A new parameterisation for the rebound effect of super-coarse dust particles over continental surfaces, relying on Zhang (2001), was added, which reduces dry deposition for those particles. In addition, sedimentation, previously limited to super-coarse dust and sea salt, is now applied broadly to all aerosol tracers, although the impact is mainly significant in the stratosphere, where sedimentation is the dominant sink.

The aerosol optical properties of aerosols have received several updates in Cy48R1. The inclusion of a specific SOA species distinguishes primary from secondary OM. The new set of optical properties for OM, based on Brown et al. (2018), leads to higher extinction, particularly in low-relative-humidity conditions. Notably, the refractive index used in Cy48R1 results in more absorbing organic matter in the UV and near-UV regions, characterised as brown carbon. Further, a scaling factor (1.375) applied to the mass extinction of sulfate aerosol, previously based on the molar mass ratio of ammonium sulfate to sulfate, has been removed as ammonium is now a separate species (since cycle 46R1).

Nitrate and ammonium aerosols were added to the aerosol species in Cy46R1 whilst simultaneously the SO₂ precursor was removed. However, the update to the new species as part of the aerosol data assimilation process was not correctly included (see Benedetti et al., 2009, for a full description of the aerosol assimilation process). This was fixed in Cy48R1, leading to increments now being added to both fine and coarse nitrate and ammonium.

2.1.2 Assimilation updates

The assimilated products in the operational CAMS system are visualised in Fig. S2. The assimilation of TROPOMI total column CO became operational on 28 April 2023 in Cy48R1 (only the last months of the e-suite period). The impact of this inclusion was tested from July to December 2021 (Inness et al., 2022a). The results showed an 8 % increase in CAMS total column CO. The assimilation impact was significant during periods of high CO emissions from boreal wildfires in July and August 2021. While TROPOMI CO assimilation enhanced column constraints, it had limited influence on individual plumes transported across continents and oceans above 500 hPa.

The CAMS aerosol data assimilation system has depended on the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument for more than 10 years. To ensure forecast continuity, as the MODIS instrument is ageing, the assimilation of AOD from NOAA VIIRS AOD from the Suomi National Polar-orbiting Partnership (S-NPP) and the NOAA-20 satellite was tested (Garrigues et al., 2023); it was activated in Cy47R3 in February 2023 and is also active in Cy48R1. Experiments assimilating VIIRS on top of MODIS or in place of MODIS in the IFS cycle were carried out from June to November 2020. A comparison with AERONET revealed that both experiments resulted in overall lower bias, notably in Europe, Africa and Southeast Asia, with substantial improvements over desert and maritime aerosol sites.

The assimilation of data from different satellite instruments can introduce biases compared to each other or the model. To address this, a bias correction scheme, known as variational bias correction (VarBC), is employed. VarBC introduces additional degrees of freedom, represented as bias parameters, into the 4D-Var cost function's observational term. Observational datasets that do not use VarBC are considered anchors and are crucial for preventing drifts in the fields (Inness et al., 2013). Note that in Cy48R1 the anchor for CO was changed from the Measurements Of Pollution in The Troposphere (MOPITT) instrument (Terra) to the Infrared Atmospheric Sounding Interferometer (IASI; Metop-C) and TROPOMI (Sentinel-5P), while the new anchor for AOD is now VIIRS (on the NOAA-20 satellite).

Cy48R1 includes a new volcanic SO₂ tracer (VSO2) in addition to the SO₂ tracer. VSO2 is currently not coupled to the chemistry but uses an e-folding lifetime of 7 d. TROPOMI SO₂ data with layer height information are assimilated into the VSO2 tracer following the method described in Inness et al. (2022a).

On 15 December 2022, still in Cy47R3, an update of the background error covariance wavelet file was implemented to use the correct NWP background errors. The update leads to considerably improved NWP forecast scores. As a consequence, changes to AOD and upper-tropospheric and lower-stratospheric ozone are also expected. The comparisons presented below cover the period October 2022–June 2023. The impact of this change is observed in ozone when comparing the first 2 months with the months in 2023.

2.1.3 Emission updates

CAMS emissions are available from gridded inventories per sector (Denier van der Gon et al., 2023), except the emissions from dust and sea salt aerosols, which are calculated online (ECMWF, 2023). Most emission inventories are at a monthly resolution, capturing the seasonal cycle. Only the emissions for biomass burning coming from GFAS v1.4 (Kaiser et al., 2012) are provided at a daily temporal resolution, including injection heights. Specifically in Cy48R1, the model uses anthropogenic emissions from CAMS-GLOB-ANT v5.3 (Soulie et al., 2023), aviation emission from CAMS-GLOB-AIR v1.1, biogenic emissions from a climatology constructed from CAMS-GLOB-BIO v3.1 (Sindelarova et al., 2022) and natural emissions of dimethyl sulfide (DMS) over the ocean from CAMS-GLOB-OCE v3.1. For reference, in cycle 47R3 the anthropogenic emissions were based on CAMS-GLOB-ANT v4.2 and biogenic emissions on CAMS-GLOB-BIO v3.1.

Natural emissions use a monthly mean climatology. For varying volcanic SO₂ emissions, a climatology is constructed based on recent satellite-based inventories (Carn et al., 2017). In Cy48R1, a sector-specific treatment for any of the emissions is introduced, allowing sector-specific diurnal cycle profiles and injection heights; see Sect. 3.1.1 in ECMWF (2023) and Guevara et al. (2021) for more details.

3 Observations used for the validation of the CAMS system

The CAMS service includes activities dedicated to the validation of the global and regional (European) service products. The latest validation results for the CAMS-global near-real-time service (the o-suite) products can be found in Warneke et al. (2024), available in ECMWF (2024d), and the activity is described in Eskes et al. (2015). All CAMS validation reports for the global service products and the verification websites can be found in ECMWF (2024g).

The CAMS validation activity makes use of about 65 measurement datasets. It covers concentrations from the surface up to the stratosphere, using a wide range of instruments and measurement techniques, including surface in situ, surface remote sensing, aircraft and balloon in situ, and satellite observations.

The observational datasets used for the evaluation of the Cy48R1 upgrade are summarised in Table 1. A description of all these diverse measurement datasets is beyond the scope of this paper. More details on the observation networks, instruments, measurement datasets and quality control can be found in the CAMS observations document (Eskes et al., 2023a) and in the list of references included in this document.

Table 1 Observational datasets used for the evaluation of the CAMS Cy48R1 e-suite. AE denotes the Ångström exponent, and CSA denotes the Canadian Space Agency; other network abbreviations can be found in the main text.

The CAMS daily forecast and analysis products are evaluated on a regular 3-monthly basis. In practice this implies that only datasets can be used that are available within 1 month of real time. With several networks, such as NDACC, IAGOS or EEA surface observations, special arrangements (contracts) have been made such that near-real-time unvalidated data can be used. For instance, the IAGOS NRT data are only available for operational users such as the weather and air quality services (e.g. CAMS). These data are inspected and validated by the instrument principal investigator (PI) using a semi-automatic approach and become available as preliminary Level 1 (L1) data with a time delay of a few days, as described in Nédélec et al. (2015) for ozone and CO and in Petzold et al. (2015).

Many of the forecast-minus-observation results shown below make use of the following three metrics: the modified normalised mean bias (MNMB; a symmetric and normalised form to express the mean bias), the fractional gross error (FGE; a symmetric and normalised absolute mean difference) and the correlation (R). The scoring recommendations and metrics are discussed in Tsikerdekis et al. (2023).

4 Results: changes in atmospheric composition introduced by Cy48R1

In this section we will summarise the main findings of the comparison of the e-suite (Cy48R1) and o-suite (Cy47R3), where both are evaluated with the independent observational datasets discussed above. This is presented for individual trace gases and aerosol properties for the available observational datasets and for regions of interest. The evaluation is done for the period 1 October 2022 to 27 June 2023, when forecast results for both cycles are available. The corresponding e-suite control run (e-control) and o-suite control run (o-control), without the assimilation of the atmospheric composition satellite data, were also evaluated. The improvement in performance of e-suite compared to the o-suite is summarised in Table 2 for all of the observational datasets used for the evaluation. The individual entries of this scorecard are discussed in the subsections below.

Table 2 Scorecard for the relative performance of the e-suite compared to the performance of the o-suite against observations. Meaning of the “Relative score” symbols: ($++$) the e-suite performs significantly better than the o-suite, (+) the e-suite shows small improvements, (n) (neutral) no significant difference between the o-suite and e-suite, (−) score is somewhat degraded in the e-suite, and ​​​​​​​($--$) the e-suite performs significantly worse than the o-suite. Remote: remote sensing from surface station.

¹ Based on the average statistics of ACE-FTS, SAGE III-ISS, OMPS LP, Aura MLS and ozonesondes. ² Based on data from a network of stations collected by FMI.

4.1 Ozone (O₃)

The ozone concentrations simulated by the CAMS e-suite and o-suite were evaluated using several surface measurements provided by WMO-GAW, NOAA Earth System Research Laboratories (ESRL), IASOA, the European Environment Agency air quality database (EEA-Airbase), the China National Environmental Monitoring Centre (CNEMC) and the US AirNow services; profile measurements from IAGOS and ozonesondes; and satellite total column observations from the Infrared Atmospheric Sounding Interferometer (IASI).

The comparison with surface observations over Europe (EEA-Airbase), Fig. 1, shows that MNMB of the e-suite is higher than that of the o-suite by up to 15 %, especially over central and northern Europe, changing some small underestimates into small overestimates. The e-suite temporal correlation improved slightly compared to the o-suite but not everywhere. The bias in the e-suite control run is slightly lower and improved compared to the o-suite control run, in agreement with comparisons with WMO-GAW and ESRL Global Monitoring Laboratory (ESRL-GML) observations.

Figure 1 Spatial distribution of the surface O₃ evaluation over Europe, for the e-suite minus EEA-Airbase observations (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (g: FGE; h: R) for the period 1 October to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

The surface ozone validation using observations from the China National Environmental Monitoring Centre, Fig. 2, shows that both the e-suite and the e-suite control runs reduce the negative bias observed for the o-suite and o-suite control runs over the northeastern region extending from Shanghai to Beijing. A slight improvement is also found over the megacity of Guangzhou in the south. This indicates less titration of ozone due to improved NO₂ in the e-suite and e-suite control run – see also Sect. 4.3 below – which is likely linked to the anthropogenic emission update. The performance of the e-suite remained almost similar to that of o-suite in central and western China. The correlation on average is about 0.75 and is similar for all experiments.

Figure 2 Spatial distribution of the O₃ evaluation over China, for the e-suite minus CNEMC observation (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (g: FGE; h: R) for the period 1 October to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

The evaluation of surface ozone using the AirNow ground-based stations over North America, Fig. 3, shows that the overestimation is slightly higher in the e-suite than in the o-suite (MNMB from +13 % to +22 %), while the temporal correlation (R) and FGE improved, especially for the control run. The higher ozone bias may be related to a slight underestimation of NO₂ in the e-suite compared to the o-suite (Fig. S3). The differences between the assimilation and control runs demonstrate the impact of the assimilation.

Figure 3 Spatial distribution of the O₃ evaluation over the USA and Canada, for the e-suite minus AirNow observations (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (g: FGE; h: R) for the period 1 October to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

Surface ozone from the e-suite and o-suite was also compared to WMO-GAW, ESRL-GML and Arctic IASOA surface station observations (Fig. S4). During October 2022 to February 2023, there was no significant change in the surface ozone bias between the CAMS e-suite and o-suite, with relative biases of less than −40 % for most of the stations. An exception is the South Pole Observatory (SPO) station over the Antarctica, where the e-suite shows a higher negative bias (−50 %) than in the o-suite (−40 %).

The e-suite and o-suite have been evaluated with IAGOS aircraft measurements of tropospheric ozone. The time series of the daily profiles at Frankfurt Airport show that the e-suite and the o-suite perform similarly in general (Fig. 4). However, in the free troposphere (between 850 and 350 hPa, approximately between 1.5 and 8.5 km) the e-suite control run shows a reduced bias compared to the other runs from both the e-suite and the o-suite from the beginning of the evaluation period until April. For May–June, the e-suite control run develops a negative bias and the results of the other e-suite and o-suite runs are similar. This is more clearly depicted in the time series of the monthly scores by vertical layers at Frankfurt, which shows a better performance (MNMB and FGE) in the e-suite control run in the free troposphere in January–April and the degradation for May and June. The two controls runs show a seasonal difference, which is compensated for by the assimilation. At most airports worldwide, the bias in the lower troposphere (pressure > 850 hPa) is slightly larger for the e-suite than for the o-suite, in particular over airports located in western Africa and eastern Asia (not shown). In the free troposphere, the bias is about the same between the e-suite and the o-suite for most visited airports for the analysis or the 1 d forecast (Fig. S5). For the control run we observe larger differences, and the o-control shows mainly positive biases and the e-control mainly negative biases compared to IAGOS; see also Fig. 4. These differences are reduced and results are improved by the data assimilation in the free troposphere.

Figure 4 Ozone comparisons using aircraft profile observations from IAGOS (http://www.iagos.org, last access: 7 June 2024). Time series of modified normalised differences, CAMS minus IAGOS, in the daily profiles of ozone at Frankfurt between October 2022 and 27 June 2023 for the e-suite (top left) and the o-suite (middle left) and their respective control runs (right). The fractional gross error differences between the e-suite and the o-suite are shown in the bottom row.

In the upper troposphere (evaluations based on flight level data with a potential vorticity below 2), ozone is overestimated and the results from the e-suite and the o-suite are very similar for all runs (Fig. S6), except for the o-suite control run, which presents slightly larger biases than the other runs over the northern Atlantic and North America.

Tropospheric ozone profiles have also been compared to ozonesonde observations; see Fig. 5. The Northern Hemisphere profile difference can be considered statistically robust (p value for tropospheric columns is below 0.01), and little difference is seen between the o-suite and e-suite in the troposphere. The e-suite control run shows a large negative bias in the upper troposphere and lower stratosphere (below 100 hPa), while almost no bias is found in the lower troposphere, in agreement with the IAGOS comparisons. For the other regions, the number of profiles is much lower, and the differences between the o-suite and e-suite are mixed: in the tropics and Antarctica, the bias in the lower troposphere is smaller for the e-suite, while the opposite is found in the Arctic and Southern Hemisphere mid-latitudes. The spread in the differences is similar for both runs. Note that the ozonesonde results in the troposphere, including the change in bias around March–April, are consistent with the IAGOS plots. Note that there is no significant change in the number of sondes around this time.

Figure 5 Comparisons against ozonesondes. Time–pressure curtain plot of ozone profile MNMB for the e-suite analysis (a), the o-suite analysis (b), the e-suite control run (c) and o-suite control run (d) against sonde profiles for the Northern Hemisphere mid-latitudes. The horizontal black line represents the tropopause. The vertical black line in the o-suite plot indicates the 15 December change in the background error covariance in Cy47R3.

The global maps of monthly mean total column ozone for October 2022 are compared with satellite observations from IASI; see Fig. 6, top row. Overall, the performance is somewhat improved in the e-suite, with slightly better distributions of the ozone columns compared to the o-suite. A bias feature over the tropical Pacific Ocean in October 2022 has disappeared in the e-suite. This feature may be attributed to the issue with the background covariances in Cy47R3, which was solved on 15 December. The positive bias at low latitudes in October is smaller in the e-suite than in the o-suite. For December 2022, the difference between the e-suite and the o-suite is less prominent. The positive bias above the Pacific Ocean is improved in the e-suite and remains mostly below 5 %. There is hardly a difference between the e-suite and o-suite for June 2023.

Figure 6 Global maps of monthly mean total column ozone (Dobson units) compared with satellite observations from IASI. The figure shows the e-suite result (top left) and the IASI observations (top right) for October 2022, with the relative bias of the e-suite (left column) and the relative bias of the o-suite (right column) below with respect to IASI (%) for October 2022 (second row), December 2022 (third row) and June 2023 (bottom).

4.2 Carbon monoxide (CO)

The simulated carbon monoxide of the CAMS e-suite and o-suite was evaluated using surface observations from the WMO-GAW, EEA-Airbase, CNEMC and AirNow networks; vertical profiles from IAGOS aircraft and NDACC FTIR measurements; and satellite total column retrievals from MOPITT and IASI.

Over China, the comparison with surface CO observations indicates an overall reduction in CO concentrations in the e-suite, which leads to a reduced positive bias and better performance in the megacities over northeastern China and particularly over Shanghai, Hangzhou and Beijing (Fig. 7). The negative bias and FGE increased in the e-suite in the rest of China, including central China. Overall correlation range between 0.3 and 0.8 for most stations, with a mean of 0.53 for the e-suite. The e-suite shows similar correlations to the o-suite. In the densely populated northeastern part of China, correlations exceed 0.6 over most stations. The reduction in CO can be linked to the emission update in Cy48R1.

Figure 7 Spatial distribution of the CO surface comparisons over China, for the e-suite minus CNEMC observations (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (g: FGE; f: R) for the period 1 October 2023 to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

Compared to surface CO observations from five WMO-GAW stations (Hohenpeißenberg, Jungfraujoch, Sonnblick, Zugspitze and Monte Cimone, located in Europe), there was no significant change in the bias between CAMS e-suite and o-suite runs, which does not exceed ±10 % in general (Fig. S7). The correlation for the e-suite has slightly improved compared to the o-suite over Europe and the Cabo Verde station in the tropics.

The comparison of the CO mixing ratios to EEA-Airbase observations over Europe shows that the e-suite performs better than the o-suite in terms of bias and correlation over most stations (Fig. S8). In all cases note that e-control scored better than o-control as well; hence the detected improvements at the surface mainly originate from the model changes rather than assimilation changes.

The e-suite has been evaluated with IAGOS measurements of CO (Level 1 data) at different airports. The time series of the daily profiles (curtain plots) of MNMB at Frankfurt Airport are presented in Fig. 8. The e-suite shows a smaller negative bias than the o-suite at all altitudes for the analysis and the 1 d forecast. This is also shown in the time series of the monthly scores by layers at Frankfurt, with an analysis and forecast improvement for the e-suite in both the lower troposphere (LT) and the free troposphere (FT) (Fig. S9). However, CO is still underestimated by both the e-suite and the o-suite, with a larger bias in the LT than in the FT. According to the fractional gross error (FGE) monthly values, the e-suite improvement over the o-suite is about −0.05 for most of the months in the LT compared to about −0.02 in the FT, while for correlation results, no notable difference is found (Fig. S9). The control runs from the e-suite and o-suite show a different seasonal pattern of the bias with a notable increase in the bias starting in early spring for both models. Like for the assimilated runs, the bias from the control run is smaller for the e-suite than for the o-suite but only until March. This can also be seen in the monthly scores time series (Fig. S9). For the remaining months the performance is similar for both control runs in the lower troposphere and slightly better for the o-suite control run in the free troposphere. For all other airports worldwide, the bias (MNMB) in the low troposphere is very similar for the e-suite and the o-suite. However and like at Frankfurt, at most visited airports the absolute differences (FGE) are better in the free troposphere for assimilated runs of the e-suite compared with those of the o-suite (Fig. S10). Regarding the control run results, they are similar for both models. In the upper troposphere, at cruise altitude, CO is underestimated by both the e-suite and the o-suite runs. Like in the free troposphere, there is a clear improvement in the bias in the upper troposphere over all regions in the e-suite compared to the o-suite and in particular for the analysis (Fig. S11).

Figure 8 CO comparisons using aircraft profile observations from IAGOS (http://www.iagos.org, last access: 7 June 2024). Time series of modified normalised differences, CAMS minus IAGOS, in the daily profiles of ozone at Frankfurt between October 2022 and 27 June 2023 for the e-suite (a) and the o-suite (c) and their respective control runs (b, d). The fractional gross error differences between the e-suite and the o-suite are shown in panels (e) and (f).

The results of the comparison with satellite CO column retrievals from the IASI and MOPITT instruments are shown in Fig. 9. Averaging kernels have been used in these comparisons. The IASI total column CO observations are well reproduced by the CAMS e-suite in terms of absolute amounts and spatial distribution. The e-suite relative bias for December 2022 with reference to IASI stays mostly within 20 %, with wide areas below 5 %, some negative bias above the southern Africa biomass burning area (up to 20 %), a positive bias around 60° S and a negative bias over Antarctica. The e-suite performs much better than the e-control, which shows negative biases for the major part of the globe up to 20 % regularly and peak amounts above South Africa exceeding 40 %. The o-suite mostly has a negative bias compared to IASI, up to 20 % regularly and up to 30 % over the Pacific. Rows four and five of Fig. 9 show the e-suite relative biases for March and June 2023, which support the previous findings also for the other seasons. The MOPITT comparison looks very different, as overall biases are smaller for the o-suite than for the e-suite. The e-suite shows widespread positive biases of up to 20 % which are not present in the o-suite. Only the negative biases of the o-suite of up to 20 % above northern Africa improved to remain mostly below 10 % in the e-suite. Similar observations are made for the additional months from different seasons. These results for CO total columns demonstrate changes in the bias correction implemented in the e-suite, which now uses IASI-C and TROPOMI as reference. Note that TROPOMI CO data were not assimilated in the e-suite until 28 April, so the impact of TROPOMI has not been explicitly evaluated here.

Figure 9 Global maps of monthly mean total column CO comparing satellite observations from IASI (left column) with MOPITT (right column) validation including the satellite observation (first row), the relative bias of the o-suite result with respect to the satellite data (second row) and the relative bias of the e-suite result (third row) for December 2022, as well as the e-suite relative bias for March and June 2023 (fourth and fifth row, respectively).

In 2023, Canada suffered from a very extreme fire season, which started as early as May, with intense fires in June, and continued during the summer. The e-suite shows pronounced enhancements in CO in June, linked to the large GFAS fire emissions over Canada inserted in the model. As shown in Fig. 9, the amount of CO produced is underestimated by up to 20 % compared to IASI, and the largest relative error in the global map is located over Canada in June. However, the e-suite compares well with MOPITT over Canada, demonstrating the differences between these two satellite products. Negative CO biases over Canada compared to IASI (but positive compared to MOPITT) are also found in July and August, as shown in the JJA-2023 validation report available in ECMWF (2024d).

Over the Arctic Sea we observe a large region with negative biases in March compared to IASI. Note that IASI data have less sensitivity in spring as compared to summer above the Arctic due to lower thermal contrast, which may explain part of the differences. The MOPITT results seem to indicate a similar underestimate, so a negative bias in the analysis is likely.

Comparisons were also made against CO partial columns and profiles from the FTIR instruments, part of the surface remote sensing network, NDACC (Fig. 10). The scores for 13 NDACC stations are presented in Tables S1 and S2 in the Supplement. For tropospheric CO columns, the bias for the e-suite is reduced compared to the o-suite for almost all sites. Although correlations are similar for the o-suite and e-suite, the ratio of the standard deviation in the tropospheric columns for the e-suite and the FTIR time series is higher compared to the o-suite. The bias at the tropical sites has switched sign and is now positive. For stratospheric CO columns, the bias for the e-suite is reduced significantly for the southern hemispheric sites and is now of the order of the measurement uncertainty. The Southern Hemisphere correlations are also higher for the e-suite (in particular for the Antarctic Arrival Heights site). For the Northern Hemisphere, the e-suite and o-suite perform similarly. The o-suite has a degraded performance during this period in the Southern Hemisphere (Wollongong and Lauder stations), where the tropospheric columns show an increased negative bias compared to the months before October 2022 and the stratospheric columns show a strong (+20 %) positive bias. A direct comparison between the o-suite and e-suite should therefore be interpreted with care. The figure shows that for the e-suite, a reduced positive bias remains in the stratosphere for this period. This can be considered an improvement compared to the o-suite performance prior to October 2022 (negative bias < −20 %) and past October 2022 (positive bias > +20 %).

Figure 10 Curtain plots of the CO profile relative difference in the CAMS products compared to FTIR observations from NDACC for the two southern hemispheric stations Lauder (left) and Wollongong (right). Shown are the differences with the e-suite (top row), o-suite (second row), e-control (third row) and o-control (bottom row). Model profiles are smoothed with the FTIR averaging kernels. The horizontal black line is the tropopause. The vertical black line indicates 15 December, when the change in the background error covariance was implemented.

As of 22 May 2023, wildfires had been raging across Canada's Alberta Province for 3 weeks. IAGOS aircraft profiles of CO over Montréal and Calgary picked up the pollution plumes from these events and have been compared with the CAMS results (Fig. S12). For most profiles studied during this event, the e-suite performed better than the o-suite, with values of the CO mixing ratios higher than those provided by the o-suite, although the forecast model still faces difficulties in capturing the high concentrations in the plumes, while the altitude is simulated with success.

4.3 Nitrogen dioxide (NO₂)

The CAMS configurations were also compared with surface observations in China, Europe and North America.

Over China (Fig. 11), the e-suite and e-suite control run significantly reduce the Cy47R3 positive offset in surface NO₂ over most regions in eastern China, which also leads to a reduced bias in surface ozone as discussed in Sect. 4.1. The correlation of the e-suite and e-suite control run is similar (slightly reduced) to that of the o-suite and o-suite control run. The mean correlation coefficient averaged over all stations is 0.52 for the e-suite and 0.54 for the o-suite.

Figure 11 Spatial distribution of the NO₂ surface comparisons over China, for the e-suite minus CNEMC observations (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (f: FGE; h: R) for the period 1 October to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

The comparison with surface NO₂ observations in Europe obtained from EEA-Airbase shows that the e-suite performs better than the o-suite in terms of bias for a majority of the stations especially in central and northern Europe (Fig. S13). Interestingly, this coincides with the regions where a slightly higher bias was found for ozone in the e-suite, indicating that a chemical regime changes in the model. In terms of correlations, the e-suite and o-suite performed almost equally.

The evaluation of surface NO₂ using the AirNow ground-based stations over North America shows that the underestimation in e-suite and e-suite control run is higher than in the o-suite and o-suite control run (from about −30 % to −40 %), while the temporal correlation (R) slightly improves (from 0.49 to 0.54) (Fig. S3).

The CAMS e-suite Cy48R1 tropospheric NO₂ column data are compared to the TROPOMI scientific IUP, University of Bremen (IUP Bremen), tropospheric NO₂ product; see Fig. 12. Three different months are selected for performance demonstration, including those periods with the highest and lowest solar elevation for the two hemispheres. Overall, the e-suite results correspond well to the observational data in terms of spatial and temporal variations and absolute amounts. However, there are also apparent differences between model and measurements. The largest differences are mainly observed above regions with major emissions. Locations strongly affected by anthropogenic pollution such as hotspots in China can show strong positive biases partly above 100 %. Relative to the o-suite, however, the e-suite performs better compared to TROPOMI especially above strongly polluted regions such as eastern China. Here, the positive bias in the o-suite is reduced in the e-suite as well as above other hotspot regions. This effect is most visible in the December maps (left column of Fig. 12). This improvement is likely due to the updated emissions used in Cy48R1. Most areas with strong positive biases are smaller in their spatial extent and smaller in their bias values in the e-suite compared to the o-suite. In addition, there are a few confined regions where the e-suite shows positive biases not present in the o-suite comparison such as above the North Sea. This can mainly be explained by the lower anthropogenic emissions in CAMS-GLOB-ANT v5.3 as applied in the e-suite, compared to CAMS-GLOB-ANT v4.2 in the o-suite.

Figure 12 Monthly mean global maps of tropospheric NO₂ column densities for TROPOMI satellite observations (first row), the e-suite (second row), the difference between the e-suite and satellite (third row), the relative difference between the e-suite and satellite (fourth row), and the relative difference between the o-suite and satellite (fifth row) for December 2022 (left column), March 2023 (middle column) and June 2023 (right column). Units: 1×10¹⁵ molec. cm⁻². Note that for the relative bias the regions with background values below 5×10¹⁴ molec. cm⁻² are not included in the analysis.

Some typical biomass burning areas such as southern Africa or South America show negative biases around 40 % compared to TROPOMI. Negative biases are seen also above parts of the USA, the northern Atlantic, northern Pacific and parts of Asia. In contrast, positive biases are found over boreal fires. No noteworthy degradation in performance is seen when moving from the previous o-suite to the e-suite results. This is also true concerning the background regions: both the e-suite and the o-suite perform well in representing background values close to zero. The negative biases for some oceanic background regions are slightly larger for the e-suite than for the o-suite. The negative biases above the oceans on the Northern Hemisphere show up in areas where the absolute tropospheric NO₂ column is small and close to background levels. Slight differences in absolute amounts thus cause comparably large relative bias values. Overall, the performance of the e-suite is slightly better than that of the o-suite.

4.4 Sulfur dioxide (SO₂)

The comparison to SO₂ surface observations from the China National Environmental Monitoring Centre shows that the e-suite–e-suite control run significantly reduces the positive bias observed for the o-suite–o-suite control run over most regions in eastern China, as indicated by the difference in FGE between the e-suite and o-suite (Fig. 13). However, a high overestimation (> 50 %) is still observed in the e-suite for many of the stations, indicating a remaining possible overestimation of SO₂ emissions in the updated CAMS_GLOB_ANT inventory. Note that the temporal correlation (R) is slightly reduced in the e-suite compared to the o-suite.

Figure 13 Spatial distribution of the SO₂ surface comparisons over China, for the e-suite minus CNEMC observations (a: MNMB; b: R), the o-suite minus observations (c: MNMB; d: R), the e-suite-minus-o-suite differences (e: FGE; f: R) and the e-control-minus-o-control differences (g: FGE; h: R) for the period 1 October to 27 June 2023. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

The comparison of the SO₂ mixing ratio against EEA-Airbase observations over Europe shows that the e-suite–e-suite control run performs better than the o-suite–o-suite control run in terms of bias for most stations (Fig. S14). The correlations are almost equal.

4.5 Formaldehyde (HCHO)

The tropospheric formaldehyde (HCHO) column from the CAMS e-suite is compared to the TROPOMI scientific IUP Bremen tropospheric HCHO product (Eskes et al., 2023a). Monthly mean global maps and differences are shown in Fig. 14.

Figure 14 Monthly mean global maps of tropospheric HCHO column densities for TROPOMI satellite observations (first row), the e-suite (second row), the difference between the e-suite and satellite (third row), the relative difference between the e-suite and satellite (fourth row), and the relative difference between the o-suite and satellite (fifth row) for December 2022 (left column), March 2023 (middle column) and June 2023 (right column). Units: 1×15¹⁵ molec. cm⁻². Regions with mean background values below 3×10¹⁵ molec. cm⁻² are excluded from the analysis (white areas).

The e-suite overestimates the tropospheric HCHO columns over South America and northern Australia by more than 60 % in December and March, and there is also an overestimation over Indonesia. The allocation of the HCHO columns over northern and southern Africa in the two months is well reproduced by the model, but there are biases in certain time periods, such as the overestimation north of the Equator and an underestimation in the south for June 2023.

In comparison to the o-suite, the overestimation above South America and northern Australia is stronger in the e-suite, visible in the relative biases of the e-suite and the o-suite in rows four and five of Fig. 14. This could be the effect of the updated isoprene chemistry, which induced higher HCHO production. Above northern Africa, the HCHO values show a larger positive bias, while over southern Africa the negative bias is slightly less pronounced in the e-suite. Overall, the performance is similar but slightly degraded for the e-suite.

4.6 Stratospheric ozone (O₃)

Stratospheric ozone from the CAMS e-suite Cy48R1 and o-suite have been compared against satellite limb profiles and ozonesondes, considering the analyses, the fifth-day forecasts and the o-suite control run; see Fig. 15. The full e-suite period is considered, from 1 October 2022 to 27 June 2023.

The main conclusions are as follows:

• In the lower stratosphere, at pressures higher than 10 hPa, the e-suite agrees better with the observations than the o-suite, with lower biases, lower FGE and higher correlations for the 9-month period evaluated.

• The e-suite agrees much better with observations than the o-suite in the tropics.

• The e-suite 5 d forecast agrees better with independent observations than the o-suite during ozone hole conditions.

• In the upper stratosphere (at pressure lower than 10 hPa), the e-suite displays a larger negative bias than the o-suite, while the correlations are improved in the e-suite in the upper stratosphere in the tropics.

• The control run of the e-suite shows a higher negative bias than the o-suite control run in the upper stratosphere above the 10 hPa level.

The differences between the o-suite and the e-suite are largely due to the introduction of full stratospheric chemistry, which is demonstrated by comparing the control runs of the o-suite and the e-suite.

Figure 15 The e-suite and o-suite compared to a multi-instrument mean (MIM), consisting of observed ozone profiles from ACE-FTS v4.1, Aura MLS v4.2, OMPS LP v2.5, SAGE III-ISS v5.2 and ozonesondes. Top row: normalised mean difference between the MIM ozone observed profiles and the o-suite analyses (solid red line), the o-suite fifth-day forecast (solid green line), the e-suite analyses (dashed red line) and the e-suite fifth-day forecast (dashed green line). The figure refers to the period October 2022, which is during the Antarctic ozone hole conditions. Five latitude bands are considered, from left to right: 90–60° S, 60–30° S, 30° S–30° N, 30–60° N and 60–90° N. The corresponding standard deviation of the differences and the correlation coefficient between observed profiles and the e-suite and o-suite runs are shown in the second and the third row, respectively.

Stratospheric ozone has also been compared to lidar profile observations from NDACC for the full e-suite period of 9 months; see Fig. 16. Above 10 hPa, the e-suite mean profile deviates from the o-suite, showing a negative bias compared to the lidar observations that increases with altitude. Between 50 and 10 hPa, the e-suite overestimates the ozone concentration. The negative bias feature observed around 20 hPa in the o-suite at Mauna Loa is no longer present in the e-suite. The dispersion (standard deviation, SD) in the differences is slightly, but systematically, improved for the e-suite. In general, the conclusions for the lidar comparisons are very similar to those for the satellite comparisons.

Figure 16 Mean ozone profile differences for e-suite (dashed red), o-suite (solid red), e-suite control run (dashed blue) and o-suite control run (solid blue) compared to the stratospheric ozone lidar observations of four stations from NDACC.

4.7 Other trace gases in the stratosphere

Other species simulated in the stratosphere in the e-suite have been compared with satellite limb profiles considering 3-hourly first-day forecast. The new CAMS cycle 48R1 introduces full stratospheric chemistry, and meaningful comparisons with the o-suite are not possible. However, reasonable concentrations of these species are important because of their impact on ozone. The species considered are as follows: CCl₄, CFC-11, CFC-12, HCFC-22, ClO, ClONO₂, HCl, CH₄, H₂O, HNO₃, N₂O, N₂O₅, NO₂, NO_x (NO+NO₂) and O₃.

There is a relatively good agreement between the e-suite and observations for long-lived species (CCl₄, CFC-11, CFC-12, CH₄, HCFC-22 and N₂O) and HCl between 10 and 200 hPa; see Fig. 17. The chemical H₂O tracer in the CAMS e-suite shows a negative bias. Even though not perfect, CAMS e-suite NO₂ (and NO_x) agrees relatively well with observations, which was not the case in the previous o-suite cycles. Above the 10 hPa level, the concentrations are overestimated compared to the Atmospheric Chemistry Experiment Fourier Transform Spectrometer (ACE-FTS), and there is room for improvement. For ClO, the agreement is good with MLS and less good with ACE-FTS, but this dataset is supposed to be less reliable than MLS for this species.

Figure 17 Comparison of O₃, H₂O, NO₂, NO_x, CCl₄, CFC-11 and CFC-12 from the e-suite first-day forecast (blue lines) with ACE-FTS observations (black dots) between 1 October 2022 and 27 June 2023. Five latitude bands are considered, from left to right: 90–60° S, 60–30° S, 30° S–30° N, 30–60° N and 60–90° N.

Stratospheric NO₂ columns have been compared with TROPOMI retrievals; see Fig. 18. The observational NO₂ stratospheric column is represented well by the e-suite in terms of absolute amounts, latitudinal variation and temporal changes. Over the largest part of the globe, column amounts agree to within 2×10¹⁵ molec cm⁻², deviating by less than 10 %. The e-suite performs well in reproducing the general strong increase in stratospheric NO₂ at high latitudes in the summer hemisphere.

Figure 18 Monthly mean NO₂ stratospheric column amounts retrieved using TROPOMI observations (a, b, c) compared to the e-suite results (d, e, f). Panels (g), (h) and (i) show the differences between the e-suite and TROPOMI. Results are shown for December 2022 (a, d, g), March 2023 (b, e, h) and June 2023 (c, f, i). The STREAM-B algorithm was used to estimate the stratospheric contribution to the total retrieved column (Eskes et al., 2023a).

4.8 UV radiation

While no changes to the UV code were implemented in Cy48R1, the surface UV index (UVI) forecasts are affected by the changes in the optical depth of trace gases (most notably stratospheric ozone), aerosols and thick clouds. The net effect of these factors on the hourly forecast performance of the new model cycle was estimated using ground-based UVI measurements as the reference data at 38 stations, located in Europe, Israel, Thailand, Australia, New Zealand and the Antarctic.

Figure 19 illustrates the statistical improvement of Cy48R1 compared with Cy47R3 when evaluated against the ground-based observations in terms of MNMB, FGE and R. MNMB and FGE included all available hourly UVI forecasts, while R included only UV forecasts close to local noon in order to emphasise the importance of atmospheric composition (instead of the solar zenith angle, SZA) and to highlight the time of day when UV radiation is typically most intense and hazardous. Overall, the statistics (mean MNMB = 5  %, mean FGE = 0.30 and mean R = 0.82) indicate that CAMS UV forecasts are of good quality in Cy48R1. The mean values of R and FGE indicate no significant changes in correlation and scatter between the old and the new model cycles; however the mean MNMB increases slightly from 2 % to 6 %. This finding is supported by comparing the zonal mean of daily maximum UVI between the two model cycles, which increased by +0 % to +4 % in Cy48R1 compared to Cy47R3, with the largest increase of +2 % to +4 % occurring between 50° S and 60° N.

Figure 19 Hourly UV index values from the CAMS e-suite and o-suite evaluated against ground-based UV measurements from 38 stations located in Europe, Israel, Thailand, Australia, New Zealand and the Antarctic. Scatterplots of MNMB, FGE and R between measurements and the o-suite (Cy47R3, x axis) and measurements and the e-suite (Cy48R1, y axis). Each dot represents a single measurement station, in either the northern latitudes (red) or the southern latitudes (blue). The time range is from 1 October 2022 to 27 June 2023.

4.9 Aerosol

Cy48R1 introduces a redistribution of aerosol optical depth per species (Fig. 20, first column). Starting from Cy48R1, the secondary organic aerosol (SOA) optical depth is now provided separately, whereas prior to this update, the SOA optical depth was included as part of the organic matter (OM) optical depth. For this report, the OM optical depth in the e-suite includes the SOA in order to ensure comparability with the OM optical depth in the o-suite. Compared to the o-suite, there is less AOD for the e-suite, particularly due to a reduction in sulfate optical depth (Fig. 20, second column). For the e-suite, there is less black carbon optical depth, particularly over central Africa. Over Southeast Asia and the Sahel, nitrate optical depth decreases by about 50 % in the e-suite compared to the o-suite. Contrarily, the sea salt and ammonium optical depth increases, while dust and organic matter changes are regionally dependent. The Ångström exponent (AE), a parameter which is indicative of the aerosol size distribution, is considerably lower globally (reduction by −0.12) in the e-suite, especially over the Middle East and Sahara, while it increased over South Africa, South America and Australia.

Figure 20 AOD (first row), AOD for each species (second row to eighth row) and Ångström exponent (ninth row) for the e-suite (first column), differences of e-suite − o-suite (second column) and the differences of e-suite − e-control (third column) for the period 1 October 2022 to 27 June 2023. Species included are dust (DU), sea salt (SS), organic matter plus secondary organic aerosol (OM + SOA), black carbon (BC), sulfate (SU), nitrate (NI) and ammonium (AM). The percentage at the bottom-left corner of first column displays the relative contribution of each species optical depth to AOD.

These changes are related to (1) the introduction of two new secondary organic aerosol tracers (anthropogenic and biogenic) along with their respective precursor gas tracers (2) modifications to dust emissions and removal simulation that increased the global dust mass burden by a factor of 2, (3) a review of aerosol optical properties for dust and brown carbon, and (4) improvements to secondary inorganic aerosol simulation. Note that the AOD and AE mostly increase due to assimilation, except for dust (Fig. 20g, h, j).

The evaluation of daily aerosol optical depth (AOD) and the Ångström exponent (AE) against the ground-based network of AERONET version 3 Level 1.5 stations shows that e-suite overestimates less compared to the o-suite for both parameters (Figs. 21 and 22). The e-suite improves in terms of AOD MNMB globally (from +21 % in the o-suite to +7 % in the e-suite), especially over North America, Europe, East Asia and the Middle East. Over Southeast Asia and the Sahara, the MNMB performance remains almost unchanged. Notably for South Asia, a deterioration in performance (higher underestimation) is observed compared to the o-suite. The correlation (R) performance remains unchanged (approximately 0.70 in both the o-suite and the e-suite). Note that both control runs underestimate AOD (not shown).

Figure 21 Spatial distribution of the daily aerosol optical depth MNMB, FGE and R for the e-suite minus observations (a, b), the o-suite minus observations (c, d), the e-suite-minus-o-suite differences (e, f) and the e-control-minus-o-control differences (g, h) for the period 1 October to 27 June 2023 using AERONET v3 Level 1.5 as a reference. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

Figure 22 Spatial distribution of the daily Ångström exponent MNMB, FGE and R for the e-suite minus observations (a, b), the o-suite minus observations (c, d), the e-suite-minus-o-suite differences (e, f) and the e-control-minus-o-control differences (g, h) for the period 1 October to 27 June 2023 using AERONET v3 Level 1.5 as a reference. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

The AE improves in terms of global bias (mean bias goes from +0.28 in the o-suite to +0.19 in the e-suite), with regional improvements in Europe and North America as well as East China (Fig. 22). The e-suite updated dust emissions and deposition fluxes, resulting in a coarser aerosol size distribution (smaller AE) and dust mass burden over the deserts in the e-suite control run. Results specifically related to dust changes are discussed in Sect. 4.11. The correlation (R) of AE improves globally (from 0.34 in the o-suite to 0.44 in the e-suite) and remains unchanged in Europe and East Asia, with a small improvement in South Asia, the Sahara and the Middle East and a considerable improvement over South America and North America, though R still remains below 0.5 in the latter case (MET Norway, 2024).

4.10 Particulate matter

Global daily near-real-time data of particulate matter, under 10 and 2.5 µm (PM₁₀ and PM_2.5, respectively) in diameter, from the surface observational networks AirNow (North America, U.S. Environmental Protection Agency, EPA), EEA-Airbase (Europe, European Environmental Agency, EEA) and CNEMC (China National Environmental Monitoring Centre), are used to evaluate e-suite and o-suite experiments for the period October 2022 to June 2023 (Fig. 23). Note that all stations are considered for the AirNow and CNEMC networks, including urban stations; hence CAMS may not be able to fully capture the very high PM_2.5 and PM₁₀ measured with observations at a local urban scale due to the coarse spatial resolution of the model. In this section we focus on the evaluation results for PM_2.5. The results for PM₁₀ were found to be very similar to those of PM_2.5. The PM₁₀ results are presented on the AeroVal website (MET Norway, 2024) and are discussed in Eskes et al. (2023b).

Figure 23 Modified normalised mean bias of CAMS minus observations (top left) and correlation (top right) of PM_2.5 and PM₁₀ based on EEA-NRT-rural (Europe), AirNow (North America) and CNEMC (East Asia) monitoring stations. The average time series for all stations over East Asia (second row) and the MNMB maps for the (a) o-suite and (b) e-suite are also shown. The mean (MN) and standard deviation (SD) for all stations are depicted for each map.

Over North America and Europe, the e-suite improves PM_2.5 in terms of MNMB (−3.2 % and 8.9 %, respectively) compared to the o-suite, which exhibited a small overestimation of about +9.5 % and +16.2 %. In contrast, over China MNMB of the e-suite (−19.9 %) exhibits a higher and negative bias when considering the performance over all available stations compared to the o-suite (+3.8 %). The spatial distribution of MNMB over China forms a clear dipole pattern. The eastern side, which encompass most of China's megacities, mostly overestimates PM_2.5, while in the western part, which contains less populated, high-altitude regions, PM_2.5 is underestimated. The PM_2.5 of the e-suite outperforms that of the o-suite in the eastern part of the country, with MNMB lower than 25 % at most stations, while the opposite is the case for the western part, where the e-suite displays MNMB that is more negative than −50 % at most stations. The measured PM_2.5 over China displays a peak in January, which is strongly underestimated by both the e-suite and the o-suite (Fig. 20). The correlation for the e-suite and o-suite remains unchanged in Europe (about 0.47), slightly improves over China (from 0.55 to 0.57) and is better in North America (from 0.39 to 0.55). The reduction in sulfates (Fig. 20), the reduction in the PM_2.5 positive bias (Fig. 23) and the reduced positive bias of SO₂ over China (Fig. 13) are linked to the reduced SO₂ emissions in the e-suite.

4.11 Aerosol coarse

AOD coarse (AODc) values of the e-suite and o-suite were evaluated against the AERONET spectral deconvolution algorithm (SDA) version 3 Level 1.5 daily data (Fig. 24). Overall, the e-suite AODc performs better than the o-suite AODc in terms of FGE globally (from 1.09 to 0.77) and for all regions (except the Middle East and the Pacific–Australia–New Zealand). The same applies to MNMB and R (not shown). Since the AE is indicative of the aerosol size distribution, its mean absolute bias (MAB) reveals similar results.

Figure 24 Fractional gross error (FGE) of AOD coarse based on AERONET SDA (spectral deconvolution algorithm) version 3 Level 1.5 daily data (a, c, e) and the Ångström exponent between 440 and 870 nm mean absolute bias (MAB) based on AERONET version 3 Level 1.5 daily data (b, d, f). The monthly time series (bold lines) along with the daily time series of observations (thin lines) for northern Africa (c, d) and the Middle East (e, f) are also depicted.

Over and around arid areas, AODc is represented mainly by aerosol dust. The new dust emission increased dust optical depth (DOD) over the Middle East and increased/decreased the DOD over the northern/southern part of the Sahara (Fig. 20e). Over northern Africa, the increasing dust concentration in spring, which increases AODc and decreases AE, is represented slightly better by the e-suite compared to the o-suite. In contrast, in the Middle East the e-suite displays too high an AODc (too low an AE) compared to the observations. In particular, the simulated AODc and AE of the e-suite are better than those of the o-suite over AERONET stations that are located over the westward transport of Saharan dust in the North Atlantic (e.g. Cabo Verde), as well as over other regions that are affected less by dust (e.g. southern Europe) (MET Norway, 2024).

5 Conclusions

The upgrade of the ECMWF/CAMS global system to Cy48R1 on 27 June 2023 involved many system changes to the composition modelling, emissions and assimilation which are listed in Sect. 2.1. Cy48R1 represents a major upgrade and is the result of 2 years of model and data assimilation development. The upgrade introduces, among many other changes, a comprehensive stratospheric chemistry scheme with the addition of 63 gas species; contains important emissions updates; implements changes to the modelling of dust aerosol resulting in a redistribution of aerosol particles towards larger sizes; adds an explicit representation of secondary organic aerosol; and revisits isoprene and the chemistry of aromatics. In Cy48R1 the assimilation of TROPOMI CO and VIIRS AOD is introduced.

The validation results for 47 comparisons (measurement datasets) in total are summarised in the scorecard of Table 2, which compares the relative performance of the new Cy48R1 configuration to the previous Cy47R3 system that was operational until 27 June 2023. The judgements presented in this summary table are qualitative, based on expert judgement and the investigation of maps and scores. The presentation of one quantitative score and significance assessment performed using all datasets is, in practice, not feasible or meaningful. Each entry in the table has its own story, which is detailed in the subsections of the Results section (Sect. 4). The datasets used for the assessment are very heterogeneous. Some datasets are very sparse (e.g. IAGOS, where an extended data record is only available over Frankfurt) or cover only a limited region (e.g. surface concentrations from air quality networks), while other networks are dense, such as AERONET. Satellite data are available globally, but retrieval products for separate instruments (or the same instrument) often show significant differences, and biases and uncertainties are not easy to assess. There are strong regional and seasonal differences in the scores, reflecting differences in sources, processes or aspects like aerosol composition.

The main results for the trace gases may be summarised as follows:

• The performance of CO has generally improved compared to all observations. The comparison against MOPITT is the exception. Previously, the assimilation system was anchored to MOPITT, while since Cy48R1 this anchor has been IASI. Furthermore, TROPOMI CO observations are now assimilated in Cy48R1. This anchoring change explains why the comparison with MOPITT is worse in the e-suite compared to the o-suite. However, the independent observations show that these changes are improvements.

• NO₂ and SO₂ show improved validation results against multiple observations, both at the surface and in columns observed from space. This may be linked to the upgrade of the emissions, which more realistically describe the emission trends in recent years.

• CAMS surface ozone and tropospheric ozone perform similarly in the e-suite and o-suite. However, ozone improvements are observed in eastern China and the USA at the surface, probably linked to the emission update and changes in the precursors like NO₂.

• Stratospheric ozone below the 10 hPa level and total column ozone have improved compared to observations, especially in the tropics. This may be linked to the inclusion of a full stratospheric chemistry scheme in Cy48R1. The bug fix in the formulation of the background covariances in Cy47R3 in December 2022 had a positive impact, especially for ozone in the tropics.

• The comparison of a large number of trace species in the stratosphere against ACE-FTS and MLS shows realistic concentration profiles. This provides confidence in the implementation of stratospheric chemistry in IFS-COMPO.

• The overestimation of HCHO in the tropics is more pronounced in the e-suite compared to TROPOMI. This may be related to the updated isoprene chemistry.

• The UV evaluation shows only minor changes.

The results for the aerosols may be summarised as follows:

• The aerosol optical depth evaluation shows improvements for most regions.

• The Ångström exponent, which is a measure representative of the aerosol size distribution, shows some improvement on the global scale, but over the Middle East the performance deteriorates, showing too large a fraction of coarse particles.

• Particulate matter (PM₁₀ and PM_2.5) at the surface is reduced in the e-suite, which leads to lower bias in the eastern part of China compared to the o-suite. The e-suite and o-suite perform similarly over North America and Europe.

It is important to note that the e-suite data are available for a period of about 9 months, with more emphasis on the Northern Hemisphere winter season. Therefore some results may not be fully representative of the entire year of 2023 or of other years. The assimilation of TROPOMI CO in the e-suite was switched on only at the end of April 2023, and its impact could not be fully evaluated.

In summary, 55 % of the evaluation datasets show an improved performance of Cy48R1 compared to the previous operational CAMS system, of which two improvements are indicated as major; 28 % of the comparisons are neutral; and 17 % indicate a degradation for the Cy48R1 e-suite compared to the Cy47R3 o-suite. This clearly indicates the overall success of the recent upgrade of the CAMS global system to Cy48R1.

The evaluation of the CAMS products with independent observations is continually developing. Apart from further establishing interfaces with the major observation networks, CAMS is actively acquiring and testing (surface) data from South America, Africa and Asian countries other than China. The scorecard presented in this paper provides a qualitative summary of the results. More quantitative scorecards are being developed for the quarterly o-suite validation reports; see Tsikerdekis et al. (2023).

The next upgrade of CAMS to Cy49R1 is planned for quarter 4 of 2024.