Comparative assessment of TROPOMI and OMI formaldehyde observations and validation against MAX-DOAS network column measurements

The TROPOspheric Monitoring Instrument (TROPOMI), launched in October 2017 on board the Sentinel-5 Precursor (S5P) satellite, monitors the composition of the Earth's atmosphere at an unprecedented horizontal resolution as fine as 3.5 × 5.5 km². This paper assesses the performances of the TROPOMI formaldehyde (HCHO) operational product compared to its predecessor, the OMI (Ozone Monitoring Instrument) HCHO QA4ECV product, at different spatial and temporal scales. The parallel development of the two algorithms favoured the consistency of the products, which facilitates the production of long-term combined time series. The main difference between the two satellite products is related to the use of different cloud algorithms, leading to a positive bias of OMI compared to TROPOMI of up to 30 % in tropical regions. We show that after switching off the explicit correction for cloud effects, the two datasets come into an excellent agreement. For medium to large HCHO vertical columns (larger than 5 × 10¹⁵ molec. cm⁻²) the median bias between OMI and TROPOMI HCHO columns is not larger than 10 % (< 0.4 × 10¹⁵ molec. cm⁻²). For lower columns, OMI observations present a remaining positive bias of about 20 % (< 0.8 × 10¹⁵ molec. cm⁻²) compared to TROPOMI in midlatitude regions. Here, we also use a global network of 18 MAX-DOAS (multi-axis differential optical absorption spectroscopy) instruments to validate both satellite sensors for a large range of HCHO columns. This work complements the study by Vigouroux et al. (2020), where a global FTIR (Fourier transform infrared) network is used to validate the TROPOMI HCHO operational product. Consistent with the FTIR validation study, we find that for elevated HCHO columns, TROPOMI data are systematically low (−25 % for HCHO columns larger than 8 × 10¹⁵ molec. cm⁻²), while no significant bias is found for medium-range column values. We further show that OMI and TROPOMI data present equivalent biases for large HCHO levels. However, TROPOMI significantly improves the precision of the HCHO observations at short temporal scales and for low HCHO columns. We show that compared to OMI, the precision of the TROPOMI HCHO columns is improved by 25 % for individual pixels and by up to a factor of 3 when considering daily averages in 20 km radius circles. The validation precision obtained with daily TROPOMI observations is comparable to the one obtained with monthly OMI observations. To illustrate the improved performances of TROPOMI in capturing weak HCHO signals, we present clear detection of HCHO column enhancements related to shipping emissions in the Indian Ocean. This is achieved by averaging data over a much shorter period (3 months) than required with previous sensors (5 years) and opens new perspectives to study shipping emissions of VOCs (volatile organic compounds) and related atmospheric chemical interactions.

1 Introduction

Satellite observations of tropospheric formaldehyde (HCHO) columns have been used for years to support air quality and chemistry–climate-related studies from the regional to the global scale. Formaldehyde is an intermediate gas in almost all oxidation chains of non-methane volatile organic compounds (NMVOCs), leading to the production of carbon monoxide (CO) and eventually carbon dioxide (CO₂). NMVOCs are, together with nitrogen oxides (NO_x), CO and methane (CH₄), among the most important precursors of tropospheric ozone. NMVOCs also produce secondary organic aerosols and influence the concentrations of hydroxyl radical (OH), the main tropospheric oxidant. The major HCHO source in the remote atmosphere is CH₄ oxidation. Over the continents, the oxidation of other NMVOCs emitted from vegetation, fires, traffic and industrial sources results in important and localized enhancements of the HCHO levels. Because of its short lifetime (of the order of a few hours), HCHO in the boundary layer can be related to the release of a large number of short-lived volatile hydrocarbons. Furthermore, HCHO observations provide information on the chemical oxidation processes in the atmosphere, including CO chemical production from CH₄ and NMVOCs, the oxidation of isoprene into HCHO, which allows the quantification of midday OH (Wells et al., 2020), and the tropospheric ozone production regimes that depend on the HCHO to NO₂ ratios (Jin et al., 2020).

Satellite observations of formaldehyde columns in the troposphere have been extensively reported in the literature from a number of nadir UV sensors, e.g. the Global Ozone Monitoring Experiment (GOME; Chance et al., 2000; Palmer et al., 2001; De Smedt et al., 2008), SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY (SCIAMACHY; Wittrock et al., 2006; De Smedt et al., 2008, 2010), Ozone Monitoring Instrument (OMI; González Abad et al., 2015; De Smedt et al., 2015, 2018; Kaiser et al., 2018; Levelt et al., 2018), Global Ozone Monitoring Experiment-2 (GOME-2; De Smedt et al., 2012, 2015; Vrekoussis et al., 2010; Hewson et al., 2013; Hassinen et al., 2016) and Ozone Mapping and Profiler Suite (OMPS; Li et al., 2015; González Abad et al., 2016). They are used in many studies related to air quality and climate change (e.g. Stavrakou et al., 2014, 2015, 2016, 2018; Fortems-Cheiney et al., 2012; Marais et al., 2012; Mahajan et al., 2015; Choi and Souri, 2015; Zhu et al., 2016; Chan Miller et al., 2017; Jin et al., 2017; Barkley et al., 2017; Cao et al., 2018; Khan et al., 2018; Surl et al., 2018; Shen et al., 2019; Su et al., 2019; Zyrichidou et al., 2019; Jin et al., 2020; Souri et al., 2020; Wells et al., 2020; Franco et al., 2021; Opacka et al., 2021).

Launched on board of the European Copernicus Sentinel-5 Precursor (S5P) satellite on 13 October 2017, the TROPOspheric Monitoring Instrument (TROPOMI, Veefkind et al., 2012) is designed for the daily monitoring of the troposphere at the global scale. Compared to its predecessor OMI, its spatial resolution is about 16 times better with at least the same signal-to-noise ratio per ground pixel. The improved TROPOMI capabilities for the observation of HCHO have been illustrated for the detection of fire plumes and their transport (Alvarado et al., 2020; Theys et al., 2020) and the detection of rapid changes in anthropogenic emissions related to the COVID crisis in China and India (Levelt et al., 2021; Sun et al., 2021). The TROPOMI observations extend the historical time series of midday observations performed using OMI. Both datasets are used in combination for long-term trend studies (Li et al., 2020). It is therefore important to evaluate their level of agreement and to report on the best practices to combine datasets from different sensors.

The TROPOMI vertical column product requirements specify a single-measurement precision of 12 × 10¹⁵ molec. cm⁻², 4 × 10¹⁵ molec. cm⁻² at 20 km spatial resolution and a systematic uncertainty lower than 40 %–80 % (ESA, 2021). The Copernicus user requirements, primarily defined for NMVOC measurements, are more stringent. For the environmental air quality theme, the required maximum uncertainty is defined as 60 % or 1.3 × 10¹⁵ molec. cm⁻² (least stringent), at the spatial resolution of 20 km and with a revisit time of 2 h. The space and time resolutions are less stringent for the climate theme (30 % or 1.3 × 10¹⁵ molec. cm⁻², 50 km, 3 d) (Bovensmann et al., 2011; Langen et al., 2017).

Given these rather strict product requirements and the diversity of the NMVOC species, lifetimes and sources (biogenic, biomass burning or anthropogenic), a validation approach addressing a large variety of conditions worldwide (tropical, temperate and boreal forests, urban and sub-urban areas) is needed as are continuous measurements in order to obtain good statistics and capture the seasonal variations. Vigouroux et al. (2020) validated the operational TROPOMI HCHO product using a global network of Fourier transform infrared (FTIR) instruments. The study concluded that overall the HCHO product fulfils the requirements of the TROPOMI mission. Compared to the FTIR data, the TROPOMI HCHO columns present a negative bias over high concentrations sites (−31 % for HCHO columns larger than 8 × 10¹⁵ molec. cm⁻²) and a positive bias for clean sites (+26 % for HCHO columns lower than 2.5 × 10¹⁵ molec. cm⁻²). Based on clean sites, an upper limit of 1.3 × 10¹⁵ molec. cm⁻² was estimated for the deviation of daily observations at a spatial resolution of 20 km. It was also pointed out that this level of random uncertainty, although reaching the Copernicus user requirements, is about twice as large as the expected theoretical noise (individual pixel precision divided by the square root of the number of observations). However, Vigouroux et al. (2020) do not address the consistency of TROPOMI HCHO with other satellite products and MAX-DOAS (multi-axis differential optical absorption spectroscopy) HCHO observations.

The present paper is a follow-up of De Smedt et al. (2018), where the HCHO retrieval algorithm applied to both OMI and TROPOMI sensors was presented. Here we concentrate on a global study of 3 years of HCHO observations with TROPOMI, and we analyse their consistency with OMI data. Throughout the paper, we discuss the improved capabilities of TROPOMI for the detection of HCHO at different temporal and spatial scales, from background conditions to high emissions. We start with a few illustrations of the TROPOMI capabilities for HCHO monitoring from space (Sect. 3). We then provide a detailed comparison with the OMI QA4ECV HCHO dataset (Sect. 4). In Sect. 5, a global network of MAX-DOAS instruments is used to validate the OMI and TROPOMI HCHO datasets. Finally, in Sect. 6, we illustrate the enhanced capability of TROPOMI for the detection of very small HCHO emissions with the identification of a signal over shipping lanes in the Indian Ocean.

2 HCHO datasets

2.1 OMI instrument and QA4ECV HCHO product

The Aura satellite was launched in July 2004, in a low-Earth polar orbit crossing the Equator at 13:30 LT. On board Aura, the Ozone Monitoring Instrument (OMI) is a nadir-viewing imaging spectrometer that measures the solar radiation backscattered by the Earth's atmosphere and surface over the wavelength range from 270 to 500 nm (Levelt et al., 2006). Operational Level 2 (L2) products include vertical columns of O₃, SO₂, NO₂, HCHO, BrO and OClO as well as cloud and aerosol information. OMI has a 2600 km wide swath (divided into 60 across-track positions or rows), providing near-daily global coverage. However, due to a detector row anomaly that occurred after a few years of operation, an increasing number of rows had to be filtered out leading to gradual degradation of the coverage. The OMI ground pixel size varies from 13×24 km² at nadir to 28×150 km² at the edges of the swath.

The OMI QA4ECV HCHO product was developed by a European consortium (BIRA, IUP, MPIC, KNMI, WUR) (De Smedt et al., 2017, https://doi.org/10.18758/71021031) in the framework of the EU-FP7 QA4ECV project. A detailed step-by-step study was performed for HCHO and NO₂ retrievals as part of a community effort to homogenize GOME, SCIAMACHY, GOME-2 and OMI, leading to state-of-the art European products (http://www.qa4ecv.eu, last access: 18 August 2021). For this study, we use the version 1.2 of the OMI HCHO dataset that now spans 15 years (2005–2020; Boersma et al., 2018; Lorente et al., 2017; Nightingale et al., 2018; Zara et al., 2018). Note that within QA4ECV, a homogenized dataset of NO₂ and HCHO MAX-DOAS reference measurements (QA4ECV_MAXDOAS, https://uv-vis.aeronomie.be/groundbased/QA4ECV_MAXDOAS/index.php, last access: 18 August 2021) was also developed for satellite validation (see Sects. 2.4 and 5).

2.2 TROPOMI instrument and the HCHO operational product

On board the S5P platform, which – like Aura – flies in a low-Earth afternoon polar orbit with a local overpass time of 13:30 LT, the TROPOMI instrument is based on an imaging spectrometer measuring in the ultraviolet (UV), visible (VIS), near-infrared (NIR) and shortwave infrared (SWIR) spectral regions (Veefkind et al., 2012). Operational L2 products include vertical columns of O₃, SO₂, NO₂, HCHO, CO and CH₄ as well as cloud and aerosol information. TROPOMI has a 2600 km wide swath (divided into 450 across-track positions or rows), providing near-daily global coverage. The spatial resolution at nadir, originally of 3.5 × 7 km² (across-track × along-track) has been refined to 3.5 × 5.5 km² on 6 August 2019, by a change in the along-track integration time. The size of the pixels remains more or less constant towards the edges of the swath (the largest pixels are ∼ 14 km wide) (L1b ATBD, https://sentinel.esa.int/documents/247904/2476257/Sentinel-5P-TROPOMI-Level-1B-ATBD, last access: 18 August 2021; L1b readme file, http://www.tropomi.eu/sites/default/files/files/publicSentinel-5P-Level-1b-Product-Readme-File.pdf, last access: 18 August 2021).

The retrieval algorithm of the TROPOMI HCHO L2 product is directly inherited from the QA4ECV OMI algorithm with the aim to create a consistent time series of early afternoon observations. For this study, we use a modified version of the TROPOMI level-2 HCHO operational data product, which starts in April 2018 (phase E2, reprocessed (RPRO) + Offline (OFFL), product versions 1.1.[5–8] + 2.1.3, https://doi.org/10.5270/S5P-tjlxfd2). Product versions are described in the Product Readme File (http://www.tropomi.eu/sites/default/files/files/publicSentinel-5P-Formaldehyde-Readme.pdf, last access: 18 August 2021).

2.3 HCHO retrieval algorithm for OMI and TROPOMI

The HCHO retrieval algorithm was fully described in De Smedt et al. (2018), and the successive adaptations of the algorithm are reported in the S5P product ATBD. Here we only provide a short description of the algorithm, which is based on a three-step DOAS method. First, the fit of the slant columns (N_s) is performed in the UV part of the spectra, in the fitting interval 328.5–359 nm. The HCHO cross section is from Meller and Moortgat (2000). All cross sections have been pre-convolved for every row separately with an instrumental slit function adjusted after TROPOMI launch. For the OMI product, the slit function of each row is adjusted daily and the cross sections are reconvolved accordingly. The DOAS reference spectrum is updated daily with an average of Earth radiances measured in the equatorial Pacific region from the previous day. The fit therefore results in a differential slant column, corresponding to the HCHO excess over sources compared to the remote background. In a second step, the conversion from slant to tropospheric vertical columns (N_v) is performed using a look up table of vertically resolved air mass factors (M) calculated at 340 nm with the radiative transfer model VLIDORT v2.6 (Spurr, 2008). Entries for each ground pixel are the observation geometry, the surface elevation and reflectivity, and clouds treated as reflecting surfaces and a priori tropospheric HCHO profiles. The surface albedo is taken from the monthly OMI albedo climatology at the spatial resolution of 1^∘ × 1^∘ (minimum Lambertian equivalent reflectivity, LER, Kleipool et al., 2008). A priori vertical profiles are provided by the TM5-MP daily analysis, at the spatial resolution of 1^∘ × 1^∘ (Williams et al., 2017). A cloud correction based on the independent pixel approximation (Boersma et al., 2004) is applied for cloud fractions (CFs) larger than 0.1. Finally, to correct for any remaining global offset and possible stripes arising between the rows, a background correction is performed based on the HCHO slant columns in the Pacific Ocean (N_s,0). For the TROPOMI operational product, N_s,0 is based on the 4 previous days. For this study, and for the OMI product, we perform the correction on the current day in order to further reduce the stripes. To compensate for a background HCHO level in the equatorial Pacific (due to the methane oxidation), a vertical column of HCHO ($N_{\mathrm{v},\mathrm{0}}^{\mathrm{CTM}}$) is taken from the TM5 model in the reference region. The resulting tropospheric HCHO vertical column can be written as follows:

$$\begin{array}{}\text{(1)}& N_{\mathrm{v}}={\displaystyle \frac{N_{\mathrm{s}}-N_{\mathrm{s},\mathrm{0}}}{M}}+{\displaystyle \frac{M_{\mathrm{0}}}{M}}{N}_{\mathrm{v},\mathrm{0}}^{\mathrm{CTM}},\end{array}$$

with M₀ the air mass factor in the reference sector. Intermediate quantities and auxiliary data are all stored in the L2 files (see the product user manual for TROPOMI and OMI). Several diagnostic variables are provided together with the measurements. The column averaging kernels and the a priori profiles are given for each observation. The tropospheric column uncertainty is resolved into its random (precision) and systematic components (accuracy) and is provided for every individual pixel.

The main difference between the OMI and TROPOMI algorithms lies in the cloud product that is used to compute air mass factors. While the QA4ECV OMI product is based on the O₂–O₂ absorption feature around 477 nm and considers a fixed cloud albedo of 0.8 (version 2.0, Veefkind et al., 2016), the TROPOMI product uses the S5P operational cloud product in CRB (Cloud as Reflecting Boundary) mode (OCRA/ROCINN-CRB; Loyola et al., 2018). The S5P ROCINN algorithm is based on the O₂ A band around 760 nm and simultaneously retrieves cloud height and cloud albedo. Systematic differences between the cloud parameters will result in differences in the air mass factors, influencing the comparisons. To mitigate the impact of this difference between OMI and TROPOMI, we also switch off the cloud correction by replacing the cloud-corrected air mass factor (AMF) by an equivalent clear-sky AMF (M_clear, no cloud correction applied) also provided in the L2 product. Based on Eq. (1), the following simple transformation can be applied:

$$\begin{array}{}\text{(2)}& N_{\mathrm{v}\mathit{\_}\mathrm{clear}}={\displaystyle \frac{M}{M_{\mathrm{clear}}}}{N}_{\mathrm{v}}.\end{array}$$

Note that this transformation has an effect on observations with cloud fractions comprised between 0.1 and 0.4. Indeed, no cloud correction is applied for CF < 0.1 and observations with CF > 0.4 are filtered out from the analysis.

2.4 MAX-DOAS datasets

MAX-DOAS instruments retrieve the abundance of atmospheric trace species in the lowermost troposphere (Hönninger et al., 2004; Wagner et al., 2004; Wittrock et al., 2004; Heckel et al., 2005). Based on DOAS analyses (Platt and Stutz, 2008) of the scattered sky light under different viewing elevations, high sensitivity close to the surface is obtained for the smallest elevation angles, whereas measurements at higher elevations provide information on the rest of the column. MAX-DOAS measurements have been used in several studies to validate satellite HCHO columns (Vigouroux et al., 2009; Franco et al., 2015; De Smedt et al., 2015; Chan et al., 2019, 2020; Ryan et al., 2020; Kumar et al., 2020). However, a global network of MAX-DOAS instruments has not been used yet for the validation of HCHO columns from space.

Ground-based data used in this study are presented in Table 1. Apart from the QA4ECV MAX-DOAS dataset, which relies on harmonized HCHO retrievals (Pinardi et al., 2013; QA4ECV D3.8, http://www.qa4ecv.eu/sites/default/files/QA4ECV_D3.8_v1.0_web.pdf and D3.9, http://www.qa4ecv.eu/sites/default/files/D3.9.pdf, http://www.qa4ecv.eu/node/9#overlay-context=, last access: 18 August 2021), the MAX-DOAS data sets used here were generated by instrument principal investigators using non-harmonized settings. The conversion to vertical columns and/or vertical profiles relies on methods of various complexity levels. Table 1 includes details about the retrieval strategy adopted by the different teams. These include the following:

• GA: geometrical approximation; the vertical column is determined using a single-scattering approximation adequate for moderately high-elevation angles α (typically 30^∘) so that a simple geometrical air mass factor (AMF ≡ SCD$/$VCD = $\mathrm{1}/\sin \left(\mathit{\alpha }\right)$; SCD stands for slant column density, and VCD stands for vertical column density) (Hönninger et al., 2004; Brinksma et al., 2008; Ma et al., 2013) can be used;

• QA4ECV: the vertical column is calculated using tropospheric AMFs based on climatological profiles and aerosol loads as developed during the QA4ECV project (QA4ECV_MAXDOAS_readmefile, http://uv-vis.aeronomie.be/groundbased/QA4ECV_MAXDOAS/QA4ECV_MAXDOAS_readme_website.pdf, last access: 18 August 2021); these data are less sensitive to relative azimuth angle than the purely geometric approximation presented above;

• OEM: vertical profile algorithms using an optimal estimation method (Rodgers, 2000); these make use of a priori vertical profiles and associated uncertainties (Frieß et al., 2006; Clémer et al., 2010; Hendrick et al., 2014; Gielen et al., 2017; Wang et al., 2019a; Friedrich et al., 2019; Bösch et al., 2018);

• PP: vertical profile algorithms based on parameterized profile shape functions; these make use of analytical expressions to represent the trace gas profile using a limited number of parameters (Irie et al., 2008, 2011; Li et al., 2010; Vlemmix et al., 2010; Wagner et al., 2011; Beirle et al., 2019).

Both OEM and parameterized profiling approaches provide vertical profiles of aerosols and HCHO with good sensitivity in the 0–4 km altitude range, in which one to three independent pieces of information in the vertical dimension are available (Vlemmix et al., 2015; Frieß et al., 2016, 2019). Recent intercomparison studies (Vlemmix et al., 2015; Frieß et al., 2019; Tirpitz et al., 2021) show that both OEM and parameterized inversion approaches lead to consistent results in terms of tropospheric vertical columns but to larger differences in terms of profiles. The accuracy of the MAX-DOAS technique depends on the SCD retrieval noise, the uncertainty of the HCHO absorption cross sections, the choice of the a priori profile shape and the uncertainty of the tropospheric AMF calculation. MAX-DOAS HCHO slant columns from several instruments have been compared during international large-scale campaigns (CINDI-1 and 2, e.g. Pinardi et al., 2013; Kreher et al., 2020) showing relatively large median differences and larger noise compared to other slant column product comparisons (e.g. NO₂). For HCHO, the slant column precision depends strongly on the signal-to-noise performance of the DOAS instrument with significantly better results for low-noise research-grade MAX-DOAS instruments (Pinardi et al., 2013; Kreher et al., 2020). The estimated total uncertainty on HCHO VCD is of the order of 30 % to 60 % in polluted conditions. This includes both random (∼ 5 % to 30 % depending on instrumental signal-to-noise ratio) and systematic (20 %) slant column contributions (Pinardi et al., 2013).

Table 1 MAX-DOAS HCHO datasets included in the validation exercise. GA stands for geometrical approximation, OEM for optimal estimation method and PP for parameterized profiling.

2.5 Data use and method

For this study, unless specified otherwise, we filter the satellite data based on the quality assurance values (QA) (Product Readme File, http://www.tropomi.eu/sites/default/files/files/publicSentinel-5P-Formaldehyde-Readme.pdf). QA > 0.5 filters out most observations presenting an error flag or a solar zenith angle larger than 70^∘, a cloud radiance fraction (CRF) at 340 nm larger than 0.6, an air mass factor smaller than 0.1, surface reflectivity larger than 0.2 or an activated snow/ice flag. It should be noted that, in the first versions of the operational product, the QA values were not correctly assigned over snow/ice regions, above 75^∘ of SZA (solar zenith angle) and sometimes over cloudy scenes. This issue has been corrected from version 2.1.3 (July 2020). For this study, we therefore reassigned QA values using the above-mentioned filters.

We calculated daily gridded data at a resolution of 0.05^∘ × 0.05^∘ in latitude and longitude, both for OMI and TROPOMI, using the Harp atmospheric toolbox (https://atmospherictoolbox.org/harp/, last access: 18 August 2021). Throughout the paper, daily and monthly averages are obtained from daily grids. For each day, we require the region to be filled with a least 50 % of valid grid cells, with a minimum of 10 TROPOMI observations (two OMI observations).

For the satellite–satellite and the satellite–ground-based comparisons, we calculate the median of the absolute differences (absolute bias) and the median of the relative differences (relative bias) in each region or station (relative either to TROPOMI in the case of satellite–satellite or to the MAX-DOAS columns in the case of satellite–ground-based). The corresponding median absolute-value deviations (MADs) of the absolute and relative differences are a robust estimate of the combined observation and comparison variability. The MAD is defined as the median of the absolute-value deviations from the data's median:

$$\begin{array}{}\text{(3)}& \mathrm{MAD}=k\cdot \mathrm{median}\left(\mathrm{abs}\right(\mathrm{Diff}\mathit{\_}i-\mathrm{median}\left(\mathrm{Diff}\mathit{\_}i\right)),\end{array}$$

where the factor k=1.4826 is used to ensure a correspondence with the 1σ standard deviation for normal distribution. The bias is regarded as statistically significant if it exceeds $\mathrm{ErrB}=\mathrm{2}\cdot \mathrm{MAD}/\sqrt{N}$, where N is the number of collocated pairs (days or months). We also derive correlation, slope and offset of the linear regression using the robust Theil–Sen estimator (Sen, 1968) as done in Vigouroux et al. (2020).

3 TROPOMI HCHO tropospheric columns

As an illustration of the data product, Fig. 1 displays the global seasonal distribution of tropospheric HCHO columns derived from TROPOMI observations between March 2018 and February 2021. The overall seasonality of the HCHO columns is largely driven by the emissions of NMVOCs from the vegetation and by the interannual variability of surface temperatures and solar radiation. As can be seen, in the south-eastern US for example, the seasonal amplitude is very important and dominated by biogenic emissions during summertime. On top of biogenic emissions, wildfires present a large variability. Since 2018, many fire events have occurred worldwide and can be traced, e.g., in HCHO columns during summer 2018 and 2020 in the western US or during summer 2019 in Siberia. After a decrease of about 10 years (De Smedt et al., 2015), South America experienced two intense fire seasons in 2019 and 2020. The year 2020 was also marked by the huge Australian and Californian wildfires, respectively, in January and October 2020, detectable in the seasonal maps. In comparison to biogenic and pyrogenic emissions of natural origin, the contribution due to anthropogenic NMVOC emissions to the total HCHO columns is generally lower. Although their oxidation is also enhanced by sunlight, anthropogenic emissions show less seasonality than natural emissions, and their detection is therefore generally easier in annual maps. This is illustrated in Fig. 2, which presents 3-year averages of HCHO columns over Asia, the Arabic Peninsula, the US, and Central and South America, providing detailed information about the spatial distribution of HCHO at the regional and urban scale. Europe and Africa are shown in the Supplement (Fig. S1 in the Supplement). Note that the colour scale has been adapted to the regions. Large urban areas are clearly visible in the HCHO distribution in Asia, the Middle East and South America. With a lower magnitude, US cities are also clearly detectable, such as Houston, Dallas or Los Angeles. HCHO levels are noticeably lower in Europe, but some urban areas are visible in the southern countries.

Figure 1 Seasonal maps of TROPOMI HCHO tropospheric columns during the first 3 years of measurements (March 2018–February 2021), on a spatial grid of 0.05^∘ in latitude and longitude. Observations are filtered using the qa_values > 0.5 (max scale: 15 × 10¹⁵ molec. cm⁻²). Modified Copernicus Sentinel-5P satellite data, OFFL L2 HCHO product, BIRA-IASB/DLR/ESA/EU.

The quality of the TROPOMI observations also allows observing HCHO columns on a much shorter timescale with an unprecedented definition. Daily observations of fire plumes are a clear step forward in the satellite remote sensing of HCHO. They can be observed over much longer distances than before, thanks to the daily global coverage, coupled with the finer spatial resolution and the improved signal-to-noise ratio, allowing the detection of lower columns transported further away (Alvarado et al., 2020; Theys et al., 2020). It is not only wildfires but also important anthropogenic emission plumes that can be observed on a daily basis, for example on the eastern coast of Saudi Arabia. A few illustrations are given in Fig. S2. The TROPOMI performances for the observations of HCHO are discussed more quantitatively throughout the paper in terms of precision and bias as a function of the HCHO levels and of the temporal and spatial scales.

Figure 2 Multi-annual regional maps of TROPOMI HCHO tropospheric columns (March 2018–February 2021), on a spatial grid of 0.05^∘ in latitude and longitude. Observations are filtered using the qa_values > 0.5. Modified Copernicus Sentinel-5P satellite data, OFFL L2 HCHO product, BIRA-IASB/DLR/ESA/EU.

4 Comparison between OMI and TROPOMI measurements

In this section, we evaluate the consistency between OMI and TROPOMI HCHO tropospheric columns. In addition, we present the gain in precision obtained with TROPOMI. The analysis relies on 32 months of simultaneous measurements from April 2018 to December 2020, allowing for a meaningful comparison at different scales. We first compare the precision obtained on individual measurements and then proceed with a comparison of the precisions achieved when averaging data at different spatial and temporal scales.

4.1 HCHO slant column precision

The random uncertainty of the tropospheric HCHO column is dominated by the error on the fitted slant column densities (SCDEs), which is directly related to the signal-to-noise ratio (SNR) of the measurement. From this point of view, TROPOMI performs significantly better than previously launched nadir UV–VIS satellite instruments. In the spectral range of HCHO retrievals (328.5–359 nm), the SNR of the TROPOMI spectra exceeds pre-flight requirements that were based on OMI specifications (Kleipool et al., 2018; Ludewig et al., 2020).

Figure 3 presents global maps of SCDE averaged over 3 months during summer 2019, from OMI and TROPOMI. From the improved SNR of TROPOMI in the UV range, TROPOMI HCHO SCDEs of individual observations are about 25 % lower than OMI ones. Over remote areas, the TROPOMI SCDE is about 6 × 10¹⁵ molec. cm⁻², while it is 8 × 10¹⁵ molec. cm⁻² for OMI. Slant column density errors are also improved over emission areas and at larger SZAs. Contrary to OMI, the effect of the South Atlantic Anomaly is absent in TROPOMI SCDE. This probably results from a better shielding of the instrument against extra-terrestrial high-energy radiation. The implemented iterative spike algorithm (De Smedt et al., 2018) is also more efficient because of the lower noise level of the instrument. Note however that over mountains, TROPOMI SCDEs are higher than OMI ones. The most obvious effect is observed over the Himalayas, but other chains such as the Andes or the Rocky mountains are also affected. This effect has been identified as a scene inhomogeneity effect (Richter et al., 2018, 2020). The effect is also visible along the borders of bright lakes or white surfaces. OMI retrievals are also affected by scene inhomogeneity effects, but the larger size of the ground pixels and the larger mean SCDE values make its detection more difficult. We note that in the 3-year averaged maps of the HCHO tropospheric columns, some collocated artefacts appear (Fig. 2, e.g. the white sands in the US, Tuz Golu lake in Turkey or Lake Mackay in Australia). Most of the snow/ice scenes are eliminated by the quality assurance values. The observations could however be better filtered over mountains and along the lake borders or even corrected during the fit of the slant columns as demonstrated for NO₂ and glyoxal (Lerot et al., 2021). The relatively coarse albedo climatology also needs to be updated with a TROPOMI-based product, better defined in space and time (Loyola et al., 2020).

Figure 3 Average HCHO slant column density fitting error (SCDE) retrieved from OMI (a) and TROPOMI (b) in JJA 2019, on a spatial grid of 0.05^∘ in latitude and longitude.

The OMI SCDEs have been very stable over the years, showing a limited increase of about 5 % between 2005 and 2019 (De Smedt et al., 2018). However, the number of valid OMI observations has decreased by about 30 % during the same period (−50 % at large SZA) due to the row anomaly. In order to evaluate the stability of the TROPOMI HCHO retrievals during the first 3 years, Fig. 4 presents the time series of the TROPOMI HCHO slant column errors in the remote Pacific Ocean as a function of latitude and instrumental rows. As expected, we observe an increase in the noise for large SZAs and for the 25 first and last rows of the scan, which have a different detector binning (L1b ATBD, https://sentinel.esa.int/documents/247904/2476257/Sentinel-5P-TROPOMI-Level-1B-ATBD). The fact that the algorithm makes use of daily updated radiances as reference for the DOAS fit allows for very stable results in time and across the rows. Only the change in pixel size in August 2019 (L1b readme file, http://www.tropomi.eu/sites/default/files/files/publicSentinel-5P-Level-1b-Product-Readme-File.pdf) resulted in a moderate step increase in the SCDE of about 15 %. These values are compared to the observed standard deviation of the slant columns in the same regions (see Fig. S3). We observe a very good agreement between the SCDEs and the standard deviation, indicating that they give a good representation of the random errors.

The reported uncertainty in the tropospheric vertical columns due to random errors corresponds to the SCDE divided by the AMF for each observation. In the equatorial Pacific, the TROPOMI vertical column precision is about 5 × 10¹⁵ molec. cm⁻², while it is 7 × 10¹⁵ molec. cm⁻² for OMI. It is larger over continental emissions, where the AMFs are generally smaller than 1.

Figure 4 TROPOMI HCHO slant column density errors (SCDEs) as a function of the latitude (a) or the detector row (b). The step increase on 6 August 2019 reflects the change in the TROPOMI pixel size (indicated with the black line).

4.2 HCHO tropospheric columns

Figure 5 presents the yearly averaged OMI and TROPOMI HCHO vertical columns (N_{v_clear}) for 2019. Even at this level of averaging, the lower noise level of TROPOMI is very clear, especially for low to medium HCHO levels. We observe an overall good agreement of the columns both in magnitude and in their spatial distribution. Differences in TROPOMI and OMI yearly averages range from +2 × 10¹⁵ molec. cm⁻² over the tropics to −2 × 10¹⁵ molec. cm⁻² over midlatitude regions. Differences tend to increase with latitudes. However, as the quality of the TROPOMI observations is improved at large solar zenith angles, more data in winter months are kept in the TROPOMI dataset, which can influence yearly averaged columns at those latitudes. In order to provide quantitative comparisons, we calculated daily and monthly averaged columns in 35 regions covering a broad range of emission levels and observation conditions (large black boxes in Fig. 5). As the regions are large, many observations are included (on average 500 per day for OMI, 12 500 per day for TROPOMI). To obtain daily and monthly comparison pairs, we keep coincident days of observations and follow the methodology presented in Sect. 2.5.

Figure 5 Average HCHO tropospheric column (N_{v_clear}) retrieved from OMI (first line) and TROPOMI (second line) in 2019. Limits of the regions selected for the comparisons are shown on the TROPOMI map. Differences between OMI and TROPOMI maps are shown on the last panel. The same grid is used for both datasets (0.05^∘). Data are filtered using the product quality flags. The large black boxes on the TROPOMI maps represent the regions used in the comparisons (see Figs. 6 and 7).

An example of a time series over equatorial Africa is presented in the first panel of Fig. 6, where monthly averaged N_{v_clear} is shown, and comparison numbers are provided in the figure. In the equatorial African region, the seasonal cycle is marked by two peaks during the dry seasons and two minima during the wet seasons. In 2019, the minimum was particularly low, observed in both the OMI and TROPOMI time series, while the maxima tend to increase over the years. More examples of time series can be found in Fig. S4. In all the regions, the seasonal and interannual variability of the HCHO columns are observed very consistently with OMI and TROPOMI.

Figure 7 presents the absolute and relative biases between OMI and TROPOMI HCHO tropospheric columns for all regions. Numbers are provided for daily averaged columns applying a cloud correction (N_v) or not (N_{v_clear}). Regions are sorted as a function of the averaged TROPOMI HCHO column. At this large spatial scale, the regions over equatorial Africa, northern China and northern India present the largest annual columns worldwide, with median levels larger than 10 × 10¹⁵ molec. cm⁻². Tropical regions in South America, Africa and Asia present elevated levels of HCHO as well, with annual averaged columns larger than 8 × 10¹⁵ molec. cm⁻².

Looking at N_v comparisons, it appears that the OMI HCHO columns present a positive bias compared to TROPOMI from 17 ± 2.5 % for the columns larger than 5 × 10¹⁵ molec. cm⁻², to 30 ± 5 % for the lower columns. This bias exceeds 50 % in northern latitudes (> 45^∘) and low-emission (< 2 × 10¹⁵ molec. cm⁻²) regions of Canada and Alaska. However, when comparing N_{v_clear}, the biases are strongly reduced below 10 % in all regions where the HCHO levels are larger than 5 × 10¹⁵ molec. cm⁻², and the TROPOMI columns are found to be slightly larger than OMI on average ($-\mathrm{3}\pm \mathrm{1.2}$ %). In northern midlatitudes/moderate-emission (2–5 × 10¹⁵ molec. cm⁻²) regions such as Europe, central and western US, north-western Canada, Siberia, or Tibet, OMI columns present a remaining bias of about 15 ± 3 %, while in the regions of Canada and Alaska, a larger bias of about +30 ± 7 % remains. Note that we observe biases lower than 10 % in the Maghreb and southern Australia regions, despite their relatively low columns or low latitudes.

Figure 6 Examples of monthly and yearly averaged HCHO columns (N_{v_clear}) retrieved from OMI (October 2004–December 2020, in red) and TROPOMI (2018–December 2020, in black) at two different spatial scales selected for the comparison: a large region of equatorial Africa and a circle of 20 km radius over New Delhi in India. Absolute and relative biases between OMI and TROPOMI HCHO monthly averaged columns are given at the top of each panel as are the median deviations of the OMI and TROPOMI averaged columns. (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻²]).

We conclude that biases up to 30 % related to the cloud correction are observed over tropical regions where the clouds are the highest in altitude (Africa, South America, South Asia), and a smaller but systematic effect, up to 15 %, is observed over midlatitude polluted regions such as China, India, the US or Europe. We also note that the differences between N_v and N_{v_clear} are mainly significant for the OMI HCHO columns. It has been reported that the cloud pressures retrieved from TROPOMI and from OMI present a bias (OMI clouds are higher in altitude, Compernolle et al., 2021). This translates into OMI cloud-corrected air mass factors generally smaller than TROPOMI AMFs by 5 % to 30 %, depending on the cloud altitude, and therefore in a positive bias of the OMI HCHO VCD compared to the TROPOMI product. It is therefore important to keep in mind that the use of different cloud products may introduce inconsistencies, which may be resolved by using clear HCHO VCDs (N_{v_clear}).

Figure 8 shows the linear regression between OMI and TROPOMI monthly averaged columns, considering all regions together. The relation between OMI and TROPOMI is provided for N_v and N_{v_clear}. This shows that switching off the cloud correction in the OMI and TROPOMI HCHO products allows to significantly improve not only the slope (from 0.87 to 0.92) and the intercept (from 1.52 to 0.48 × 10¹⁵ molec. cm⁻²), but also the data scatter, i.e. the Pearson R correlation (from 0.74 to 0.98). When considering large-scale comparisons, the agreement between OMI and TROPOMI N_{v_clear} is therefore very satisfactory.

Figure 7 Absolute and relative biases between OMI and TROPOMI HCHO daily averaged tropospheric columns using cloud-corrected AMF (N_v, a, b) or clear-sky AMF (N_{v_clear}, c, d) for the large regions represented on Fig. 5. Regions are sorted as a function of the median TROPOMI HCHO column. Values of the averaged HCHO columns are provided on the top axis as are the numbers of common days taken for the comparison and the latitude of the region. The median OMI (red) and TROPOMI (black) columns are plotted together with the absolute differences (in blue). Error bars represent the median deviations of the columns or the median absolute deviations of the differences (MAD, in grey). Statistical ErrB is also plotted for the relative bias (in blue). Pink areas indicate 10 % and 20 % bias. (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

Figure 8 Scatter plots of OMI versus TROPOMI columns for the monthly means of collocated data. Results are shown for N_v (a) and N_{v_clear} (b). The correlation, slope and intercept of a linear regression using the robust Theil–Sen estimator are given in each panel and plotted as a blue line. The black dotted line is the 1 : 1 line. The colour indicates the latitude of the region. (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

When averaging data over large regions, the dispersion due to random uncertainties is greatly reduced compared to individual observations. As summarized in Table 2, the median absolute deviations of the monthly averaged columns are equivalent for OMI and TROPOMI (1.8 × 10¹⁵ molec. cm⁻²), while the MADs of their differences are significantly lower (0.5 × 10¹⁵ molec. cm⁻²). This indicates that at this spatiotemporal resolution, the natural variability dominates the dispersion of the averaged observations. Looking at the daily averaged columns, the TROPOMI median deviation is lower than for OMI (2.2/2.7) but still larger than the MAD of their differences (1.5).

The improved spatial resolution of TROPOMI should allow for a better detection of localized HCHO columns. To address this question, we performed the same comparisons as for the large regions but looking at smaller areas of 20 km radius around cities. Figure 9 presents the absolute and relative biases of the monthly averaged HCHO columns (N_{v_clear}) for a large number of cities. At this spatial scale, Jakarta is the location with the largest median HCHO level (> 18 × 10¹⁵ molec. cm⁻² over the 2018–2020 period). Indian, Chinese and other Asian cities follow as well as Mexico, Monterrey or Kinshasa (> 12 × 10¹⁵ molec. cm⁻²). Sao Paulo, Tehran and Cairo also present noticeably elevated HCHO levels (> 9 × 10¹⁵ molec. cm⁻²). An example over New Delhi is presented in the second panel of Fig. 6 and more examples can be found in Fig. S5.

When comparing OMI and TROPOMI N_{v_clear} around the cities, the same general behaviour as in the large regions can be observed. OMI presents a positive bias (20 ± 15 %) compared to TROPOMI for low to medium HCHO levels, while for medium to high levels, the agreement is very good on average (−1 ± 10 %). There are nevertheless a few exceptions where TROPOMI HCHO columns are significantly larger than the OMI ones. This is the case at La Réunion, Paramaribo, Nairobi, Bujumbura, Sao Paulo, Monterrey, Mexico or Jakarta. Those cities are located along marine coasts or lakes, at higher altitude, or are surrounded by mountains. In those cases, the finer spatial resolution of TROPOMI clearly improves the detection of the HCHO signal. For most other locations, however, the impact of the improved spatial resolution of TROPOMI on the HCHO columns is not detectable in the column magnitudes, when compared to OMI observations. This is likely related to the nature of the HCHO production that mostly is secondary from the oxidation of NMVOCs with various lifetimes (Stavrakou et al., 2015; Bauwens et al., 2016). Except for regions where the topography presents sharp discontinuities, this causes a natural spread of the HCHO columns at a scale larger than the TROPOMI spatial resolution.

Note however that at this spatial resolution (20 km radius), the level of noise is larger than for the regional averages and the TROPOMI-averaged columns are significantly more stable than the OMI ones, as evidenced by their median deviations (see Table 2). On a daily basis, the OMI columns present a dispersion of 7.8 × 10¹⁵ molec. cm⁻², while the TROPOMI dispersion is about twice smaller (3.7 × 10¹⁵ molec. cm⁻²). In this case, the MAD of the differences (7.1 × 10¹⁵ molec. cm⁻²) is dominated by the noise in OMI observations. Note that these estimates still include the natural variability of the columns themselves. If an area of 20 km radius in the remote equatorial Pacific is considered, the observations represent constant background values and the seasonal variability is further reduced. In such conditions, the dispersion of the OMI daily observations is 3.5 × 10¹⁵ molec. cm⁻² but only 1 × 10¹⁵ molec. cm⁻² for TROPOMI. We show in the next section that validation with ground-based measurements brings further information on the satellite column precision.

Figure 9 Absolute and relative biases between OMI and TROPOMI HCHO monthly averaged tropospheric columns using clear-sky AMF (N_{v_clear}) within 20 km radius circles around selected cities, sorted as a function of the median TROPOMI HCHO column. Values of the averaged HCHO columns are provided on the top axis as are the numbers of months taken for the comparison and the latitude of the region. The median OMI (red) and TROPOMI (black) columns are plotted together with the absolute differences (in blue). Error bars represent the median absolute deviations (MADs) of the columns and of the differences (in grey). Statistical ErrB is also plotted for the relative bias (in blue). Pink areas indicate 10 % and 20 % bias. (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

Table 2 Median absolute deviation of the OMI and TROPOMI daily and monthly averaged columns (N_{v_clear}), in large regions and in a 20 km radius area. MADs of differences between OMI and TROPOMI columns are also given in the last column.

5 Validation with a global MAX-DOAS network

Here, we present a validation exercise based on a network of 18 ground-based MAX-DOAS instruments. This effort complements the study of Vigouroux et al. (2020), which relied on a network of FTIR instruments. Compared to the FTIR instruments, the MAX-DOAS instruments provide a higher sensitivity in the boundary layer, where the bulk of HCHO is located. The MAX-DOAS network covers stations where the level of HCHO is significant, from medium to very large HCHO columns, while the FTIR network includes a larger number of remote stations. In this study, we validate in parallel the OMI and TROPOMI datasets. We first focus on a direct comparison of the satellite and MAX-DOAS tropospheric columns. The effect of the vertical smoothing is investigated in the next subsection for three stations.

5.1 Direct comparisons of tropospheric columns

For each station in Table 1, we consider daily averages of the satellite columns in a radius of 20 km around the instruments. We average MAX-DOAS columns between 11:00 and 16:00 local time. We keep coincident days of observations (OMI/MAX-DOAS, TROPOMI/MAX-DOAS) to obtain daily and monthly comparison pairs. Note that the time periods used for the comparison are not the same for OMI and TROPOMI and vary between the stations. To obtain the validation results, we follow the methodology presented in Vigouroux et al. (2020) (see Sect. 2.5).

Figure 10 Absolute (a, blue line) and relative biases (b) between MAX-DOAS and TROPOMI HCHO daily averaged tropospheric columns in a circle of 20 km radius around the stations. Regions are sorted as a function of the median MAX-DOAS HCHO column. In (a), the median MAX-DOAS (red) and TROPOMI (black) columns are plotted together with the differences. Error bars (in grey) represent the median absolute deviations (MADs) of the columns and of the differences. Statistical ErrB is also plotted for the relative bias (in blue). Pink areas indicate 20 % and 40 % bias. The correlation between the daily observations is given in (b) (grey circles). (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm-².)

Figure 11 Same as Fig. 10 for daily averaged MAX-DOAS and OMI HCHO.

Figures 10 and 11 present the absolute and relative biases of the daily averaged columns as a function of the median MAX-DOAS HCHO column, respectively, for TROPOMI and OMI. A more detailed description for each station and for individual time series is presented afterwards. The values of the biases are similar for OMI and TROPOMI, except for the lowest columns in Uccle and Fukue, where OMI presents larger positive biases exceeding +20 %. In agreement with Vigouroux et al. (2020), TROPOMI columns do not present a significant bias for the range of HCHO levels from 4 to 8 × 10¹⁵ molec. cm⁻². Note that, in contrast to FTIR data, the range of values covered by our MAX-DOAS network does not extend to columns lower than 4 × 10¹⁵ molec. cm⁻². We observe that the stations in De Bilt and Cabauw tend to show somewhat stronger negative biases even for medium levels of HCHO, which might point to a network inhomogeneity. For larger HCHO columns (> 8 × 10¹⁵ molec. cm⁻²) and in agreement with the FTIR results, we observe that negative biases tend to increase for large HCHO columns such that the underestimation of the satellite columns reaches about −40 % for the largest columns. On the upper plot, the error bars represent the median absolute deviations of the columns and of their differences. It appears clearly that the MADs obtained with TROPOMI are substantially lower than those obtained with OMI. Note that the type of MAX-DOAS instrument (in particular its signal-to-noise ratio) may also influence the observed MAD at the different stations.

Figures 12, 13 and 14 present more detailed results for the stations in Europe, in Japan and Australia, and in China, India, Thailand and in Mexico, respectively. In each plot, the time series of the MAX-DOAS, OMI and TROPOMI data are displayed together. Results of the daily statistical analysis are given in each panel. At European stations, which show medium-range HCHO levels, we obtain contrasting results. With a mean HCHO column of 4.5 × 10¹⁵ molec. cm⁻², Uccle is one of the stations with the lowest columns of the network presented in this paper. While OMI values show a positive bias (13 ± 15 %) and a poor correlation (0.3) with the MAX-DOAS, TROPOMI appears to be biased low ($-\mathrm{10}\pm \mathrm{6}$ %) but much better correlated (0.82) with the MAX-DOAS data. As opposed to Uccle, the observed biases in De Bilt, Cabauw and Mainz are largely negative (from −40 % to −50 %). The correlations found with TROPOMI are nevertheless much better than with OMI. Note that the median MAX-DOAS HCHO value in Mainz is larger than 10 × 10¹⁵ molec. cm⁻², which is quite high for a European site. The results in Munich have been presented in detail in Chan et al. (2020). They are closer to what is found in Uccle, with a small positive bias for TROPOMI (1±3 %) and for OMI (6±13 %). Similarly in Madrid, OMI and TROPOMI results are very consistent with a mean bias of, respectively, 8±16 % and 10±6 %. In Thessaloniki, the negative bias is $-\mathrm{12}\pm \mathrm{5}$ %, but the correlation is poorer than in Madrid.

Figure 12 Time series of MAX-DOAS HCHO columns (blue), OMI N_{v_clear} (red) and TROPOMI N_{v_clear} (black) at European sites. Thick lines show monthly median values, and dots represent daily median values. Mean relative bias, median absolute deviations and correlations between the time series are provided for the daily averaged data. (Pmolec. cm⁻² = 10¹⁵ molec. cm⁻².)

In Fig. 13, we show three Japanese stations operated by the CHIBA University. Mean HCHO levels in Japan are comparable to values found at European sites. In Chiba and Kasuga, TROPOMI and MAX-DOAS columns are strongly correlated (about 0.7), but on the island of Fukue the correlation is poor due to a lack of variability at this site. At all these sites, TROPOMI shows small biases relative to MAX-DOAS data ($-\mathrm{9}\pm \mathrm{4}$ % in Chiba, 3±4 % in Kasuga, 8±8 % in Fukue). The HCHO observations in Broadmeadows, in northern Melbourne, have been published by Ryan et al. (2020). We find a bias of $-\mathrm{12}\pm \mathrm{6}$ % for TROPOMI and a good correlation of about 0.7. Quite unusually, the seasonal amplitude of the MAX-DOAS time series at this station is smaller than observed with OMI and TROPOMI.

Figure 13 Same as Fig. 12 in Japan and Australia.

Stations with large HCHO levels in China, India, Thailand and Mexico are presented in Fig. 14. In China, we show the results of two instruments in Xianghe and one instrument in Beijing. With the USTC instruments, we find small biases of $-\mathrm{4}\pm \mathrm{4}$ % and $-\mathrm{5}\pm \mathrm{5}$ % and correlations larger than 0.8. With the BIRA-IASB instrument in Xianghe, the correlation is also excellent. The MAX-DOAS columns are larger than the ones obtained with the USTC instrument, and we find a significant negative bias of the TROPOMI data of $-\mathrm{27}\pm \mathrm{2}$ %. However, this larger bias is in better agreement with the results found for equivalent stations in India and with FTIR validation results in Xianghe (Vigouroux et al., 2020). This result illustrates the actual uncertainty related to the ground-based measurements themselves and the need for further harmonization of the MAX-DOAS network. Correlations in India and Thailand are of about 0.7, while the biases are consistently negative ($-\mathrm{21}\pm \mathrm{2}$ % in Mohali, $-\mathrm{38}\pm \mathrm{4}$ % in Pantnagar, $-\mathrm{21}\pm \mathrm{2}$ % in Phimai). The situation is more complex at the UNAM site in Mexico. There, the correlation is poor (0.3), and a negative bias of $-\mathrm{29}\pm \mathrm{3}$ % is found. These results are however more dependent on the radius considered around the station and on the selection of the MAX-DOAS observations (Rivera Cárdenas et al., 2021) (see Sect. 5.4).

Figure 14 Same as Fig. 12 at Chinese, Indian, Thai and Mexican sites.

Finally, Fig. 15 presents scatter plots of the satellite against MAX-DOAS columns, considering all the stations and for daily and monthly comparisons. Table 3 summarizes the validation results. The best agreement is found with monthly TROPOMI columns, for which we find a slope of 0.64 and a positive offset of 1.7 × 10¹⁵ molec. cm⁻² compared to the MAX-DOAS columns. Slopes and biases for the large columns are found to be close for OMI and TROPOMI datasets. The improvement with TROPOMI can be seen in the correlation, offset and bias values obtained for the lower columns as well as in the precision of the daily validation results. On average, the OMI biases are found to be statistically non-significant for the lowest columns. When considering monthly averaged data, the correlation between MAX-DOAS and satellite columns improves from 0.74 with OMI to 0.85 with TROPOMI (+15 %). More importantly, it improves from 0.45 to 0.76 when considering daily observations (+68 %). The daily offset is reduced by 60 % from OMI to TROPOMI (3.1 to 1.9 × 10¹⁵ molec. cm⁻²). In low-emission conditions, the MADs of the differences provide an upper limit of the precision of the satellite measurements. If we consider HCHO levels below 8 × 10¹⁵ molec. cm⁻² (medium level, but the low range is not represented here), the precision of the daily TROPOMI HCHO observations is estimated to be 3 × 10¹⁵ molec.cm⁻², which represents an improvement of more than a factor 2 compared to OMI. The precision of monthly TROPOMI observations reaches 1.4 × 10¹⁵ molec. cm⁻², which is close to the Copernicus user requirements.

Figure 15 Scatter plots of OMI (a, c) and TROPOMI (b, d) versus MAX-DOAS data for the daily (a, b) and monthly (c, d) medians of collocated data. The correlation, slope and intercept of a linear regression using the robust Theil–Sen estimator is given in each panel and plotted as a blue line. The black dotted line is the 1 : 1 line. The colour indicates the latitude of the station. (Pmolec. cm⁻² = 10¹⁵ molec. cm⁻².)

Table 3 Summary of validation results for OMI and TROPOMI when considering all collocated pairs (daily or monthly means) together. Values for HCHO columns lower or larger than 8 × 10¹⁵ molec. cm⁻² are given in brackets.

5.2 Sensitivity tests

We performed a few sensitivity tests, in order to evaluate the robustness of the validation results. First, we used different radii around the stations (from 10 to 100 km), in order to detect possible spatial resolution effects. Results are presented in Fig. 16, for the TROPOMI case. At most stations, the bias shows a marginally small dependency on the radius. Again, this points to the large natural dispersion of the HCHO columns. We find an important exception at the UNAM station in Mexico, where the bias clearly increases with the radius (−30 % at 10 km, −50 % at 100 km). At this location, the correlation and MADs are also improved at 10 km (not shown). In Beijing and Broadmeadows, we do observe an increase in the bias at 100 km resolution, but the values at 10 and 20 km are mostly equivalent. We performed the same test with OMI and found consistent results, except that the lower sampling does not allow using a 10 km radius area.

Figure 16 Median monthly bias as a function of the radius taken around the validation sites. Pink areas indicate 40 % bias.

We also evaluated the impact of clouds using two further tests: (1) comparing the daily TROPOMI validation results for N_v and N_{v_clear}; (2) using a much stricter cloud filter on cloud radiance fractions (CRF) of 20 % instead of 60 % (equivalent to an effective cloud fraction of 10 % instead of 40 %). With this strict cloud filter, there is no difference between N_v and N_{v_clear}. Results are summarized in Table 4. These tests indicate that the TROPOMI HCHO validation results do not change significantly when a cloud correction is applied, although the N_{v_clear} results are slightly better. Using a more stringent cloud filter reduces the number of observations. The bias for the lowest columns becomes positive (from −10 to +3 %), and the offset is increased (from 1.9 to 2.6 × 10¹⁵ molec. cm⁻²), while the negative bias for the largest columns remains equivalent. These numbers will have to be re-evaluated using only version 2 of the TROPOMI level 2 products, available since July 2020, when enough data will be available. However, we note that this limited impact of the cloud correction on the HCHO columns appears to be consistent with previous satellite datasets, independently of the cloud product, as already observed with GOME-2 and OMI, using version 1 of the O₂–O₂ cloud product (De Smedt et al., 2015).

Table 4 Summary of daily validation results for TROPOMI when considering all collocated pairs when using (baseline) N_{v_clear} (first column), (1) when using N_v (second column) or (2) when using a strict cloud filter (third column).

5.3 Effect of vertical smoothing

Three MAX-DOAS stations (Uccle, Xianghe BIRA-IASB and UNAM) provide retrieved and a priori vertical profiles together with corresponding averaging kernels (GEOMS format, https://evdc.esa.int/tools/data-formatting-templates/, last access: 18 August 2021). This allows taking into account the different vertical sensitivity of MAX-DOAS and TROPOMI measurements when making comparisons. We follow the methodology from Rodgers and Connor (2003) described in detail in Vigouroux et al. (2020). It consists of two steps: first taking into account the different a priori profiles used to retrieve these two data sets (Eq. 2 of Vigouroux et al., 2020) and then smoothing the ground-based profiles using TROPOMI averaging kernels (Eq. 3 of Vigouroux et al., 2020).

In Table 5, we give the MAD and biases obtained before and after application of the methodology, for the daily mean comparisons. Note that the numbers at each site are slightly different than the ones obtained in Sect. 5.1 (Figs. 12 and 14) because the collocated pairs are constructed slightly differently: each collocated pixel of the satellite must be compared to MAX-DOAS before the daily average because the TROPOMI averaging kernel differs for each pixel.

We see in Table 5 that at the cleanest site (Uccle) the effect of the smoothing is small, while at the more polluted sites Xianghe and UNAM, the biases are strongly reduced by about 20 %. This result is in agreement with previous MAX-DOAS validation studies (De Smedt et al., 2015; Wang et al., 2019b), but also with aircraft and regional model comparisons (Zhu et al., 2020; Su et al., 2020). The effect of the smoothing is also clearly seen in Fig. 17 where the scatter plots of daily comparisons between TROPOMI and MAX-DOAS are shown before and after vertical smoothing. The strong effect of the smoothing is usually not observed with FTIR comparisons because TROPOMI and FTIR measurements have similar vertical sensitivity, which rapidly drops in the atmospheric layers lower than 3 km (Vigouroux et al., 2020), while MAX-DOAS shows an opposite sensitivity that is maximal at the surface and generally becomes negligible above 3 km (Vigouroux et al., 2009; De Smedt et al., 2015; Wang et al., 2019a). An illustration of typical averaging kernels for OMI, TROPOMI and the MAX-DOAS instrument in Xianghe is provided in Fig. S6. As the observation angles and overpass times are very close for OMI and TROPOMI, their measurements come with a similar vertical sensitivity. This highlights the importance of taking into account the different a priori profiles and averaging kernels when comparing techniques having different vertical sensitivity.

Table 5 Effect of a priori substitution and vertical smoothing on the daily comparisons of TROPOMI and MAX-DOAS data.

Figure 17 Scatter plots of TROPOMI versus MAX-DOAS data for the daily means of collocated data before (a) and after (b) vertical smoothing of the MAX-DOAS profile in Uccle, Xianghe and UNAM, Mexico. The correlation, slope and intercept of a linear regression using the robust Theil–Sen estimator is given in each panel and plotted as a blue line. The black dotted line is the 1 : 1 line (Pmolec. cm⁻² = 10¹⁵ molec. cm⁻²).

6 Detection of weak HCHO columns over shipping lanes

As shown above, TROPOMI HCHO observations feature an unprecedented level of precision allowing for an improved detection of small columns at short timescales. Here, we present a case study to illustrate the ability of TROPOMI to detect small HCHO signals related to shipping emissions. When inspecting TROPOMI maps averaged over several months, weak lines of HCHO columns become visible over the background, especially in the Indian Ocean (see e.g. Fig. 5). This becomes even clearer when saturating the continental HCHO columns by setting a lower maximum scale, as in Fig. 18, which shows HCHO columns seasonally averaged over the months December, January and February between 2018 and 2021.

Figure 18 Seasonal DJF map of TROPOMI HCHO tropospheric columns between December 2018 and February 2021, on a spatial grid of 0.05^∘ in latitude and longitude. Observations are only filtered using the provided qa_values > 0.5 (max scale: 8 × 10¹⁵ molec. cm⁻²).

The detection of shipping emissions with satellite observations has often been reported for NO₂ (see for example Beirle et al., 2004; Richter et al., 2004, 2011; Boersma et al., 2015; Georgoulias et al., 2020) and more recently also for SO₂ based on OMI measurements (Theys et al., 2015). In the case of HCHO, however, only one study pointed to the identification of a shipping lane signal detected in a 7-year average of ERS-2 GOME data in the ship track corridor from Sri Lanka to Singapore (Marbach et al., 2009).

Here, we study two lines: (1) from Sri Lanka to Singapore and (2) from Madagascar to Singapore. We perform an analysis and several sensitivity tests in order to gain confidence and information on the enhanced HCHO. As illustrated in Fig. 19a (line 1) and Fig. 20a (line 2), in each box, we average the HCHO columns along the ship track to obtain a spatial cross section, and we bin the data as a function of the distance from the line (distances are expressed in degrees per 0.5^∘ bin). The background level is not constant, for example due to continental outflow in the Bay of Bengal, and needs to be removed. To do so, we fit a straight line through the column values at the edges of the box and subtract this line from the signal. This allows us to isolate a differential column and to evaluate its absolute and relative magnitude compared to the background (respectively, shown in Figs. 19b, c and 20b, c). For comparison, we perform the same analysis using TROPOMI NO₂ tropospheric columns from the operational product (NO₂ ATBD, https://sentinel.esa.int/documents/247904/2476257/Sentinel-5P-TROPOMI-ATBD-NO2-data-products, last access: 18 August 2021, Van Geffen et al., 2020). Although only about half as wide, the localization of the NO₂ peak is found to be well aligned with the HCHO signal. Along the line from Sri Lanka to Singapore, we find a similar column enhancement and plume width as in Marbach et al. (2009).

In order to exclude a possible indirect AMF effect caused by the TM5 a priori profiles, the same analysis is done based on background-corrected slant columns (bc-SCDs). We also restrict the analysis to clear-sky observations by using a strict cloud filtering of CRF < 20 %. Furthermore, we use the wind vector information provided in the TROPOMI L2 product from version 2 onwards (from August 2020) to select only clear-sky observations with low-wind conditions (qa > 0.5, CRF < 20 %, W<5 m s⁻¹). Finally, we add to the analysis a climatology of HCHO observations based on OMI measurements (2005–2009).

Figure 19 Box average for the first selected line between Sri Lanka and Singapore between December 2020 and February 2021. The x axis represents the distance (south–north) in degrees from the shipping lane. Panel (a) shows the HCHO (in black) and NO₂ (in blue) tropospheric columns, binned per distance from the line centre. The fitted lines are used to remove the background contribution. The two bottom panels present the absolute (b) and relative (c) column deviations from the background line. The analysis is performed on the slant and the vertical columns (circles/lines), using a stricter cloud filtering (CRF < 20 %, black dotted line) and an additional filter on the wind velocity (W<5 m s⁻¹, green dotted line) and finally on OMI observations averaged between 2005 and 2009 (red). (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

Figure 20 Same as Fig. 19 for the second selected line between Madagascar and Singapore.

Using this approach, we analysed HCHO datasets for each season between MAM 2018 and DJF 2021. The absolute and relative magnitude of the largest detected signal is plotted as a function of the season in Figs. 21 and 22. Along the two lines, the signal is detected in the slant columns of HCHO and NO₂ as well. This excludes the possibility of an artefact coming from the TM5 a priori profiles. The signal remains detectable in clear-sky observations and is even increased along the second line. We observe a similar effect of the wind speed filtering (last two seasons). Selecting only low-wind conditions clearly enhances the signal along line 2 and during SON along line 1. The magnitude of the detected HCHO signal is larger along line 1 (from 0.2 to 0.7 × 10¹⁵ molec. cm⁻², 15 %) compared to line 2 (from 0.1 to 0.3 × 10¹⁵ molec. cm⁻², 8 %). We find that the absolute magnitude of the HCHO signal is larger than the NO₂ signal by a factor of 3 to 10, but the relative increase in the NO₂ columns is significantly larger: 60 % along line 1 and 15 % along line 2. Both lines show a clear seasonality, particularly in the HCHO columns, with a maximum during the DJF seasons seen in the OMI climatology and in the TROPOMI 3-month averages. The HCHO signal presents a clear drop in JJA along line 1. This is related to the wind direction and strength, which bring the line signal closer to the HCHO continental outflow, making its detection more difficult. The OMI data need to be averaged over several years in order to detect a significant signal. While the first line is well detected in the 5-year OMI climatology, the second line presents a smaller magnitude and a larger variability and cannot be detected in the most recent years of OMI measurements.

Figure 21 Seasonal variation in the absolute (a) and relative (b) column deviations of the TROPOMI HCHO (black), OMI 2005–2009 climatology HCHO (red) and TROPOMI NO₂ (blue) tropospheric columns along the Sri Lanka–Singapore line. For each season, the maximum deviation compared to the background is provided. The results of the analysis are given for the slant and the vertical columns (circles/lines), using a stricter cloud filtering (CRF < 20 %, black dotted line) and an additional filter on the wind velocity (W<5 m s⁻¹, green dotted line). (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

Figure 22 Same as Fig. 21 along the Madagascar–Singapore line.

Using TROPOMI HCHO observations averaged over 3 months, it is therefore possible to detect a signal as small as 0.1 × 10¹⁵ molec. cm⁻² (with a median deviation of 0.03 × 10¹⁵ molec. cm⁻²), after removal of the background contribution. Note that along the first line a similar analysis can also be performed on a monthly basis. While we show several pieces of evidence that the signal is related to shipping emissions, its source is not studied here. As discussed in Marbach et al. (2009) it could be due to secondary HCHO production via the atmospheric oxidation of NMVOCs emitted from ship engines but also to enhanced CH₄ oxidation by elevated levels of OH radicals within the ship plumes. Model analysis suggests that the second hypothesis is the main factor responsible for the elevated HCHO levels (Song et al., 2010). Other HCHO lines can be detected as well in the tropics, although weaker in magnitude or closer to the continental outflow (in the south-west of Africa or in the west of India). More advanced techniques to separate the signal from the background and to account for wind dispersion effects could help in detecting more shipping lanes but also weak continental emissions (Beirle et al., 2004).

7 Conclusions

Owing to its high spatial resolution resulting in many measurement points, coupled with an improved signal-to-noise ratio at the single-pixel level, TROPOMI allows us to monitor HCHO tropospheric columns from space with an unprecedented definition. The global and regional maps show a clear reduction in the noise compared to previous sensors, allowing for the detection of weaker HCHO signals and the monitoring of HCHO variations on a much shorter timescale.

Figure 23 Estimated precision of OMI (in red) and TROPOMI (in black) HCHO columns at different spatial and temporal scales (20 km, 100 km, regions, day, month). The median deviation of the satellite HCHO columns are provided for continental emissions (plain circles) and in the remote reference sector (white circles). Validation estimates are plotted at 20 and 100 km (MAD of differences between satellite and MAX-DOAS columns, triangles). The theoretical noise (dotted lines) corresponds to single-measurement precision divided by the square root of observations. The dashed blue line is the TROPOMI product requirement, based on a single-measurement precision of 12 × 10¹⁵ molec. cm⁻². The horizontal blue line at 1.3 × 10¹⁵ molec. cm⁻² represents the Copernicus user requirement. (Pmolec. cm⁻² = 1 × 10¹⁵ molec. cm⁻².)

We have evaluated the TROPOMI HCHO operational product against the QA4ECV OMI HCHO dataset and against a network of 18 ground-based MAX-DOAS instruments. The gain in precision at different spatial and temporal scales was estimated by (1) comparing the median deviation of the averaged columns and (2) validating the data using MAX-DOAS column network measurements. Both methods include additional noise components from temporal variation, spatial variation and ground-based column precision. Results are summarized in Fig. 23 where precision estimates are provided for observations over regions with enhanced continental emissions and for background conditions, as a function of the time resolution (daily or monthly averages) and of the spatial resolution (from 20 km to regional scale). At 20 and 100 km resolution, both the median deviation approach and the validation results lead to very consistent estimates of the precision. The theoretical noise is also represented in the figure; it decreases as the squared root of the number of observations included in the averages. In remote conditions, the median deviation of the averaged columns follows closely the theoretical noise until reaching a threshold. If we consider a large region in the reference sector, all estimates converge towards a limit of about 0.2 × 10¹⁵ molec. cm⁻² (day) to 0.1 × 10¹⁵ molec. cm⁻² (month) both for OMI and TROPOMI. Over continental emission sources, the reduction in the noise is counterbalanced by the HCHO natural variability and by other sources of pseudo-noise which depend on the spatial and temporal scales of the observations. The largest improvement brought by TROPOMI is found for daily observations at 20 km resolution, for which a gain in precision by a factor of 3 is obtained compared to OMI. The product and Copernicus user requirements for precision are also represented in the figure. Both are reached with TROPOMI using daily averaged data at the resolution of 20 km if we consider the dispersion in remote regions. However, over continental emissions, local variability effects added up to the estimated precision that reaches a threshold of about 2 × 10¹⁵ molec. cm⁻².

For the HCHO absolute values, we show that OMI and TROPOMI observations agree very well for moderate to large HCHO levels (columns larger than 5 × 10¹⁵ molec. cm⁻²) for which the bias between both datasets is smaller than 10 %. For lower columns, however, OMI observations present a remaining bias of about +20 % compared to TROPOMI. This good agreement is obtained by considering vertical columns calculated with air mass factors not corrected for cloud effects (clear VCD). This allows us to avoid biases related to differences in the cloud products. For all applications that require combining the OMI and TROPOMI observations for low to moderate cloud fractions, we therefore advise using clear VCDs. Validation results confirm the good agreement between the OMI and TROPOMI datasets and a similar underestimation of both products in the highest range of the HCHO levels (−25 % on average for columns larger than 8 × 10¹⁵ molec. cm⁻²). For medium columns, OMI presents a slight overestimation compared to MAX-DOAS data, which is not observed for TROPOMI. Sensitivity tests show that validation results obtained with the TROPOMI HCHO columns are weakly dependent on the cloud correction. They also depend weakly on the radius considered around the station, with a few exceptions such as Mexico city or coastal stations. By contrast, the vertical smoothing (tested at three stations) has a strong effect on the comparison with MAX-DOAS. After taking into account the different a priori profiles and averaging kernels, the bias for large HCHO columns is strongly reduced by about 20 %.

Comparing OMI and TROPOMI monthly averaged HCHO columns, we do not observe significant differences related to the spatial resolution, except in regions surrounded by natural boundaries where the benefit of the finer spatial resolution of TROPOMI is clearly apparent. The weak sensitivity to the spatial resolution of HCHO measurements can be understood when considering that HCHO is a secondary product from the degradation of NMVOCs with various lifetimes, which results in a general spread of the HCHO spatial distributions. The large number of TROPOMI observations allows us to perform validation at a resolution as small as 10 km on a daily basis with a sufficient precision, which is not possible with OMI. It is clear that TROPOMI brings a significant improvement in the temporal resolution of the observations. At most of the validation sites, TROPOMI allows for daily validation results as robust as those obtained with OMI on a monthly basis.

The number of ground-based stations providing MAX-DOAS HCHO observations is constantly growing, providing a large range of observation conditions and for some of them allowing comparisons of the performances of several satellite datasets over several years. Note however that the lower range of HCHO levels is underrepresented as are some of the largest emission regions such as South America or Africa. Following the validation study of Vigouroux et al. (2020) based on an FTIR network of instruments, this study illustrates again the added value of using a large network of instruments to draw more robust conclusions. FTIR and MAX-DOAS networks are complementary to each other and could be combined to cover as many conditions as possible. Similarly to what was achieved for the FTIR network, the MAX-DOAS HCHO datasets would benefit from further homogenization efforts.

Finally, to illustrate the benefit of TROPOMI for the detection of small HCHO signals, we present a case study addressing the detection of shipping lanes in the Indian Ocean. Using simultaneous observations of tropospheric NO₂ and meteorological wind field data, we present strong evidence for an HCHO production in regions affected by shipping emissions. Owing to the fine spatial resolution and high spatial sampling of TROPOMI, such small signals can now be observed from space on a seasonal basis.