Effects of a priori profile shape assumptions on comparisons between satellite NO₂ columns and model simulations

A critical step in satellite retrievals of trace gas columns is the calculation of the air mass factor (AMF) used to convert observed slant columns to vertical columns. This calculation requires a priori information on the shape of the vertical profile. As a result, comparisons between satellite-retrieved and model-simulated column abundances are influenced by the a priori profile shape. We examine how differences between the shape of the simulated and a priori profiles can impact the interpretation of satellite retrievals by performing an adjoint-based four-dimensional variational (4D-Var) assimilation of synthetic NO₂ observations for constraining NO_x emissions. We use the GEOS-Chem adjoint model to perform assimilations using a variety of AMFs to examine how a posteriori emission estimates are affected if the AMF is calculated using an a priori shape factor that is inconsistent with the simulated profile. In these tests, an inconsistent a priori shape factor increased root mean square errors in a posteriori emission estimates by up to 30 % for realistic conditions over polluted regions. As the difference between the simulated profile shape and the a priori profile shape increases, so do the corresponding assimilated emission errors. This reveals the importance of using simulated profile information for AMF calculations when comparing that simulated output to satellite-retrieved columns.

1 Introduction

Satellite observations provide a wealth of information on the abundance of trace gases in the troposphere (Fishman et al., 2008). The next generation of satellite instruments, including the upcoming geostationary constellation of TEMPO (Chance et al., 2013; Zoogman et al., 2017), Sentinel-4 (Bazalgette Courrèges-Lacoste et al., 2011; Ingmann et al., 2012), and GEMS (Bak et al., 2013; Kim, 2012), will provide information on NO₂ and other air-quality-relevant pollutants on unprecedented spatial and temporal scales. Insight into processes that affect atmospheric composition, including emissions (Streets et al., 2013), lifetimes (Fioletov et al., 2015; de Foy et al., 2015; Laughner and Cohen, 2019), and deposition (Geddes and Martin, 2017; Kharol et al., 2018), can be gained by interpreting this information with atmospheric chemistry models.

There are three main stages in retrieving trace gas abundances from ultraviolet and visible solar backscatter radiance measurements: calculating a light-path slant column by fitting observed spectra to known spectral signatures of trace gases, removing the stratospheric portion of the column, and converting the slant column to a vertical column density using an air mass factor (AMF). AMFs are calculated using a radiative transfer model and are a function of viewing geometry, surface reflectance, clouds, and radiative transfer properties of the atmosphere. AMF calculations also require an a priori estimate of the trace gas vertical profile and are sensitive to the profile shape (Eskes and Boersma, 2003; Palmer et al., 2001). Uncertainties in AMF calculations are the dominant source of uncertainty in satellite NO₂ retrievals over polluted regions (Boersma et al., 2007; Martin et al., 2002) largely due to sensitivity to surface reflectance, clouds, aerosols, and a priori profile information (Lorente et al., 2017).

Boersma et al. (2016) highlighted the issue of representativeness errors in comparing model-simulated values with UV–Vis satellite-retrieved columns. Vertical representativeness errors arise from the satellite's altitude-dependent sensitivity due to atmospheric scattering and can degrade the quality of model–measurement comparisons beyond errors that arise from either modeling or measurements alone. A consistent accounting of the altitude-dependent sensitivity is necessary to limit these errors.

Two common methods are used to account for vertical representativeness. In one method, observed slant columns are converted to vertical columns using an air mass factor calculated with scattering weights to represent instrument vertical sensitivity and shape factors to represent the vertical profile (Palmer et al., 2001). Another commonly used method employs an AMF provided with the retrieval to convert slant columns to vertical columns and then applies an averaging kernel to the simulated profile to resample the simulated profile in a manner that mimics the satellite vertical sensitivity (Eskes and Boersma, 2003). In this method both the averaging kernel and the retrieval AMF are calculated using an a priori NO₂ profile that may have a different shape than the simulated profile, which may introduce errors in the observation–simulation comparison (Zhu et al., 2016).

A common application of comparisons between satellite-observed columns and model simulations is to constrain NO_x emissions (e.g., Ding et al., 2018; Ghude et al., 2013; Lamsal et al., 2011; Martin et al., 2003; Vinken et al., 2014). One such approach is the use of four-dimensional variational (4D-Var) data assimilation, which seeks to minimize a cost function that accounts for the difference between simulated and retrieved values. As the cost function is a difference between observed and simulated NO₂ columns, it is susceptible to vertical representativeness errors resulting from inconsistent a priori vertical profile information.

In this work we examine how a priori profile assumptions impact satellite–model comparisons and use the GEOS-Chem adjoint as a case study to assess how this impact can affect the interpretation of satellite observations. Section 2 provides the mathematical framework for AMF calculations and satellite–model comparisons. Section 3 describes the adjoint model and synthetic observations for the case study. Section 4 discusses the results.

2 Mathematical frameworks

2.1 AMFs and averaging kernels

The air mass factor translates the line-of-sight slant column abundances (Ω_s) retrieved from satellite-observed radiances into vertical column abundances (Ω_v). An air mass factor is the ratio of Ω_s to Ω_v and depends on the atmospheric path as determined by geometry, NO₂ vertical profile (n), surface reflectance, and radiative transfer properties of the atmosphere. Here we use M(n) to represent an air mass factor derived using the vertical number density profile n:

$$\begin{array}{}\text{(1)}& M\left(\mathbf{n}\right)={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s}}}{{\mathrm{\Omega }}_{\mathrm{v}}}}.\end{array}$$

In the method described by Palmer et al. (2001), a radiative transfer model is used calculate scattering weights w(z) (also known as box air mass factors) which characterize the sensitivity of backscattered radiance I_B to the abundance of a trace gas at altitude z:

$$\begin{array}{}\text{(2)}& \mathit{w}\left(z\right)=-{\displaystyle \frac{\mathrm{1}}{M_{\mathrm{G}}}}{\displaystyle \frac{{\mathit{\alpha }}_{\mathrm{a},z}}{{\mathit{\alpha }}_{\mathrm{eff}}}}{\displaystyle \frac{\partial \ln \left(I_{\mathrm{B}}\right)}{\partial \mathit{\tau }}},\end{array}$$

where α_a,z is the temperature-dependent absorption cross section (m² per molecule), α_eff is the effective (weighted average) absorption cross section (m² per molecule), and ∂τ is the incremental trace gas optical depth. M_G represents a geometric path correction accounting for the satellite viewing geometry:

$$\begin{array}{}\text{(3)}& M_{\mathrm{G}}=\sec {\mathit{\theta }}_{\mathrm{o}}+\sec \mathit{\theta },\end{array}$$

where θ is the solar zenith angle and θ_o is the satellite viewing angle. This information is then combined with an a priori NO₂ shape factor (i.e., normalized vertical profile),

$$\begin{array}{}\text{(4)}& \mathbf{S}\left(z\right)={\displaystyle \frac{\mathbf{n}\left(z\right)}{{\mathrm{\Omega }}_v}},\end{array}$$

typically calculated with an atmospheric chemistry model to provide an air mass factor via

$$\begin{array}{}\text{(5)}& M\left(\mathbf{n}\right)=M_G{\int }_{\mathrm{0}}^{\mathrm{tropopause}}\mathbf{w}\left(z\right)\mathbf{S}\left(z\right)\mathrm{d}z,\end{array}$$

where S(z) is calculated using vertical profile n(z). An attribute of the formulation of Palmer et al. (2001) is the independence of atmospheric radiative transfer properties w(z) and the vertical trace gas profile S(z). The AMF definition in Eq. (1) combined with Eq. (4) indicates that a slant column can be calculated from a known vertical profile via

$$\begin{array}{}\text{(6)}& {\mathrm{\Omega }}_{\mathrm{s}}={\int }_{\mathrm{0}}^{\mathrm{tropopause}}\mathbf{w}\left(z\right)\mathbf{n}\left(z\right)\mathrm{d}z.\end{array}$$

In an alternative formulation, the air mass factor is represented as part of an averaging kernel. As formulated by Rodgers and Connor (2003), the averaging kernel (A) provides the information needed to relate the retrieved quantity $\widehat{n}$ to the true atmospheric profile n:

$$\begin{array}{}\text{(7)}& \widehat{\mathbf{n}}-{\mathbf{n}}_{\mathrm{a}}=\mathbf{A}(\mathbf{n}-{\mathbf{n}}_{\mathrm{a}}),\end{array}$$

where n_a is an assumed a priori profile of number density. The elements of the column averaging kernel are related to the scattering weights by

$$\begin{array}{}\text{(8)}& \mathbf{A}\left(z\right)={\displaystyle \frac{\mathbf{w}\left(z\right)}{M\left({\mathbf{n}}_{\mathrm{a}}\right)}},\end{array}$$

where M(n_a) is an air mass factor calculated using a priori vertical profile information. It is important to note that, unlike scattering weights, averaging kernels depend on the a priori assumed vertical profile shape.

A lexicon is given in Table 1 as notation used to describe these treatments has varied across the literature. We choose M for air mass factor as a single letter is clearer in equations, w for scattering weights to maintain the original formulation of Palmer et al. (2001), n for number density following IUPAC recommendations, and Ω for column densities as is common in radiative transfer literature.

Table 1 Lexicon comparing notation used in this paper to that used in previous studies.

Figure 1 shows examples of typical shape factor, scattering weight, and averaging kernel profiles for a range of atmospheric conditions. NO₂ shape factors have significant variability; shape factors peak near the surface in urban regions due to local pollution sources but peak in the upper troposphere in more remote regions due to lightning. The shape of a scattering weight profile depends strongly on surface reflectance and cloud conditions. Sensitivity in the lower troposphere increases over reflective surfaces. Clouds increase sensitivity above due to their reflectance but shield the satellite from observing the atmosphere below. Averaging kernels have similarities with scattering weights but depend on both the shape of the prior and the satellite sensitivity. As AMF calculations are a convolution of the shape factor and the scattering weight profiles, these shapes affect NO₂ retrievals. For these examples, the AMF for a clear-sky observation with surface reflectance of 0.01 can range from 0.7 in an urban region to 1.7 in a remote region. This large difference demonstrates the importance of the assumed profile shape to the retrieval process.

Figure 1 (a) Shape factor profiles from a GEOS-Chem simulation for July 2010. Shown are a global average, as well as typical urban (Beijing), rural (Midwest USA), and remote (tropical Pacific) profiles. (b) Typical Ozone Monitoring Instrument (OMI) scattering weight profiles for varying surface reflectance and cloud height. (c) Averaging kernels calculated using the same shape factors and scattering weights (clear-sky surface reflectance is 0.01; cloudy uses a cloud height of 1 km).

2.2 Comparing satellite observations to simulated values

The following section expresses mathematically how satellite–model comparisons are made using various a priori profiles.

2.2.1 Using scattering weights

Following Palmer et al. (2001), a retrieved vertical column (${\widehat{\mathrm{\Omega }}}_{\mathrm{v},\mathrm{o}}$) is estimated using an observed slant column Ω_s,o and a simulation-based air mass factor M(n_m), which can be calculated with Eq. (5) using the model-simulated NO₂ profile (n_m):

$$\begin{array}{}\text{(9)}& {\widehat{\mathrm{\Omega }}}_{\mathrm{v},\mathrm{o}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{M\left({\mathbf{n}}_{\mathrm{m}}\right)}}.\end{array}$$

The difference Δ_m between the estimated retrieved column and the model-simulated vertical column (Ω_v,m) is

$$\begin{array}{}\text{(10)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{m}}={\mathrm{\Omega }}_{\mathrm{v},\mathrm{m}}-{\widehat{\mathrm{\Omega }}}_{\mathrm{v},\mathrm{o}},\text{(11)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{m}}=\left({\sum }_{\mathrm{0}}^{\mathrm{tropopause}}{\mathit{n}}_{\mathrm{m}}\right)-{\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{M\left({\mathbf{n}}_{\mathrm{m}}\right)}}.\end{array}$$

Equation (11) describes how this comparison is used in practice. However, we can rearrange this expression in terms of model (Ω_s,m) and observed (Ω_s,o) slant columns using the definition of air mass factor:

$$\begin{array}{}\text{(12)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{m}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{m}}}{M\left({\mathbf{n}}_{\mathrm{m}}\right)}}-{\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{M\left({\mathbf{n}}_{\mathrm{m}}\right)}},\text{(13)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{m}}={\displaystyle \frac{\mathrm{1}}{M\left({\mathbf{n}}_{\mathrm{m}}\right)}}\left({\mathrm{\Omega }}_{\mathrm{s},\mathrm{m}}-{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}\right).\end{array}$$

2.2.2 Using averaging kernels

Comparison of simulated and retrieved columns using the averaging kernel is described by Eskes and Boersma (2003) and in the retrieval documentation in Boersma et al. (2011). The averaging kernel is applied to the simulated profile in order to sample the simulated column in a manner that reflects the retrieval sensitivity:

$$\begin{array}{}\text{(14)}& {\widehat{\mathrm{\Omega }}}_{\mathrm{v},\mathrm{m}}={\sum }_{\mathrm{0}}^{\mathrm{tropopause}}{\mathbf{An}}_{\mathrm{m}}.\end{array}$$

The resampled simulated column is then compared to the retrieved vertical column (Ω_v,o) using the a priori-based air mass factor M(n_a) supplied with the retrieval dataset:

$$\begin{array}{}\text{(15)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{a}}={\widehat{\mathrm{\Omega }}}_{\mathrm{v},\mathrm{m}}-{\mathrm{\Omega }}_{\mathrm{v},\mathrm{o}},\text{(16)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{a}}=\left({\sum }_{i=\mathrm{0}}^{\mathrm{tropopause}}{\mathbf{A}}_{\mathbf{i}}{\mathbf{n}}_{\mathrm{m},i}\right)-{\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{M\left({\mathbf{n}}_{\mathrm{a}}\right)}}.\end{array}$$

Equation (16) describes how this method is used in practice. To facilitate the comparison with Eq. (13), Eq. (16) can be rewritten using an alternative formulation relating averaging kernels to scattering weights:

$$\begin{array}{}\text{(17)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{a}}=\left({\sum }_{i=\mathrm{0}}^{\mathrm{tropopause}}{\displaystyle \frac{{\mathbf{w}}_i{\mathbf{n}}_{\mathrm{m},i}}{M\left({\mathbf{n}}_{\mathrm{a}}\right)}}\right)-{\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{M\left({\mathbf{n}}_{\mathrm{a}}\right)}},\text{(18)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{a}}={\displaystyle \frac{\mathrm{1}}{M\left({\mathbf{n}}_{\mathrm{a}}\right)}}\left({\mathrm{\Omega }}_{\mathrm{s},\mathrm{m}}-{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}\right).\end{array}$$

By comparing Eqs. (13) to (18), it is evident that the underlying difference between the two approaches is the choice of a priori profile information used to calculate the AMF, as the averaging kernel method is not independent of a priori profile assumptions. This bias could be addressed by replacing the a priori-based AMF in Eq. (18) with a simulation-based AMF using the following relationship (Boersma et al., 2016; Lamsal et al., 2010):

$$\begin{array}{}\text{(19)}& M\left({\mathbf{n}}_{\mathrm{m}}\right)=M\left({\mathbf{n}}_{\mathrm{a}}\right){\displaystyle \frac{\sum {\mathbf{An}}_{\mathrm{m}}}{\sum {\mathbf{n}}_{\mathrm{m}}}}.\end{array}$$

It should be noted that both the averaging kernel and scattering weight methods are equivalent for comparisons that examine ratios of retrieved and modeled columns:

$$\begin{array}{}\text{(20)}& {\displaystyle }{r}_{\mathrm{m}}={\displaystyle \frac{\widehat{{\mathrm{\Omega }}_{\mathrm{v},\mathrm{o}}}}{{\mathrm{\Omega }}_{\mathrm{v},\mathrm{m}}}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}/M\left({\mathbf{n}}_{\mathbf{m}}\right)}{\sum {\mathbf{n}}_{\mathrm{m}}}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{\sum {\mathbf{n}}_{\mathrm{m}}}}{\displaystyle \frac{\sum {\mathbf{n}}_{\mathrm{m}}}{\sum {\mathbf{wn}}_{\mathrm{m}}}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{\sum {\mathbf{wn}}_{\mathrm{m}}}},\text{(21)}& {\displaystyle }{r}_{\mathrm{a}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{v},\mathrm{o}}}{\widehat{{\mathrm{\Omega }}_{\mathrm{v},\mathrm{m}}}}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}/M\left({\mathbf{n}}_{\mathrm{a}}\right)}{\sum {\mathbf{An}}_{\mathrm{m}}}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}/M\left({\mathbf{n}}_{\mathrm{a}}\right))}{\sum {\mathbf{wn}}_{\mathrm{m}}/M\left({\mathbf{n}}_{\mathrm{a}}\right)}}={\displaystyle \frac{{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}}{\sum {\mathbf{wn}}_{\mathrm{m}}}}.\end{array}$$

For ratios, both methods are dependent on geophysical assumptions used to calculate scattering weights but are independent of a priori profile information. Lastly, some studies (e.g., Qu et al., 2017) may directly assimilate slant column densities rather than vertical column densities using

$$\begin{array}{}\text{(22)}& {\displaystyle }{\mathrm{\Delta }}_{\mathrm{s},\mathrm{a}}={\widehat{\mathrm{\Omega }}}_{\mathrm{s},\mathrm{m}}-{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}\text{(23)}& {\displaystyle }=\left({\sum }_{i=\mathrm{0}}^{\mathrm{tropopause}}{\mathbf{w}}_i{\mathbf{n}}_{\mathrm{m},i}\right)-{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}.\end{array}$$

This approach is also still dependent upon the scattering weights but not upon external a priori profile information. Overall, the choice of approach may be influenced by whether or not scattering weights are available from either the NO₂ retrieval product or radiative transfer calculations applied to the model. In contrast, use of Eqs. (11) or (16) is applicable when these are not explicitly available or provided.

3 Tools and methodology

3.1 GEOS-Chem and its adjoint

The GEOS-Chem chemical transport model (http://www.geos-chem.org, last access: 28 January 2020) is used to create synthetic NO₂ observations and for their analysis. The GEOS-Chem version used here is version 35j of the GEOS-Chem adjoint model. GEOS-Chem includes a detailed oxidant–aerosol chemical mechanism (Bey et al., 2001; Park et al., 2004) and uses assimilated meteorological fields from the Goddard Earth Observation System (GEOS-5), with 47 vertical levels up to 0.01 hPa and a horizontal resolution of $\mathrm{4}{}^{\circ }\times \mathrm{5}{}^{\circ }$. Global anthropogenic NO_x emissions are provided by the Emission Database for Global Atmospheric Research (EDGAR) inventory (Olivier et al., 2005), with regional overwrites over North America (EPA/NEI99), Europe (EMEP), Canada (CAC), Mexico (BRAVO, Kuhns et al., 2005), and East Asia (Streets et al., 2006). Other NO_x sources include biomass burning (GFED2, Van der Werf et al., 2010), lightning (Murray et al., 2012), and soils (Wang et al., 1998). This model has been used previously to constrain NO_x emissions (Cooper et al., 2017; Henze et al., 2009; Qu et al., 2017, 2019; Xu et al., 2013; Zhang et al., 2016).

The GEOS-Chem adjoint (Henze et al., 2007, 2009) is used here to perform a 4D-Var data assimilation. The adjoint seeks to iteratively minimize a cost function generally defined by the difference between satellite-retrieved and simulated columns (Δ, from either Eq., 11, if using a simulation-based air mass factor or Eq., 16, if using the retrieval a priori-based air mass factor):

$$\begin{array}{}\text{(24)}& J={\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}{\mathrm{\Delta }}^{\mathbf{T}}{\mathbf{S}}_{\mathrm{o}}^{-\mathrm{1}}\mathrm{\Delta }+{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}{\mathit{\gamma }}_{\mathrm{R}}{\left(\mathbf{E}-{\mathbf{E}}_{\mathrm{a}}\right)}^{\mathbf{T}}{\mathbf{S}}_{\mathrm{E}}^{-\mathrm{1}}(\mathbf{E}-{\mathbf{E}}_{\mathrm{a}}),\end{array}$$

where E and E_a are the a posteriori and a priori emissions, S_o and S_E are the retrieval and a priori emission error covariance matrices, and γ_R is a regularization parameter that allows for weighting the cost function towards the retrieved columns or a priori emissions. Tests performed here required 20–30 iterations to minimize the cost function.

3.2 Experiment outline

In this study we perform 4D-Var data assimilation experiments to infer surface NO_x emissions using synthetic NO₂ observations. We use synthetic observations built from known emission inventories to provide a “truth” that can be used to evaluate the inversion results. To demonstrate how a priori profile information can propagate in an assimilation, we use either the model profile (Δ_m, Eq. 11) or an a priori profile (Δ_a, Eq. 16) in the cost function. A 1-week spin-up window at the start of each adjoint iteration is used to allow NO_x to reach a steady state. Observation error covariances S_o are described as a relative error of 30 % of the slant column density plus an absolute error of 10¹⁵ molec cm⁻², which is representative of typical satellite-retrieved NO₂ column uncertainties (Boersma et al., 2007; Martin et al., 2002). We omit the a priori emissions constraint in the cost function (i.e., set γ_R=0) to isolate the impact of the observations.

3.2.1 Synthetic observations

Synthetic observations (Obs₅) are created using a GEOS-Chem simulation where random Gaussian noise with a standard deviation of 5 % is added to the anthropogenic NO_x emissions. Additional tests using observations where noise with a standard deviation of 30 % is added (Obs₃₀) are also used. No additional noise is added to the individual observations to isolate the impact of AMF errors against additional sources of uncertainty. Figure 2 shows the standard (a priori) anthropogenic NO_x emissions and the changes used to create the true emissions for the synthetic observations.

Figure 2 (a) Anthropogenic NO_x emissions for July 2010 used in GEOS-Chem. (b) Ratio of true emissions used to create Obs₅ synthetic observations to a priori NO_x emissions.

For these tests, we use one observation per hour per $\mathrm{4}{}^{\circ }\times \mathrm{5}{}^{\circ }$ grid box for a period of 2 weeks in July 2010. Observations consist of synthetic slant columns (Ω_s,o) created by applying scattering weights to the synthetic vertical profiles using Eq. (6). Scattering weights are calculated using the LIDORT radiative transfer model (Spurr, 2002) by providing LIDORT with the observation conditions of OMI observations during July 2010, which are used to represent typical viewing conditions of low-earth-orbit satellite observations, and aerosol profiles from the GEOS-Chem base simulation. To represent typical conditions, these representative scattering weight profiles for each grid box are used to produce the synthetic slant columns. Tests performed for all $\mathrm{4}{}^{\circ }\times \mathrm{5}{}^{\circ }$ grid boxes used here indicate that the mean relative difference between an air mass factor calculated using an average scattering weight profile and the average of air mass factors using observation-specific scattering weight profiles is less than 4 %.

3.2.2 Shape factors

To test the impact of a priori profile information, seven different tests are performed using seven different NO₂ profile shapes for AMF calculations:

• Case SF_M – the GEOS-Chem model-simulated profile (n_m), updated at each iteration of the adjoint run.

• Case SF_prior – the a priori GEOS-Chem simulated profile, without updating.

• Case SF_n30 – an a priori profile created by a GEOS-Chem simulation where global anthropogenic NO_x emissions were perturbed with random Gaussian noise with a standard deviation of 30 %; in cases where this results in negative emissions, a value of zero is used.

• Case SF_diffem – an a priori profile created by a GEOS-Chem simulation where regional emission overwrites are turned off.

• Case SF_finer – an a priori profile created by a GEOS-Chem simulation run at finer ($\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$) resolution.

• Case SF_trop – an a priori profile that assumes the NO₂ profile shape is uniform from the surface to the tropopause (∼200 hPa).

• Case SF_BL: an a priori profile that assumes the NO₂ profile shape is uniform from the surface to the boundary layer (∼800 hPa).

An advantage of using scattering weights and the simulated shape factor in a 4D-Var framework is that it allows for the shape factor, and thus the AMF, to be updated at each iteration. When a priori profiles from an external source are used it is not possible for them to update during the inversion. The SF_M and SF_prior cases test the impact that iterative updates to the AMF have on a posteriori estimates. The additional cases test for the impact of using an averaging kernel based on a priori profile assumptions that are inconsistent with the model. In practice, averaging kernels and a priori profiles included in retrieval datasets are generally derived from chemical transport models that have different physical processes, emissions, or spatial resolutions. The SF_n30 and SF_diffem tests are representative of inversions that use a priori profile information from a different chemical transport model with similar resolution but different emissions. The SF_finer test represents an inversion that uses a priori profiles from a chemical transport model with a different horizontal resolution. The SF_BLand SF_trop tests do not represent any modern retrieval algorithms but are used as extreme examples of an a priori that assumes no spatial variability. The SF_BL profile is representative of polluted regions as indicated by the typical urban profile in Fig. 1, while the SF_trop profile is representative of a typical rural profile. Table 2 provides global mean AMFs for these test cases, which range from 1.3 to 2.1, and the resulting global mean observed vertical columns, which range from 0.9 to 1.5×10¹⁵ molec cm⁻². Global mean observed vertical columns are 33 % higher for SF_2×25 than for SF_M and up to 66 % higher for SF_BL. Global mean observed vertical columns for SF_n30 and SF_diffem are similar to SF_M, although individual observations may differ by up to 18 % for SF_n30 and 28 % for SF_diffem.

Table 2 Global mean air mass factors and synthetic observation vertical column density for shape factors tested here.

4 Results

Figure 3 shows root mean square errors (RMSEs) for the a posteriori emissions estimated by the 4D-Var assimilations of Obs₅ synthetic observations. All tests successfully reduce the a priori emission error by an order of magnitude or more. The SF_M has the lowest RMSE, indicating that it can best estimate the true emissions. The next lowest RMSE is for the SF_prior test, which uses the same initial model shape factor but does not update during the adjoint iterations, followed by the SF_finer, SF_diffem, SF_n30, SF_trop, and SF_BL tests.

Figure 3 Global root mean square error (RMSE) values for 4D-Var estimates of NO_x emissions for tests using various shape factors in AMF calculations.

Figure 4 Difference between root mean square error (RMSE) of adjoint tests for Obs₅ synthetic observations. Root mean square differences between the a posteriori emissions estimates (molec cm⁻² s⁻¹) are inset.

Figure 4 shows maps of the difference in RMSE between the SF_M test and the other tests for Obs₅ observations. The SF_M test has a lower RMSE than the other tests in 65 %–72 % grid boxes where the difference is nonzero. Again, the SF_prior test is closest to the SF_M test with a root mean square difference of 2.9×10⁷ molec cm⁻² s⁻¹, followed by SF_finer (3.6×10⁷ molec cm⁻² s⁻¹), SF_n30 (3.8×10⁷ molec cm⁻² s⁻¹), SF_diffem (4.0×10⁷ molec cm⁻² s⁻¹), SF_trop, (7.8×10⁷ molec cm⁻² s⁻¹), and SF_BL (9.0×10⁷ molec cm⁻² s⁻¹).

Table 3 summarizes additional error statistics focused on grid boxes with significant emission sources. Errors in a posteriori emission estimates are correlated with the true emissions in the SF_trop and SF_n30 tests, and they are weakly correlated in the SF_BL, SF_prior, and SF_diffem tests, indicating that these tests are not constraining the emissions well. Differences between tests are more significant over polluted regions where AMF errors are more influential; for example, in the regions with the highest NO_x emissions, RMSE values indicate SF_M outperforms SF_n30 by 30 % and SF_trop by > 80 %. Another sign of adjoint inversion quality is a low variance in errors. While the posterior error is reduced relative to the a priori error in all tests, error standard deviations are 30 % higher for SF_n30 and 90 % higher for SF_trop compared to SF_M. The global maximum error for the SF_trop test is 30 % higher than for the SF_M test. All metrics indicate that the SF_M test best represents the true emissions.

Table 3 Summary of error statistics for adjoint tests. Values marked with asterisks (^*) indicate that correlation is not statistically significant (p > 0.05). For comparisons, mean true emissions for grid boxes with emissions > 10¹⁰ molec cm⁻² s⁻¹ are 4.9×10¹⁰, and mean true emissions for boxes with emissions > 10¹¹ molec cm⁻² s⁻¹ are 1.6×10¹¹ molec cm⁻² s⁻¹.

Tests using Obs₃₀ observations and the SF_Mand SF_trop shape factors were also performed. Despite the difference between a priori observed vertical columns using these shape factors as indicated by Table 2, these assimilations produced similar a posteriori results, with RMSE of 2.9×10⁸ molec cm⁻² s⁻¹ for SF_M and 2.8×10⁸ molec cm⁻² s⁻¹ for SF_trop.

5 Discussion and conclusions

Accounting for the vertical profile dependence of satellite observations is essential to accurately interpret those observations. This work examines how the choice of shape factor affects differences between simulated and satellite-retrieved quantities in a data assimilation framework. Examination of the mathematical frameworks behind two common methods for comparing simulated and retrieved columns highlights how the method introduced by Palmer et al. (2001) facilitates separation of observation sensitivity (scattering weights) from the profile shape (shape factor) enabling the model–retrieval comparison to be independent of a priori profile assumptions.

Figure 5 Scatterplot of adjoint test results. X axis represents the deviation of the shape factor from the model-simulated shape factor (root mean square difference). Y axis represents the a posteriori emissions error from the adjoint inversion.

In these case studies, vertical representativeness errors were best reduced by using a shape factor that was consistent with the model simulation. This was especially true in polluted regions where the AMF errors dominate observation uncertainties, as deviations between the tests were largest in these regions. The further the shape factor deviated from the model state the larger the inversion errors became, as indicated by Fig. 5. The SF_finer test indicates that although using a finer-resolution model to generate a priori profiles is desirable for a more accurate retrieval, consistency between the simulation profile and the a priori shape factor is of greater importance explicitly for the purpose of simulation–observation comparisons to constrain emissions at the simulation resolution. Comparing the SF_M and SF_prior tests shows that allowing for the shape factor to update during the iterative adjoint process further reduces the RMSE by 10 %. However, even without allowing for shape factor updates, using a shape factor that is consistent with the initial model state produces a more accurate inversion result than using other assumed profile shapes.

The case study presented here demonstrates that the shape factor source can have a strong influence on adjoint inversion results. However, the magnitude of this influence can vary. Inversion tests performed using synthetic observations based on random 30 % perturbations to emissions were insensitive to the AMF, despite large differences in a priori vertical column densities. In these tests, the cost function was more sensitive to the larger difference between the observed and simulated slant columns (i.e., ${\mathrm{\Omega }}_{\mathrm{s},\mathrm{m}}-{\mathrm{\Omega }}_{\mathrm{s},\mathrm{o}}$ in Eqs. 13 and 19) than to the AMF. This indicates that while the cost function is mathematically dependent on the AMF, the inversion is less sensitive to vertical representativeness errors in cases where emissions are poorly constrained, as is the case in recent adjoint inversion studies (e.g., Qu et al., 2017). However, the choice of AMF will become increasingly important to adjoint inversions as emission inventories improve. Furthermore, omitting the a priori emissions constraint in the cost function and omitting noise in the observations in these tests to isolate the impact of the AMF effectively assume poorly constrained a priori emissions and ideal observations. In practice, cost function sensitivity to AMF choice may be buffered when a priori emissions uncertainties and observational noise are considered.

As it is beneficial for a consistent shape factor to be used when comparing satellite-retrieved values to model-simulated results, it will be useful for data products to provide the information required for this method to the user community. This is most straightforward when scattering weights (rather than averaging kernels) are provided alongside retrieved column data, as scattering weights and shape factors are independently calculated; however, simulation-based air mass factors can be calculated using the averaging kernel and a priori-based air mass factor via Eq. (19).

In summary, when comparing a model simulation to a satellite-retrieved NO₂ column in a data assimilation environment utilizing column differences, calculating the AMF using the simulated shape factor allows for better accuracy in inversion results. This demonstration can provide general guidance for other methods of interpreting satellite observations with models, as using the simulated shape factor assures consistency in the vertical representativeness between model and retrieval.