Site representativity of AERONET and GAW remotely sensed aerosol optical thickness and absorbing aerosol optical thickness observations

Remote sensing observations from the AERONET (AErosol RObotic NETwork) and GAW (Global Atmosphere Watch) networks are intermittent in time and have a limited field of view. A global high-resolution simulation (Goddard Earth Observing System Model (GEOS-5) Nature Run) is used to conduct an OSSE (observing system simulation experiment) for AERONET and GAW observations of AOT (aerosol optical thickness) and AAOT (absorbing aerosol optical thickness) and estimate the spatiotemporal representativity of individual sites for larger areas (from 0.5 to 4^∘ in size).

GEOS-5 NR and the OSSE are evaluated and have shown to have sufficient skill, although daily AAOT variability is significantly underestimated, while the frequency of AAOT observations is overestimated (both resulting in an underestimation of temporal representativity errors in AAOT).

Yearly representation errors are provided for a host of scenarios: varying grid-box size, temporal collocation protocols and site altitudes are explored. Monthly representation errors show correlations from month to month, with a pronounced annual cycle that suggests temporal averaging may not be very successful in reducing multi-year representation errors. The collocation protocol for AEROCOM (AEROsol Comparisons between Observations and Models) model evaluation (using daily data) is shown to be suboptimal and the use of hourly data is advocated instead. A previous subjective ranking of site spatial representativity (Kinne et al., 2013) is analysed and a new objective ranking proposed. Several sites are shown to have yearly representation errors in excess of 40 %.

Lastly, a recent suggestion (Wang et al., 2018) that AERONET observations of AAOT suffer a positive representation bias of 30 % globally is analysed and evidence is provided that this bias is likely an overestimate (the current paper finds 4 %) due to methodological choices.

1 Introduction

As the temporal sampling of observations is often intermittent and their field of view limited, the ability of observations to represent the weather or climate system is negatively affected (Nappo et al., 1982). This adverse effect can be described through a representation error, which allows comparison to, e.g. observational errors or model errors.

Representation errors have been receiving more attention recently, in a variety of fields: solar surface radiation (Hakuba et al., 2014b, a; Schwarz et al., 2017, 2018), sea surface temperatures (Bulgin et al., 2016), trace gases (Sofieva et al., 2014; Coldewey-Egbers et al., 2015; Lin et al., 2015; Boersma et al., 2016), water vapour (Diedrich et al., 2016), cloud susceptibility (Ma et al., 2018) and even climate data (Cavanaugh and Shen, 2015; Director and Bornn, 2015). In the field of aerosol, most work has been on the representativity of satellite measurements (Kaufman et al., 2000; Smirnov, 2002; Remer et al., 2006; Levy et al., 2009; Colarco et al., 2010, 2014; Sayer et al., 2010; Geogdzhayev et al., 2014), either using satellite data or model data. A new development is the use of local spatially relatively dense measurement networks (Shi et al., 2018; Virtanen et al., 2018).

As aerosols are known to vary over short timescales and spatial scales (Anderson et al., 2003; Kovacs, 2006; Santese et al., 2007; Shinozuka and Redemann, 2011; Weigum et al., 2012; Schutgens et al., 2013), aerosol studies are likely to experience large representation errors. Indeed, Schutgens et al. (2016b) (S16b hereafter) showed that representation errors due to temporal sampling in both satellite and AERONET (AErosol RObotic NETwork) observations were of similar magnitude as actual model errors and often larger than observational errors. Similarly, Schutgens et al. (2016a) (S16a hereafter) showed that the narrow field of view of in situ measurements could lead to large differences from area averages (monthly root mean square differences of 10 %–80 % for 210×210 km², depending on the type of measurement and the location of the site). Recently, Schutgens et al. (2017) (hereafter S17) considered the combined impact of spatiotemporal sampling on the representativeness of remote sensing data (both satellite and ground based). They provide representation uncertainty estimates and optimal strategies when dealing with different observing systems (ground networks, polar orbiting satellites with varying revisit times or geostationary satellites).

In this paper, a global 1-year high-resolution simulation of the atmosphere (Goddard Earth Observing System Model (GEOS-5) Nature Run) is used to conduct an Observing System Simulation Experiment to estimate representation errors for remote sensing measurements of aerosol optical thickness (and its absorptive counterpart) as observed by the global AERONET and GAW (Global Atmosphere Watch) networks. In S16a and S17, regional high-resolution simulations covering a month were used to study representation errors. This prevented an analysis of such errors worldwide and on longer timescales. In addition, the limited spatiotemporal domains made evaluation of the high-resolution simulation difficult. These issues are addressed in the current study. Note that the current paper does not replace previous work (which also considers satellite, in situ and flight measurements) but extends it. In addition, the current study allows us to evaluate a recent suggestion by Wang et al. (2018) that representation errors in AERONET absorbing aerosol optical thickness (AAOT) observations are positively biased (by ∼30 %) which would help to explain the observed underestimation of AAOT in global models (Bond et al., 2013).

Representation errors are not only determined by observational sampling but also by how these observations are put to use. If observations are used to evaluate models, different protocols (or strategies) exist to temporally collocate model data and observations. For instance, within AEROCOM (AEROsol Comparisons between Observations and Models), an oft-used strategy is daily collocation: daily averages of observations are collocated with daily model data. The different sampling of model and observations throughout the day is ignored (e.g. most remote sensing observations only observe a small part of the diurnal cycle). In contrast, hourly collocation uses hourly model data that are collocated with hourly averages of observations. S17 showed that in the case of remote sensing observations daily collocation allows significantly larger representation errors than hourly collocation. A third protocol would be yearly collocation, which is seldom used these days in model evaluation as it yields large representation errors (S16b). However, if remote sensing observations are used to construct a yearly climatology, effectively, a yearly collocation protocol is used.

In data assimilation, the representation error is often (but not always) thought to include effects from incorrectly modelled subgrid processes. In this paper, the representation error is purely thought of as resulting from the different sampling by observations and models.

Section 2 describes the high-resolution simulation data and AERONET observations used in this study. The observing system simulation experiment (OSSE) for estimating representation errors is briefly explained in Sect. 3 but more details can be found in S17. An evaluation of the high-resolution simulation with a particular focus on its use in an OSSE is given in Sect. 4. While the simulation shows deviations from AERONET observations, the agreement is deemed sufficient to study representation errors. Representation errors in AERONET aerosol optical thickness (AOT) and AAOT are studied in Sect. 5. A ranking of AERONET sites in terms of their representativity is given in Sect. 6. As may be expected, the paper finishes with a summary of the conclusions (Sect. 7).

2 Data

2.1 GEOS-5 Nature Run

The GEOS-5 Nature Run (G5NR hereafter) is a 2-year global, non-hydrostatic simulation from June 2005 to May 2007 at a 0.0625^∘ grid resolution (∼7 km near the Equator). Not just a simulation of standard meteorological parameters (wind, temperature, moisture, surface pressure), G5NR includes tracers for common aerosol species (dust, sea salt, sulfate, black and organic carbon) and several trace gases: O₃, CO and CO₂. The simulation is driven by prescribed sea surface temperature and sea ice, daily volcanic and biomass burning emissions, as well as monthly high-resolution inventories of anthropogenic sources (Putman et al., 2014). As it is a nature run (i.e. no meteorological nudging), the meteorology in G5NR can deviate substantially from the actual weather in 2006.

Aerosols in GEOS-5 are calculated using the Goddard Chemistry, Aerosol, Radiation, and Transport (GOCART) module (Chin et al., 2002) that uses 15 tracers to describe externally mixed species of organic carbon, black carbon, sulfate, sea salt and dust. Biomass burning emissions are obtained from QFED (Quick Fire Emissions Dataset) (Koster et al., 2013) with a diurnal cycle imposed online. Anthropogenic emissions of organic and black carbon use EDGAR-HTAP (Emissions Database for Global Atmospheric Research – Hemispheric Transport of Air Pollution) emissions (Janssens-Maenhout et al., 2012) which were rescaled to match AEROCOM phase II emissions. Non-shipping anthropogenic SO₂ emissions come from EDGAR v4.1.

Evaluation of G5NR (Gelaro et al., 2015) against NASA/GMAO MERRA (Modern-Era Retrospective analysis for Research and Applications) aerosol reanalysis (da Silva et al., 2012) suggests that global organic carbon, black carbon and sulfate AOT are underestimated by 30 %–40 %, while dust AOT is overestimated by ∼50 %. Global sea-salt AOT is similar to MERRA within 10 %. Hence, Castellanos et al. (2019) derived global rescaling factors for aerosol-speciated AOT in G5NR but these are not used in the current study (true scaling factors are unlikely to be global, it is unclear what to do about AAOT, and the focus here is on relative errors anyway). Comparison with AEROCOM models shows that G5NR sulfate lifetimes are quite low (at 2.7 d), while the other species fairly agree with the AEROCOM multi-model mean. G5NR shows reasonable cloud fractions compared to CERES-SSF (Clouds and the Earth’s Radiant Energy System-Single Scanner Footprints), although in the equatorial/subtropical region (30^∘ S–30^∘ N), G5NR has a deficit of partially cloudy scenes. In addition, there are too few clouds off western continental coasts and the southern branch of the ITCZ is too strong. CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) data suggest the G5NR cloud fraction is too low, especially over equatorial/subtropical lands in the Northern Hemisphere, and too high in the northern polar region.

For this study, the following hourly G5NR data for 2006 were obtained; see Table 1.

Table 1 G5NR data used in this study.

^* Top of atmosphere.

2.2 AERONET observations and geolocations

AERONET (Holben et al., 1998) data were obtained from https://aeronet.gsfc.nasa.gov (last access: 3 March 2018). For 2006, AOT from direct sun version 3 L2.0 (Giles et al., 2019; Smirnov et al., 2000) and AOT and AAOT from inversion version 2 L1.5 and L2.0 (Holben et al., 2006) were logarithmically interpolated to values at 550 nm and averaged over an hour. For all years, starting from 1992, geolocation data were obtained for all sites (1144 in total).

The direct sun dataset contains only AOT (at multiple wavelengths). These observations are based on direct transmission measurements of solar light and have high accuracy of ±0.01 (Eck et al., 1999; Schmid et al., 1999). The inversion dataset contains both AOT and AAOT (at multiple wavelengths) and these observations are based on measurements of scattered solar light from multiple directions. This inversion uses radiative transfer calculations (Dubovik and King, 2000) and yields larger errors than the direct sun measurements. In particular, Dubovik et al. (2000) showed that single scattering albedo (SSA) errors decrease with increasing AOT and estimated SSA errors of ±0.03 for water-soluble aerosol at AOT at 440 nm ≥0.2 and for dust or biomass burning aerosol at 440 nm ≥0.5. Consequently, one important distinction between inversion L1.5 and L2.0 data is a minimum threshold of AOT at 440 nm ≥0.4 used in the latter (improved cloud screening is another distinction). Inversion L2.0 is a subset of the L1.5 dataset. For an intercomparison of AERONET SSA with flight campaign data, see Schafer et al. (2014).

In the current study, only AOT at 550 nm is used and the inversion L2.0 AOT at 440 nm criterion is adapted to AOT at 550 nm≥0.25. This is the minimum value of AOT at 550 nm present in actual inversion L2.0 data but also corresponds to AOT at 440 nm=0.4 for small particles (Ångström exponent of 2.1). As a result, the OSSE in this paper is rather lenient when it comes to selecting valid observations similar to inversion L2.0.

2.3 GAW geolocations

GAW geolocation data were obtained from NILU (Norwegian Institute for Air Research). Two networks were used: the GAW-AOT network which comprises 29 sun-tracking sun photometers that measure AOT, and the GAW-ABS network which comprises 81 filter instruments that measure surface properties. The real GAW-ABS network is not capable of measuring a columnar (A)AOT but here we will assume it does, similar to AERONET, and consider its representation errors.

3 Method: analysis of representation errors

The representation error is defined as the difference between a perfect observation (i.e. no observational error) and a truth value (area average); see also S16a and S17. Here, a self-consistent high-resolution simulation will be used to generate both observation and truth in a so-called observing systems simulation experiment. The representation error may refer to instantaneous values or time averages. This work concerns itself mostly with yearly averages (and some monthly averages). For instantaneous and daily error values, see S16a and S17. The mapping from G5NR data to the data used in this study is given in Table 2.

Table 2 Mapping from G5NR data to data used in this study.

Perfect observations are generated from the high-resolution simulation by choosing the data at the location of an AERONET or GAW site and subsampling those data in time according to certain conditions for solar zenith angles (SZAs), cloud fraction and AOT. Table 3 lists the threshold conditions for which observations will be possible. Values for SZA and AOT are inferred from real AERONET data files. The maximum cloud fraction was tuned to obtain similar temporal coverage of observations as real AERONET data (see Sect. 4 and Fig. 3) but the impact of tuning is small.

Table 3 Conditions for valid AERONET observations as simulated in this study.

The truth is generated from the high-resolution simulation by averaging AOT and AAOT over a large area (0.5 to 4^∘ grid boxes) and further averaging in time. Here, we should distinguish three different protocols depending on how one intends to use the observations; see Table 4. In the case of a gridded climatology derived from observations, the truth should be an average over a continuous long-term time range (say a year). In the case of model evaluation, it is possible to resample model data to the times of the observations. For example, within the AEROCOM community, a daily collocation protocol is often used, where daily model data are used for days with observations only (irrespective of the temporal sampling of those observations throughout the day). To assess representation errors in this case, the truth needs to be sampled accordingly to days with observations before yearly averages are determined. The same protocols were also explored in S17.

Table 4 Collocation protocols.

The current methodology differs slightly from S17 in the following:

1. A different model is used to construct the OSSE.

2. Previously, SZA was assumed to be sufficiently high for a fixed fraction of the day (10 h). In the current work, SZA is calculated from downwelling TOA SW radiation and will vary with geolocation and time of day.

3. Previously, the truth was generated for grid boxes centred on the observations. In the current work, those grid boxes are assumed regularly spaced from 0 to 360^∘ longitude and −90 to 90^∘ latitude. The AERONET and GAW sites can be located anywhere within those grid boxes (at their real geolocation).

4. Previously, the high-resolution simulation had a constant grid size of (about) 10 km. In the current work, the grid size varies but has a constant angular size of 0.0625^∘ (∼7 km at the Equator).

The last point implies that the simulation grid box used for the observation decreases towards zero as we approach the poles. Since this is clearly undesirable (field of view will remain on the order of several kilometres), we will limit our analysis of representation errors to latitudes below 60^∘. The exception is Fig. 5.

Our methodology allows separation of the factors that determine the representation error: spatial extent of the grid box and observational intermittency due to low SZA, high cloud fraction or low AOT. We will not present such causal analysis in this paper (see S17 instead) but will refer to it to explain results.

To show the distributions of representation errors, box-and-whisker plots using the 2 %, 9 %, 25 %, 75 %, 91 % and 98 % quantiles will be used in this paper (in addition, the median is shown as a bar and the mean as a circle). For a normal distribution, these quantiles will be equally spaced. Any skewness or extended wings in a distribution will be readily visible. In addition to quantiles, the values of mean error and the mean signless error will be provided. The mean signless error is deemed more relevant than the standard deviation as (1) it includes biases; (2) the errors are seldom normally distributed, and a standard deviation is very sensitive to larger errors (“outliers”). For a normal distribution with a mean of 0 and a standard deviation of 1, the mean signless error is ∼0.8. The correlation used in this paper is the Pearson correlation coefficient that assesses linear relationships. Regression slopes were calculated with a robust ordinary least squares (OLS bisector from the Interactive Data Language (IDL) sixlin regressor function; Isobe et al., 1990). This regressor is recommended when there is no proper understanding of the errors in the independent variable; see also Pitkänen et al. (2016).

4 Evaluation of G5NR and OSSE

In this section, G5NR is evaluated with real AERONET observations of AOT and AAOT, with special focus on its usefulness in an OSSE. As G5NR generates its own meteorology that deviates from 2006, one might expect differences between simulation and observations. Simulated data were nevertheless collocated to the time of the observations (within the hour) to ensure the same temporal sampling throughout the days, the months and the year.

The mean and standard deviation of AOT and AAOT per site are shown in Fig. 1a–c. In general, simulated site-mean AOT shows good agreement with the observations with correlations around 0.75 and slopes around 0.84. Simulated site-mean AAOT does not agree as nicely with the observations but there is still correlation (0.48) (the evaluation of AAOT will be affected by large measurement errors). The agreement in standard deviation suggests that simulated and observed AOTs (and AAOT) show similar temporal variation. But the global agreement also suggests that the simulation captures spatial variation rather well. This is also true on shorter length scales, as an analysis by region shows in Table 5. Europe appears to be the exception but this is mostly due to a few southern sites. As the table shows, without those sites, correlation increases significantly. This may be related to the overestimation of dust and underestimation of carbonaceous and sulfate aerosol in G5NR (Gelaro et al., 2015), which will affect north–south gradients in AOT in Europe. DRAGON (Distributed Regional Aerosol Gridded Observation Networks; see Holben et al., 2018) campaigns might allow evaluation of the spatial distribution of simulated AOT at even smaller length scales (tens of kilometres) but are not available for 2006.

Figure 1 Evaluation of the G5NR simulation of AOT and AAOT with AERONET data. The top row (a, b, c) shows evaluation against three different datasets. Each dot represents the yearly mean or standard deviation for a single AERONET site (with at least 100 observations in 2006); the mean value is shown in red and the standard deviation in blue. The coloured text summarizes the statistics over all data points in the figure. The bottom row (d, e, f) shows the impact of the minimum required number of observations per site on those summary statistics (for means) is shown. Colours relate lines to axes and have different meanings than those in the top row. The solid red line indicates correlation, the dashed red line indicates slope, the solid blue line indicates mean, and the dashed blue line indicates standard deviation. In all figures, hourly G5NR model data were collocated in time and space with AERONET observations before calculating site statistics.

Table 5 Correlation in modelled and observed yearly site-mean AOT.

^* Southern AERONET sites removed from analysis.

Figure 1a–c were created using only sites that provide a minimum of 100 real observation throughout 2006. Figure 1d–f show how this criterion affects results. As the minimum number of observations per site increases, so do the correlations, probably due to a reduction in statistical noise (partly due to different simulated and actual meteorology). But the overall bias also increases. This criterion selects for sites with lower cloudiness (higher number of observations) until predominantly northern African and Saudi Arabian sites are left for a minimum of 500 observations per site. The increase in bias is thus likely due to the overestimation of dust AOT that was mentioned earlier.

Note that AAOT is here evaluated with L1.5 data. The L2.0 data have a minimum AOT threshold which results in fewer observations and fewer available sites overall. Although L1.5 is considered a less reliable product, the evaluation with L2.0 (which now uses a minimum of 30 observations per site) yields a similar but slightly poorer result for G5NR (see Fig. S1 in the Supplement) and over a shorter range of values.

Figure 2 shows mean values per site for the daily difference in maximum and minimum AOT. Again, good agreement for simulated AOT is seen but AAOT compares rather poorly. However, its correlation is still above 0.6 and it is clear that the simulation underestimates daily AAOT variation. The impact of AAOT measurement error on daily variation is likely reduced, as the variation is a difference between two measurements (Thomas F. Eck and Oleg Dubovik, personal communication, 2019).

Figure 2 Evaluation of the G5NR simulation of AOT and AAOT with AERONET data. Each dot represents the yearly average of daily variation (maximum minus minimum value) for a single AERONET site (with at least 100 observations in 2006). The grey text summarizes the statistics over all data points in the figure. In all figures, hourly G5NR model data were collocated in time and space with AERONET observations before calculating site statistics.

Figure 3 shows the temporal coverage (or frequency of observation) per site as a function of latitude. G5NR's simulated coverage is calculated using the conditions described in Table 3 (and explained in Sect. 3). This coverage would be 100 % if observations are available 24 h d⁻¹, 365 d yr⁻¹. In practice, it cannot be higher than 50 % due to the day–night cycle and will be less due to cloudiness or low AOT.

Figure 3 Evaluation of the temporal coverage predicted by the OSSE with AERONET observations. Each dot represents temporal coverage (or frequency of observation) for a single AERONET site (with at least 100 observations in 2006, at least 30 observations for inversion L2.0). The grey dots are real AERONET data; the red dots are simulated by the methodology described in Sect. 3. The numbers in the graph are temporal coverages estimated by hemisphere.

The bimodal structure that is visible in both the simulation and observations is due to SZA variation (which reduces coverage towards the poles) and cloudiness (which reduces coverage near the Equator). Simulated and real coverage per site are not expected to agree well due to meteorological differences and downtimes from site maintenance. Still, the results suggests that the OSSE predicts similar frequency of direct sun observations as actually observed.

However, the OSSE also simulates more inversion observations in the Northern Hemisphere than actually occur. This suggests there are additional factors in observational coverage that are not accounted for in Table 3. One factor is that real inversion measurements are attempted less frequently (several times per day) than direct sun measurements (several times per hour). Other factors may include inversion failure at low SZA (real observations show that inversion data generally have larger SZA than direct sun data even though inversion data are generally closer to the Equator) and overestimation of dust AOT in G5NR (largest overestimates of coverage occur for Saharan and Saudi Arabian sites). Yet another issue is that successful inversion requires a high degree of azimuthal symmetry in the measurements. In essence, this is a built-in check on the magnitude of spatial representation errors which will lower temporal coverage of the observations but is not considered in the OSSE due to lack of information. Finally, instrument malfunction and maintenance are not taken in to account, although that would affect AOT coverage as well.

In all, it seems that G5NR can realistically simulate spatial and temporal variation in AOT and AAOT, at least on the scales accessible by the available observations. There is some underestimation of daily AOT variation and significant underestimation of daily AAOT variation. G5NR can also be used to fairly realistically simulate frequency of observation (temporal coverage), although it will overestimate this for the inversion products in the Northern Hemisphere.

5 Results

5.1 Representation errors in yearly AOT

Figure 4 shows yearly representation errors for AERONET direct sun L2.0 AOT observations as a function of model grid-box size for the three collocation protocols (see Table 4). As grid-box size changes from 4 to 0.5^∘, errors for hourly collocation are more than halved from 13 % to 5 %, while those for daily collocation change only from 17 % to 12 %. In contrast, errors for yearly collocation (∼22 %) are dominated by temporal sampling and do not depend much on grid-box size. Smaller representation errors for hourly collocation can also be seen in a regional analysis; see Fig. S2. The hourly collocation is especially beneficial when using the inversion L2.0 AOT product, which allows large representation errors due to the condition of a minimum AOT at 440 nm (≥0.4) for valid observations (see Fig. S3), even though it results in a global 9 % bias.

Figure 4 Yearly representation errors for AOT from AERONET direct sun L2.0 for different model grid-box sizes. The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Numbers on top are the means of the errors and means of the signless errors.

The impact of collocation protocol can also be shown through the total number of sites that yield errors larger than, say, 10 %: 821 (yearly), 653 (daily), 235 (hourly) out of 1108 AERONET stations in total, for a grid box of $\mathrm{1}{}^{\circ }\times \mathrm{1}{}^{\circ }$.

In addition to larger representation errors in general, the yearly and daily collocation protocols also allow significant biases across the AERONET network. Regionally, spatial patterns with east–west or north–south gradients in the representation errors exist; see Figs. 5 and S4. Such patterns are absent or at least much reduced for hourly collocation.

Figure 5 Yearly representation errors (%) for AOT from AERONET direct sun L2.0 in North America, for two different collocation protocols (a daily; b hourly) and a model grid-box size of 1^∘.

The biases in regional and global distributions of representation errors for yearly and daily collocations are strongly affected by cloudiness. Higher humidity in the cloudy part of a grid box increases AOT through hygroscopic growth. The area averages used to calculate representation errors have been derived for the entire grid box (all sky), both clear and cloudy parts. Representation errors for clear-sky parts of grid boxes are lower for the yearly and daily collocation protocols; see Fig. 6. In certain situations, it seems more realistic to use only the clear part of the grid box in calculating representation errors, e.g. when the grid-box average stands in for an aggregated satellite product. In this paper, the focus will be on the all-sky representation error.

Figure 6 Yearly representation errors for AOT from AERONET direct sun L2.0 using all-sky or clear-sky conditions and model grid-box size of 4^∘ (a) or 1^∘ (b). The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Numbers on top are the means of the errors and means of the signless errors.

Table 6 shows absolute values of the yearly representation errors for different collocation protocols (yearly and hourly) and grid-box sizes. The statistical metrics provided are the mean of the signless representation error over all AERONET sites and the 90 % quantile of the signless representation error (an indication of the large representation errors possible for some sites). Using absolute values allows a comparison with the AERONET AOT measurement error of 0.01 (Eck et al., 1999; Schmid et al., 1999). This is the error for individual measurements and not that of a yearly average, which is likely to be much smaller. Clearly, representation errors are larger than measurement errors.

Table 6 Absolute representation errors for AERONET sites.

Results so far suggest that the daily collocation is a significant improvement over the yearly collocation. This is in contrast to S17 (their Fig. 7), where the representation errors for daily and monthly collocation were found to be similar. The absence of diurnal (anthropogenic) emission profiles in G5NR may cause underestimation of representation errors for the daily collocation in the current study.

It is interesting to compare the representation errors of two different networks (AERONET and GAW). AERONET was not designed with representativity in mind but the GAW network was. Nevertheless, Fig. 7 suggests that GAW sites exhibit slightly larger representation errors than AERONET. In particular, GAW error statistics are strongly skewed to negative values. In the G5NR OSSE, GAW sites are located at higher altitudes and more often on isolated mountains than AERONET sites (G5NR site altitudes correlate very well with real altitudes (R=0.98) but tend to underestimate by 28 m on average, with a standard deviation of 171 m). A look at yearly representation errors for the hourly collocation reveals a systematic altitude dependence; see Fig. 8. A high-altitude site on an isolated mountain will observe a shorter atmospheric column than the surrounding grid box (most of which is at lower altitudes), which will cause a negative representation error. Note that AERONET sites do not show this dependence on altitude for 1^∘ grid boxes, probably because they are located more often on mountains surrounded by similar mountainous terrain.

Figure 7 Yearly representation errors for AOT from AERONET direct sun L2.0 and GAW and a model grid-box size of 1^∘. The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Numbers on top are the means of the errors and means of the signless errors.

Figure 8 Yearly representation errors for AOT from AERONET direct sun L2.0 (red circles) and GAW (black squares) as a function of site altitude, for a model grid-box size of either 4^∘ or 1^∘, using hourly collocation.

Finally, a comparison is made with a previous study into AERONET representation errors (Kinne et al., 2013). Using a range score r (see Table 7), they ranked sites according to their representativity for larger domains. This ranking is subjective in that it is non-quantitative, based on personal knowledge of the sites and only defines representativity in broad terms. The range scores are only available for sites that had at least 5 months of data before 2008. Using the methodology of this paper, representation errors were calculated for all sites of a certain range score; see Fig. 9. For large grid boxes of 4^∘ (∼450 km near the Equator), the impact of the range score on representation error is quite small. While there is a visually arresting change in the error distribution for r>1 (wide flanks are changed into a broader centre), the mean signless error barely changes. This rather weak dependence on range score suggests that Kinne et al. (2013) overestimated the size of the domains (≥500 km for r>1) of which their sites were representative. On the other hand, for a grid box of 1^∘, a substantial reduction in representation error can be seen for r≥1 sites. However, this only occurs for the hourly collocation; Kinne et al. (2013) did not consider the temporal sampling of the observations which causes large representation errors. An alternative ranking of representativity will be introduced in Sect. 6.

Table 7 Range scores for AERONET sites in Kinne et al. (2013).

Figure 9 Yearly representation errors for AOT from AERONET direct sun L2.0 for different range scores r by Kinne et al. (2013), for a model grid-box size of either 4^∘ or 1^∘. The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Numbers on top are the means of the errors and means of the signless errors.

5.2 Representation errors in monthly AOT

Surprisingly, monthly representation errors are not that much larger than yearly errors; see Fig. 10. If monthly errors for the same site were independent and random, one would expect them to be $\sim \sqrt{\mathrm{12}}\approx \mathrm{3.5}$ larger than yearly errors, but that is not the case. As a matter of fact, monthly errors are strongly correlated from month to month, throughout the year; see Fig. 11. The increase in correlation with January after September is probably due to yearly cycles in meteorology and emissions and very likely to be a realistic aspect of representation errors. The implication of this is that multi-year averages may not reduce representation errors as strongly as one would hope.

Figure 10 Yearly and monthly representation errors for AERONET direct sun L2.0 AOT, for a model grid-box size of 1^∘. In contrast to Fig. 4, only sites that provide observations 12 months out of the year are used in this analysis. The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Obviously, for the brown bar on the right, a monthly protocol was used. Numbers on top are the means of the errors and means of the signless errors.

Figure 11 Correlation in monthly representation errors with errors for January, for AERONET direct sun L2.0 AOT, for a model grid-box size of 1^∘. Only sites that provide observations 12 months out of the year are used in this analysis. The colours indicate different collocation protocols: monthly (brown), daily (orange) and hourly (red).

This analysis also provokes the question whether representation errors (per site) should be seen as mostly biases or random errors (see also Schwarz et al., 2018). Preliminary analysis suggests that both cases can occur. That is, some sites show large variations (including sign changes) in representation error from month to month, and as a consequence, a strongly reduced yearly representation error. Here, monthly representation errors may be interpreted as mostly random. Other sites show monthly representation errors with not much variation and as a consequence yearly representation errors are similar to the monthly errors. There, the representation error is better characterized as a bias. A proper analysis of this would require significantly longer time series of data than are currently available. Further discussion of this can be found in Sect. 6.

5.3 Representation errors in AAOT

The discussion of representation errors for inversion L1.5 AAOT will be shorter than that for direct sun L2.0 AOT, as the main conclusion is identical: the hourly collocation yields smaller representation errors than the other protocols; see Fig. 12. Note also that representation errors in AAOT are of a similar magnitude as for AOT. One obvious difference is that AAOT representation errors tend to be positively biased, while the AOT errors were negatively biased. While the latter was due to cloudiness as discussed before, the positive bias for AAOT is more difficult to explain. It appears that a combination of conditions (location of the sites, necessity of daylight, clear skies and a minimum AOT of 0.03) together conspire to create these positive biases. Only over the Amazon can a simple explanation be found: the clear-sky condition prevents many observations outside the biomass burning season, explaining large positive biases for yearly collocation (see also Fig. S5, discussed later).

Figure 12 Yearly representation errors for AAOT from AERONET inversion L1.5 for different model grid-box sizes. The colours indicate different collocation protocols: yearly (brown), daily (orange) and hourly (red). Numbers on top are the means of the errors and means of the signless errors.

Even more than for AOT, representation errors for AAOT are very similar for the daily and hourly collocations. As discussed before, this is likely due to the absence of diurnal (anthropogenic) emissions profiles. The daily variation of AAOT is strongly underestimated by G5NR (see Sect. 4 and Fig. 2).

For completeness' sake, an analysis of AAOT representation errors for different regions (Fig. S5), different products (Fig. S6), different networks (Fig. S7) and different range scores by Kinne et al. (2013) (Fig. S8) is given in the Supplement. Overall, the conclusions are very similar to those for AOT.

The similarity in general behaviour of representation errors for AOT and AAOT should not be taken to mean that these errors are identical per site. As discussed in Sect. 6, representation errors for AOT and AAOT at individual sites can be very different. Ultimately, this is due to the different sources of AOT and AAOT, which leads to different spatiotemporal distributions in the atmosphere.

5.4 Comparison to recent results from Wang et al. (2018)

Recently, Wang et al. (2018) suggested that the observed underestimation of AAOT by AEROCOM models (Bond et al., 2013) may be due to spatial representation errors. Spatial representation errors are entirely due to the narrow field of view of AERONET observations (i.e. the intermittent temporal sampling of these observations is ignored). Their analysis found that AERONET inversion L1.5 AAOT representation errors exhibit a global bias of 30 % for $\mathrm{2}{}^{\circ }\times \mathrm{2}{}^{\circ }$ model grid boxes, which would help explain the aforementioned underestimation by the global models. As AERONET sites need to be serviceable, they are often found near roads and urban build-up, i.e. near sources of absorbing aerosol. Compared to the larger area of global model grid boxes, these sites would quite naturally observe larger AAOT. Thus, Wang et al. (2018) concluded that at least part of the underestimation of modelled AAOT is an artefact created by the location of the AERONET sites.

Wang et al. (2018)'s idea is quite persuasive, and indeed one can see evidence of such positive representation errors in Fig. 13 where sites in major cities like London, Paris, Madrid and Barcelona clearly exhibit positive representation errors. (For another example, see Fig. 3b in S17 concerning surface black carbon concentrations.) But Wang et al. (2018)'s study found such biases for the majority of AERONET sites, not just a few located in big cities. As a matter of fact, the current study shows no evidence of this global bias of 30 %. Instead, it finds a global bias of only 9 %, dominated by a few sites with large positive representation errors (median bias over all sites: 4 %).

Figure 13 Black carbon emissions over France (Europe) with the representation errors in AAOT from AERONET inversion L1.5 superimposed. The top colour bar (white–black) represents emissions (kg m⁻² s⁻¹), and the bottom colour bar (blue–red) represents relative representation errors (%). Only spatial representation errors are shown; i.e. the temporal sampling of observations is ignored.

Wang et al. (2018) performed an analysis very much like the one in this study with one crucial difference. As they did not have a global simulation at high resolution like G5NR, they downscaled results from a standard global simulation at $\mathrm{2.5}{}^{\circ }\times \mathrm{1.27}{}^{\circ }$ resolution. The downscaling was accomplished with the help of a high-resolution ($\mathrm{0.1}{}^{\circ }\times \mathrm{0.1}{}^{\circ }$) black carbon emission map (Wang et al., 2016). It is possible to simulate this procedure using the high-resolution G5NR black carbon emission maps and AAOT simulations (the AAOT simulation was first coarsened over $\mathrm{2}{}^{\circ }\times \mathrm{2}{}^{\circ }$) and explain the different results in Wang et al. (2018) and the current study.

Figure 14 shows AAOT spatial representation errors as estimated by the current study and by Wang et al. (2018)'s methodology as simulated with G5NR data. A global bias of 25 %, not very different from the original 30 % mentioned in Wang et al. (2018), is found for the Wang et al. (2018) analysis, whose representation errors yield a strongly skewed distribution over all sites. In contrast, the present study yields a more symmetric distribution with a much smaller bias. Unlike in the Wang et al. (2018) analysis, this bias is dominated by just a few sites with large positive representation errors.

Figure 14 Yearly representation errors for AAOT from inversion L1.5 AERONET as estimated in this paper or using the methodology from Wang et al. (2018) and a model grid-box size of 2^∘. The representation error shown is the spatial representation error (Schutgens et al., 2017); i.e. temporal sampling of observations is ignored.

The analysis above is a self-consistent evaluation of Wang et al. (2018)'s methodology. Using high-resolution black carbon emission data to downscale coarse model AAOT fields ignores redistribution of absorbing aerosol due to small-scale (at and below the coarse model's grid box) advective and turbulent transport as well as removal by local precipitation (Wang et al., 2018 were aware of this limitation but could not assess its impact). It also ignores local orography and the contribution of absorbing dust to AAOT. The result is that there is very little correlation between representation errors as estimated by the two methods; see Fig. 15. As a matter of fact, representation errors from the current study do not show a systematic dependence on emission distributions, unlike the representation errors from Wang's methodology.

Figure 15 Comparison of yearly representation errors for AAOT from AERONET inversion L1.5 as estimated in this paper or using the methodology from Wang et al. (2018) and a model grid-box size of 2^∘. The representation error shown is the spatial representation error (Schutgens et al., 2017); i.e. temporal sampling of observations is ignored. Also shown are the Pearson linear correlation (PCorr) and rank correlation (RCorr) between the data. The dashed line shows y=x.

6 A ranking of representativity for the AERONET sites

A ranking of AERONET and GAW sites in terms of their spatial representativity for AOT and AAOT can be found in Schutgens (2019). Only sites below 60^∘ latitude are considered, and temporal sampling of observations is ignored. The latter was done for two reasons: (1) as discussed in Sects. 2 and 4, temporal sampling of observations is considered less accurately modelled by the OSSE than spatial variability; (2) both S17 and the current study show that once hourly collocation is used, the remaining representation error is similar although slightly larger than the spatial representation error.

Relative representation errors are classed according to bins: 0 %–5 % (rank 1), 5 %–10 % (rank 2), 10 %–20 % (rank 3), 20 %–40 % (rank 4), 40 % and higher (rank 5). The accuracy of this ranking depends of course on the skill of G5NR and the OSSE but also on statistical noise due to the use of a single year of data. The latter source of uncertainty was assessed using a block bootstrap method (Efron, 1979) on the time series per site. Typically, more than 85 % of all resampled time series yield a representation error in the same class as the original time series. For large grid boxes (4^∘) and small errors (<10 %), this may drop down to 66 % of the resampled time series. For those resampled time series that yielded a different ranking, this ranking was only off by 1. It then seems that statistical noise does not prevent a robust classification of yearly relative spatial representation errors. The impact of G5NR and OSSE skill on the classification can currently not be assessed.

Compared to the subjective ranking by Kinne et al. (2013), the new ranking is objective because the rank is related to a well-defined representation error that is quantified bottom-up from known emission sources and calculated meteorology. That in itself is of course no guarantee for accuracy.

Inspection of the rankings turns up several interesting points. Analysis in the previous sections determined a few “rules” for the behaviour of representation errors (e.g. errors decrease when the grid-box size decreases) but these can easily be “broken” for specific sites: a smaller grid box may actually lead to larger representation errors (e.g. AOE_Baotou, Ascension_Island, Aras_de_los_Olmos), monthly errors may be substantially larger than yearly errors (e.g. ARM-Darwin, BORDEAUX). Also, representation errors for AOT and AAOT may be very different: Bayfordbury shows small yearly representation errors for AOT but large errors for AAOT, while Mace_Head shows the opposite.

7 Conclusions

Remote sensing observations from the AERONET and GAW networks are intermittent in time and have a limited field of view. Consequently, such observations have a limited ability to represent (absorbing) aerosol optical thickness, or (A)AOT, over larger areas. The resulting spatiotemporal representation error is here analysed using a high-resolution simulation of global aerosol (G5NR, ∼7 km resolution near the Equator). Using G5NR, an OSSE was constructed that simulates the frequency of AERONET observations taking solar zenith angle, cloud fraction and AOT values into account.

This work extends previous work on temporal representation with global low-resolution models (Schutgens et al., 2016b) to spatiotemporal representation. It also extends previous work on spatiotemporal representation with regional high-resolution simulations (Schutgens et al., 2016a, 2017) to the global domain. The current work is more limited in scope than the previous studies and only considers ground-based remote sensing observations. For satellite remote sensing, see Schutgens et al. (2016b, 2017). For in situ measurements, see Schutgens et al. (2016a, 2017).

G5NR and the OSSE are evaluated and found to show significant skill in AOT and reasonable skill in AAOT. AERONET mean AOT per site, as well as yearly and daily variability, was estimated quite correctly, usually within a factor of less than 2. Considering that G5NR generates its own meteorology, G5NR AOT correlated very well (R≈0.75) with the observations. Similarly, the OSSE was surprisingly good at simulating the overall pattern of observational coverage (frequency of AOT observation). Results were not as good for AAOT but still acceptable. Yearly AAOT variability was slightly underestimated, while daily AAOT variability was severely underestimated. The latter is possibly related to the absence of diurnal anthropogenic emission profiles in G5NR. For representativity studies that take diurnal variations into account, see Schutgens et al. (2016a, 2017). In addition, the OSSE tended to overestimate the frequency of AAOT observations per site (although this was shown to have no impact on representation errors).

Both yearly and monthly representation errors are provided for observations from ground sites that attempt to represent larger areas (from 0.5 to 4^∘ in size). The monthly representation errors are shown to be strongly correlated throughout the year. For some sites, this is an expression of a bias but that is not universally the case. In any case, monthly representation errors can not be treated as independent and this has (negative) consequences for the reduction of representation errors in multi-year averages. Other conclusions are (1) AERONET-derived climatologies allow for substantial representation errors (yearly collocation allows errors of typically 20 %; see Fig. 4); (2) AEROCOM evaluation protocol is suboptimal (daily collocation can show errors of 25 % in coherent regional patterns). Instead, hourly collocation is advocated; (3) the representativity of AERONET and GAW sites was shown to be not very different, although AERONET sites seem to be more affected by nearby sources, while GAW sites seem more affected by their altitude. Finally, a subjective ranking (Kinne et al., 2013) of the spatial representativity of sites was analysed and shown to broadly agree with the current study, although it appears to overestimate represented spatial domain sizes and judges several sites as less representative than the current analysis. A new objective ranking is also presented.

While the current study's focus is on strategies for model evaluation with original (“all points”) AERONET data, it does allow recommendations to be made for the optimal aggregation level of observational data. Hourly products are preferred to daily or monthly products, as they allow users to perform hourly collocation which in turn yields significantly smaller representation errors. This should hold for both satellite and AERONET data.

Spatial representation errors have been used to reconcile observations and global simulations of AAOT. Bond et al. (2013) showed that global models tend to significantly underestimate AAOT but Wang et al. (2018) suggested that AERONET AAOT observations may suffer from a global 30 % representation bias. In contrast, the current analysis finds a much smaller bias of 9 %, which is moreover strongly influenced by a few sites with large positive representation errors due to their proximity to black carbon sources. Judiciously excluding those sites significantly reduces the bias even further (4 %). The large positive representation errors found by Wang et al. (2018) are shown to be due to methodological choices that limit the realism of their OSSE.

Several questions remain and seem interesting for follow-up studies:

1. How can we evaluate the representativity rankings?

2. How do OSSE errors affect estimated representation errors?

3. How will diurnal emission profiles impact results?

4. Can representation errors at any site be decomposed in a bias and random error (possibly with temporal correlations over several months)?

5. What are representation errors like in multi-year averages?