EARLINET evaluation of the CATS Level 2 aerosol backscatter coefficient product

We present the evaluation activity of the European Aerosol Research Lidar Network (EARLINET) for the quantitative assessment of the Level 2 aerosol backscatter coefficient product derived by the Cloud-Aerosol Transport System (CATS) aboard the International Space Station (ISS; Rodier et al., 2015). The study employs correlative CATS and EARLINET backscatter measurements within a 50 km distance between the ground station and the ISS overpass and as close in time as possible, typically with the starting time or stopping time of the EARLINET performed measurement time window within 90 min of the ISS overpass, for the period from February 2015 to September 2016. The results demonstrate the good agreement of the CATS Level 2 backscatter coefficient and EARLINET. Three ISS overpasses close to the EARLINET stations of Leipzig, Germany; Évora, Portugal; and Dushanbe, Tajikistan, are analyzed here to demonstrate the performance of the CATS lidar system under different conditions. The results show that under cloud-free, relative homogeneous aerosol conditions, CATS is in good agreement with EARLINET, independent of daytime and nighttime conditions. CATS low negative biases are observed, partially attributed to the deficiency of lidar systems to detect tenuous aerosol layers of backscatter signal below the minimum detection thresholds; these are biases which may lead to systematic deviations and slight underestimations of the total aerosol optical depth (AOD) in climate studies. In addition, CATS misclassification of aerosol layers as clouds, and vice versa, in cases of coexistent and/or adjacent aerosol and cloud features, occasionally leads to non-representative, unrealistic, and cloud-contaminated aerosol profiles. Regarding solar illumination conditions, low negative biases in CATS backscatter coefficient profiles, of the order of 6.1 %, indicate the good nighttime performance of CATS. During daytime, a reduced signal-to-noise ratio by solar background illumination prevents retrievals of weakly scattering atmospheric layers that would otherwise be detectable during nighttime, leading to higher negative biases, of the order of 22.3 %.

1 Introduction

The Cloud-Aerosol Transport System (CATS) is a satellite-based elastic backscatter lidar developed to provide near-real-time, vertically resolved information on the vertical distribution of aerosols and clouds in the Earth's atmosphere (McGill et al., 2015). Developed at NASA's Goddard Space Flight Center, CATS is based on the Cloud Physics Lidar (CPL; McGill et al., 2002) and the Airborne Cloud-Aerosol Transport System (ACATS; Yorks et al., 2014), designed to operate aboard the high-altitude NASA ER-2 aircraft. CATS operated as a scientific payload aboard the Japanese Experiment Module – Exposed Facility (JEM-EF), utilizing the International Space Station (ISS) as a space science platform (Yorks et al., 2016). Starting from 10 February 2015, CATS provided aerosol and cloud profile observations along the ISS flight track for more than 33 months, until 30 October 2017, when the system suffered an unrecoverable fault.

CATS was developed to meet three main scientific goals. The primary objective was to measure and characterize aerosols and clouds on a global scale. The spaceborne lidar orbited the Earth at an altitude of approximately 405 km and a 51^∘ inclination. The use of the ISS as an observation platform facilitated, for the first time, global lidar-based climatic studies of aerosols and clouds at various local times (Noel et al., 2018; Lee et al., 2018). In addition, near-real-time data acquisition of the CATS observations was developed towards the improvement of aerosol forecast models (Hughes et al., 2016). A secondary objective was related to the need for long-term and continuous satellite-based lidar observations to be available for climatic studies. The first spaceborne lidar mission, the Lidar In-Space Technology Experiment (LITE; McCormick et al., 1993) in 1994, was succeeded by the joint NASA and Centre National d'Études Spatiales (CNES) Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) mission in June, 2006 (Winker et al., 2007). Since 2009, the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument (Winker et al., 2009) aboard CALIPSO operates on the secondary backup laser. The launch of the post-CALIPSO missions, the joint European Space Agency (ESA) and JAXA's satellite Earth Cloud, Aerosol and Radiation Explorer (EarthCARE; Illingworth et al., 2015) and NASA's Aerosols, Clouds, and Ecosystems (ACE) are planned for 2021 and post-2020 respectively. The CATS project was partially intended to fill a potential gap in global lidar observations of aerosol and cloud profiling. The third scientific objective of CATS was to serve as a low-cost technological demonstration for future satellite lidar missions (McGill et al., 2015). Its scientific goal to explore different technologies was fulfilled through the use of photon-counting detectors and of two low-energy (1–2 mJ) and high repetition rate (4–5 kHz) Nd:YVO4 lasers (multi-beam and high spectral resolution lidar – HSRL; UV demonstrations), aiming to provide simultaneous multiwavelength observations (355, 532, and 1064 nm). Additional gains of the CATS project were related to the exploitation and risk reduction of newly applied laser technologies to pave the way for future spaceborne lidar missions (high repetition rate, injection seeding, and wavelength tripling at 355 nm).

CATS performance has been validated against ground-based AERONET (Holben et al., 1998) measurements and evaluated against satellite-based aerosol optical depth (AOD) retrievals of the Aqua and Terra Moderate Resolution Imaging Spectroradiometer (MODIS; Levy et al., 2013) and active CPL (McGill et al., 2002) and CALIPSO CALIOP (Winker et al., 2009) profiles of the extinction coefficient and AOD at 1064 nm. Lee et al. (2018) compared daytime quality-assured CATS V2.01 vertically integrated extinction coefficient profiles (1064 nm) and AERONET AOD (1020 nm) values, spatially (within 0.4^∘ longitude and latitude) and temporally (±30 min) collocated, and found a reasonable agreement, with a correlation of 0.64. A comparative analysis of CATS and MODIS C6.1 Dark Target (DT) AOD retrievals, through spectral interpolation between 0.87 and 1.24 µm channels, reported a correlation of 0.75 and slope of 0.79 over ocean. In addition, Lee et al. (2019) evaluated AOD and extinction coefficient profiles from CATS through intercomparison with CALIOP. Regarding AOD, analysis of 2681 CATS and CALIOP collocated observation cases (within 0.4^∘ longitude and latitude and ±30 min ISS and CALIPSO overpass difference) showed correlations of 0.62 and 0.52 over land and ocean respectively during daytime (1342 cases) and 0.84 and 0.81 over land and ocean respectively during nighttime (1339 cases). Comparison of CATS and CALIOP collocated extinction coefficient profiles based on the closest Euclidian distance on the Earth's surface shows also good shape agreement despite an apparent CALIOP underestimation in the lowest 2 km height. CATS and CALIOP observations were used by Rajapakshe et al. (2017) to study the seasonally transported aerosol layers over the SE Atlantic Ocean. The performed comparative analysis reported on similar geographical patterns regarding above-cloud aerosol (ACA), cloud fraction (CF), and ACA occurrence frequency (ACA_F) between CATS and CALIOP retrievals. However, the authors reported also on differences between CATS and CALIOP vertical aerosol distributions, with the ACA bottom height identified by CATS being lower than the respective of CALIOP. Noel et al. (2018) implemented measurements from CATS to investigate the diurnal cycle and variations in clouds over land and ocean. The authors showed that both CATS and CALIOP profiles and CF agree well on both the vertical patterns and values at 01:30 and 13:30 LT, over both land and ocean, with minor differences of the order of 2 %–7 % throughout all cloud profiles. Finally, CATS depolarization measurements, which are critical in the processing algorithms of aerosol-subtype classification, were investigated in the case of desert dust, smoke from biomass burning, and cirrus clouds (Yorks et al., 2016) and were found to be consistent and in good agreement with depolarization measurements from previous studies and historical datasets implementing CPL (Yorks et al., 2011) and CALIOP (Liu et al., 2015).

Overall, CATS retrievals have been evaluated and found to be in reasonable agreement with ground-based AERONET, airborne CPL, and satellite-based MODIS and CALIOP measurements. However, for the quality assessment of CATS backscatter coefficient profiles, a large-scale and dense network of ground-based lidar systems is needed in order to facilitate high-quality collocated and concurrent measurements. This necessity is largely related to the ISS orbital characteristics, the CATS near-nadir viewing (0.5^∘ off nadir), the lidar narrow footprint (14.38 m diameter), and the limited number of ISS overpasses. The European Aerosol Research Lidar Network (EARLINET) consists of a unique infrastructure for assessing the validation needs for spaceborne lidar missions. EARLINET operates in the framework of Aerosols, Clouds and Trace Gases Research Infrastructure (ACTRIS) as a pan-European effort to develop a coordinated lidar research infrastructure (Pappalardo et al., 2014) of advanced Raman lidar systems and is characterized by extensive geographical coverage.

In this paper, we utilize EARLINET for the evaluation of the CATS Level 2 aerosol backscatter coefficient product at 1064 nm. The paper is structured as follows: in Sect. 2 we introduce aspects of CATS and EARLINET relevant to the study. and additionally the comparison methodology is presented and discussed. Specific study cases are evaluated and discussed in Sect. 3. Section 4 presents the generic intercomparison results between CATS and EARLINET, while the concluding remarks on the CATS–EARLINET backscatter coefficient evaluation are summarized in Sect. 5.

2 Data and methodology

2.1 CATS

The CATS elastic backscatter lidar was designed to provide near-real-time measurements of the vertical profiles of aerosol and cloud optical properties at three wavelengths (355, 532, and 1064 nm). As a payload of the JEM-EF on the ISS, CATS was designed to operate two high repetition rate lasers in three different modes and at four instantaneous fields of view (iFOVs). Mode 1 was designed as multi-beam backscatter and depolarization configuration at 532 and 1064 nm, where a beam splitter would produce two footprints of 14.38 m diameter on the Earth's surface to the left-side FOV (LSFOV) and the right-side FOV (RSFOV) of the ISS orbit track, separated by approximately a distance of 7 km. Mode 2 was designed as a demonstration of the HSRL to provide backscatter profiles at 532 nm and backscatter and depolarization ratio profiles at 1064 nm (forward FOV – FFOV). Mode 3 was designed to operate and provide backscatter at 355, 532, and 1064 nm and at a depolarization ratio at 532 and 1064 nm. CATS was a technological demonstration designed to operate on orbit between 6 months and 3 years. Due to a failure in the CATS optics at the 355 nm wavelength, CATS did not operate in Mode 3, while the use of Mode 1 was limited between 10 February 2015 and 21 March 2015 due to a failure in the electronics of laser 1. Nevertheless, the successful long-term operation of Mode 2, between February 2015 and October 2017, allowed CATS to fulfill its science objectives.

CATS processing algorithms (Pauly et al., 2019) rely heavily on the processing algorithms developed in the framework of the CPL, ACATS, and CALIPSO lidar systems (Palm et al., 2002; Yorks et al., 2011; Hlavka et al., 2012), while the products of CATS are provided in different levels of processing. CATS Level 1B data include vertical profiles of total and perpendicular attenuated backscatter signals, which are range-corrected, calibrated, and annotated with ancillary meteorological parameters based on previous work using CPL and CALIPSO (McGill et al., 2007; Powell et al., 2009; Vaughan et al., 2010). Level 2 products provide the vertical distribution of aerosol and cloud properties (depolarization ratio, backscatter, and extinction coefficient profiles at 1064 nm – FFOV), with a horizontal and vertical resolution of 5 km and 60 m respectively. In addition, Level 2 data include geophysical parameters of the identified atmospheric layers (vertical feature mask – feature type and aerosol subtype), the required horizontal averaging, and information on the feature type classification confidence (Yorks et al., 2019). In addition to CATS Level 2 feature type (namely clear air, cloud, aerosol, and totally attenuated), the algorithm provides the confidence level of the feature type classification, which is similar to the CALIOP cloud–aerosol discrimination (CAD) algorithm (Liu et al., 2004, 2009). The CATS feature type score is a multidimensional probability density function (PDF) developed based on multiyear CPL observations that discriminates cloud and aerosol features, assigning an integer between −10 and 10 for each detected atmospheric layer.

In this study, we used CATS Level 2 V2.01 profiles (Palm et al., 2016). A comprehensive overview of the CATS instrument and CATS scientific goals is given by McGill et al. (2015) and Yorks et al. (2016), while detailed information related to CATS datasets and a CATS lidar quick-look browser can be found in the CATS Data Release Notes, Quality Statements and Theoretical Basis, available at https://cats.gsfc.nasa.gov/ (last access: 20 December 2018).

2.2 EARLINET

EARLINET (https://www.earlinet.org/index.php?id=earlinet_homepage, last access: 20 December 2018) was founded by the European Commission (Bösenberg et al., 2001) as a research project within the framework of the Fifth Framework Programme (FP5). Currently the network activity is integrated and constitutes a major component of the ACTRIS research infrastructure (https://www.actris.eu/, last access: 20 December 2018). The main objective of EARLINET is to establish an extended, coordinated, and continental network of sophisticated ground-based Raman lidar systems (Ansmann et al., 1992). The vertical distribution of aerosols in the atmosphere as well as their temporal evolution are provided by high-resolution EARLINET measurements over Europe. The long-term continuous operation of EARLINET infrastructure has fostered a quantitative, comprehensive, and statistically significant database of the distribution of aerosol on a continental scale (Bösenberg et al., 2003; Pappalardo et al., 2014).

Since the beginning of the initiative in 2000, EARLINET has significantly increased its observing and operational capacity. Currently, EARLINET is composed of 29 operating lidar stations distributed over Europe (Fig. 1), including seven admitted or joining stations. EARLINET stations are classified as “active”, “not permanent”, “joining”, and “not active”. An EARLINET station is classified as active on the condition that measurements are performed regularly and simultaneously with the other stations composing the lidar network, and accordingly, contribute with uploading the performed measurements to the EARLINET database (https://www.earlinet.org/, last access: 20 December 2018). Lidar observations in the framework of EARLINET are performed according to a common schedule – on preselected dates. The schedule involves three measurements per week, namely one during daytime at around local noon (Monday, 14:00 ± 1 h) and two during nighttime (Monday and Thursday, sunset + 2/3 h) to enable Raman extinction retrievals. In addition to the preselected dates of the operation schedule, dedicated measurements are performed to monitor special events such as major volcanic activity (Ansmann et al., 2010, 2011; Pappalardo et al., 2013; Perrone et al., 2012; Sicard et al., 2012; Wang et al., 2008), long-range transport of Saharan dust (Ansmann et al., 2003; Solomos et al., 2017, 2018), and smoke particles (Ortiz-Amezcua et al., 2017; Janicka et al., 2017; Stachlewska et al., 2018). Some of the EARLINET systems perform 24/7 continuous measurements as, for example, the PollyXT systems (Engelmann et al., 2016; Baars et al., 2016). The quality assurance and improvement of the performance of the EARLINET systems are tested through the intercomparison of both the infrastructure (Wandinger et al., 2016) and the optical products (Böckmann et al., 2004; Pappalardo et al., 2004). In addition, the homogenization of the lidar data in a standardized output format is facilitated, and an automatic algorithm is developed to further address the quality assurance of the lidar measurements (the Single Calculus Chain – SCC; D'Amico et al., 2015, 2016; Mattis et al., 2016). The SCC has been used in near-real time to show the potential operationality of the network in a 72 h continuous measurement exercise in 2012 (Sicard et al., 2015).

Figure 1 Distribution of EARLINET lidar stations over Europe and western Asia. Green dots: stations used in the intercomparison. ISS orbits between February 2015 and September 2016 are overlaid in red for daytime and in blue for nighttime overpasses.

Due to its implicit characteristics, EARLINET is an optimum tool for supporting satellite-based lidar missions with extensive experience in satellite calibration and validation activities. EARLINET and CALIPSO (Winker et al., 2009) correlative measurements are regularly performed in order to investigate the quality of CALIOP observations, to test the presence of possible biases, and to assess different aspects of spaceborne lidar measurements (e.g., Pappalardo et al., 2010; Mamouri et al., 2009; Mona et al., 2009; Perrone et al., 2011; Wandinger et al., 2011; Amiridis et al., 2013; Grigas et al., 2015; Papagiannopoulos et al., 2016). Similarly, ESA validation programs of the Atmospheric Laser Doppler Instrument (ALADIN) aboard Aeolus (Stoffelen et al., 2005; Ansmann et al., 2007) and the ESA–JAXA EarthCARE (Illingworth et al., 2015) are highly dependent on ground-based EARLINET correlative measurements. In addition, EARLINET supports the homogenization of the different satellite missions. CALIOP is a two-wavelength polarization-sensitive lidar that operates at 532 and 1064 nm, while the ESA's ALADIN aboard Aeolus and the ESA–JAXA Atmospheric Lidar (ATLID) aboard EarthCARE operate at 355 nm, and NASA's CATS lidar operates at 532 and 1064 nm in Mode 1 and 1064 nm in Mode 2 (Yorks et al., 2014). EARLINET supports the continuity of satellite lidar missions through the calculation of aerosol-dependent spectral conversion factors between different wavelengths to homogenize different missions at different operating wavelengths in order to provide a long-term 3-D climatic record from space (Amiridis et al., 2015; Chimot et al., 2017; Marinou et al., 2017; Proestakis et al., 2018).

2.3 CATS–EARLINET correlative measurements

2.3.1 Comparison methodology

To obtain a significant number of collocated and concurrent CATS–EARLINET cases, a large number of EARLINET stations contributed to the CATS evaluation activity. Figure 1 shows the geographical distribution of the active EARLINET stations during the study over Europe and Asia, including the daytime and nighttime overpasses of ISS within the evaluation period, between February 2015 and September 2016, encompassing the first 20 months of CATS operation. The green circles denote the stations participating in the CATS–EARLINET intercomparison activity (namely – in alphabetical order – Athens-NOA, Athens-NTUA, Barcelona, Belsk, Bucharest, Cabauw, Dushanbe, Évora, Hohenpeißenberg, Lecce, Leipzig, Potenza, Thessaloniki, and Warsaw). All participating stations operate high-performance multiwavelength lidar systems. Six of the contributing stations (Athens-NOA, Cabauw, Dushanbe, Évora, Leipzig, and Warsaw) are part of the PollyNET subnetwork (http://polly.tropos.de/, last access: 17 September 2019), operating 24/7 portable, remote-controlled multiwavelength-polarization Raman lidar systems (PollyXT; Baars et al., 2016; Engelmann et al., 2016). Due to the geographical distribution of EARLINET stations, the evaluation activity accounts for a large variety of aerosol types (marine, urban, desert dust, and smoke). Table 1 provides the locations of the EARLINET stations contributing to this analysis along with the surface elevation and the respective identification codes.

Table 1 Contributing EARLINET lidar stations, including identification codes, geographical coordinates, and elevation.

In order to quantitatively address the accuracy and representativeness of CATS retrievals, we follow the methodology introduced by EARLINET for CALIOP validation, which is based on correlative independent measurements (Pappalardo et al., 2010). For the validation of spaceborne lidar observations, of fundamental significance is the spatial and temporal variability in the atmospheric scene. The effect of the distance between ground-based lidar measurements and space-based lidar measurements was investigated in the framework of the CALIPSO validation. In particular, EARLINET-based studies attribute an introduced discrepancy of the order of 5 % to the intercompared signal analysis when the horizontal distance between the EARLINET stations and the spaceborne lidar footprint is below 100 km (Mamouri et al., 2009; Mona et al., 2009; Pappalardo et al., 2010; Papagiannopoulos et al., 2016). In the context of the applied validation criteria, we selected CATS measurements within 50 km horizontal distance between the EARLINET stations and the ISS subsatellite overpass position. In addition, the correlative measurements should be as close in time as possible. EARLINET contributed with performed measurements as close in time as possible, typically with a starting time or stopping time of the performed measurements window within 90 min of the ISS overpass. The CATS–EARLINET cases considered in the assessment of the accuracy and representativeness of CATS backscatter coefficient profiles are provided in Table 2, including the name of the EARLINET station, the EARLINET measurements window, the ISS overpass time, and the ISS minimum distance between the corresponding EARLINET station and the lidar footprint of CATS and the daytime and nighttime information.

Table 2 ISS–CATS and EARLINET cases considered in the evaluation process of CATS backscatter coefficient profiles at 1064 nm.

The number of available cases for the intercomparison is subject to a certain number of constraints. First and foremost, the orbital inclination of the ISS does not allow overpasses close to EARLINET stations north of 52.2^∘ latitude. Second, the ISS crossing time and ground track over an area are highly variable, enhancing the probability of the overpass time to fall outside of the predefined common and fixed schedule of EARLINET measurements. In addition, to account for contamination effects of multiple scattering and specular reflection in the intercomparison process, only cloud-free atmospheric scenes are used. Cases with detected cirrus clouds, either at the EARLINET range-corrected signal quick looks or at the ISS–CATS backscatter coefficient profiles and feature type profiles, are not considered in the study. Initially, the presence of clouds is investigated through the implementation of the CATS backscatter coefficient and depolarization time–height images and EARLINET range-corrected signal. Cases for which the retrieval of EARLINET temporally averaged profile is not feasible due to the presence of clouds, and/or CATS cases in which the presence of clouds propagated into the CATS spatially averaged profile, are discarded from the analysis. Regarding CATS, the “Sky_Condition” flag is also used to screen cloudy (no aerosols) and hazy or cloudy (both clouds and aerosols) profiles from the analysis. The “Feature_Type_Score” parameter stored in the Level 2 data was additionally used to remove aerosol cases of medium or low confidence in the comparison process (“Feature_Type_Score” $\ge -\mathrm{1}$). Applying all match-up selection criteria resulted in a total of 47 correlative CATS–EARLINET cases suitable for quantitatively addressing the accuracy and representativeness of the CATS Level 2 backscatter coefficient product at 1064 nm. CATS requirements applied in the methodology are summarized in Table 3.

Table 3 List of CATS quality-assurance thresholds applied in the EARLINET comparison.

2.3.2 Particle backscatter coefficient retrievals from ground-based lidars at 1064 nm

In order to evaluate the CATS Level 2 aerosol backscatter product at 1064 nm, we utilized backscatter coefficient profiles calculated either with the SCC algorithm or, in case of PollyXT lidar systems, with independently developed user-assisted retrieval algorithms (Baars et al., 2016). The EARLINET backscatter coefficient profiles used in this study are calculated with the SCC version 4 algorithm (for the stations that are not part of PollyNET) and with the methodology described in Haarig et al. (2017; for the stations that are part of PollyNET). The SCC algorithm (D'Amico et al., 2015, 2016; Mattis et al., 2016) is developed with the concept of sustaining the homogeneity of aerosol products derived from different EARLINET lidar systems while satisfying the need for coordinated, quality-assured measurements. It consists of five different modules, including one for handling the pre-processing of raw lidar data by applying all the necessary instrumental corrections to the signal and a module for providing the final aerosol optical products, namely the particle backscatter and extinction coefficient. In particular, the SCC algorithm calculates the backscatter coefficient with the iterative method (Di Girolamo et al., 1995), using only the elastic lidar channels. To calculate the b_1064 nm with these methods, an assumption of the lidar ratio value is required (as a profile or a height-independent value, representative of the corresponding atmospheric scene) and the selection or determination of a reference height (R₀), usually chosen at an altitude range with the minimum aerosol contribution. All methods applied within the SCC have been tested against synthetic (Mattis et al., 2016) and real lidar data (D'Amico et al., 2015). The comparison showed that by using only the signal from the elastic channels, the mean relative deviation in the calculation of the aerosol backscatter coefficient at 1064 nm is less than 30  % (Althausen et al., 2009; Baars et al., 2012; Engelmann et al., 2016; Hänel et al., 2012), thus meeting the quality-assurance requirements of EARLINET. None of the lidar systems participating in the present study are equipped with a rotational–vibrational Raman channel excited by the 1064 nm, as, for example, recently reported by Haarig et al. (2017). In the case of PollyXT lidars, for the daytime backscatter coefficient calculations, the Fernald–Klett method (Klett, 1981; Fernald, 1984) is implemented, assuming a height-independent lidar ratio. For the nighttime calculations, the Raman channel at 607 nm is additionally used (Baars et al., 2016). Specifically, the basic lidar equation at 1064 nm can be described by

$$\begin{array}{}\text{(1)}& \begin{array}{rl}& P^{\mathrm{1064}}\left(R\right)=C^{\mathrm{1064}}{\displaystyle \frac{O\left(R\right)}{R^{\mathrm{2}}}}\left({\mathit{\beta }}_{\mathrm{par}}^{\mathrm{1064}}\left(R\right)+{\mathit{\beta }}_{\mathrm{mol}}^{\mathrm{1064}}\left(R\right)\right)\\ & \exp \left(-\mathrm{2}\underset{\mathrm{0}}{\overset{R}{\int }}\left[a_{\mathrm{mol}}^{\mathrm{1064}}\left(r\right)+a_{\mathrm{par}}^{\mathrm{1064}}\left(r\right)\right]\mathrm{d}r\right),\end{array}\end{array}$$

and the corresponding lidar equation at 607 nm can be described by

$$\begin{array}{}\text{(2)}& \begin{array}{rl}& P^{\mathrm{607}}\left(R\right)=C^{\mathrm{607}}{\displaystyle \frac{O\left(R\right)}{R^{\mathrm{2}}}}\left({\mathit{\beta }}_{\mathrm{mol}}^{\mathrm{607}}\left(R\right)\right)\\ & \exp \left(-\underset{\mathrm{0}}{\overset{R}{\int }}\left[a_{\mathrm{mol}}^{\mathrm{532}}\left(r\right)+a_{\mathrm{par}}^{\mathrm{532}}\left(r\right)+a_{\mathrm{mol}}^{\mathrm{607}}\left(r\right)+a_{\mathrm{par}}^{\mathrm{607}}\left(r\right)\right]\mathrm{d}r\right).\end{array}\end{array}$$

A solution for the particle backscatter coefficient at 1064 nm is obtained using the ratio

$$\begin{array}{}\text{(3)}& {\displaystyle \frac{P^{\mathrm{607}}\left(R_{\mathrm{0}}\right)P^{\mathrm{1064}}\left(R\right)}{P^{\mathrm{1064}}\left(R_{\mathrm{0}}\right)P_{{}^{\mathrm{607}}}\left(R\right)}},\end{array}$$

where P⁶⁰⁷ and P¹⁰⁶⁴ stand for the power received from a distance R, with respect to the lidar system, at 607 nm and 1064 nm respectively. The constant C at 607 or 1064 nm contains all range-independent system parameters. The overlap function O(R), which is less than unity for the altitude range where the laser beam is not completely inside the receiving telescope field of view (Wandinger and Ansmann, 2002), is assumed to be identical between the two channels, which is the case for PollyXT systems, which use one beam expander for all three emitted wavelengths. β_mol and β_par represent molecular and particle backscattering respectively, whereas α_mol and α_par are the molecular and particle extinction coefficients.

Figure 2 (a) Nighttime ISS orbit over Athens, Greece, on 1 February 2016 (blue line). The concentric white circles denote regions of 10, 20, 30, 40, and 50 km from the location of PollyXT NOA lidar system (white dot). Red color in the ISS footprint indicates CATS observations within 50 km distance from the NOA PollyXT lidar system. (b) CATS backscatter coefficient at 1064 nm on 1 February 2016, 17:24 UTC. The white box depicts CATS observations used for the profile intercomparison. (c) PollyXT NOA range-corrected signal time-series at 1064 nm. The white box delineates the temporal averaging of the lidar signals (17:45–19:30 UTC), while the red line denotes the ISS overpass at 1 February 2016, 17:24 UTC – the closest distance time. (d) CATS (blue line) and PollyXT NOA (red line) mean profiles and standard deviations of backscatter coefficient at 1064 nm (0–6 km).

Finally, in order to perform the intercomparison between CATS and EARLINET profiles, the high resolution of EARLINET profiles was lowered to match the vertical resolution of CATS profiles (i.e., 60 m). The objective of obtaining profiles of similar vertical resolution was addressed through computing the EARLINET mean backscatter coefficient value from all EARLINET bins within each CATS 60 m backscatter coefficient height range. The computed EARLINET profiles of similar vertical resolution to CATS followed, with high accuracy, the characterizes and tendencies, both qualitative and quantitative, of the initial EARLINET profiles despite the loss of vertical resolution (Iarlori et al., 2015).

2.4 Demonstration of the comparison methodology for a case study over Athens

To illustrate the evaluation methodology for the CATS Level 2 aerosol backscatter coefficient at 1064 nm, a pair of collocated and concurrent CATS and EARLINET lidar observations is shown in Fig. 2. The example refers to a nighttime ISS overpass of the coastal city of Athens, Greece, on 1 February 2016. During that period, the PollyXT NOA system was operating in a 24/7 mode in Athens, at the premises of the National Observatory of Athens, to fulfill the needs of an ACTRIS Joint Research Activity (JRA) related to aerosol absorption (Tsekeri et al., 2018). At the same time, on Monday 1 February 2016, the lidar station operating at the National Technical University of Athens (NTUA) was performing nighttime measurements according to the EARLINET schedule of regular and simultaneous measurements in order to enable Raman extinction retrievals. The closest distances between the CATS footprint of the ISS overpass and the locations of the EARLINET-at (NTUA) and EARLINET-no (NOA) stations were approximately 18.58 and 23.3 km at 17:24 UTC respectively (Fig. 2a). The vertical distribution of aerosols and clouds is shown in the CATS 1064 nm backscatter coefficient quick look (Fig. 2b) and the PollyXT NOA lidar range-corrected signal at 1064 nm between 1 February 2016 at 12:00 UTC and 2 February 2016 at 00:00 UTC (Fig. 2c). The temporal averaging window of the ground-based lidar signal is shifted to a few minutes after the ISS overpass (17:45–19:30 UTC), due to routine and automatic depolarization calibration measurements conducted with PollyXT NOA system at the exact time of the overpass (Engelmann et al., 2016), while for the EARLINET-at system the temporal averaging window between 18:20:51 and 19:57:41 UTC was used. Both CATS and PollyXT NOA quick looks advocate for the horizontal and vertical homogeneity of the scene. For the comparison of CATS and EARLINET observations, the latest are regridded to the CATS Level 2 vertical resolution (60 m). Accordingly, the CATS spatially averaged and the EARLINET (NOA–NTUA) temporally averaged backscatter coefficient profiles are qualitatively compared (Fig. 2d). The observed disagreements between the two EARLINET profiles are related to differences between the two systems, the different surface elevations of the locations of the two stations (86 m for EARLINET-no and 212 for EARLINET-at), and the different overlap regions. The horizontal bars in the CATS profile (Fig. 2d) correspond to the standard deviation of the spatially averaged backscatter coefficient profiles.

Figure 3 (a) Nighttime ISS orbit over the EARLINET Leipzig station on the 13 September 2016 at 03:37:49 UTC and of closest distance between the footprint of CATS and the EARLINET Leipzig station of 3.79 km. The white dot denotes the location of Leipzig lidar system, while the blue line shows the lidar footprint of CATS. (b) CATS backscatter coefficient at 1064 nm. (c) CATS (blue line) spatially and EARLINET Leipzig (red line) temporally averaged backscatter coefficient profiles (1064 nm). The implemented EARLINET Leipzig time window of cloud-free measurements was between 00:00:00 and 02:30:00 UTC. The horizontal blue and red lines denote the variability (1 standard deviation) in the CATS- and EARLINET-measured atmospheric scenes respectively.

The comparison of the mean backscatter coefficient profiles retrieved by CATS and the two corresponding EARLINET NOA and NTUA profiles presented in Fig. 2 is an initial demonstration of the good agreement between the two products. The CATS instrument reproduces the observed aerosol features in terms of aerosol load as well as their vertical distribution (Fig. 2d). The assessment of CATS backscatter coefficient is performed in the region between 0.5 km a.g.l. of the EARLINET sites to account for overlap effects between the laser beam and the telescope (Wandinger and Ansmann, 2002), topographic effects, surface returns, and differences of atmospheric samples within the planetary boundary layer (Fig. 2d – shaded area iii) and 10 km a.s.l.. An upper limit of 2 Mm⁻¹ sr⁻¹ is applied to the aerosol backscatter coefficient values in order to account for cloud features possibly misclassified as aerosols (Fig. 2d – shaded area ii). Finally, cases of EARLINET backscatter coefficient values below the CATS minimum detectable backscatter limit at 1064 nm are not included in the comparison when the corresponding CATS backscatter coefficient is reported to be zero (Fig. 2d – shaded area i). The latter constraint is applied to account for very thin detected layers from ground-based lidar systems with backscatter values below the CATS minimum detection limit due to the low signal-to-noise ratio (SNR) values. The discussed constraints are employed because of our basic objective to quantitatively assess the representativeness and accuracy of the detected CATS aerosol features while preventing possible contaminations (e.g., presence of clouds) from propagating into the CATS–EARLINET dataset.

3 Results

3.1 CATS–EARLINET correlative cases

To illustrate strengths and limitations of CATS products, we discuss in detail three selected cases of collocated and concurrent CATS–EARLINET observations close to the (EARLINET) stations of Leipzig, Évora, and Dushanbe. The three study cases represent different atmospheric conditions with an increasing degree of difficulty in the detection of representative aerosol layers by CATS.

3.1.1 Case I: ISS–CATS over Leipzig – 13 September 2016 at 03:37 UTC

The first overpass considered here shows a representative case study of a nighttime ISS orbit on 13 September 2016 (blue line), at a minimum distance of 3.78 km from the EARLINET Leipzig, Germany, PollyXT lidar system (indicated by a white dot), at 03:37 UTC (Fig. 3a). The CATS particulate backscatter coefficient cross section at 1064 nm (Fig. 3b) shows the presence of aerosols up to 2.6 km a.s.l. The CATS feature-mask algorithm classifies all of the detected layers as aerosols (not shown). The ground-based lidar measurements at the Leipzig station between 00:00 and 12:00 UTC did not report any cloud features either, including cirrus clouds. CATS spatially averaged and Leipzig temporally averaged profiles were derived from CATS profiles within horizontal distance of 50 km, between the Leipzig station and the ISS footprint, and Leipzig measurements within 90 min of the ISS overpass respectively (Fig. 3c). The direct comparison of the backscatter coefficient profiles, measured from the EARLINET Leipzig station (red line) and CATS (blue line), along with their standard deviations (horizontal error bars), indicate also the presence of aerosol up to 2.6 km a.s.l. The intercompared profiles between ISS–CATS and EARLINET Leipzig station are characterized by high agreement, although discrepancies are also present. To the uppermost part of the profiles, between 2.5 and 3 km a.s.l., due to the higher SNR, the Leipzig lidar is capable of detecting tenuous atmospheric features of low backscatter coefficient values. Although the case presented and discussed in Fig. 3 corresponds to a nighttime ISS overpass, the case is representative for cloud-free and relative homogeneous atmospheric scenes in terms of aerosols, for both daytime and nighttime solar background illumination, demonstrating the overall high performance of CATS under such conditions.

3.1.2 Case II: ISS–CATS over Évora – 31 May 2016 at 19:43 UTC

Small biases between the EARLINET and CATS backscatter coefficient are also identified in specific cases. CATS particulate backscatter coefficient profiles are available for the identified atmospheric features and not as full profiles, as in the case of the attenuated backscatter profiles. The feature classification algorithm, assuming no cloud or aerosol layers are detected and no overlaying opaque layers are present, classifies the atmospheric layers as clear air. Clear-air segments though are not pristine and aerosol-free, as they frequently contain tenuous particulate layers (Kim et al., 2018). Layers of atmospheric features that are not detected contain either fill values (0.0 km⁻¹ sr⁻¹) or are marked as invalid in cases when the calculation of the particulate backscatter coefficients was not possible (−999.9). This scheme of assigning appropriate backscatter coefficient values to the detected atmospheric features (e.g., aerosol and clouds) propagates through many of the Level 2 products in the comparison of CATS Level 2 data and thus in the assessment of the representativeness of CATS observations. Consequently, the comparison of CATS Level 2 backscatter coefficient profiles against EARLINET observations is only possible over the detected atmospheric features. In addition, the identification of the atmospheric features strongly depends on the calibrations of the CATS lidar system and to the level of the background signal – solar illumination conditions – due to the different SNRs between daytime and nighttime.

Figure 4 (a) Daytime ISS orbit over the EARLINET Évora station on the 31 May 2016 at 19:43:31 UTC and of closest distance between the footprint of CATS and the EARLINET Évora station of 39.42 km. The white dot denotes the location of Évora lidar system, while the red line shows the lidar footprint of CATS. (b) CATS backscatter coefficient at 1064 nm. (c) CATS (blue line) spatially and EARLINET Évora (red line) temporally averaged backscatter coefficient profiles (1064 nm). The implemented EARLINET Évora time window of cloud-free measurements was between 19:29:56 and 19:59:35 UTC. The horizontal blue and red lines denote the variability (1 standard deviation) in the CATS- and EARLINET-measured atmospheric scenes respectively.

Figure 4 shows a daytime ISS match-up, on 31 May 2016 (red line), at a minimum distance of 39.4 km from the EARLINET station of Évora, Portugal (indicated by a white dot) at 19:43:41 UTC, during a time window of cloud-free atmospheric conditions (Fig. 4a). The CATS particulate backscatter coefficient cross section at 1064 nm (Fig. 4b) shows the absence of aerosol and/or cloud features, while the Évora temporally averaged profile during the cloud-free window (Fig. 4c) indicates the presence of thin aerosol layers in the altitude range between 1 and 2.5 km a.s.l. The aerosol layer detected by the Évora PollyXT lidar system is characterized by backscatter coefficient values lower than 0.3 Mm⁻¹ sr⁻¹. Although CATS is characterized by relatively low minimum detection thresholds (Yorks et al., 2016), CATS capabilities are limited in terms of detecting similarly tenuous aerosol layers at levels that lie below the detection thresholds (e.g., CATS 7.2 minimum detectable backscatter 1064 nm – night: $\mathrm{5.00}\times {\mathrm{10}}^{-\mathrm{5}}\pm \mathrm{0.77}\times {\mathrm{10}}^{-\mathrm{5}}$ km⁻¹sr⁻¹; day: $\mathrm{1.30}\times {\mathrm{10}}^{-\mathrm{3}}\pm \mathrm{0.24}\times {\mathrm{10}}^{-\mathrm{3}}$ km⁻¹ sr⁻¹ – for cirrus clouds; Yorks et al., 2016). The detection limitation of CATS may propagate in scientific studies implementing CATS through introduced underestimations and possible biases.

3.1.3 Case III: ISS–CATS over Dushanbe – 25 May 2015 at 18:53 UTC

The assessment of accuracy of CATS Level 2 against EARLINET collocated and concurrent observations is performed on the basis of backscatter coefficient profiles because this product constitutes the CATS Level 2 parameter with the lowest influence of a priori assumptions (e.g., lidar ratio). In addition CATS Level 2 provides the feature classification of the detected layers and associated confidence level of the classification. The cloud–aerosol discrimination though is not performed perfectly. Thus misclassified aerosol layers may be classified as clouds and vice versa. In the framework of the study, for the assessment process of the CATS Level 2 aerosol quality, strict cloud filtering is applied. In particular, cloud-contaminated profiles (sky condition 2 and 3) and aerosol layers characterized by medium or low classification confidence (Feature_Type_Score $\ge -\mathrm{1}$) are filtered. The strict cloud screening is applied because of our basic idea to establish the accuracy of CATS aerosol backscatter coefficient profiles based on intercomparison against EARLINET, preventing any contamination of cloud features from propagating into the dataset.

Figure 5 (a) Nighttime ISS orbit over the EARLINET Dushanbe station on the 25 May 2015 at 18:53:19 UTC and of closest distance between the footprint of CATS and the EARLINET Dushanbe station of 24.3 km. The white dot denotes the location of Dushanbe lidar system, while the blue line shows the lidar footprint of CATS. (b) CATS backscatter coefficient at 1064 nm. (c) CATS (blue line) spatially and EARLINET Dushanbe (red line) temporally averaged backscatter coefficient profiles (1064 nm). The implemented EARLINET Dushanbe time window of cloud-free measurements was between 18:00:00 and 20:00:00 UTC. The horizontal blue and red lines denote the variability (1 standard deviation) in the CATS- and EARLINET-measured atmospheric scenes respectively.

As discussed in the case of Leipzig overpass, on average, the agreement between CATS Level 2 backscatter coefficient profiles and EARLINET is good, especially under relative homogeneous cloud-free atmospheric conditions. Under complex atmospheric conditions, though, of coexistent and adjacent aerosol and cloud features, the impact of the CATS feature type score on the CATS aerosol retrievals becomes significant. Figure 5 shows the CATS footprint for the nighttime ISS orbit, on 25 May 2015 (blue line), at a minimum distance of 24.3 km from the EARLINET Dushanbe, Tajikistan, station (Hofer et al., 2017), at 18:53:19 UTC (Fig. 5a). This EARLINET station is located in a natural basin surrounded by mountain ridges of variable height, between 0.7 and 4 km a.s.l. The CATS particulate backscatter coefficient cross section at 1064 nm (Fig. 5b) shows the predominant presence of aerosols up to 3.6 km a.s.l., adjusted to broken thin clouds. These cloud characteristics though are not consistent with the observations performed at Dushanbe station between 13:00 and 23:00 UTC on 25 May 2015 that reported the absence of cloud features below 6 km. CATS lidar profile and the EARLINET Dushanbe profile yield different behavior in terms of backscatter coefficient (Fig. 5c). The Dushanbe lidar reports a weak presence of aerosols, up to approximately 4 km a.s.l. The backscatter comparison against CATS profile reveals enhanced discrepancies in segments of the CATS profile, denoted by the high backscatter coefficient values (> 2 Mm⁻¹ sr⁻¹). The cloud features that cause the observed discrepancies are classified by the CATS CAD algorithm as aerosol layers, contaminating the CATS profile despite the strict cloud screening. Features with an invalid CATS CAD score, although not frequently observed, may impact the quality of the column AOD and related climatological studies. In addition, complex topography in terms of geographical characteristics, erroneous mean backscatter coefficient profiles due to the high variability in aerosol load in the planetary boundary layer, the horizontal distance between the CATS lidar footprint and the ground-based lidar stations and surface returns further enhance these discrepancies, especially in the lowermost part of the profiles. Based on this analysis and comparisons with CALIPSO, the CATS cloud–aerosol discrimination algorithm was updated for the V3-00 Level 2 data products (released in the end of 2018) to improve the accuracy of the feature type and feature type score, especially during daytime.

3.2 CATS–EARLINET comparison statistics

In this section an overall assessment of the CATS backscatter coefficient product at 1064 nm is given, using the entire dataset of CATS–EARLINET collocated profiles. To address quantitatively the accuracy and representativeness of the satellite-based lidar retrievals, the estimation of possible biases in the CATS backscatter coefficient is performed. Towards this assessment, in the comparison of CATS against EARLINET, we implement the CATS_i EARLINET_i residuals for each pair of observations “i”, as a statistical indicator of CATS average overestimation or underestimation of the aerosol load, in terms of backscatter coefficient values.

Figure 6 shows the distributions of CATS_i EARLINET_i backscatter coefficient differences. On average, the agreement is good, demonstrating the high performance of CATS, with mean and median residual values close to zero and typically within 0.4 Mm⁻¹ sr⁻¹. The intercomparison between CATS satellite-based and EARLINET ground-based lidar retrievals reveals the presence of negative biases in the CATS 1064 nm backscatter coefficient profiles. The CATS_i EARLINET_i differences, for all the available 21 daytime (Fig. 6a) and 26 nighttime (Fig. 6b) cases of paired correlative observations, show an underestimation of the CATS retrievals, being more pronounced during daytime than nighttime. In the case of daytime observations, the calculated mean (median) CATS difference from EARLINET is −0.123 Mm⁻¹ sr⁻¹ (−0.095 Mm⁻¹ sr⁻¹). In the case of nighttime observations, the corresponding mean (median) difference from EARLINET is −0.031 Mm⁻¹ sr⁻¹ (−0.065 Mm⁻¹ sr⁻¹). The observed standard deviation (SD) is 0.431 Mm⁻¹ sr⁻¹ over daytime and 0.342 Mm⁻¹ sr⁻¹ during nighttime. During daytime, minimum and maximum CATS–EARLINET residual values of −1.802 and 1.189 Mm⁻¹ sr⁻¹ are observed, while the corresponding minimum and maximum values for nighttime are −1.348 and 1.149 Mm⁻¹ sr⁻¹. The CATS_i EARLINET_i daytime mean absolute bias and root-mean-square error (RMSE) statistical indicators (Binietoglou et al., 2015) of daytime observations are 0.323 and 0.448 Mm⁻¹ sr⁻¹, while the respective statistical indicators for the nighttime cases are 0.249 and 0.343 Mm⁻¹sr⁻¹. CATS performance is also quantified through the linear correlation coefficient between the CATS and EARLINET backscatter coefficient distributions, with correlation coefficients of 0.54 and 0.69 during daytime and nighttime respectively. The correlations between CATS and EARLINET distributions are not very good, as expected due to the significant influence of the topography, the high inhomogeneities within the local planetary boundary layer (PBL), and the effect of the horizontal distance and temporal measurement differences. The fractional bias values for daytime and nighttime are −0.676 and −0.773 respectively, while the fractional gross error ranges between 1.061 for daytime and 0.999 for nighttime cases. Overall, the agreement between CATS and EARLINET is good. On average though, slight underestimations of CATS compared to EARLINET are observed: 6.3 % during nighttime and 22.3 % during daytime. The intercomparison statistical values between CATS and EARLINET are summarized in Table 4.

Figure 6 Distributions of the differences between CATS Level 2 and the corresponding EARLINET backscatter coefficient measurements, calculated for (a) daytime (21 collocated cases) and (b) nighttime (26 collocated cases).

Table 4 CATS–EARLINET comparison statistics on mean bias, median, mean absolute bias, standard deviation, root-mean-square error (RMSE), and minimum and maximum values on the observed backscatter coefficient profiles at 1064 nm (Mm⁻¹ sr⁻¹) for daytime and nighttime correlative cases.

Figure 7 reports the mean aerosol backscatter coefficient profiles at 1064 nm as provided by CATS and EARLINET daytime (Fig. 7a) and nighttime (Fig. 7b) lidar observations. On average, the mean aerosol backscatter coefficient profiles reveal similar characteristics between CATS and EARLINET, although the comparisons are subject to the different number of available cases: 21 and 26 for daytime and nighttime respectively. Both CATS and EARLINET daytime and nighttime backscatter coefficient profiles yield higher values close to the surface level, gradually decreasing with altitude. Especially in the range between the full overlap region of the laser beam and the telescope of the EARLINET systems (approximately 1 km) and the middle free troposphere (∼6 km a.s.l.), the mean backscatter coefficient profile of CATS is well within the standard deviation of the scenes provided by EARLINET. Nonetheless, discrepancies are also evident. CATS, as a result of the high spatial atmospheric variability, yields usually higher values of standard deviation than EARLINET. In addition, at altitudes higher than 6 km a.s.l., the CATS mean backscatter coefficient profile yields zero or close-to-zero values, while EARLINET shows the presence of elevated aerosols, with rather low mean backscatter values, which are lower than 0.2 Mm⁻¹ sr⁻¹.

Figure 7 CATS (blue line) and EARLINET (red line) mean profiles of backscatter coefficient at 1064 nm for (a) daytime and (b) nighttime. The horizontal lines represent the SD of CATS (blue colour) and EARLINET (red colour) profiles.

The CATS Level 2 backscatter coefficient product evaluation study shows that CATS agrees reasonably well with ground-based EARLINET measurements, although they are generally biased low. To assess the ability of the CATS lidar to detect aerosol features and optical properties and to shed light on the origin of observed CATS–EARLINET discrepancies, the conducted CALIOP validation studies offer an unprecedented basis. This is due to the similar viewing geometry between CATS and CALIOP and to the similarities between Level 1B and Level 2 processing algorithms (McGill et al., 2015; Yorks et al., 2016, 2019).

Since CALIPSO joined the A-Train constellation of Earth observation satellites in June 2006 (Winker et al., 2007), several studies have been conducted to validate and evaluate CALIOP Level 1B, Level 2, and Level 3 products against ground-based, airborne, and spaceborne measurements. Airborne NASA Langley HSRL (Hair et al., 2008) and CPL (McGill et al., 2002) flights, of close spatial and temporal coincidence with the CALIPSO satellite, documented the high performance of CALIOP, although with the presence of low negative biases (Burton et al., 2010, 2013; McGill et al., 2007; Rogers et al., 2011, 2014). Kacenelenbogen et al. (2014) reports on the detection of aerosols above cloud (AAC) in only 151 of 668 CALIOP HSRL coincident airborne cases (23 %). The use of ground-based Raman lidar observations also reports that CALIOP Level 1B and Level 2 products are biased low (Mamouri et al., 2009; Mona et al., 2009; Pappalardo et al., 2010; Tesche et al., 2013). In terms of columnar measurements, the conducted validation activities based on collocated observations between CALIOP and AERONET (Dubovik et al., 2000) showed CALIPSO AOD underestimations (Amiridis et al., 2013; Omar et al., 2013; Schuster et al., 2012). In addition, evaluation studies of AOD observations from the passive spaceborne MODIS (Remer et al., 2005) show that CALIOP provides reasonably well-known climatic features, although with apparent AOD underestimations (Amiridis et al., 2013; Kittaka et al., 2011; Oo and Holz, 2011; Redemann et al., 2012). The magnitude of the documented agreements and biases in the detection of aerosol features varies from study to study with respect to the different CALIOP versions. Substantial improvement in the detection of aerosol features is documented in the latest CALIPSO version 4 (AMT CALIPSO special issue).

4 Discussion

Overall, CATS, much like CALIOP, observes the vertical distribution of atmospheric aerosol backscatter coefficient reasonably well, although with slight underestimations. The observed discrepancies in the compared CATS–EARLINET profiles are attributed to several sources.

First, the retrieval accuracy of CATS Level 2 data products, such as the aerosol and cloud backscatter and extinction coefficient profiles, the vertical feature mask, and the integrated parameters (e.g., AOD), depends crucially on the calibration of the lidar system and the calibration region (Kar et al., 2018). CATS total attenuated backscatter from molecules and particles in the atmosphere is performed in the calibration region between 22 and 26 km, starting with V2-08 of the Level 1B data (Russell et al., 1979; Del Guasta, 1998; McGill et al., 2007; Powell et al., 2009). Uncertainties in the CATS Level 1B backscatter calibration are attributed to random and systematic errors (CATS Algorithm Theoretical Basis Document – ATBD). Random errors result mainly from normalizing the 1064 nm lidar signal to modeled molecular signal and are dominated by lidar noise. On the contrary, systematic errors result from a number of different sources, including uncertainties in the CALIOP stratospheric scattering ratios and molecular backscatter coefficient values generated from the Goddard Earth Observing System (GEOS) atmospheric general circulation model and assimilation system used to calculate molecular and ozone atmospheric transmission (Rienecker et al., 2008) and from the non-ideal performance of CATS. The total uncertainty due to the CATS calibration constants is estimated at between 5 % and 10 % (CATS ATBD).

Secondly, CATS detection and classification schemes, similar to CALIOP, provide Level 2 aerosol products only in regions where aerosol features are detected and identified. This implies that optically thin aerosol layers can go undetected by CATS due to weak backscattering intensities below the CATS detection thresholds (Kacenelenbogen et al., 2014; Thorsen et al., 2015). To increase the detection of tenuous aerosol layers, CATS incorporates an iterated horizontal averaging scheme (5 and 60 km; Yorks et al., 2019). Failures of spaceborne lidar instruments and algorithms to detect tenuous aerosol layers (Toth et al., 2018) result in range bin backscatter coefficient assignments to 0.0 Mm⁻¹ sr⁻¹. The faint undetected aerosol layers do not contribute to the CATS aerosol backscatter profiles and, consequently, neither contribute to extinction coefficient profiles nor to estimates of CATS AOD, which is similar to CALIOP AOD (Kim et al., 2018; Rogers et al., 2014; Thorsen and Fu, 2015). The detection sensitivity is attributed to the solar background and sunlight illumination conditions due to the significantly lower CATS SNR during daytime than nighttime (Rogers et al., 2014). The undetected aerosol layers, although of low aerosol load, introduce negative biases in the CATS–EARLINET comparison. The total uncertainty, the sum of the systematic and random errors, in the CATS ATBD at 1064 nm is estimated at 10 %–20 % for nighttime data and 20 %–30 % for daytime data.

Another source of discrepancy between CATS and EARLINET is attributed to the effect of horizontal distance between the ground-based lidar systems and the space-based lidar footprint. Studies performed in the framework of EARLINET attribute an introduced discrepancy of the order of 5 % to the intercompared profiles, when the horizontal distance is below 100 km (Mamouri et al., 2009; Pappalardo et al., 2010; Papagiannopoulos et al., 2016). The different – opposite – viewing geometry (upward for EARLINET and downward for CATS and CALIPSO) and the different transmittance terms are further sources of discrepancies (Mona et al., 2009). In addition, enhanced disagreements observed between CATS and EARLINET in the lowermost part of the mean backscatter coefficient profiles are attributed to the high spatial and temporal variability in the aerosol content within the PBL, to the complexity of the local topography, and to surface returns.

Finally, regarding the utility of CATS in climatic studies, another common reason of satellite-based lidar overestimations or underestimations is attributed to the absence of detailed aerosol properties in the classification of the detected aerosol layers. The aerosol-subtype classification scheme frequently results in aerosol layer misclassifications, as has been shown in the case of coincident HSRL CALIPSO under-flights (Burton et al., 2012). Misclassified aerosol layers incorporate erroneous values of lidar ratio. Possible underestimation or overestimation of aerosol backscatter coefficient profiles, considered with erroneous aerosol-subtype classification, introduces biases in corresponding extinction coefficient profiles and eventually in total columnar AOD retrievals. The CATS V3.00) Level 2 data products improve errors in cloud–aerosol typing identified in these CATS–EARLINET comparisons. Furthermore, Wandinger et al. (2010), based on CALIOP extinction coefficient profiles in case of dust aerosol layers and collocated ground-based Raman lidar measurements, showed that multiple-scattering effects can result in negative biases if not considered in the algorithm inversion schemes. Data users should be aware of these multiple-scattering effects and cloud–aerosol typing errors when using the CATS data.

5 Summary and conclusions

This study implements independent retrievals carried out at several EARLINET stations to qualitatively and quantitatively assess the performance of NASA's CATS lidar operating aboard the ISS from February 2015 to October 2017. We compared satellite-based CATS and ground-based independent measurements over 12 high-performance EARLINET stations across Europe and one located in Central Asia. Our analysis is based on the first 20 months of CATS operation (February 2015–September 2016). Comparison of CATS Level 2 and EARLINET backscatter coefficient profiles at 1064 nm is allowed only in cases in which the maximum distance between the ISS overpass and the EARLINET stations is below 50 km. EARLINET contributed with observations as close in time as possible, typically with a starting time or stop time of the measurements within 90 min of the ISS overpass. The analysis was restricted to cloud-free profiles to avoid possible cloud contamination of the intercompared aerosol backscatter coefficient profiles.

In the quantitative assessment of the performance of CATS, 47 collocated, concurrent, and cloud-free measurements of CATS the EARLINET were identified (21 daytime and 26 nighttime), offering a unique opportunity for the evaluation of the spaceborne lidar system. The results of the generic comparison are encouraging, demonstrating the overall good performance of CATS, although with negative biases. The agreement, as expected due to higher SNR, is better during nighttime operation, with observed underestimation of 22.3 % during daytime and 6.1 % during nighttime respectively.

In addition to the generic comparison, three CATS–EARLINET comparison cases were examined to demonstrate the system's performance under different study conditions. The comparison showed that under cloud-free, relative homogeneous atmospheric aerosol conditions, the spatially averaged CATS backscatter coefficient profiles are in good agreement with EARLINET, independent of light conditions. The deficiency of CATS though to detect tenuous aerosol layers, due to the inherent limitations of space-based lidar systems, may lead to systematic deviations and slight underestimations of the total AOD in climatic studies. In addition, the CATS V2.01 feature type score misclassification of aerosol layers as clouds, and vice versa, in cases of coexistent and/or adjacent aerosol and cloud features, may lead to non-representative, unrealistic, and cloud-contaminated aerosol profiles. While CATS feature identification will improve in V3.01 data products, the most crucial reason for the observed discrepancies between CATS and EARLINET in the lowermost part of the profiles is related to the complexity of the topography and the geographical characteristics. Especially in the case of large elevation or slope differences, the effects of both inadequate sampling lower than the maximum elevation and of the different atmospheric sampling volumes result in large AOD biases and unrealistic AOD values.

The qualitative and quantitative agreement between CATS and EARLINET reported in this study is encouraging, especially during nighttime; this is agreement that will hopefully facilitate further studies implementing CATS observations in the future. CATS, for a period of almost three years, provided an unprecedented global dataset of vertical profiles of aerosols and clouds, much like CALIOP, taking advantage, though, of the unique orbital characteristics of the ISS. The ISS enabled CATS to provide for the first time satellite-based lidar measurements of the diurnal evolution of aerosols and clouds over the tropics and midlatitudes and to be more specific to latitudes below 52^∘. Since CALIPSO and Aeolus (and in the future also EarthCARE) are polar sun-synchronous satellites of a fixed equatorial crossing time (01:30 and 13:30 LT for CALIOP; 06:00 and 18:00 LT for ALADIN), it is expected that, at least for the near future, the CATS dataset will remain the only available satellite-based lidar source of nearly global diurnal measurements of atmospheric aerosols and clouds. In addition, while CALIOP is a two-wavelength lidar system operating at 532 and 1064 nm with depolarization capabilities at 532 nm, CATS provided satellite-based aerosol and cloud depolarization profiles at 1064 nm, thus in a different wavelength. This dataset, much like the CALIOP dataset, is especially useful for studies of the three-dimensional distribution of non-spherical aerosol particles in the atmosphere (e.g., mineral dust and volcanic ash) and, especially since it is an active sensor, over regions of high reflectivity (e.g., deserts and ice). Future studies including the exploitation of CATS unique observations may help the scientific community shed new light on physical processes of aerosols and clouds in the Earth's atmosphere.