An evaluation of the liquid cloud droplet effective radius derived from MODIS, airborne remote sensing, and in situ measurements from CAMP²Ex

The cloud drop effective radius (R_e) of the drop size distribution derived from passive satellite sensors is a key variable used in climate research. Validation of these satellite products has often taken place under stratiform cloud conditions that favor the assumption of cloud horizontal homogeneity used by the retrieval techniques. However, many studies have noted concerns with respect to significant biases in retrieved R_e arising from cloud heterogeneity, for example, in cumulus cloud fields. Here, we examine data collected during the 2019 “Cloud, Aerosol and Monsoon Processes Philippines Experiment” (CAMP²Ex), which, in part, targeted the objective of providing the first detailed evaluation of R_e retrieved across multiple platforms and techniques in a cumulus and congestus cloud region. Our evaluation consists of cross-comparisons of R_e between the Moderate Resolution Imaging Spectroradiometer (MODIS) onboard the Terra satellite, the Research Scanning Polarimeter (RSP) onboard the NASA P-3 aircraft, and in situ measurements from both the NASA P-3 and Learjet aircraft that are all taken in close spatiotemporal proximity to the same cloud fields. A particular advantage of our approach lies in the capability of the RSP to retrieve R_e using a bi-spectral MODIS approach and a polarimetric approach, which allows for the evaluation of bi-spectral and polarimetric R_e retrievals from an airborne perspective using the same samples.

Averaged over all P-3 flight segments examined here for warm clouds, the RSP polarimetric method, the in situ method, and the bias-adjusted MODIS method of Fu et al. (2019) show a comparable median (mean ± standard deviation) for the R_e samples of 9.6 (10.2 ± 4.0) µm, 11.0 (13.6 ± 11.3) µm, and 10.4 (10.8 ± 3.8) µm, respectively. These values are far lower than the values of 15.1 (16.2 ± 5.5) µm and 17.2 (17.7 ± 5.7) µm from the bi-spectral retrievals of RSP and MODIS, respectively. Similar results are observed when R_e is segregated by cloud-top height and in detailed case studies. The clouds sampled during CAMP²Ex consist of mostly small (mean transect length ∼ 1.4 km) and low clouds (mean cloud-top height ∼ 1 km), which had more numerous small clouds than the trade wind cumuli sampled in past field campaigns such as Rain in Shallow Cumulus over the Ocean (RICO) and the Indian Ocean Experiment (INDOEX). The overestimates of R_e from the RSP bi-spectral technique compared with the polarimetric technique increased as cloud size and cloud optical depth decreased. Drizzle, cloud-top bumpiness, and solar zenith angle, however, are not closely correlated with the overestimate of bi-spectral R_e. For shallow clouds that dominated the liquid cloud cover for the CAMP²Ex region and period, we show that 3-D radiative transfer and cloud heterogeneity, particularly for the optically thin and small clouds, appear to be the leading cause of the large positive biases in bi-spectral retrievals. Because this bias varies with the underlying structure of the cloud field, caution continues to be warranted in studies that use bi-spectral R_e retrievals in cumulus cloud fields.

1 Introduction

Satellite-retrieved cloud properties have been critical in advancing the understanding of the role of clouds in the Earth's climate system. However, the role of clouds in a changing climate still remains a dominant source of uncertainty in climate change predictions (IPCC, 2013). Efforts to improve the accuracy of our satellite record of cloud properties continue to be called for (Ohring et al., 2005; NASEM, 2018). This includes the record of the cloud droplet effective radius (R_e) of the drop size distribution. Satellite-retrieved R_e, owing to its wide spatial coverage and continuous monitoring record, has been applied for a wide range of studies, such as estimating aerosol–cloud interactions (e.g., Menon et al., 2008; Ross et al., 2018; IPCC, 2013) and evaluating model parameterizations (e.g., Ban-Weiss et al., 2014; Suzuki et al., 2013). The (by far) most dominant approach for retrieving R_e from space has been based on the bi-spectral technique of Nakajima and King (1990), which simultaneously retrieves cloud optical thickness (COT) and R_e from visible–near-infrared (VNIR) and shortwave infrared (SWIR) radiances. It has been applied to sensors such as the Advanced Very High Resolution Radiometer (AVHRR; Rossow and Schiffer, 1991), the Moderate Resolution Imaging Spectroradiometer (MODIS; Platnick et al., 2003), and newer sensors such as the Visible Infrared Imaging Radiometer Suite (VIIRS; Cao et al., 2014) and the Advanced Himawari Imager (AHI; Bessho et al., 2016). Therefore, the longest records (spanning nearly 4 decades) of observations for cloud optical and microphysical properties are derived from the bi-spectral technique. Given its legacy and likely continued use in the future, it is essential to assess the error characteristics of the bi-spectral approach to advance the understanding of climate science, particularly as it applies to cloud feedbacks (e.g., Tan et al., 2019) and aerosol–cloud interactions (e.g., Menon et al., 2008; Gryspeerdt et al., 2019).

There have been numerous studies aimed at understanding the error characteristics of R_e retrieved using the bi-spectral technique. The largest errors are expected to occur whenever nature substantially deviates from the assumptions used by the aforementioned technique, such as horizontally homogeneous clouds (i.e., 1-D radiative transfer is used as the forward model in this retrieval algorithm), vertically homogeneous clouds, and a single-mode drop size distribution. Evaluations of R_e from past field campaigns (e.g., Nakajima et al., 1991; Platnick and Valero, 1995; Painemal and Zuidema, 2011; McBride et al., 2012; Witte et al., 2018) show a ∼ −0.2 to 3 µm (∼ −2 % to 40 %) bias for MODIS and MODIS-like instruments, mostly for marine stratiform clouds under high sun conditions – conditions that are most favorable for the 1-D assumption (e.g., Loeb et al., 1998; Di Girolamo et al., 2010). Moreover, 3-D radiative transfer simulations suggest larger biases in the cumulus cloud fields that can reach ∼ 100 % (e.g., Marshak et al., 2006), with the bias closely related to cloud heterogeneity and solar zenith angles (SZAs). Under low sun conditions, Ahn et al. (2018) recently compared MODIS R_e with airborne in situ measurements over the Southern Ocean and reported a bias of 8–13 µm for non-drizzling clouds. A global perspective of the bias in MODIS R_e was provided by Liang et al. (2015), who estimated zonal mean biases ranging from 2 to 11 µm by fusing data from MODIS and the Multi-angle Imaging SpectroRadiometer (MISR; Diner et al., 1998). Their approach was further extended to regional estimates of the bias across the globe by Fu et al. (2019), who showed dependence of the R_e bias on the cloud regime (i.e., larger bias in more cumuliform regimes). Fu et al. (2019) showed that the largest R_e biases (up to +10 µm) occur over the western tropical Pacific, which is also curiously the region where MODIS pixels detected as cloudy have the largest failures rates (up to 40 %) in retrieving cloud optical and microphysical properties (Cho et al., 2015). As liquid-water clouds in this region are dominated by cumulus and cumulus congestus clouds, a field campaign that, in part, targets the evaluation of R_e retrievals for these clouds was warranted.

The “Cloud, Aerosol and Monsoon Processes Philippines Experiment” (CAMP²Ex; Di Girolamo et al., 2015), which took place in the Philippines and its surrounding waters from August to October of 2019, offers an opportunity to evaluate and understand satellite-derived cloud optical and microphysical properties in a heterogeneous environment. Remote sensing and in situ measurements of the clouds and aerosol fields were retrieved by the NASA P-3 and Learjet aircraft platforms. In this study, we focus on evaluating remotely sensed R_e retrievals for warm cumulus and congestus clouds sampled during CAMP²Ex. Over the past several decades, satellite retrievals have not been evaluated in cumulus cloud fields, largely because of the difficulties involved with doing so. The fast-changing nature and complex cloud-top structures of these clouds pose challenges for good cloud-top coordination between satellite observations and airborne/in situ measurements. CAMP²Ex provided tight coordination between Terra overpasses and the P-3 aircraft that carried the Research Scanning Polarimeter (RSP; Cairns et al., 1999). The RSP provides bi-spectral and polarimetric retrievals of R_e. The polarimetric R_e is retrieved from multi-angle polarized radiances that are sensitive to single scattering. Past studies have indicated that the accuracy of polarimetric retrievals is less affected by the assumptions of plane-parallel and homogeneous clouds than the bi-spectral technique (Bréon and Doutriaux-Boucher, 2005; Alexandrov et al., 2012, 2015). In this study, we rely on the RSP polarimetric R_e to assess the RSP bi-spectral R_e and MODIS R_e. In addition, R_e derived in situ from the P-3 and the Learjet platforms can also help to assess the performance of both the RSP-retrieved R_e and MODIS-retrieved R_e. There are several merits in cross-evaluating remotely sensed R_e through comparison of data from different techniques and platforms: (1) RSP alone allows us to assess the performance of the bi-spectral technique against the polarimetric technique without concerns regarding spatial and temporal co-location mismatches; (2) comparing the MODIS bi-spectral R_e against RSP bi-spectral R_e can further assess the impact of measurement resolution (i.e., satellite vs. airborne) on the retrievals; and (3) P-3 R_e derived in situ can assess the performance of the RSP polarimetric R_e from the same airborne platform, whereas the Learjet R_e derived in situ can further supplement the R_e derived in situ from a different airborne platform. Along with RSP, the P-3 carried the High-Spectral-Resolution Lidar – Generation 2 (HSRL-2; Hair et al., 2008; Burton et al., 2018), which provided measurements of aerosol properties and cloud-top height (CTH) values, and the Airborne Third Generation Precipitation Radar (APR-3; Durden et al., 2020), which provided precipitation information. Together, these instruments help to further investigate underlying relationships between the R_e differences (difference between RSP bi-spectral and polarimetric R_e) and potential impact factors such as 3-D effects and drizzle. Thus, the objective of this study is to better understand the error characteristics of satellite-retrieved R_e and provide insights into future satellite instrumental designs by comparing bi-spectrally retrieved satellite R_e with that from aircraft remote sensing and in situ measurements. In doing so, this study addresses the following questions:

1. What are the microphysical and macrophysical properties of warm cumulus and congestus clouds sampled from a variety of observing systems during CAMP²Ex?

2. What are the relative errors between R_e values retrieved using the bi-spectral techniques of MODIS and RSP, the bias-corrected MODIS R_e technique of Fu et al. (2019), the RSP polarimetric technique, and in situ cloud probes?

3. How do these relative errors depend on factors such as cloud horizontal and vertical heterogeneity and drizzle?

This paper is structured as follows: in Sect. 2, the dataset and the methodology used in this analysis are presented; in Sect. 3, we first provide an overview of the sampled clouds' characteristics and then examine the detailed behaviors of individual cloud fields while also focusing on the differences in the retrieved R_e from different techniques; in Sect. 4, we further examine the dependence of the observed R_e differences between the RSP polarimetric R_e and bi-spectral R_e on various impact factors (e.g., 3-D effects, sub-pixel heterogeneity, and drizzle) and discuss the consistency of the representativeness of the R_e retrieved using different techniques during CAMP²Ex; finally, conclusions are provided in Sect. 5.

2 Data and methodology

2.1 CAMP²Ex dataset

CAMP²Ex was focused on the Philippines and its nearby waters (approximately 6–23^∘ N, 116–128.5^∘ E). A total of 19 research flights with the NASA P-3 and 13 flights with the Stratton Park Engineering Company (SPEC) Learjet were flown during CAMP²Ex, 12 of which were joint missions. Sampled cloud fields include tropical storm convective cores, cold pools, and broken shallow cumulus and congestus clouds. Frequent cirrus and altostratus clouds were also present during the flights. The P-3 platform was equipped with an array of instruments that included remote sensing instruments such as the RSP, HSRL-2, APR-3, and the SPN-S spectral pyranometer (Badosa et al., 2014; Norgren et al., 2022). In situ probes such as the fast cloud droplet probe (FCDP; O'Connor et al., 2008) and two-dimensional stereo (2D-S) probe (Lawson et al., 2006) were also installed on the P-3. The SPEC Learjet carried similar cloud microphysical probes to those of the P-3. There were 14 research flights (RFs; Fig. 1) of the P-3 that were coordinated with Terra MODIS overpasses. Terra MODIS was chosen for the analysis rather than Aqua MODIS or VIIRS because the overpass time of the latter two sensors occurs in the afternoon when cirrus is more frequent and when the aircraft was returning to base; therefore, the sampling was not favorable. In addition, we applied the bias-adjustment technique of Fu et al. (2019), which was specifically developed for Terra MODIS R_e.

Figure 1 Flight tracks for 14 P-3 research flights (RFs) with Terra MODIS overpass coordination over the CAMP²Ex region. Dots indicate remote sensing legs with valid RSP retrievals; hollow circles indicate in situ legs with a FCDP number concentration > 10 cm⁻³.

2.1.1 RSP cloud retrievals

RSP (Cairns et al., 1999) is a multi-angle multi-spectral polarimeter that provides along-track scans at up to 152 views between view zenith angles of about ±60^∘. It measures total and polarized reflectance at nine visible and shortwave infrared channels. RSP retrieves both polarimetric and bi-spectral R_e. The instrument retrieves polarimetric R_e using polarized reflectance of the cloud bow with scattering angles ranging between 137 and 165^∘. The shape of the cloud bow is dominated by the single-scattering properties of cloud particles, which are less susceptible to uncertainties caused by 3-D radiative effects and aerosol loading (Alexandrov et al., 2012). The polarimetric technique uses a pre-calculated lookup table of single-scattering polarized phase functions with various R_e, V_e (effective variance), and scattering angles. Polarimetric R_e is retrieved by applying a parametric fitting to determine the relation between the phase function and the observed polarized reflectance. For the bi-spectral technique, like MODIS, the RSP uses the nadir reflectance at 865 nm (channel with negligible absorption by water) and at 1588 and 2260 nm (channels with strong absorption by water) to retrieve R_e and COT from a lookup table of pre-calculated reflectance of the two channels as a function of R_e, COT, and sun-view geometry. In this analysis, we mostly focus on the bi-spectral R_e retrievals from 2260 nm. Note the maximum R_e for both polarimetric and bi-spectral lookup tables is 30 µm. For COT, the standard COT product from the RSP is retrieved using total reflectances and polarimetric R_e; the RSP also reports the bi-spectral retrieved COT. The RSP retrievals are reported at ∼ 0.8 s intervals (∼ 1.2 Hz), which (depending on the aircraft platform altitude and air speed) results in spatial resolutions of ∼ 120 m during CAMP²Ex.

One of the merits of using the RSP for this evaluation study is its capability to provide co-located polarimetric and bi-spectral R_e retrievals. Thus, the comparison between the RSP bi-spectral and polarimetric retrievals does not need to consider uncertainty resulting from sampling and co-location (a common issue with cross-platform comparisons). Using RSP retrievals alone provides a comparison between the bi-spectral and polarimetric retrieval techniques. The RSP polarimetric retrievals have been examined in other field campaigns, showing good agreement of better than 1 µm compared to in situ measurements in stratocumulus cloud fields (e.g., Alexandrov et al., 2018; Painemal et al., 2021). Here, we extend its evaluation to cumulus cloud fields sampled during CAMP²Ex.

The RSP retrieves CTH using a multi-angle parallax approach (Sinclair et al., 2017). In addition, a simple cloud mask based on reflectance thresholds is reported, and RSP reports cloud-top height retrievals whenever the cloud mask is valid. As we will show in Sect. 4.1, we also make use of valid (nonzero) RSP CTH retrievals to organize cloud properties in cloud elements, where a contiguous set of CTH retrievals is labeled as one cloud element. Means and standard deviations of retrieved quantities are computed for each cloud element. This further allows us to relate cloud properties to cloud macrophysics, such as cloud length (characterized by the RSP transect length) and cloud-top bumpiness (characterized by the standard deviation of CTH) at a cloud element level.

2.1.2 SPEC in situ measurements

The SPEC provided an array of in situ cloud probes for CAMP²Ex on the NASA P-3 and the SPEC Learjet. During CAMP²Ex, the NASA P-3 often targeted clouds using stacked tracks of in situ cloud legs within the cloud field and cloud remote sensing legs above the cloud field, whereas the Learjet provided only in situ measurements. The Learjet was equipped with in situ instruments only, and data from this platform are used to characterize the cloud microphysical properties. For this study, the SPEC in situ instruments include the fast forward-scattering spectrometer probe (FFSSP; Brenguier et al., 1998), the fast cloud droplet probe (FCDP; O'Connor et al., 2008), and the two-dimensional stereo (2D-S) probe (Lawson et al., 2006). The FFSSP and FCDP are similar scattering probes that retrieve droplet number concentrations from the forward scattering of a laser impinging on cloud droplets and provide the droplet size distribution in 21 size bins ranging from 1.5 to 50 µm in diameter. The two probes share the same electronics and differ slightly with respect to the design of the probe tips to reduce shattering. The FFSSP was only installed on the Learjet, whereas the FCDP was installed on both the Learjet and P-3. The 2D-S is an optical array probe that uses two orthogonal laser-beams to record images of particles and nominally provides size distributions for diameters ranging from 10 to 3000 µm. We combined the FCDP/FFSSP and 2D-S cloud droplet size distributions for diameters from 1 to 1280 µm in order to cover cloud droplet and drizzle sizes. The “break point” to combine the FCDP/FFSSP and 2D-S particle distribution is fixed at 40 µm. Sensitivity tests were carried out using various break points from 25 to 45 µm. We found that the choice of break point does not introduce differences greater than 1 µm in the derived R_e for ∼ 90 % of the 1 Hz samples used in this study. The FCDP/FFSSP and 2D-S number concentrations are combined at a 1 Hz temporal resolution. Only drop size distributions with total number concentrations greater than 10 cm⁻³ and temperatures greater than 0 ^∘C are included in this study following thresholds used to define warm cloud in previous studies (e.g., McFarquhar and Heymsfield, 2001). The value of R_e from the combined size distributions is calculated as

$$\begin{array}{}\text{(1)}& R_{\mathrm{e}}={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^N{n}_i{r}_i^{\mathrm{3}}}{{\sum }_{i=\mathrm{1}}^N{n}_i{r}_i^{\mathrm{2}}}},\end{array}$$

where n_i is the number concentration (no. cm⁻³) for individual size bins, N is the number of bins, and r_i is the bin-center radius.

The CAMP²Ex data also archive an R_e product for full-length cloud passes computed from size distributions summed from all samples belonging to the cloud pass. These size distributions use the FCDP/FFSSP, 2DS, and the High-Volume Precipitation Spectrometer (HVPS; Lawson et al., 1993) to extend the size distribution out to 3–5 mm (in diameter). The multiple probes' size distributions are stitched together using break points that vary from different cloud passes. When compared to our R_e derived at 1 Hz using only FFSSP/FCDP and 2D-S, our cloud-averaged R_e values compared favorably to the cloud pass R_e stored in the database: the median differences were within 1 µm for both P-3 and Learjet data across all flights but with a smaller tail in the R_e distribution towards larger values – particularly for the Learjet samples, which targeted deeper clouds compared with the P-3. While acknowledging this difference, we used the R_e derived at 1 Hz from the FFSSP/FCDP and 2D-S, as it has a horizontal resolution similar to the RSP retrievals at 1.2 Hz. The effects of precipitation on our understanding of RSP bi-spectral and polarimetric R_e retrievals are examined here using coincident APR-3 airborne radar data discussed below.

2.1.3 Ancillary data

Apart from the RSP, other remote sensing instruments onboard the P-3 platform provided information about the sampled cloud fields and the surrounding environment that may influence retrieval accuracy. For instance, cirrus above the aircraft can lead to large biases in the bi-spectral retrieved cloud properties, as their absorbing effect is not modeled in the retrieval (e.g., Chang and Li, 2005). To identify the presence of above-aircraft cirrus, we utilize the measurements from the SPN-S (airborne prototype spectral Sunshine Pyranometer; Norgren et al., 2022). The SPN-S was mounted on top of the P-3 for measuring downwelling spectral total and diffuse irradiances at wavelengths ranging from 380 to 1000 nm. We derived direct-beam transmittances at 860 nm with the assumption that the solar direct beam is attenuated as prescribed by the Beer–Lambert law. Proper plane attitude adjustment has been applied to the SPN-S data (Bannerhr and Glover, 1991). By co-locating the SPN-S transmittance with the cloud retrievals from the Advanced Himawari Imager (AHI) (temporal difference < 10 min and spatial difference < 5 km), we found that the co-located samples have a SPN-S transmittance of less than 0.95 when the AHI cloud-phase flag indicates cirrus clouds. Thus, a direct-beam transmittance of 0.95 is used to filter out possible above-aircraft cirrus contamination.

The APR-3 is used to detect in-cloud drizzle in this study. The APR-3 is a Doppler, dual-polarization radar system operating at three frequencies (13, 35, and 94 GHz). It was mounted looking downward from the P-3, and it performed cross-track scans which covered a swath that is within the ±25^∘ scan range. The 94 GHz channel's sensitivity to cloud liquid water has led to it being used in many studies to detect drizzle (e.g., Tanelli et al., 2008; Dzambo et al., 2019; Lebsock and L'Ecuyer, 2011). In our analysis, we discovered that Version 2.3 of the APR-3 CAMP²Ex data included numerous segments containing calibration errors that showed up as large along-track discontinuities in the background noise. This affected about 10 % of the total APR-3 data and was, therefore, removed in our analysis.

The HSRL-2 (Burton et al., 2018) is a three-wavelength lidar that makes measurements of the atmosphere at 355, 532, and 1064 nm. It retrieves CTH and aerosol properties, such as extinction coefficient, backscatter, and aerosol optical density (AOD). In our analysis, we take advantage of the capability of the HSRL-2 to provide high-resolution CTH information at 2 Hz, to supplement the RSP in providing cloud macrophysics characteristics of the CAMP²Ex sampled clouds. As we will show in Sect. 4.2.1, we also use HSRL-2 2 Hz CTH to investigate clear-sky contamination for the RSP cloud element analysis.

All of the instruments on the P-3 platform were temporally synchronized with the meteorological and navigation information provided by the National Suborbital Research Center (NSRC).

Compared with past field campaigns, one advantage of CAMP²Ex is the availability of the continuous monitoring from the Advanced Himawari Imager (AHI) on the Himawari 8 geostationary satellite. AHI provides moderate-resolution (1 km) reflectances over the entire CAMP²Ex region at 10 min intervals. This is important for post-campaign data processing, as it provides a continuous view of a cloud field's evolution through each research flight.

2.2 MODIS cloud retrievals

The main goal of this study is to evaluate and understand the performance of bi-spectral R_e during CAMP²Ex, including those retrieved by satellites. The satellite R_e retrievals in this study come from MODIS onboard the Terra satellite. Terra is in a sun-synchronous orbit and has an Equator-crossing time of 10:30 LT (local time). The R_e retrieved from Terra MODIS represents the longest, single-platform, global record of R_e. In our analysis, we used the MODIS Collection 6.1 Level 2 (L2) Cloud Product at 1 km resolution (MOD06 V6.1; Platnick et al., 2017). For R_e and COT, only the standard product from fully cloudy pixels was included, thereby excluding partially cloudy pixels. Only liquid-water clouds were considered based on the cloud-phase flag provided in the MOD06 product. Only MODIS granules that overlapped with the CAMP²Ex sampling regions during individual P-3 research flights are included. In this analysis, we focus on the R_e and COT retrieved using the 0.86 and 2.1 µm channels, as these channels are the most widely used and RSP has a similar channel at 2.26 µm. Some recent studies have discussed the validity of comparing the MODIS 2.1 µm channel to the 2.26 µm channel from VIIRS, AHI, and RSP (e.g., Platnick et al., 2020; Zhuge et al., 2021). It has been pointed out that the inconsistency in the spectral response function of the two wavelengths can lead to differences of ∼ 1–2 µm between the R_e derived from the two wavelengths, which is much smaller than the R_e bias estimates of up to 10 µm reported in Fu et al. (2019). We also examined the MODIS R_e product derived using its 3.7 µm channel and found the differences between the R_e products derived from the MODIS 2.1 and 3.7 µm channels to be consistent with what has been reported in previous studies (e.g., Zhang and Platnick, 2011; Fu et al., 2019). Thus, they are not included in the figures and tables in the following sections. However, we provide a brief summary of these differences at the end of Sect. 4.3.

2.3 Bias-adjusted MODIS cloud retrievals

The MODIS R_e bias estimates presented in Fu et al. (2019) are also evaluated via comparison against the CAMP²Ex dataset. As a continuation of Liang et al. (2015), Fu et al. (2019) fused MISR Level 1B (L1B) radiance data and MODIS L2 cloud R_e to retrieve COT at the nine MISR viewing angles. Liang et al. (2015) revealed that the COT retrievals show a local minimum around the cloud-bow scattering direction (∼ 140^∘), and this feature was prominent in multiyear climatologies of MODIS cloud COT values and COT retrieved from MISR. They showed that this minimum is due to an overestimate in the MODIS R_e product and that the value of the climatological R_e bias could be estimated by extrapolating the R_e correction derived from the cloud-bow feature to retrievals at all scattering angles. The local minimum of COT was prevalent in a multiyear climatology carefully stratified by scattering angle, latitude, and SZA rather than apparent at a ∼ 1 km pixel resolution within an L2 MODIS granule because of large spatial variability in scattering angle, SZA, and cloud heterogeneity within an L2 granule. To correct for the R_e bias, Fu et al. (2019) used 8 years of MISR and MODIS data that were further stratified by MISR nadir τ and cloud heterogeneity to produce climatology estimates of corrected MODIS R_e between 60^∘ N and 60^∘ S globally at a 2.5^∘ resolution for the months of January and July. In the current study, we apply the July regional correction factors from Fu et al. (2019) at 2.5^∘ to the MODIS L2 granules over the CAMP²Ex domain to allow for better comparison with R_e derived from other techniques under similar seasonal conditions. This enables one to test the robustness of the correction. The average of the July correction factors over the CAMP²Ex domain is ∼ 0.6. The correction factors over this region range from 0.25 to 0.97 depending on latitude, τ, and cloud heterogeneity. We are interested in evaluating the capability of regional bias corrections to capture the actual variability at its original resolution (i.e., MODIS 1 km retrieval) as we compare to field measurements from CAMP²Ex.

2.4 Co-location technique

One major challenge for constructing the evaluation framework is the co-location between different platforms. In CAMP²Ex, the P-3 performed both remote sensing and in situ sampling during the same flight; therefore, simultaneous sampling from both methods is not possible. Furthermore, CAMP²Ex targeted mostly cumulus and congestus clouds that have faster evolution and a shorter lifetime compared with stratocumulus clouds. A sawtooth flight pattern, commonly used in field campaigns targeting stratocumulus regions (e.g., Curry et al., 2000; Painemal and Zuidema, 2011; Witte et al., 2018; McFarquhar et al., 2021; Redemann et al., 2021), was not employed during CAMP²Ex. However, while a strict point-to-point comparison is not achievable, we adopted the following approach to co-locate MODIS, RSP, and in situ measurements from a statistical standpoint.

A valid co-location between MODIS and the P-3 occurs based on a spatial and temporal matching criterion. For the case-by-case comparisons presented in Sect. 3.3, all samples within the tightest rectangular box circumscribing the P-3 flight path that fell within ±1.5 h of the MODIS overpass time are included in the comparison. This time window was chosen based on the examination of all of the 10 min AHI imagery and forward/nadir videos from the P-3 to maintain a balance between ensuring a significant number of samples and ensuring that the airborne remote sensing and MODIS observe the same cloud features. The sensitivity of our results to tighter temporal windows (e.g., 30 min and 1 h) was tested and did not alter the patterns observed in our results. Of the 19 P-3 research flights, 14 research flights had successful overlap with Terra MODIS overpasses (Fig. 1, Table 1).

Table 1 CAMP²Ex P-3 research flights (RFs) with successful coordination between P-3 and MODIS. RFs in bold indicate successful overlap between RSP sampling and MODIS.

When comparing remotely sensed R_e with R_e derived in situ, one limitation lies in the simplified representation of clouds in the algorithms. Current passive remote sensing assumes clouds to be homogeneous in both the horizontal and vertical directions, but this representation of clouds is different from reality. In nature, clouds tend to have an R_e profile that increase with height (e.g., McFarquhar et al., 2007; Arabas et al., 2009), although it is relatively constant in the horizontal direction at a given height level (e.g., Khain et al., 2019; Pinsky and Khain, 2020; Zhang et al., 2011). The vertical variability in R_e is often observed from R_e derived in situ at various levels throughout a cloud. For remotely sensed R_e, however, satellite-retrieved bi-spectral R_e is viewed as a vertically weighted R_e with peak weighting near cloud top (e.g., McFarquhar and Heymsfield, 1998; Platnick, 2000; Nakajima et al., 2010). For the polarimetric R_e retrievals, the vertical weighting is more strongly peaked and closer to the cloud top compared with the bi-spectral technique. This is because the polarimetric signature is dominated by single-scattering contributions, with a mean penetration optical depth of ∼ 0.5 and negligible contributions from levels below an optical depth of ∼ 3 from cloud top (Miller et al., 2018). Thus, to directly compare R_e retrieved in situ with satellite or airborne remotely sensed R_e, many studies have used in situ measurements at the cloud top to evaluate satellite R_e (e.g., Painemal and Zuidema, 2011; Witte et al., 2018; Gupta et al., 2022). This requires the determination of the altitude of cloud tops during the in situ legs, which is simple for stratiform cloud with the aircraft performing sawtooth flight patterns at cloud top but not so for cumulus cloud. Here, we made use of all in situ measurements throughout various levels of the cloud fields. While we know the altitude in which the aircraft penetrated cloud, we do not have coincident measurements of co-located cloud top. We exclude in situ samples for which the co-located AHI brightness temperature at 11 µm is below 273 K. This removes deeper convective clouds sampled by the aircraft that are not observed in the warm clouds sampled by passive remote sensing (i.e., RSP). As the two airborne platforms are equipped with similar SPEC probes, despite the differences in the platform and sampling, the two in situ datasets serve to complement each other, providing additional information that is key to the evaluation of remotely sensed bi-spectral R_e. We pay special attention to these sampling issues in our comparison of R_e measured in situ with remotely sensed R_e.

3 Results

3.1 General cloud characteristics of CAMP²Ex

We begin by providing an overview of some general cloud characteristics derived using the remote sensing data collected by the P-3 for all of the research flights. Only oceanic liquid-water clouds are included based on the RSP cloud-top liquid index (van Diedenhoven et al., 2012). Cloud segments overlaid with cirrus are removed based on SPN-S transmittance < 0.95. Segments where the P-3 was banking (i.e., roll angle > 3^∘ and roll angle difference < 1^∘) are also removed. Here, data are organized into cloud elements. Figure 2a and b show the respective probability distribution functions (PDFs) and cumulative distribution functions (CDFs) for cloud element mean R_e and COT retrieved using the RSP polarimetric and bi-spectral (using the 2260 nm channel) techniques. While the COT distributions among the two techniques are in good agreement, the two R_e distributions are quite different: the polarimetric R_e distribution mode occurs at ∼ 6 µm, whereas the bi-spectral R_e mode occurs at ∼ 12 µm. The median polarimetric R_e is 7.0 µm, and the median bi-spectral R_e is 16.1 µm. The median COT is 3.5 for the standard RSP COT retrievals and 4.2 for the bi-spectral COT retrievals. Figure 2c provides the PDFs of RSP mean CTH and HSRL-2 mean CTH, with HSRL-2 CTH from 2 Hz samples. Similar to past studies that compared RSP CTH with airborne-lidar-retrieved CTH (e.g., Sinclair et al., 2017), the RSP and HSRL-2 CTH distributions in Fig. 2c are in excellent agreement, both showing that ∼ 60 % of the cloud elements sampled have mean cloud tops < 1 km. Figure 2d and e show the PDFs of cloud element transect lengths and clear lengths (between cloud elements) derived from the RSP and HSRL-2 CTH mask. We see that 50 % of cloud elements sampled by RSP have transect lengths less than 0.6 km, with a mean length of 1.4 km, while 50 % of HSRL-2-derived cloud elements have transect lengths less than 0.5 km, with a mean of 1.3 km. A total of 50 % of the clear lengths derived from RSP and HSRL-2 are less than 1.0 km (mean length of 2.6 km) and 0.8 km (mean length of 1.9 km), respectively. We note that these clouds sampled by the RSP and HSRL-2 have more numerous small clouds than the trade cumuli sampled during INDOEX (the Indian Ocean Experiment; Lelieveld et al., 2001) using a multichannel radiometer (MCR) and during RICO (Rain in Shallow Cumulus Over the Ocean; Rauber et al., 1997) using ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer; Abrams et al., 2000), both of which had 50 % of the total cloud fraction contributed by cloud-area-equivalent diameters of less than 2 km and with about half the amount of clouds (McFarquhar et al., 2004; Zhao and Di Girolamo, 2007). Given that the mean cloud-area-equivalent diameter is approximately 1.1 to 1.7 times that of a random linear transect (e.g., Barron et al., 2020; Romps and Vogelmann, 2017), the clouds sampled by RSP and HSRL-2 during CAMP²Ex are much smaller than the trade cumuli sampled during INDOEX or RICO. As such, the 1 km resolution MODIS pixels are expected to have a considerable amount of sub-pixel clouds during CAMP²Ex. We speculate that the reason for the maximum failure rate in MODIS cloud microphysical retrievals occurring over the western tropical Pacific, as reported by Cho et al. (2015), may be the high frequency of small clouds in this region relative to anywhere else. Finally, Fig. 2f shows the PDF and CDF of the derived APR-3 W-band maximum reflectivity within individual RSP cloud elements. The APR-3 W-band maximum reflectivity median is at −9.24 dBZ. Past studies have shown that a threshold of W-band column maximum reflectivity of ∼ −15 dBZ is associated with the transition from non-drizzle to light drizzle (e.g., Dzambo et al., 2019; Wang and Geerts, 2003). From Fig. 2f, ∼ 20 % of the valid APR-3 W-band maximum reflectivity values are less than −15 dBZ, indicating that most of the cloud elements sampled by RSP may have some degree of drizzle somewhere in the cloud. Overall, Fig. 2 reveals that most clouds observed by the P-3 remote sensors are small, optically thin, and often contained pockets of drizzle somewhere within the cloud. Most of the clouds were low clouds with tops under 2 km.

Figure 2 Probability distribution function (PDF, solid line) and cumulative distribution function (CDF, dashed line) for cloud element mean values of (a) the RSP polarimetric and bi-spectral R_e, (b) the RSP polarimetric and bi-spectral COT, (c) CTH from RSP and HSRL-2, (d) cloud element transect length from RSP and HSRL-2, (e) clear segment length (between cloud elements) from RSP and HSRL-2, and (f) APR-3 W-band maximum reflectivity within a cloud element, using warm oceanic liquid cloud segments from all P-3 research flights.

3.2 RSP cloud microphysics statistics

The ability to retrieve both co-located polarimetric and bi-spectral R_e from the RSP allows us to compare the performance of the two techniques without further concerns regarding sampling differences. Figure 3 shows 2-D histograms of RSP polarimetric and bi-spectral R_e as well as R_e differences (the difference between bi-spectral and polarimetric R_e) as a function of COT and CTH, using all 1.2 Hz samples passing the above P-3 cirrus filter for oceanic cloud samples during all flights. The differences between bi-spectral COT and COT as a function of polarimetric R_e as well as COT as a function of CTH are also shown. Several key features are displayed in Fig. 3. Figure 3a shows that most of the bi-spectral R_e measurements are larger than the polarimetric R_e. A linear regression shows that the correlation between the two R_e measurements is 0.38, with a bias (difference) of 6 µm and root-mean-square error (RMSE) of 8.2 µm. Figure 3b shows a rapid increase in R_e difference as retrieved optical depths decrease below 5. In other words, the largest R_e differences are associated with optically thin clouds, which is consistent with the findings from the deployment of the RSP during ORACLES (ObseRvations of Aerosols above CLouds and their intEractionS; Miller et al., 2020). For CAMP²Ex, the differences between the two R_e retrievals has a mean of 6.0 µm with a maximum of 26 µm, compared with a mean difference of ∼ 1 µm and a maximum of 15 µm for ORACLES. The likely reason for the much larger R_e differences in CAMP²Ex is the greater cloud heterogeneity in the oceanic regions around the Philippines compared with stratocumulus cloud sampled in ORACLES. COT retrievals from the two techniques do not show large differences, as indicated in Fig. 3c. Most of the COT differences are less than 2 (∼ 20 %), which is similar to the results in Miller et al. (2020). Finally, when the R_e differences are binned by CTH (Fig. 3d), the R_e differences decrease as CTH increases for low to mid-level clouds (CTH < 4 km). As seen from Fig. 3e, COT increases with CTH, which would also result in the liquid water path increasing with CTH. Beyond 4 km, no clear trend in R_e difference related to CTH is observed, perhaps because the population is largely alto-clouds (as evident in Fig. 3e).

Figure 3 RSP 2-D density histogram of (a) polarimetric R_e vs. bi-spectral R_e, (b) RSP COT vs. R_e difference (bi-spectral R_e – polarimetric R_e), (c) polarimetric R_e vs. COT difference (bi-spectral – cloud bow), (d) CTH vs. R_e difference (bi-spectral R_e – polarimetric R_e), and (e) RSP COT vs. CTH. The black solid lines in panels (a)–(d) plot the median with respect to each horizontal bin, and black dashed lines indicate the interquartile range.

This comparison indicates that the bi-spectral R_e values are considerably larger than the polarimetric R_e. However, without further examining the details of the macrophysics and the microphysics of the sampled cloud fields, it is difficult to comment on possible causes for the observed R_e differences. Therefore, it is necessary to focus on individual cloud fields to study the characteristics of each cloud field (Sect. 3.3) and then relate the observed R_e differences to other observed properties, such as cloud macrophysics and the presence of drizzle. Possible causes of the differences between the two R_e retrieval techniques are further explored in Sect. 4.

3.3 Individual case studies

In our analysis above, we examined the general cloud characteristics and RSP bi-spectral and polarimetric retrieval differences over all 19 P-3 RFs from CAMP²Ex. Here, we provide a few case studies to illustrate detailed intercomparisons between remote sensing (satellite and aircraft) and in situ retrievals of R_e during CAMP²Ex. Cases were selected when there was a good overlap between the MODIS, RSP, and in situ sampling over cirrus-free liquid-phase cloud fields over ocean. Table 2 provides the details of selected cases, including the geolocation, MODIS overpass time, selected co-location time period of RSP, and in situ time period for the P-3 and Learjet platforms. For each case, we compare R_e sampled by RSP, in situ R_e from the P-3 platform, in situ R_e from the SPEC Learjet platform, R_e sampled by MODIS, and the bias-adjusted MODIS R_e from Fu et al. (2019) to evaluate the performance of bi-spectral R_e against polarimetric R_e and in situ R_e measurements. For a shallow cumuli case from RF17, the co-located high-resolution ASTER (also onboard Terra) data allow us to highlight the representativeness of MODIS L2 cloud retrievals in sub-pixel cloud fields. In an effort to keep this discussion concise, we discuss only RF02 here; additional figures and discussions for the RF07, RF12, and RF17 case studies are given in the Supplement.

Table 2 Individual case details showing the research flight (RF) designation, approximate domain-center geolocation, MODIS overpass time (UTC), RSP, P-3 in situ and Learjet in situ time periods (UTC), and a brief description of sampled cloud fields. NA – not available.

3.3.1 RF02, 27 August 2019

During RF02, shallow convection was observed near 18.6^∘ N, 116.9^∘ E, as shown in the MODIS RGB image (Fig. 4a) during the Terra overpass at 03:05 UTC. The P-3 first entered the area depicted in Fig. 4a around 03:00 UTC on a low-altitude leg (∼ 500 m), sampling below the shallow cumulus field. Between 03:00 and 03:30 UTC, the P-3 conducted several upward ascents into level legs to sample clouds in situ. Several high-altitude remote sensing legs were flown between 03:30 and 04:30 UTC, sampling along a cumulus cloud line between 17 and 19^∘ N and between 116 and 117^∘ E, as indicated in Fig. 4a. This cloud field occurred in the vicinity of a larger low-pressure system east of Luzon; some thin cirrus clouds are observed to the east of the sampled clouds. During the 1.5 h time period, AHI imagery indicated that the shallow convective line retained its overall pattern and distributions, exhibiting consistent cloud-top structures of typical broken shallow to moderate cumulus. Cirrus and ice clouds were filtered out from MODIS according to the MODIS L2 phase flag. For the P-3 platform, the lower cumuli were mostly not affected by cirrus, as seen from the AHI imagery and according to the SPN-S transmittance above the P-3. MODIS L2 retrievals show R_e ranging from 8 to 30 µm, associated with optically thin to moderately thick COT (1–50) and a CTH of ∼ 500 to 4000 m. The RSP bi-spectral R_e also shows a range of 8 to 30 µm, similar to MODIS. In great contrast, RSP polarimetric R_e, bias-adjusted MODIS R_e, and R_e derived in situ from P-3 all suggest a similar range of 5 to 15 µm (with only a few outliers ∼ 20 µm), which is much lower than the bi-spectral R_e retrievals. The W-band maximum reflectivity from APR-3 indicates some precipitation in the deeper clouds (CTH > 2 km).

Figure 4 (a) MODIS RGB reflectance at 03:05 UTC on 27 August 2019; the color bar indicates the P-3 altitude and flight track within ±1.5 h of the MODIS overpass time, and the green cross indicates the P-3 location at the MODIS overpass time. (b) MODIS Level 2 1 km R_e retrievals from the 2.1 µm channel. (c) MODIS Level 2 1 km bias-adjusted R_e retrievals from the 2.1 µm channel after applying the Fu et al. (2019) correction factors. (d) MODIS Level 2 1 km COT from the 2.1 µm channel. (e) RSP bi-spectral R_e retrievals from the 2.26 µm channel; in situ R_e from P-3 is displayed using circles. (f) RSP polarimetric R_e retrievals from the 0.86 µm channel; in situ R_e from P-3 is displayed using circles. (g) MODIS Level 2 1 km CTH retrievals. (h) RSP R_e curtain between 03:30 and 04:30 UTC. (i) APR-3 W-band reflectivity and RSP CTH (black dots) between 03:30 and 04:30 UTC.

The RSP bi-spectral and polarimetric R_e retrievals have the same sampling, as does MODIS R_e and the MODIS bias-adjusted R_e; therefore, they are directly comparable. However, a direct pixel-to-pixel comparison between MODIS, RSP, and R_e values sampled in situ is essentially impossible because they are not coincident in space and time. Nevertheless, the samples were collected in a fairly small spatiotemporal window over which little overall change in the cloud field occurs, as indicated by AHI imagery. We sorted the R_e retrievals into 250 m CTH bins. The R_e mean and standard deviation are computed for each height bin as a means of comparing the remote sensing techniques' ability to capture the variations in R_e with CTH, which is important when using the data to understand cloud processes. As the tops of cumulus clouds sampled in situ are hard to determine, the platform altitude was used for in situ sampling, noting that these R_e derived in situ are in-cloud measurements rather than a vertically weighted near-cloud-top R_e as obtained from remote sensing. While acknowledging the different sensitivity to cloud exterior and in-cloud microphysics for these different techniques, observations and simulations have shown, for shallow cumulus clouds, that low variation in R_e (∼ 10 %) exists between the exterior and interior of clouds at a given altitude (e.g., Khain et al., 2019; Gerber et al., 2008). Thus, we primarily focus on accounting for the systematic variation with altitude. To do this, we binned R_e retrievals from all five techniques (P-3 in situ, RSP polarimetric, RSP bi-spectral, MODIS bi-spectral, and bias-adjusted MODIS) separately as a function of binned CTH/altitudes. The results for the RF02 case are given in Fig. 5. All five techniques indicated an overall pattern of increasing R_e with height. One prominent feature of Fig. 5 is that, for mid- to low-level cloud tops (below 3.5 km), the P-3 in situ (FCDP and 2D-S), RSP polarimetric, and bias-adjusted MODIS R_e values all indicate an increasing R_e profile from ∼ 7 to ∼ 15 µm in the mean values. Thus, despite the differences in sampling and retrieval technique, the three are very consistent; the mean difference between the three R_e profiles (Table 3) are all within 2 µm. The bi-spectral R_e from RSP and MODIS, however, shows much larger values than the other three techniques, with increasing R_e profiles from ∼ 13 to ∼ 22 µm. Thus, despite sampling and resolution differences, these two bi-spectral products are consistent among themselves, with the RSP bi-spectral R_e ∼ 3 µm smaller than that from MODIS. The bi-spectral R_e values from MODIS and RSP also show much greater R_e variability at each height level (as seen from the horizontal whiskers), compared with RSP polarimetric, in situ, and bias-adjusted MODIS R_e. For CTH below ∼ 1.3 km, the RSP bi-spectral R_e suggests a decreasing R_e profile, which is essentially the opposite of the other techniques. At higher altitudes, the existence of drizzle tends to result in higher R_e values with larger variability for both RSP polarimetric and bi-spectral R_e retrievals. The APR-3 and RSP curtains in Fig. 4 also confirm the correlation between drizzle and larger R_e values for both techniques. At ∼ 3.5 km, R_e derived in situ indicated values of 13–20 µm, as it penetrated a convective cloud whose tops were higher than the P-3 by several hundred meters and visually appeared to be optically thick, as indicated by the P-3 forward video just before cloud penetration. Splashing of precipitation on the P-3 windshield was also evident from the forward video. Apart from sampling differences, some of the variability between in situ R_e and the remotely sensed R_e may also be due to entrainment and mixing (e.g., Gerber et al., 2008).

Figure 5 R_e profile (mean R_e vs. mean height) of vertically binned MODIS R_e, bias-adjusted MODIS R_e (after applying the Fu et al., 2019, correction factors), RSP polarimetric R_e, RSP bi-spectral R_e, and P-3 R_e derived in situ for the RF02 case. Horizontal whiskers indicate the standard deviation of data within each 250 m altitude bin.

Table 3 Mean bias (difference) between the RSP polarimetric technique and the other techniques for the four case studies. NA – not available

3.3.2 Summary and discussion of case studies

The analysis and discussion of RF07, RF12, and RF17 presented in the Supplement follow the figures and discussion given above for RF02. Table 3 summarizes the mean difference between the RSP polarimetric R_e and the other techniques for these cases. At lower altitudes, the Learjet and P-3 in situ and RSP polarimetric R_e values are in good agreement. While acknowledging that the definition of R_e is different for remote sensing retrievals and in situ measurements, the overall good agreement between polarimetric and in situ R_e should provide more confidence in the performance of polarimetric R_e retrievals in cumulus cloud fields. In all comparison cases, RSP polarimetric R_e also agrees well with the MODIS bias-adjusted R_e, despite the resolution and sampling differences between the two. Finally, both the RSP bi-spectral and MODIS R_e values show good agreement with each other, although with the overall largest R_e values among all techniques. The distinct separation between bi-spectral R_e and R_e from all other techniques implies an overestimate in bi-spectral R_e for cumulus clouds regions; the mean R_e differences of ∼ 5–10 µm in the individual cases match with the estimates of ∼ 6–9 µm bias in the 2.1 µm channel MODIS R_e found by Fu et al. (2019).

The impact of drizzle on R_e from these case studies was evident in remotely sensed and in situ observations. Direct comparison between in situ and remote sensing in deeper clouds containing drizzle was hampered by the fact that the in situ samples containing drizzle and large R_e values occurred at locations that were not close to cloud top according to the aircraft forward video. Therefore, we undertake a more extensive examination of the impact of drizzle on our comparison of bi-spectral and polarimetric retrievals in Sect. 4.2.2 using APR-3.

As mentioned in Sect. 2.2, we acknowledge the channel differences between the MODIS 2.1 µm and RSP 2.26 µm R_e retrievals, so we do not expect the R_e retrieved from MODIS and RSP bi-spectral to have the exact same bias. In our analysis, other contributing factors that may impact the differences between RSP 2.26 µm bi-spectral R_e and MODIS 2.1 µm R_e include (1) sampling differences, (2) channel differences in the face of vertical and horizontal variations in cloud optical properties, and (3) pixel size difference in the face of 3-D variations. Despite these factors, the RSP 2.26 µm bi-spectral R_e and MODIS 2.1 µm R_e have very similar behavior, exhibiting a large positive bias and much greater variability in R_e relative to the other techniques.

3.3.3 A brief look at the representativeness of MODIS retrievals with ASTER

As indicated in Fig. 2, CAMP²Ex samples contain lots of small clouds; this introduces possible sub-pixel issues with respect to the MODIS retrievals and calls the representativeness of MODIS-derived cloud climatologies into question. Here, we use RF17 to illustrate a few points. RF17 sampled a field of small, shallow cumuli that appear very different from the previous cases. The most prominent feature for this case is the abundance of small broken cumulus clouds in this domain (Fig. 6a). On 2 October at around 01:00 UTC, the P-3 platform entered the region at around 1.5 km altitude and then descended to ∼ 100 m above sea level to begin in situ measurements below cloud base and at various levels within clouds. Around 02:00 UTC, the P-3 started climbing from 1 to 5 km altitude to perform remote sensing sampling, with long stretches of straight legs, as shown in Fig. 6a. The aircraft exited the region at around 04:30 UTC. A Terra overpass took place at 02:40 UTC. The MODIS and ASTER imagery indicates that clouds were mostly very shallow, small cumuli in this case. To better demonstrate the representativeness problem with MODIS retrievals in this common type of cloud field, we overlaid the MODIS 1 km resolution R_e retrievals over the MODIS 250 m RGB imagery (Fig. 6a) and ASTER 15 m radiances (Fig. 6b) for the box region highlighted in red in Fig. 6a.

Figure 6a shows that, while there are numerous very small broken cumulus clouds in the scene, the MODIS L2 R_e only reported a handful of pixels with successful retrievals. For the successful MODIS R_e retrievals shown in Fig. 6b, the estimated cloud fraction from ASTER 15 m imagery ranges from ∼ 0.25 to 0.62. It clearly shows that these small cumuli are sub-pixel for MODIS L2 retrievals, with cloud variability that cannot be resolved by MODIS. It is expected that this unresolved variability leads to biases in MODIS-retrieved R_e, which we further investigate in Sect. 4.2. The MODIS R_e retrievals (Fig. 6a, d) and RSP bi-spectral retrievals (Fig. 6c) both suggest R_e values in the range of 6 to 30 µm (median R_e value of 14.0 µm for MODIS R_e and median R_e value of 12.0 µm for RSP bi-spectral R_e). The RSP polarimetric R_e, however, shows much smaller values of 4 to 7 µm (median R_e value of 5.3 µm) that also agree well with the P-3 in situ R_e (median R_e value of 6.1 µm). Similar to Fig. 5, R_e retrievals from the different techniques binned by CTH/altitude for the RF17 case are included in the Supplement (Fig. S6).

While not shown, many clouds in the scene are correctly identified by MODIS as partly cloudy (PCL) pixels and are excluded from the standard R_e product analyzed here. We examined MODIS PCL R_e retrievals within the domain in Fig. 6a; they revealed that most of the R_e values were in the 10 to 30 µm range, with a median value of 18.5 µm, which is 4.5 µm larger than the standard MODIS R_e. This is mostly due to MODIS not being able to resolve the sub-pixel variability, and the low sub-pixel cloud fraction would lead to erroneously larger R_e values. Some failed retrievals may also be attributed to the finite range of the bi-spectral lookup table (LUT). For example, Cho et al. (2015) showed that failed retrievals in the MODIS product would occur whenever the retrievals fell outside the LUT range, and this failure rate can be as high as 40 % in the Southeast Asia oceanic region. This calls the validity of the representativeness of long-term MODIS climatologies into question for regions dominated by small cumulus cloud fields.

4 Relating R_e bias to 3-D factors, sub-pixel heterogeneity, and drizzle

As the largest (i.e., > 5 µm) observed R_e bias between the bi-spectral and polarimetric technique tends to occur for small retrieved optical thicknesses (COT < 5, Fig. 3b), it is especially important to consider the uncertainties in the retrieval process that can affect the bi-spectral retrieval, even when the core assumptions of 1-D radiative transfer are met. Sources of uncertainty include instrument calibration, atmospheric correction, the surface bidirectional reflectance distribution function (BRDF), the assumed size distribution shape, and the retrieval logic. These uncertainties are derived for MODIS in Platnick et al. (2004), and they are globally validated for COT retrievals of oceanic liquid-water clouds in the limit of homogeneous clouds in Di Girolamo et al. (2010). Nevertheless, even in the case of ideal simulated 1-D retrievals, retrieved R_e values can be biased high due to the presence of multiple R_e solutions and the limitations of lookup table interpolation for low optical depth (Miller et al., 2018). However, these large R_e retrievals are much more frequent in our data (e.g., Fig. 3b) than can be reasonably explained by these sources of uncertainty, as discussed below.

For MODIS, the uncertainty in retrieved R_e can be significant (16 %–30 %) for COT < 5 (Platnick et al., 2017). Similar uncertainties can be anticipated for RSP, as RSP has a calibration uncertainty of 3 % (Knobelspiesse et al., 2019), which is similar to MODIS. Even in the worst-case scenario in which all uncertainties are systematic across the field campaign period, these uncertainties are much smaller than the factor of 2 differences observed between RSP polarimetric and bi-spectral R_e with COT < 5. This indicates that other retrieval assumptions should be investigated to understand the cause of the observed differences between the R_e retrieval techniques.

Figure 6 (a) MODIS 250 m RGB reflectance at MODIS overpass (02:40 UTC) overlaid by MODIS Level 2 liquid R_e retrievals for the RF17 P-3 flight path within 1.5 h of the MODIS overpass; the flight path is color-coded by altitude, the green cross indicates the P-3 position at the MODIS overpass time, the yellow box indicates the outline of the co-located ASTER 15 m granule, and the red box indicates the outline of the close-up view shown in panel (b). (b) Close-up view of ASTER 15 m resolution Band 3 nadir (3N) radiance, overlaid by MODIS Level 2 liquid R_e retrievals. (c) Scatterplot of RSP polarimetric and bi-spectral R_e retrievals within 1.5 h of the MODIS overpass time; the blue and red lines indicate the median value for the RSP polarimetric and bi-spectral R_e retrievals, respectively. (d) Box plots of the MODIS R_e retrievals shown in panel (a) and P-3 R_e derived in situ within 1.5 h of the MODIS overpass time; the line in the box indicates the median R_e value.

The literature contains extended discussions relating passive cloud retrieval bias in COT and R_e to the impact of sub-pixel heterogeneity and other 3-D effects (e.g., Marshak et al., 2006; Zhang and Platnick, 2011; Zinner et al., 2010; Zhang et al., 2012, 2016), sun-view geometry (e.g., Loeb and Davies, 1996; Várnai and Davies, 1999; Liang and Di Girolamo, 2013; Grosvenor and Wood, 2014), and the presence of drizzle (e.g., Zhang et al., 2012; Ahn et al., 2018). In this section, we test the hypothesis that retrieval errors from 3-D effects contribute to measurable bias in R_e retrievals from the CAMP²Ex dataset. To facilitate our investigation, we used a cloud-element-labeling technique based on RSP L2 cloud retrievals, where a cloud element is defined as a region with contiguous CTH retrievals. We then provide statistics of cloud microphysics and macrophysics for each cloud element. We use cloud macrophysical properties (e.g., cloud size, cloud-top bumpiness, CTH) and solar zenith angle (SZA) as proxies to relate to 3-D radiative effects and test the sensitivity of the R_e bias to these factors (Sect. 4.2). We also look at the impact of drizzle on R_e bias and find no apparent relationship between the two (details in Sect. 4.2.2).

4.1 RSP cloud element analysis

The cloud-element-labeling technique was developed using the RSP CTH retrievals. A contiguous set of CTH retrievals are counted as one cloud element. Places with no retrievals between the cloud elements are labeled as “clear” segments. For each cloud element, means and standard deviations of cloud properties (R_e, COT, CTH) are calculated along with the cloud elements' horizontal length. This method allows one to further relate the RSP-retrieved cloud properties to quantities such as the standard deviation of CTHs and cloud horizontal length, which can serve as proxies for cloud-top bumpiness and cloud size to further investigate the sensitivity of R_e retrievals to these factors. When developing the cloud-element-labeling technique, we compared the cloud elements derived using RSP CTHs with that derived from HSRL-2 CTHs (Fig. 2c). While the two CTHs showed very similar results, we chose to use RSP because R_e and COT are tied directly to RSP sampling.

This cloud element analysis was implemented for all research flights with good cloud sampling segments (number of cloud elements > 3), without further consideration for MODIS or in situ co-locations. Table 4 lists the 12 research flights and time periods used in our analysis. Figure 7 shows mean R_e values for each cloud element as a function of its mean CTH, with whiskers representing the standard deviations of R_e and CTH for the cloud element. Each cloud element is color-coded by its cloud transect length. A prominent feature in Fig. 7 is the high degree of correlation between the RSP polarimetric R_e and CTH means (linear correlation coefficient, r, of 0.64 averaged across all 12 cases), compared with the correlation between the RSP bi-spectral R_e and CTH means (r=0.18 averaged across all 12 cases). The variability in bi-spectral R_e is also much larger than that of polarimetric R_e across all CTH levels, particularly for the lower levels (below 2 km). Colors also indicate that lower clouds are more often found with smaller lengths (less than 5 km and often less than 1 km), indicating that low clouds are mostly very small cumuli. Figure 7 also shows that higher clouds often have higher CTH standard deviations, indicating bumpier cloud tops. Overall, the bi-spectral R_e values are much larger than the polarimetric R_e values across all CTH ranges. Across all 12 flights, the mean R_e difference between the bi-spectral and the polarimetric R_e is 11 µm.

Figure 7 RSP cloud element mean R_e vs. mean CTH for from 12 research flights with good RSP sampling of warm cumulus and congestus clouds. The dots represent the mean R_e value vs. mean CTH of each cloud element, and the whiskers indicate the standard deviations of R_e and CTH for each cloud element. Cloud elements are color-coded by their horizontal transect lengths.

Table 4 The time periods of 12 research flights used to construct the analysis shown in Fig. 7.

4.2 Sensitivity of R_e retrieval bias to potential factors

In this section, the RSP cloud element statistics throughout the entire CAMP²Ex mission are used to investigate the impact of various potential factors on the observed R_e differences between RSP polarimetric and bi-spectral retrievals. The factors investigated include COT, cloud size, cloud-top bumpiness, sub-pixel heterogeneity, SZA, and drizzle. In reality, the impacts of these factors are often intertwined. For example, clouds with smaller lengths are also shallower and optically thinner. Deeper clouds may also have larger lengths and increased likelihood of drizzle. While it is not possible to fully isolate the impact from each individual factor, we will focus our discussion primarily within the scope of 3-D radiative effects, sub-pixel heterogeneity, and drizzle.

4.2.1 Three dimensional (3-D) radiative effects and sub-pixel heterogeneity

Cloud optical thickness and cloud size

Figure 8a shows the differences between the cloud element mean polarimetric R_e and bi-spectral R_e retrievals organized as a function of cloud optical thickness. A sharp decrease in R_e differences (bi-spectral − polarimetric) from ∼ 25 to ∼ 8 µm (maximum R_e difference) is observed with increasing COT up to 9. This difference tapers off to a value of roughly 4 µm for larger COT values. This pattern is similar to Fig. 5a in Marshak et al. (2006), where they used 3-D large eddy simulation (LES) simulations to discuss the radiative effects of clouds' 3-D structure in 1-D R_e retrievals using the bi-spectral technique. Marshak et al. (2006) concluded that shadowing effects (defined as when the measured reflectance is lower than its 1-D plane-parallel equivalent reflectance) dominate over illumination effects (measured reflectance is higher than its 1-D plane-parallel equivalent reflectance) in cumulus cloud fields and lead to an overestimate in retrieved 1-D R_e. Vant-Hull et al. (2007) studied the impact of scattering angle on cumulus clouds and found that 3-D cloud structures would lead to overestimates in the 1-D bi-spectral R_e retrievals far from the backscatter direction (where shadowing effects dominate). However, near the backscatter viewing geometry (where illumination effects dominate), the opposite is true. To date, these effects in cumuliform cloud fields have only been studied through simulations. As RSP bi-spectral retrievals were never taken close to the backscatter direction, the expectation is an overall overestimate of retrieved R_e using the bi-spectral technique. If we assume the RSP polarimetric R_e as the true R_e (given how well it compared with R_e derived in situ in Sect. 3.3), Fig. 8a suggests that large overestimates in bi-spectral R_e will dominate in broken cumulus scenes (typical in CAMP²Ex), consistent with the earlier studies based on simulations. However, differences do exist. For example, we observe fewer negative values of scattered R_e differences in Fig. 8a compared with Fig. 5a of Marshak et al. (2006). This may be due to different sun-view geometries, cloud structures, and cloud microphysics between RSP retrievals in CAMP²Ex and the simulations in Marshak et al. (2006).

Figure 8 (a) Density plot of the cloud element mean R_e difference (bi-spectral – polarimetric R_e) vs. mean COT. (b) Scatterplot of the R_e difference (color) vs. transect length and COT using cloud element means.

Figure 8b shows the relationships between the R_e differences, COT, and the cloud element's transect length as measured by RSP. We see that cloud elements with the largest R_e differences (up to 25 µm) tend to be associated with the smallest transect lengths (< 1 km) and the smallest retrieved COT. As the transect length increases to 3–5 km, the large R_e differences drop rapidly, showing no further dependence on transect length. Smaller transect lengths are strongly associated with smaller clouds, with mean cloud-area-equivalent diameters ∼ 1.1 to 1.7 times larger than the mean of random linear transects though fields of cumuli (e.g., Barron et al., 2020; Romps and Vogelmann, 2017). Figure 8b reveals that clouds with the smallest sizes have the largest R_e differences between the two techniques. These smaller clouds are associated with smaller retrieved COT. These 1-D retrieved COT values may be biased low by 3-D effects, such as shadowing and leakage of photons out the side of clouds (e.g., Marshak et al., 2006). From Fig. 8b, it is safe to conclude that the RSP cloud element analysis reveals that the largest R_e differences are associated with clouds that are optically thin and small in horizontal size.

Sub-pixel heterogeneity and clear-sky contamination

Several studies have pointed out that the bi-spectral R_e retrieval has a sensitivity to instrument resolution due to (a) the nonlinear relationship between VNIR and SWIR reflectances and the COT and R_e and (b) the presence of variability in cloudy reflectances at all scales (e.g., Marshak et al., 2006; Zhang et al., 2012, 2016; Werner et al., 2018a). An important example of this effect is clear-sky contamination, in which cloudy radiances and clear-sky radiances are both present within the field of view (FOV) of the sensor. The presence of this sub-pixel clear sky can cause R_e overestimates of up to 41 % when decreasing instrument resolution from 30 m to 1 km (Werner et al., 2018b). The bias in the bi-spectral retrieval due to clear-sky contamination decreases monotonically as instrument resolution increases. This is due to the applicability of an independent column approximation as a model of the variability within the FOV due to the negligible atmospheric and surface scattering contributions over ocean surfaces to VNIR and SWIR radiance. Other reflectance variations within cloudy portions of an instrument FOV also cause a sensitivity of the bi-spectral retrieval to instrument resolution, although this is typically smaller (1–3 µm) (Zhang et al., 2016; Werner et al., 2018a). In this case, increasing instrument resolution does not necessarily cause a monotonic reduction in retrieval bias (e.g., Davis et al., 1997; Zhang et al., 2012). This is because the relationship between the heterogeneity of the optical and microphysical properties (e.g., R_e) within the cloud and the radiance field is governed by 3-D radiative transfer, not an independent column approximation. Note that polarimetric retrievals are only weakly sensitive to instrument resolution, as they are largely unaffected by clear-sky contamination (e.g., Miller et al., 2018; Shang et al., 2015). Based on these considerations, we assess the sensitivity of our bias estimate in the bi-spectral R_e due to the relative coarse resolution of the RSP by investigating the impact of clear-sky contamination using the higher-resolution HSRL-2 instrument. We may then attribute the remaining bias in R_e to the expression of cloud heterogeneity and 3-D radiative transfer, whether this occurs at resolved or unresolved (i.e., sub-pixel to HSRL-2) scales.

For CAMP²Ex, given the statistics of the sampled clouds (Fig. 2), the abundance of small (transect length) and optically thin clouds (Fig. 8) led to our investigation of clear-sky contamination of the observed R_e differences between RSP bi-spectral and polarimetric retrievals. The RSP instrument's 14 mrad instantaneous field of view (IFOV) converts to a roughly ∼ 120 m horizontal footprint for the cloud retrievals. We used HSRL-2 2 Hz data (∼ 75 m horizontal resolution) to derive the cloud fraction (CF) for each RSP cloud element to identify if RSP-identified cloud elements could contain some unresolved clear sky. The higher resolution and greater sensitivity of the HSRL-2 relative to RSP plays out in Fig. 2d and e cloud and clear length statistics, with the largest differences occurring for lengths < 200 m. The good agreement between the two instruments for in-cloud lengths > 200 m and for CTH (Fig. 2c) indicates similar sensitivity to clouds larger than the RSP footprint. Hence, using HSRL-2 CTH retrievals to derive a cloud fraction for RSP-identified cloud elements should provide a relevant sub-pixel cloud fraction for the RSP retrievals. However, this would be a minimum estimate of clear-sky contamination in RSP cloud elements, as the lidar itself may also contain unresolved clear sky. For RSP cloud elements that have HSRL-2 CTH retrievals, the HSRL-2 cloud fraction for an RSP cloud element is defined as the number of valid HSRL-2 CTH retrievals divided by the total number of HSRL-2 CTH retrievals within the RSP cloud element. We examined the CDF of RSP cloud elements against HSRL-2 CF and found the following: for clouds with transect lengths under 600 m, at least 49 % of cloud elements have CF < 0.95; for all cloud transect lengths, at least 46 % of cloud elements have CF < 0.95. This reveals that at least about half of the cloud elements have some degree of clear-sky contamination (CF < 0.95). We are interested in how the amount of clear-sky contamination can impact the observed differences between bi-spectral and polarimetric R_e retrievals.

To further investigate how R_e differences depend on the HSRL-2 CF, the cloud elements are separated into partly cloudy (CF < 0.5), mostly cloudy (0.5 < CF < 0.95), and overcast (CF > 0.95) and are then further divided into small-transect-length (< 2 km) and large-transect-length (> 2 km) populations. Figure 9 shows the PDFs of the three CF groups segregated by the two cloud size groups. For transect lengths smaller than 2 km (solid lines in Fig. 9), the three CF groups have similar distributions, with median R_e differences of 8.9, 7.3, and 8.1 µm for partly cloudy, mostly cloudy, and overcast, respectively. For transect lengths greater than 2 km (dashed lines in Fig. 9), there are no samples for CF < 0.5 (hence no red dashed histogram), and the median values of the R_e differences are 5.3 and 5.7 µm for the mostly cloudy and overcast groups, respectively. Thus, sub-pixel clear-sky contamination (derived from HSRL-2 2 Hz data) appears to account for < ∼ 1 µm in the R_e differences, with transect lengths playing a larger role as the median value of the small-size group is 2.4 µm larger than the large-size group in the R_e differences. To summarize, from the RSP cloud element analysis, the large R_e difference between the two techniques is found to be more related to optical depth and transect length than sub-pixel clear-sky contamination for the RSP warm-cloud samples observed during CAMP²Ex.

Figure 9 PDF of the R_e difference stratified by CF < 0.5 (11 % of all cloud elements), 0.5 < CF < 0.95 (34 % of all cloud elements), and CF > 0.95 (54 % of all cloud elements) for cloud elements with transect lengths less than 2 km (79 % of all cloud elements, solid) and greater than 2 km (21 % of all cloud elements, dashed). There are no samples for CF < 0.5 for transect lengths greater than 2 km.

Cloud-top bumpiness and solar zenith angle

In our cloud element analysis, we defined cloud-top bumpiness as the standard deviation of CTH for each cloud element, with the idea that it can capture the variation in cloud-top structure for each cloud element. As pointed out by Loeb et al. (1998), not accounting for sub-pixel variations in CTH in the plane-parallel assumption leads to large biases in retrieved COT, particularly at large SZAs. More recent studies have shown that polarimetric R_e retrievals are less susceptible to cloud-top bumpiness (e.g., Cornet et al., 2018). In our analysis, we used HSRL-2 2 Hz CTH to derive cloud-top bumpiness, where the 2 Hz HSRL-2 data convert to a roughly ∼ 75 m horizontal resolution. To avoid clear-sky contamination, we separated overcast cloud elements from partly and mostly cloudy cloud elements, as above. When the differences in RSP bi-spectral and polarimetric R_e retrievals are organized as a function of HSRL-2 cloud-top bumpiness, no apparent dependence was observed in the overcast cases (not shown). This might be due to the fact that the possible effects of cloud-top bumpiness on R_e differences is masked by the effect of cloud size, as the clouds with the smallest transect length also have the smallest CTH standard deviation.

Past literature that has examined the dependence of R_e bias on SZA through simulation and observation studies has typically shown SZA contributions to R_e variations of ∼ 1 to 2 µm (e.g., Zhang et al., 2012; Grosvenor and Wood 2014; Horváth et al., 2014; Ahn et al., 2018). We also examined the impact of SZA on the RSP R_e retrievals with our cloud element analysis, noting that the RSP retrievals rarely had cloud samples under low sun conditions (SZA < 60^∘), with most samples taken at a SZA of between 20 and 45^∘. Similar to the findings in Ahn et al. (2018) and Grosvenor and Wood (2014), the cloud element R_e difference does not seem to be sensitive to SZA, possibly due to the small range of SZAs under which RSP retrievals were collected, along with the complexity of the co-variability between different cloud variables and 3-D pathways (e.g., shadowing vs. illumination, leakage vs. channeling).

4.2.2 Drizzle

In our analysis, we used the maximum APR-3 W-band reflectivity over a cloud element as a proxy for in-cloud drizzle for that cloud element. We examined the sensitivity of the differences between RSP bi-spectral and polarimetric retrievals of R_e to the APR-3 reflectivity. To determine the effect of the single-mode droplet size distribution assumption in the bi-spectral technique, we further restricted the maximum of the APR-3 W-band reflectivity to ∼ 100 m from the cloud top of each cloud element. We also experimented using a column maximum W-band reflectivity, and the results are very similar to those using ∼ 100 m from the cloud top.

Figure 10 shows the R_e differences between the RSP polarimetric and bi-spectral retrievals of all of the cloud elements binned by W-band reflectivity in 5 dBZ intervals. Past studies have shown that we can be certain that W-band reflectivities < −20 dBZ contain no drizzle, those between ∼ −15 and −7.5 dBZ contain light drizzle, and those > 0 dBZ contain rain (e.g., Dzambo et al., 2019; Wang and Geerts, 2003; Sauvageot and Omar, 1987). The largest mean R_e difference of 9.2 µm (with a standard deviation of 5.6 µm) was observed for Z < −20 dBZ, and the smallest mean R_e difference of 3.1 µm (with a standard deviation of 1.7 µm) was observed for 0 < Z < 5 dBZ. Figure 10 also reveals a large range of R_e difference values (of up to 25 µm) for bins with Z < −15 dBZ, whereas a trend of decreasing range and maximum R_e difference is observed for bins with Z > −15 dBZ. While no clear sensitivity of R_e difference to W-band reflectivity is suggested, Fig. 10 indicates that observed large R_e differences, especially for bins with Z < −15 dBZ, cannot be explained by drizzle. Therefore, we conclude that it is not likely that the difference in the bi-spectral and polarimetric R_e can be explained by drizzle alone. This aligns with findings from Zhang et al. (2012) and Ahn et al. (2018).

Figure 10 RSP cloud element R_e difference using all RSP cloud elements across all RFs, binned by APR-3 W-band maximum reflectivity intervals of 5 dBZ. The orange line indicates the median for each bin, the box ends indicate the interquartile range, and the ends of the whiskers indicate the maximum and minimum values for each bin.

4.3 Consistency of the R_e retrieval representativeness from CAMP²Ex

Lastly, the consistency between R_e retrievals across all techniques is examined for the CAMP²Ex region. In doing so, we seek to gauge the representativeness of Terra MODIS R_e retrievals by comparing them to RSP (airborne remote sensing) and in situ measurements sampled across all RFs. Level 2 liquid R_e retrievals from Terra MODIS within the CAMP²Ex region (Fig. 1) from all 19 P-3 research flight days were included to derive a R_e distribution. All valid R_e retrievals from RSP and MODIS R_e are included after removing the cirrus and ice clouds indicated by the SPN-S and MODIS phase flag. For in situ measurements, again to avoid sampling differences with passive remote sensing (i.e., in situ sampling through deep convective clouds which was not sampled by RSP), samples with an 11 µm brightness temperature < 273K were removed, as indicated from AHI. Figure 11 shows the R_e distributions from MODIS, RSP, and in situ measurements. Given the difference in resolution and spectral channel, RSP polarimetric and bias-adjusted MODIS R_e values agree within 1 µm, as indicated by the median and mean values in Table 5. The two also have very similar variability, as indicated by the standard deviations. The RSP bi-spectral and the original MODIS R_e also have similar R_e statistics that agree within ∼ 2 µm of each other, but both are also 5–7 µm larger than RSP polarimetric and bias-adjusted MODIS R_e with larger standard deviations. R_e values derived in situ from the P-3 and Learjet platforms indicate median values of 11.0 and 12.4 µm, respectively, which agree with RSP polarimetric and bias-adjusted MODIS R_e values within ∼ 2 µm. Longer tails in the distributions of R_e from in situ measurements are not limited by the 30 µm cut-off in the bi-spectral retrievals LUT. Moreover, as noted in Sect. 3, they are associated with the penetration of deeper clouds at altitudes different from the CTH level observed by remote sensing, with the clouds tending to contain rain. This long tail contributes to the much larger mean and standard deviation relative to the remote sensing retrievals. Overall, R_e observations from the RSP polarimetric, the bias-adjusted MODIS R_e, the Learjet in situ platform, and the P-3 in situ platform indicated similar median R_e values of ∼ 10–12 µm, whereas MODIS R_e and RSP bi-spectral R_e show overestimates of ∼ 5–7 µm compared with the other techniques. This is also consistent with the results from the individual case studies in Sect. 3.3.

Figure 11 R_e distributions from L2 RSP polarimetric R_e, L2 RSP bi-spectral R_e, L2 MODIS R_e, L2 MODIS bias-adjusted R_e, 1 Hz P-3 R_e derived in situ, and 1 Hz Learjet (LJ) R_e derived in situ using all valid retrievals of warm oceanic clouds within the domain indicated in Fig. 1.

Table 5 R_e statistics averaged over all research flight segments for warm clouds.

To test the similarity of the different R_e distributions shown in Fig. 11, a Kolmogorov–Smirnov (K–S) test was applied to all six R_e distributions, with the results listed in Table 6. When the other five R_e distributions are compared with the RSP polarimetric R_e distribution, both the RSP bi-spectral R_e and MODIS R_e distributions have p values < 0.05, which indicates that these two R_e distributions do not belong to the same distribution as the RSP polarimetric R_e distribution. Bias-adjusted MODIS, P-3 in situ, and Learjet in situ R_e distributions, however, have p values > 0.05; therefore, we cannot reject the null hypothesis that these three R_e distributions belong to the same distribution as the RSP polarimetric R_e distribution. Among the three distributions, bias-adjusted MODIS R_e has the smallest K–S value and the highest p value, which indicates that it is overall the closest fit to the RSP polarimetric R_e distribution. The K–S results are consistent with the difference in the R_e median values from Table 5, i.e., RSP polarimetric, bias-adjusted R_e, P-3 in situ measurements, and Learjet in situ measurements have closer median R_e values that are within 1–2 µm, whereas RSP bi-spectral R_e and MODIS R_e median values are 5–7 µm larger than that from the RSP polarimetric R_e. We also examined the similarity of CTH distributions retrieved from RSP and MODIS, and we obtained a K–S statistic of 0.32 with a p value of 0.31. Therefore, we cannot reject the null hypothesis that the two CTH distributions belong to the same distribution. Similarly, when Learjet and P-3 sampling altitude distributions averaged across all flight segments are compared, the two in situ sampling distributions have a K–S statistic of 0.16 and a p value of 0.98. Thus, we conclude that the two in situ techniques sampled similar cloud fields during CAMP²Ex, and RSP and MODIS cloud samples also came from similar cloud fields during CAMP²Ex.

Table 6 Kolmogorov–Smirnov test for similarity statistics, using the RSP polarimetric R_e distribution as a reference.

While we have been using the MODIS R_e product derived from its 2.1 µm channel, we also examined the MODIS R_e derived from its 3.7 µm channel. For MODIS R_e derived from its 3.7 µm channel, the median, mode, and mean ± standard deviation are, 14.3, 12.5, and 15 ± 4.8 µm, respectively, for warm clouds over all research flights. Comparing these values to those in Table 5, we see that the MODIS 3.7 µm retrievals are ∼ 2 to 3 µm smaller than MODIS 2.1 µm retrievals; this is thought to be due to a combination of less impact from 3-D effects and differences in the vertical weighting of the two channels (e.g., Zhang and Platnick, 2011). The MODIS 3.7 µm retrievals are still ∼ 3 to 5 µm larger than the RSP polarimetric retrievals.

5 Conclusions

This paper presents the first field evaluation of satellite bi-spectral R_e retrievals in tropical cumulus cloud fields. The evaluation consists of comparison between airborne RSP bi-spectral and polarimetric retrievals of R_e as well as cross-comparison between airborne remote sensing, in situ, and satellite-retrieved R_e collected during the CAMP²Ex field campaign. Unlike previous studies that used field data for evaluating satellite bi-spectral retrievals of stratocumulus cloud fields, validation in cumulus cloud fields presents a greater challenge because they are less persistent, with fast changing cloud morphologies and complex cloud structures. Here, we take a full advantage of the capability of the RSP to provide both co-located bi-spectral and polarimetric R_e retrievals; thus, there is no sampling difference in comparing the two retrieval techniques. We show that the RSP bi-spectral R_e retrieved in CAMP²Ex cloud fields is on average overestimated by 6.0 µm compared with the RSP polarimetric R_e for the 1.2 Hz samples across the entire mission. RSP polarimetric R_e also indicates much less variability and a clear increase with CTH compared with RSP bi-spectral R_e.

MODIS R_e retrievals, which use a bi-spectral approach, are in good agreement with RSP bi-spectral R_e (median R_e difference within 2.1 µm from Table 5). The bias-adjusted MODIS R_e, based on the Fu et al. (2019) bias-correction factors, shows tight agreement with the RSP polarimetric R_e (median R_e difference within 0.8 µm from Table 5). The bias-adjusted MODIS R_e and the RSP polarimetric R_e both show increasing profiles with CTH as well as less variability compared with the original MODIS R_e. The R_e values measured in situ are in good agreement (median R_e difference within 2.8 µm from Table 5) with the RSP polarimetric and bias-adjusted MODIS R_e values. Further restricting altitudes to CTH < 2 km for shallow convection yields better agreement (median R_e within 1.7 µm) between them. Thus, these three independent techniques are in very good agreement with each other and are ∼ 5.5 to 7.6 µm smaller than the median R_e values from the bi-spectral R_e from RSP and MODIS. These agreements were found to be consistent between mission-averaged statistics (Table 5) and case-by-case comparison (Table 3). For deeper clouds containing precipitation measured in situ, in situ measures of R_e can, at times, be much larger than all of the remotely sensed R_e values.

By taking advantage of co-located RSP, APR-3, and HSRL-2 on the P-3, we further examined the differences in RSP bi-spectral and polarimetric R_e and how they relate to cloud macrophysics (cloud transect length, cloud-top bumpiness, sub-pixel cloud fraction), COT, SZA, and drizzle. We found that R_e differences (bi-spectral – polarimetric) of up to 25 µm (median 11 µm) are associated with small COT values (COT < 5). As the COT increases from 5 to 15, the R_e difference maximum decreases to ∼ 5 µm (median ∼ 3 µm). For COT values greater than 15, there is no clear dependence of R_e difference on COT. Similarly, R_e differences of up to 26 µm (median ∼ 8 µm) are associated with the smallest cloud transect lengths (< 0.5 km). For cloud transect lengths greater than 5 km, R_e differences drop to 10 µm (median ∼ 5 µm). The RSP cloud retrievals have clear-sky contamination, as revealed by higher-resolution HSRL-2 data. Clear-sky contamination is shown to have only a minor impact on R_e differences (< 1 µm) relative to fully cloudy pixels. No apparent relationships between R_e differences and SZA and cloud-top bumpiness are observed, noting that the range of SZA sampled during RSP R_e retrievals was small under moderately high sun conditions (SZA = 2 to 45^∘) and the co-variability of cloud-top bumpiness with other cloud variables. A total of 80 % of the cloud elements sampled by the RSP contained some degree of drizzle within the cloud as revealed by APR-3, and a general decreasing trend of R_e differences is observed for the lowest W-band reflectivities. Separating non-drizzle and drizzle samples by −15 dBZ reveals that, on average, cloud elements with detectable drizzle have R_e differences that are ∼ 1 µm smaller than cloud elements with no detectable drizzle.

Our analysis in Sect. 3.1 showed that most samples observed by the P-3 remote sensors came from small, optically thin, shallow clouds. The samples exhibit a large difference (∼ factor of 2) between RSP bi-spectral and polarimetric R_e retrievals. For non-drizzling shallow clouds, in situ observations compare well to the RSP polarimetric retrievals, and they show variability of within ∼ 2 µm. For these non-drizzling shallow clouds, no in situ R_e samples are as large as the RSP bi-spectral R_e. Therefore, for the shallow clouds observed by RSP during CAMP²Ex, the long-standing hypothesis of the presence of drizzle or vertical variations as major contributing factors to R_e differences between bi-spectral and polarimetric retrievals could be rejected with near certainty. Furthermore, as revealed by the HSRL-2-derived RSP cloud element cloud fraction, clear-sky contamination only has a very limited contribution (∼ 1 µm) to the observed RSP R_e differences. Thus, for shallow, non-drizzling clouds, the evidence presented herein is strongly suggestive that the dominant cause of the differences between RSP polarimetric and bi-spectral R_e observed during CAMP²Ex is due to 3-D radiative transfer and cloud heterogeneity (both resolved and unresolved by RSP) effects that lead to large positive biases in bi-spectral retrievals of R_e compared with polarimetric retrievals. For deeper clouds that contain drizzle, true in-cloud vertical variations could still be at play in explaining additional R_e differences between bi-spectral and polarimetric techniques.

For MODIS, there is a substantial number of partly cloudy pixels, as revealed by coincident, high-resolution ASTER data. These sub-pixel clouds often lead to failed MODIS retrievals of R_e, as discussed in Cho et al. (2015). Comparing the cloud macrophysical properties for CAMP²Ex reported in Sect. 3.1 with those reported for RICO and INDOEX, the CAMP²Ex shallow clouds have more numerous smaller clouds. We speculate that the reason for the maximum failure rate in MODIS cloud microphysical retrievals occurring over the western tropical Pacific, as reported by Cho et al. (2015), may be because of the high frequency of small clouds here relative to anywhere else. However, as shown in Sect. 4.3, we cannot reject the null hypothesis that the MODIS CTH distributions belong to the same distributions observed by the RSP from P-3. This provides confidence that the conclusions drawn from the RSP polarimetric and bi-spectral R_e comparison extend to MODIS as well.

This study also provides additional validation of the bias-adjusted MODIS R_e values reported in Fu et al. (2019), showing a mission-averaged mean ± standard deviation of 10.8 ± 3.8 µm compared with RSP polarimetric R_e values of 10.2 ± 4 µm, as shown in Table 5. Throughout, we used the upper-bound R_e bias adjustment factors of Fu et al. (2019), rather than present both upper- and lower-bound estimates, simply to reduce clutter within many of the figures. However, both were examined. Using the lower-bound bias adjustment factors leads to a mission mean ± standard deviation of 6.7 ± 3.2 µm. The RSP polarimetric R_e falls between the upper- and lower-bound estimates. Fu et al. (2019) showed that the largest regional R_e biases for marine liquid-water clouds occur over the western tropical Pacific, and our results seem to indicate that this may be because of a higher frequency of smaller clouds here relative to everywhere else. Our validation here, along with in situ validation of MODIS R_e from other regions (e.g., Painemal and Zuidema, 2011; Ahn et al., 2018), provides additional confidence in the global distribution of bias-adjusted MODIS R_e reported in Fu et al. (2019).