Clouds, comprising visible aggregates like atmospheric water droplets, supercooled water droplets, ice crystals, etc., cover roughly 70 % of the Earth's surface (Stubenrauch et al., 2013). They play a pivotal role in global climate change, the hydrometeor cycle, and aviation safety and serve as a primary focus in weather forecasting and climate research, particularly storm clouds (Hansen, 2007; Hartmann and Larson, 2002). From advanced geostationary (GEO) and polar-orbiting (low-Earth orbit, LEO) satellite imagers, various measurable cloud properties, such as cloud fraction, cloud phase, cloud top height (CTH), and cloud optical thickness (DCOT), are routinely retrieved. However, high-quality cloud geometric height (CGH) and cloud base height (CBH), a fundamental macrophysical parameter delineating the vertical distribution of clouds, remain relatively understudied and underreported. Nonetheless, for boundary-layer clouds, the cloud base height stands as a critical parameter depending on other cloud-controlling variables. These variables encompass the cloud base temperature (Zhu et al., 2014), cloud base vertical velocity (Zheng et al., 2020), activation of cloud condensation nuclei (CCN) at the cloud base (Rosenfeld et al., 2016; Miller et al., 2023), and cloud–surface decoupling state (Su et al., 2022). These factors significantly impact convective cloud development and ultimately the climate.

There are distinct diurnal cycle characteristics of clouds in different regions across the globe (Li et al., 2022). These diurnal cycle characteristics primarily stem from the daily solar energy cycle absorbed by both the atmosphere and Earth's surface. Moreover, vertical atmospheric motions are shaped by imbalances in atmospheric heating and surface configurations, also leading to a range of cloud movements and structures (Miller et al., 2018). Cloud base plays a pivotal role in weather and climate processes. It is critical for predicting fog and cloud-related visibility issues important in aviation and weather forecasting. For instance, lower cloud bases often lead to more intense rainfall. In climate modeling, CBH is integral for accurate long-term weather predictions and understanding the radiative balance of the Earth, which influences global temperatures (Zheng and Rosenfeld, 2015). Hence, the accurate determination of CBH and its diurnal cycle with high spatiotemporal resolution becomes very important, necessitating comprehensive investigations (Viúdez-Mora et al., 2015; Wang et al., 2020). Such efforts can provide deeper insights into the potential ramifications of clouds for radiation equilibrium and global climate systems.

However, as one of the most crucial cloud physical parameters in atmospheric physics, CBH poses challenges in terms of measurement or estimation from space. Presently, the primary methods for measuring CBH rely on ground-based observations, utilizing tools such as sounding balloons, Mie-scattering lidars, stereo-imaging cloud height detection technologies, and cloud probe sensors (Forsythe et al., 2000; Hirsch et al., 2011; Seaman et al., 2017; Zhang et al., 2018; Zhou et al., 2019, 2024). While in situ ground-based observation methods offer highly accurate, reliable, and timely continuous CBH results, they are constrained by localized observation coverage and the sparse distribution of observation sites (Aydin and Singh, 2004). In recent decades, with the rapid advancement of meteorological satellite observation technology, spaceborne observing methods that provide global cloud observations with high spatiotemporal resolution compared to conventional ground-based remote sensing methods have emerged. In this realm, satellite remote sensing techniques for measuring CBH fall primarily into two categories: active and passive methods. Advanced active remote sensing technologies like CloudSat (Stephens et al., 2002) and the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) (Winker et al., 2009) in the National Aeronautics and Space Administration (NASA) A-Train (Afternoon Train) series (Stephens et al., 2002) can capture global cloud profiles, including CBH, with high quality by detecting unique return signals from cloud layers using onboard active millimeter-wave radar or lidar. However, their viewing footprints are limited along the nadir of the orbit, implying that observation coverage remains confined primarily to a horizontal scale (Min et al., 2022; Lu et al., 2021).

In addition to active remote sensing methods, satellite-based passive remote sensing technologies can also play an important role in estimating CBH (Meerkötter and Bugliaro, 2009; Lu et al., 2021). The physics-based principles and retrieval methods for CTH have reached maturity and are now widely employed in the satellite passive remote sensing field (Heidinger and Pavolonis, 2009; Wang et al., 2022). However, the corresponding physical principles or methods for measuring CBH using satellite passive imager measurements are still not entirely clear and unified (Heidinger et al., 2019; Min et al., 2020). A recent study by Yang et al. (2021) utilized oxygen A-band data observed by the Orbiting Carbon Observatory-2 (OCO-2) to retrieve single-layer marine liquid CBH. These abovementioned passive space-based remote sensing methods, such as satellite imagery, play a key role in retrieving CBH. In terms of detection principles, the first method involves the extrapolation technique for retrieving CBH for clouds of the same type. For instance, Wang et al. (2012) proposed a method to extrapolate CBH from CloudSat using spatiotemporally matched Moderate Resolution Imaging Spectroradiometer (MODIS) cloud classification data (Baum et al., 2012; Platnick et al., 2017). The second physics-based retrieval method first approximates the cloud geometric thickness using its optical thickness. It then employs the previously derived CTH product to compute the corresponding CBH using the respective National Oceanic and Atmospheric Administration (NOAA) Suomi National Polar-orbiting Partnership/Visible Infrared Imaging Radiometer Suite (SNPP/VIIRS) products (Noh et al., 2017). Hutchison et al. (2006) and Hutchison (2002) also formulated an empirical algorithm that estimates both cloud geometric thickness (CGT) and CBH. This algorithm relies on statistical analyses derived from MODIS DCOT and cloud liquid water path products (Hutchison et al., 2006; Hutchison, 2002).

Machine learning (ML) has proven to be highly effective in addressing nonlinear problems within remote sensing and meteorology fields, such as precipitation estimation and CTH retrieval (Min et al., 2020; Håkansson et al., 2018; Kühnlein et al., 2014). In recent years, several studies have leveraged ML-based algorithms to retrieve CBH, establishing nonlinear connections between CBH and GEO satellite observations. For instance, Tan et al. (2020) integrated CTH and cloud optical property products from the Fengyun-4A (FY-4A) GEO satellite with spatiotemporally matched CBH data from CALIPSO/CloudSat. They developed a random forest (RF) model for CBH retrieval. Similarly, Lin et al. (2022) constructed a gradient boosted regression tree (GBRT) model using US new-generation Geostationary Operational Environmental Satellites - R Series (GOES-R) Advanced Baseline Imager (ABI) Level-1B radiance data and the ERA5 (the fifth-generation ECMWF) reanalysis dataset (Lin et al., 2022; Hersbach et al., 2020) (https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5, last access: 14 December 2024). They employed CALIPSO CBH data as labels to achieve single-layer CBH retrievals. Notably, the CBH quality of ML-based algorithms was found to surpass that of physics-based algorithms (Lin et al., 2022). Moreover, Tan et al. (2020) utilized Himawari-8 data and the RF algorithm to develop a novel CBH algorithm, achieving a similar high correlation coefficient (R) of 0.92 and a low root mean square error (RMSE) of 1.17 km compared with CloudSat/CALISPO data.

However, these former studies did not discuss whether both physics-based and ML-based algorithms of the GEO satellite could retrieve the diurnal cycle of CBH well. This gap in research could mainly be attributed to potential influences from the fixed LEO satellite's (with active radar or lidar) passing time in the previous CBH retrieval model (Lin et al., 2022). The diurnal cycles of CBH have not been well investigated in both GEO and LEO remote sensing research. Hence, it is crucial to thoroughly investigate the diurnal cycle features of CBH derived from GEO satellite measurements by comparing them with ground-based radar and lidar observations (Min and Zhang, 2014; Warren and Eastman, 2014). In this study, we aim to assess the applicability and feasibility of both physics-based and ML-based algorithms of GEO satellites in capturing the diurnal cycle characteristics of CBH.

The subsequent sections of this paper are structured as follows. Section 2 provides a concise overview of the data employed in this study. Following this, Sect. 3 introduces the four distinct physics- and ML-based CBH retrieval algorithms. In Sect. 4, the CBH results obtained from these four algorithms are analyzed, and comparisons are drawn with spatiotemporally matched CBHs from ground-based cloud radar and lidar. Finally, Sect. 5 encapsulates the primary conclusions and new findings derived from this study.

In this study, observations from the Himawari-8 (H8) Advanced Himawari Imager (AHI) are utilized for the retrieval of high-spatiotemporal-resolution CBH. Launched successfully by the Japan Meteorological Agency on 7 October 2014, the H8 geostationary satellite is positioned at 140.7° E. The AHI on board H8 encompasses 16 spectral bands ranging from 0.47 to 13.3 µm, featuring spatial resolutions of 0.5–2 km. This includes 3 visible (VIS) bands at 0.5–1 km, 3 near-infrared (NIR) bands at 1–2 km, and 10 infrared (IR) bands at 2 km. The H8/AHI can scan a full disk area within 10 min, two specific areas within 2.5 min, a designated area within 2.5 min, and two landmark areas within 0.5 min (Iwabuchi et al., 2018). Its enhanced temporal resolution and observation frequency facilitate the tracking of rapidly changing weather systems, enabling the accurate determination of quantitative atmospheric parameters (Bessho et al., 2016).

Operational H8/AHI Level-1B data, accessible from 7 July 2015, are freely available on the satellite product home page of the Japan Aerospace Exploration Agency (Letu et al., 2019). The Level-2 cloud products utilized in this study, including the cloud mask (CLM), CTH, the cloud effective particle radius (CER or Reff), and DCOT, are generated by the Fengyun geostationary satellite algorithm test bed (FYGAT) science product (Wang et al., 2019; Min et al., 2017) of the China Meteorological Administration (CMA) for various applications. According to previous CALIPSO validations (Min et al., 2020), the absolute bias of cloud top height retrieved by the H8 satellite is approximately 3 km, with an absolute bias of 1 to 2 km for samples below 5 km. The accuracy of CTH is crucial for estimating CBH in the subsequent algorithm. It is important to note that certain crucial preliminary cloud products, such as CLM, have been validated in prior studies (Wang et al., 2019; Liang et al., 2023). Nevertheless, before initiating CBH retrieval, it is imperative to validate the H8/AHI cloud optical and microphysical products from the FYGAT retrieval system. This validation has been carried out by using analogous MODIS Level-2 cloud products as a reference. Additional details regarding the validation of cloud products are provided in Appendix A.

In addition to the H8/AHI Level-1 and Level-2 data, Global Forecast System (GFS) numerical weather prediction (NWP) data are employed for CBH retrieval in this study. The variables include land/sea surface temperature and the vertical profiles of temperature, humidity, and pressure. Operated by the US NOAA (Kalnay et al., 1996), the GFS serves as a global and advanced NWP system. The operational GFS system routinely delivers global high-quality and gridded NWP data at 3 h intervals, with four different initial forecast times per day (00:00, 06:00, 12:00, and 18:00 UTC). The three-dimensional NWP data cover the Earth in a 0.5° × 0.5° grid interval and resolve the atmosphere with 26 vertical levels from the surface (1000 hPa) up to the top of the atmosphere (10 hPa).

As previously mentioned, the official MODIS Collection 6.1 Level-2 cloud product climate data records (Platnick et al., 2017) are utilized in this study to validate the H8/AHI cloud products (CTH, CER, and DCOT) generated by the FYGAT system. High-quality, long-term MODIS data are often used as a validation reference to evaluate the products of new satellites. MODIS sensors are on board NASA's Terra and Aqua polar-orbiting satellites. Terra functions as the morning satellite, passing through the Equator from north to south at approximately 10:30 local time (LT), while Aqua serves as the afternoon satellite, traversing the Equator from south to north at around 13:30 LT. As a successor to the NOAA Advanced Very High Resolution Radiometer (AVHRR), MODIS features 36 independent spectral bands and a broad spectral range from 0.4 µm (VIS) to 14.4 µm (IR), with a scanning width of 2330 km and spatial resolutions ranging from 0.25 to 1.0 km. Recent studies (Baum et al., 2012; Platnick et al., 2017) have highlighted significant improvements and collective changes in cloud top, optical, and microphysical properties from Collection 5 to Collection 6.

In addition to the passive spaceborne imaging sensors mentioned above, the CloudSat satellite, equipped with a 94 GHz active cloud profiling radar (CPR), holds the distinction of being the first sun-synchronous orbit satellite specifically designed to observe global cloud vertical structures and properties. It is part of the A-Train series of satellites, akin to the Aqua satellite, launched and operated by NASA (Heymsfield et al., 2008). CALIPSO is another polar-orbiting satellite within the A-Train constellation, sharing an orbit with CloudSat and trailing it by a mere 10–15 s. CALIPSO is the first satellite equipped with an active dual-channel Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) at 532 and 1064 nm bands (Hunt et al., 2009). Both CloudSat and CALIPSO possess notable advantages over passive spaceborne sensors due to the 94 GHz radar of CloudSat and the joint return signals of lidar and radar on CALIPSO. These features enhance their sensitivity to optically thin cloud layers and ensure strong penetration capability, resulting in more accurate CTH and CBH detections compared to passive spaceborne sensors (CAL_LID_L2_05kmCLay-Standard-V4-10). The joint cloud type products of 2B-CLDCLASS-LIDAR, derived from both CloudSat and CALIPSO measurements, offer a comprehensive description of cloud vertical structure characteristics, cloud type, CTH, CBH, etc. The time interval between each profile in this product is approximately 3.1 s, and the horizontal resolution is 2.5 km (along track) × 1.4 km (cross-track). Each profile is divided into 125 layers with a 240 m vertical interval. For more details on 2B-CLDCLASS-LIDAR products, refer to the CloudSat official product manual (Sassen and Wang, 2008). In this study, we consider the lowest effective cloud base height from the joint CloudSat/CALIOP data as the true values for training and validation. Note that for this study, we utilized 1-year H8/AHI data and matched them with the joint CloudSat/CALIOP data from 1 January to 31 December 2017.

To explore and identify the optimal and most robust CBH retrieval algorithm from geostationary satellite imager measurements, particularly focusing on capturing the typical diurnal cycle characteristics of CBH over land, this study employs four different retrieval algorithms (two physics-based and two ML-based algorithms). High-spatiotemporal-resolution CBHs are retrieved using the H8/AHI data from 2017 to 2019. To assess the accuracies of the retrieved CBHs, point-to-point validations are conducted using spatiotemporally matched CBHs from the joint CloudSat/CALIOP product, ground-based lidar, and cloud radar observations in China. The main findings and conclusions are outlined below.

Four independent CBH retrieval algorithms, namely physics-based GEO IDPS, physics-based GEO CLAVR-x, ML-based VIS+IR, and ML-based IR single, have been developed and utilized to retrieve CBHs from GEO H8/AHI data under the assumption of single-layer clouds. The two physics-based algorithms utilize cloud top and optical property products from AHI as input parameters to retrieve high-spatiotemporal-resolution CBHs, with operations limited to daytime. In contrast, the ML-based VIS+IR model and IR-single model algorithms use the matched joint CloudSat/CALIOP CBH product as true values for building RF prediction models. Notably, the ML-based IR-single algorithm, which relies solely on infrared band measurements, can retrieve CBH during both daytime and nighttime.

The accuracy of CBHs retrieved from the four independent algorithms is verified using the joint CloudSat/CALIOP CBH products for the year 2017. The GEO IDPS algorithm shows an R of 0.62 and an RMSE of 2.64 km. The GEO CLAVR-x algorithm provides more accurate CBHs with an R of 0.65 and RMSE of 2.91 km. After filtering samples with optical thickness less than 1.6 and brightness temperature (at the 11 µm band) greater than 281 K, the ML-based VIS+IR and ML-based IR-single algorithms achieve higher accuracy, with an R (RMSE) of 0.92 (1.21 km) and 0.91 (1.42 km), respectively. This indicates strong agreement between the two ML-based CBH algorithms and the CloudSat/CALIOP CBH product.

However, in stark contrast, the results from the physics-based algorithms (with an R and RMSE of 0.59 and 2.86 km) are superior to those from the ML-based algorithms (with an R and RMSE of 0.39 and 3.88 km) when compared with ground-based CBH observations such as lidar and cloud radar. In the comparison with the cloud radar at Beijing Nanjiao station in 2017, the R of the GEO CLAVR-x algorithm is 0.57, while the R of the GEO IDPS algorithm is 0.52. Meanwhile, notable differences are observed in the CBHs between both ML-based algorithms. Similar conclusions are also evident in the 2 d comparisons at Yunnan Lijiang station.

The CBH results from the two ML-based algorithms (R > 0.91) can likely be attributed to the use of the same training and validation dataset source as the joint CloudSat/CALIOP product. However, this dataset has limited spatial coverage and small temporal variation, potentially limiting the representativeness of the training data. In contrast, the GEO CLAVR-x algorithm demonstrates the best performance and highest accuracy in retrieving CBH from geostationary satellite data. Notably, its results align well with those from ground-based lidar and cloud radar during the daytime. However, both physics-based methods, utilizing CloudSat CPR data for regression, struggle to accurately retrieve CBHs below 1 km, as the lowest 1 km above ground level of this data is affected by ground clutter. In general, the physics-based algorithms, such as GEO CLAVR-x and GEO IDPS, demonstrate notable advantages in capturing the diurnal cycle of CBH. Unlike ML-based methods, they offer more stable error metrics, especially with higher correlation and lower RMSE during the daytime. Additionally, they are more effective at capturing significant and natural variations in CBH, providing generally higher-quality retrievals from H8/AHI data, even though challenges remain in accurately retrieving CBHs below 1 km.

Additionally, despite utilizing the same physics principles in spaceborne and ground-based lidar/radar CBH algorithms, the study by Thorsen et al. (2011) has highlighted differences in profiles between them. Therefore, this factor induced by the detection principle could contribute to the relatively poorer results in CBH retrieval by ML-based algorithms compared to ground-based lidar and radar. The analysis and discussion above suggest that ML-based algorithms are constrained by the size and representativeness of their datasets.

Ideally, we guess that including more spaceborne cloud profiling radars with varying passing times (covering the entire day) in the training dataset could improve the machine learning technique, potentially leading to a higher-quality CBH product with more comprehensive observations. The CBH product using ML-based algorithms should continue to be improved in future work. Particularly, exploring the joint ML- and physics-based method presents a promising direction, which can address the complexities and challenges in retrieving cloud properties. By integrating established physical relationships into ML models, we can potentially enhance the accuracy and reliability of predictions. This approach not only leverages the strengths of both physics-based models and data-driven techniques but also offers a pathway to more robust and interpretable solutions in atmospheric sciences. At present, we focus on developing physics-based algorithms for cloud base height for the next generation of geostationary meteorological satellites to support the application of these products in weather and climate domains.

Moreover, at night, current GEO satellite imaging instruments encounter challenges in accurately determining CBH due to limited or absent solar illumination. Because it is unable to retrieve cloud optical depth in the visible band, the current method faces limitations. However, there is potential for enhanced accuracy in deriving cloud optical and microphysical properties, as well as CBH, by incorporating day–night band (DNB) observations during nighttime in the future (Heidinger et al., 2012).