Dear editor, dear authors,

This manuscript compares two types of models for the retrieval of the cloud base height (CBH) and investigate in more depth the retrieval of its diurnal cycle. The authors use geostationnary satellite data from the Himawari-8 Advanced Himawari Imager (AHI) instrument to describe two physics-based (IDPS CBH and CLAVR-x CBH algorithms) and two machine learning (RF VIS+IR and RF IR-single algorithms) CBH retrieval methods. These methods are first introduced, then evaluated against Calipso/Cloudsat joint CBH retrievals and eventually compared to surface-based lidar and radar measurements from two instruments in China. The physics-based algorithms are built on cloud properties such as cloud optical thickness, cloud phase or cloud top height from the corresponding cloud products of the geostationnary satellite. On the other hand, the machine learning algorithms are built on measurements from infrared and visible bands, along with NWP data including for example temperature profiles or precipitable water. The paper investigates interesting questions regarding the benefits and drawbacks of the two types of models presented. However, the ambiguity about the Calipso/Cloudsat data source on which the machine learning models are trained and then evaluated alongside the two physics-based models renders the evaluation ambiguous and could benefit from some clarifications. The comparison to surface-based measurements allows for further comparison and evaluation of the four algorithms. Furthermore, additional details about the diurnal cycle of CBH  in the observations and in the retrievals should be emphasised clearly in the corresponding sections. Only the ML-based IR-single method is able to retrieve CBH during nighttime, hindering the evaluation of the full diurnal cycle of CBH from the four retrieval methods.

Detailed comments are included in the following sections, followed by more technical comments.

Specific comments:

    • Training and evaluation Cloudsat/Calipso datasets:

        ◦ In section 3.3, what is the reason for the different amounts of points in the dataset between the two methods? Just a clarification that it stems from the fact that only daytime data can be used for the VIS+IR method should be added here (if that is actually the reason).

        ◦ The random splitting of the data for training and testing might lead to spurious correlation when evaluating the method due to auto-correlation between samples. Furthermore, are the two datasets split in a way that the testing datasets contain the same samples?  Splitting the data according to spatiotemporality could circumvent such issue, but as the dataset is built on a single year it might prove difficult to properly investigate here.

        ◦ I did not quite understand in lines 351-353 if you mean that the training datasets are also used during validation with the Cloudsat/Calipso product? 

        ◦ Generally, I find it unclear on which samples the evaluation is built compared to the training samples for the ML-based algorithms. Details should be included either in section 3.3 or at the beginning of section 4 because it is crucial in the comparison of the different algorithms.

        ◦ The scores of the two ML-based methods on their respective (or maybe on a common) testing dataset should be included if different from the ones presented in Figure 2.

    • Results section:

        ◦ In section 4.1, it might be interesting to include subsections to group the different aspects evaluated during the comparison to the Cloudsat/Calipso CBH retrievals to improve readability. I would suggest subsections as follow (titles might obviously need some rewording): 4.1.1 Joint scatter plots, 4.1.2 Test case. However, the last paragraph (lines 446-457) is misplaced and should be included in section 3.3 with the description of the ML-based algorithms. The first paragraph of the section (L363-372) could even be placed as an introduction to section 4.

        ◦ Similar comment can be made regarding the structure of section 4.2. Seperating the evaluation experiments would improve readability. For example, one could put the use cases showcased in figures 5 and 6 in a first section and then the results for the whole year of 2017 (figures 7-9).

        ◦ In the subsequent analysis of the CBH retrievals, the diurnal cycle characteristics of the CBH are mentioned but never clearly explained or put in context with respect to the measurements or time series presented. Additionally, only the ML-based IR-single method is actually able to retrieve CBH during nighttime, making the evaluation of the full diurnal cycle of the retrieved CBH from the different methods impossible. This should be highlighted in the manuscript somewhere.

        ◦ In section 4.2, is there a particular reason for the choice of the dates of December 6, 2018, and January 8, 2019 to perform the validation? Similar comment is valid for the second use case with the Beijing Nanjiao station. A description of the cloud scene or of the characteristics of the measured CBH time series would be a great addition to give further background ahead of the comparison.

    • Conclusions and discussion:

        ◦ Paragraph L611: As mentioned previously, the potential auto-correlation between samples in the training and testing datasets can inflate the performance skill. A comment about how the data could be split to evaluate the generalisation skill to unseen locations or time periods could be included here. Following the subsequent analysis of the performance of the different ML-based algorithms, either the main limitation is the representativeness of the training data, or the potential overfitting of the models to the modality of the training dataset.

        ◦ Paragraph L629-636: A short comment on the avenue of research with Physics-informed ML could be interesting. ML models building on known or trusted physical relationships could potentially bridge the gap in the case of problematic and challenging retrievals for cloud properties for example.

    • L40-42: Mention on which dataset are the ML methods outperforming the other two and for which method is the R score is given.

    • L45 and L53: Same as previous comment, to which method is the R score referring?

    • The comparison to MODIS data included in Appendix A and the supplementary materials is interesting to gain insights on the joint distributions of the cloud properties and evaluate the Himawari-8 AHI instrument. However, if I am not mistaken, the methods presented in Baker at al. (2011) are based on data from VIIRS instrument. A comparison to the corresponding cloud product could also be of interest to assess the portability/transferability of the algorithms.

    • L338-339: Several hyperparameters of the random forest models are listed but only the chosen number of estimators and the maximum depth are mentioned.

    • L447: A reference or a sentence about how the importance scores are computed would be useful.

Additional specific/technical comments:

General comment: The spatial and temporal adjectives when combined should write as spatiotemporal (eg. L580) or spatiotemporally (eg. L120, L582).

Across the manuscript, when an acronym is introduced first, you should refer to it in the rest of the manuscript. For instance, cloud optical thickness is first introduced at line 128-129 without the acronym, then at line 194-195 with the acronym and again introduced at line 268 with two different acronyms. Another example is the cloud mask (CLM) mentioned line 194 and at line 198 in plain text. Similar comments follow for example for cloud geometric thickness (line 264) and cloud top height (line 265). Overall, these inconsistencies render reading sometimes a bit difficult with the constant switching between plain text and acronyms. The authors should make sure that throughout the manuscript, the acronyms are introduced first and then used consistently. The acronyms for the cloud properties could be introduced at line 71 for example.

Generally, when used in the text and not in parentheses, the figure mention should be included in full: “Figure 1 displays …” (L365), “Figure 2 presents …” (L373), “Similar to Figure 5, Figure 6 …” (L529)

L84: “… ramifications of clouds …”.

L114: The reference needs to be properly included: “A recent study by Yang et al. (2021) …”.

L119: The reference needs to be properly included: “For instance, Wang et al. (2012) …”.

L124: “… corresponding CBH …”.

L126: The reference is not included in full for Hutchinson et al. (2003? 2006? or both?).

L134: Drop the “previous”.

L143: Please include the reference for the ERA5 dataset.

L139-149: The references for Tan et al. (2020), Lin et al. (2022) and Tana et al. (2023) are included twice in the respective sentences describing their methods. The references at the end of these sentences can be omitted.

L189: “… facilitate …”.

L210: “… global high-quality …”.

L215: Include the references for the MODIS Cloud product Collection 6.1: Platnick et al. (2017).

L274: “… multi-layered cloud systems …”.

L289: Do you mean the GOES-R geostationary satellite imager?

L320: “… regression Random Forest model …”.

The equations 2-6 could be included in a different section, for example at the beginning of the method or result sections, as the metrics are used for comparing all the methods and do not only pertain to the ML-based algorithms.

L325-326: Include the formulas for the air mass predictors as a mathematical equation object.

L379: “Seaman et al. (2017) …”.

L385: Valid for other instances but the metrics should be reported with the same precision consistantly.

L447: “… importance scores …”.

L521-523: Is the verb (ie. an “are”) missing in this sentence? 

L532-534: Monthly aggregated results are stated to be in the supplements but are not included.  Furthermore, this sentence should be included earlier when the yearly dataset is presented, namely line 508.

L580: Data from 2019 is also used (comparison with Lijiang station).

L593: Clarify that the ML-based IR-single algorithm can retrieve cloud base during both day and night, “throughout the day” is a bit ambiguous.

L603-605: Clarifiy for which methods among the physics-based and the ML-based are the R and RMSE metrics obtained.

L608-609: “… notable differences are observed in the CBHs from both ML-based algorithms.” Clarify the sentence: Are the differences observed between the two ML-based methods or between the ML-based methods and the observations.

L733: Acronym of cloud top temperature.

L743 the Figure B3b is mentioned but not included in the manuscript or supplements.

Table 1 should be reformatted. Column names should be included. Combining together the common predictors used from satellite measurements for both methods would greatly help make sense of the difference in the input datasets.

Figure 3: the markers could be bigger (at least in the legend) as the colors are not very distinct between the different methods and appear quite small on the plot.

Figure 5: In the figure description, it refers to figure 6 and not figure 5.

Figure S4: All x-axis legends are misspelling MODIS as “MOIDS”.

 

The paper provides many insightful results from both physics-based and ML-based algorithms and tremendous observational data sets for satellite cloud base height retrievals. I appreciate the authors' hard work which must include large data set processing. The methods and results are fairly well described in general. However, some major aspects of the analysis and additional information should be provided and clarified, which may lead to different conclusions, depending on the details.

Comments/suggestions:

Introduction: The literature survey on CBH retrievals using satellite observations seems quite narrow, and not very clearly described even though for instance the authors address “two” groups but I cannot find what exactly that means.  The paragraphs may be reorganized for further clarification for readers. More detailed comments are below.

For the comparisons of the 4 algorithm outputs: Do the ML algorithms target the CBH of ceilings or still for the topmost layers like physics-based CBH algorithms? I cannot find the details, which might give very different results depending on which is focused on.

In particular, I cannot find sufficient discussions on multilayer cloud cases (how to treat them in each algorithm and how to consider those scenes in comparisons which may give very different analysis results and conclusions).

Section 4.1: So you use two physics-based CBH methods which are highly dependent on CTH accuracy, right? Then, it would be worth addressing the CTH dependency in the comparison results, which doesn't seem to evaluate cloud top separately here. 

Additionally, when using 2B-CLDCLASS-LIDAR cloud product, which cloud base data is used here, the topmost layer’s one or the lowest? If the lowest from CloudSat is used for the two physics-based cloud base algorithm products which are designed for the uppermost layer cloud base, the comparisons wouldn’t be complete, or you should address it.

Line 250: For the matchup method, please add references or a brief description including the temporal/spatial matchup windows (here or around line 364).

Line 380: “In their results” - Their comparison was for the topmost layers for both VIIRS and CloudSat, excluding precipitation pixels and only using CTH error is within 2 km range due to CBH's large dependency on CTH. Is the same strategy used here for the comparisons? Otherwise, it should be clearly stated similar to your Fig. 3 discussion below, and if needed, the comparison should be close to apple-to-apple approaches for all four algorithms.

Line 80-81: It seems like a jump in the discussion here. It could be improved with more clarifications and emphasis on the main goal of this study regarding the cloud base diurnal cycle.

Line 100-102: I understand more details will be in the next sessions, but it would be good to add reference papers simply here first for CloudSat, CALIPSO, A-Train, etc.

Line 114-115

Line 116: “Two primary methods” - out of physics-based methods in passive sensor observations? Which two methods exactly do the authors discuss here, cloud type dependency or CGT and CTH use for CBH? Said two, but cannot find a clear cut for the two groups. If not better, it would be ok to overall write some literature survey to obtain CBH from physics-based (or statistical-based_ methods.

Line 121-126: It looks like the sentences should be divided into two: 1) a general CBH derivation method and 2) Noh et al. which was demonstrated using VIIRS. The first description would be for a general method to derive CBH by subtracting CGT from CTH.

Line 128: This algorithm: Which one, Noh et al. or Hutchison et al., or both or the others?

Line 148: “root mean square error (RMSE)” No need to use both full name + acronym repeatedly.

Line 154-165: As mentioned earlier, it would be better that these discussions on the importance of the diurnal cycle of CBH study are addressed first before jumping into line 80-82 and line 150-153.

Line 284: Add references for CLAVR-x and ACBA.

Line 288-289: It is recently being applied for NOAA GOES-R ABI as well, as part of the NOAA Enterprise Cloud Algorithms. Correct GORS-R to GOES-R.

Line 295: This sentence should be corrected such as "This algorithm is suitable for single-layer and the topmost layer of multi-layer clouds”

Line 391-393: Please double check this statement. Do the physics-based CBH algorithms use visible band data? Or you want to say the lack of CloudSat/CALIPSO data below 1 km due to the ground clutter or something? Line 417 statement should be placed first here, too.

Line 550-551: Not very correct statement. CLAVR-x CBH uses CWP input from NWP data for nighttime CBH, although the products would be degraded due to the lack of visible band information as the authors mentioned in the paper.

Line 646: Correct it to “Colorado State University”.