Geiss et al. "Understanding the model representation of clouds based on visible and infrared satellite observations"

The paper discusses biases in the representation of clouds in convection-permitting simulations with the ICON-D2 model. The authors use a combination of visible satellite reflectances and infrared brightness temperatures to derive model shortcomings. Using satellite forward operators, observation equivalents are computed from model data which allow for a direct comparison with observations. The authors contrast uncertainties in the visible forward operator to sensitivities in model parameter setting. Based on their analysis result, the authors emphasize that the assumptions on subgrid-scale water clouds are the primary source for model biases in the visible spectrum and that the representation of these clouds need to be carefully revised to make further improvement possible.

I think the present study will become a valuable resource and I recommend the publication of the manuscript in ACP after major revision.

General Remarks

The paper is in general well written and structured. The objectives are clearly stated and all arguments are well supported. I don't see that language usage is of any concern. The manuscript discusses a relevant topic in atmospheric research, advanced analysis techniques are applied and the resulting scientific outcome is of interest for a wider audience.

General Comments

* Relationship to ISCCP-style analysis?: Combining VIS and IR as joint histograms or PDFs is not a new technique. There are a lot of different examples in literature which use joint histograms of cloud-optical thickness and cloud-top height to assess quality of climate model, global and regional weather forecasts. Two respective examples are: Zhang, M. H., et al. (2005), Comparing clouds and their seasonal variations in 10 atmospheric general circulation models with satellite measurements, J. Geophys. Res., 110, D15S02, doi:10.1029/2004JD005021.  & Otkin, J. A., & Greenwald, T. J. (2008). Comparison of WRF model-simulated and MODIS-derived cloud data. Monthly Weather Review, 136(6), 1957-1970. and much more references therein and also based on these papers. It feels like you had completely overlooked this branch of studies and their relationship to your research. Please add a comprehensive discussion on this topic in your introduction and in your results section (where it is appropriate).

* Connection to solar power prediction: To my opinion, the analysis that tries to establish a connection between satellite data and global irradiance measured at surface is the weakest part of your manuscript. I guess you try to make the argument that solar power prediction would improve if the representation of clouds (measure from space) is becoming more realistic. However, your analysis and the presented arguments do not support such a conclusion by now. I recommend you to revise the analysis in Sect. 3.3. It would be beneficial for the reader to show how biases in GHI are correlated with the biases in VIS and IR108. One would expect that lower GHI biases coincide with lower VIS biases which would support the conclusion that the use of visible satellite data is beneficial for ground-based irradiance predictions.   

* Figure Quality: Please make sure that font sizes in your figures (e.g. axis labels, legends) are sufficiently large.  Text in figures should not be significantly smaller than the text in the figure caption. Please update your figures accordingly!

Detailed Comments

L. 11: "modified ...settings": Please rephrase to make more clear that both, variations in model settings and forward operator uncertainties, have been considered.

L. 16: "VIS solar reflectance and global horizontal irradiance": Please make clear that the former in measured at TOA and the latter at surface.

L. 17: "will coincide" -> "can enable"?

L. 35: "are usually ... smaller" - Please support this statement with references!

L. 45: "minimization" -> reduction

L. 46/47: "Unfortunately, ...." Statement is very general and for sure not true for all current NWP systems. Please make it more specific and supported by references!

L. 51: "meteorological sensitive areas": Unclear what this means.

Fig. 1: Does not appear to be referenced in the right order. Labels are too small.

L. 58: "solar irradiance fluctuation" + "at surface" (or at ground). Also this statement needs to be support by a reference.

L. 65 full paragraph: This needs like a conclusion paragraph and is not in the right place here. Please rephrase and complete paragraph. This is the place where you can state your research question and outline how approach your research goal.

L. 78: "cloud climatology": Here, and everywhere else: Please avoid the term cloud climatology which is mis-leading because it refers to long-term (!) cloud statistics which is not the case in your study. Please use "time-mean statistics" (or "time-average") instead.

L. 86: "ICON-D2". Please name the model version here.

L. 96: "We performed six" + "additional"

L. 98: "The objective was to ..." Please rephrase sentence.

eq. (1): needs more explanation! I guess this is only a partial contribution to total cloud cover (might be indicated by subscript cc_turb). Is this cc contribution just added to the other contributions? What is q_sat? And where does the scheme come from (reference) and how should it be interpreted? Parameter B needs to be explained as well.

L. 125: "like the operational one." -> "like the operational one-moment scheme."

L. 129: "cloud-concentration number" -> cloud droplet number concentration"

L: 135: 2*10^4 to 4*10^4 hPa: This is definitely too large! Wrong units?

L. 146: "visible 0.6 um channel" Please specify if the visible reflectance is corrected by solar zenith angle. If yes, comparison in Fig. 8 would be inappropriate because GHI is scaled by a constant.

L. 151: "TCW" / TCI": I would prefer "LWP" and "IWP", liquid-water path & ice-water path is more commonly used.

Section 2.3 misses to tell how aerosol is treated.

Eq. (2): 

* Consistency of symbols: You use low-case q in eq. (1) for content. And you use capital R as reflectance later. I suggest to use consistent symbols.

* Is this equation consistently applied to visible and infrared? Please comment on this aspect.

* How does this method compares to the generalized effective diameter in Senf and Deneke (2017), AR, eq. (B.3)? 

L. 192/193: This is much too short! SGS clouds play an important role in your analysis. Please be much more explicit about your treatment of SGS clouds. What are the assumptions about microphysics (effective radius, adiabaticity) of SGS clouds? How does this impact cloud-optical thickness?

L. 197: "calibration offset" To my opinion, you are removing a systematic bias from the simulation which is fine in general. However, I would phrase it in that way.

L. 201: "spatial resolution" -> Please move to Sect. 2.2.

Sect. 2.4: What is the accuracy of GHI measurements?

L. 215: "... without coarsening and thinning" -> unclear

L. 220: I don't understand why you don't take the observation as a reference: eps = P(SIM) - P(OBS)?

Violin plots:  I would recommend to skip the distribution outside a certain range (<10th and >90th percentiles) to increase readability of the plots in Fig. 12. Otherwise these plots are dominated by the extremes.

L. 224: CFAD -> reference

L. 224ff "Standard atmosphere .... ": Don't understand why you choose this distinction. Much more natural would be <273 K, (273 K... 243 K), <243K which would separate liquid, mixed-phase and ice clouds.  

Fig. 4 + 5: Please use same projection as in Fig. 1 or the other way around. Please avoid histograms and use PDFs instead as you introduced PDFs as verifcation metrics.

L. 248: Fig. 4a & 4b -> wrong reference, 4b shows BTs.

Fig. 6 caption: White lines: What do they mean? "normalized by the sum" -> confusing. If you show PDFs then normalization is not a matter of choice: \int P(BT, R) dBT dR = 1!

Fig. 7: Observed BTs are higher than 300 K. Is the range > 300 K considered in the normalization of the PDFs?

Sect 3.2. Again, avoid the term "climatology".

L. 282 / L.284: There is a duplicate statement: "findings from previous studies"; "found in other studies" Please rephrase the two sentences.

Sect 3.3: 

* Please see my general comment. What is the general idea of this analysis? I guess you like to show that GHI forecasts can improve when VIS / BT forecast are more realistic, right? Why don't you show the bias in GHI vs. the bias in VIS? Otherwise, the reader get the feeling that plotting hourly average GHI values against instantaneous VIS observations is rather inappropriate (see L. 312 - 14). 

* Caption Fig. 8 "number of matches"-> unclear. 

* Meaning and usefulness of lines in Fig. 8 is also unclear. 

* Is scaling of GHI consistent with scaling of VIS radiances? See above.

L. 335: "imperfect parameterization" Again, a clearer description of the microphysics of SGS clouds would help. 

L. 337: flat plateau for grid-scale clouds: Would this mean that this VIS bias can be resolved by proceeding to even higher resolutions, e.g. hecto-scale simulations? Could you comment on this? Are there any indications in the literature?

L. 347: "it seems that the subgrid water cloud parameterisation needs to be improved" -> or its coupling to the VISOP?

L. 353: "missing RT effects" -> unclear

L. 382ff: In this paragraph, it is not clear how you treat aerosols in the reference run. 

L. 387: "Aerosol can scatter photons ..." Sentence reads weird. Please rephrase.

Interpretation of Fig. 11: It seems that carefully chosen aerosol can bring simulated VIS006 closest to the observation. Is this conclusion correct?

L. 398: "ice habit is thus not likely to cause large uncertainties..." This is only true because your scenes have a very high low-level cloud cover and semi-transparent cirrus overlays lower clouds, right?

L. 403: "simulation" -> "simulations"

L. 411 & l. 415: "pixels" I find "pixel" inappropriate for model data.

L. 420: "experiment VI" -> do you rather mean VII?

L. 424 / 425: "cloud top height is an important additional parameter" I guess you mean in addition to cloud-optical thickness? Please make this clear!

Fig. 12: 

* It is hard to see the differences here. The plot are dominated by the extremes. Please trim the range of the PDFs e.g. within (-2, 2). 

* Panel a & b should have the same size.

* y labels should be eps_n,d consistent with Sect. 2.5

L. 450: "to eliminate the errors of the reference run" + "in the IR108 channel"

L: 485: "solar satellite observations are novel for model evaluation". This might be true for RTTOV, but not for other forward operator methods. Please be more specific and discuss, if applicable, already existing advancements by others (e.g. in CRTM). 

L. 490: "well-suited to improve model cloud parameterisations for better PW power production forecasts" This statement could be better supported by your analysis. To my feeling, you can show that better VIS / IR108 forecasts ultimately lead to improvements in GHI predictions.

Review of “Understanding the model representation of clouds based on visible and infrared satellite observations” by Geiss, Scheck, de Lozar, and Weismann.

Review by Matthew Igel.

The manuscript presents satellite and model comparisons from 2 days during a 30-day ICON-D2 hindcast to motivate the use of visible and infrared analysis in tandem when assessing model clouds.  Then statistics from the full 30 days are shown to illustrate systematic model deficiencies.  An attempt is made to understand the source of these deficiencies by focusing on cloud parameters and parameterizations within ICON.  Tweaks to these schemes are used to motivate possible ways to improve ICON.

There is a lot of back and forth in the study design between weather models, radiative transfer calculations, and satellite observations. This does not seem to be atypical for the satellite community, but for those of us on the cloud process/modeling side who would seem to benefit most from this study, this back and forth presents an opportunity for confusion.  My biggest concern with the manuscript is not the methods, per se, but the logic of their presentation.  I think the overall experimental design needs to be made much clearer.  Why are the steps taken the right ones and taken in the right order?  I had to sketch out sequencing of the study for myself after reading the manuscript a second time to make sense of things.  Even then, some aspects of the manuscript felt out of place.

Major Concerns

1. The abstract contains lots of ambiguous sentences that simply can’t stand on their own. For example, the second to last sentence means something very specific to the authors (and to the reader after reading the manuscript) but seems very unclear to the uninitiated.  The same could be said of the final sentence and many others.
2. The paragraph starting on L65 seems very important, but has similar issues to the abstract. This is a somewhat roundabout study which focuses on a number of different things, so I think this paragraph which is intended to describe the logic of the methodology deserves to be better.  I would start by reiterating the goal of the study (which I infer to be): “the meteorologically forecasting relevant quantities for PV generation will rely on assimilating clouds well and on accurate cloud simulation.  This study is therefore aimed at improving our general representation of clouds in models by assessing current model performance relative to satellite observations. Etc”
3. Section 3.3 seems unnecessary. Maybe I’m missing something important, but the result of this section seems logical and the figure unsurprising.
4. The exact logic of section 4.1 needs to be explained. It’s not clear precisely how I should interpret this figure in general.  For example, if one of your test cases exactly recreated the OBS but REF didn’t, it’s not exactly clear to me what the conclusion would be.  What if REF and REF-Grid were exactly the same?  Should this analysis be used to draw conclusions about the success or failure of ICON or of the forward model?  I don’t need answers to these questions, exactly, but rather am trying to illustrate my lack of understanding of the logic of this section.
5. I am left wondering how sensitive the conclusions are to the cloud morphology of summer over northern Europe. Presumably column precipitate mass is mostly liquid during these months which leads you to the conclusions that liquid is ultimately important for (if nothing else) solar reflectance.  Do you feel your results are generally applicable in the context of a weather model that may need to simulate lots of different cloud states over the course of a year?

Minor Concerns

L96: How did you determine what is physically plausible?

L100: These seem arbitrarily chosen.  How were these chosen before the study or were they chosen as a result of initial data analysis?

L126: Similarly, why these seven (especially for VI and VII)?

L176-L191: You mean the effective radii calculated by the ICON radiation scheme, correct?  Not the geometric radii?

L196: You mean you followed the procedure of Meirink by replacing MODIS with your ICON radiances?  Why is this a necessary step?  Without it, might you have usefully inferred a model bias?

L242 and L273: Use of the word “exemplarily” feels a little out of place.

Fig. 6:  Do “difference plots” help to highlight anything that isn’t obvious simply by showing observation and simulation results side by side?

General: It feels as though there are a lot of acronyms that have been defined but are not used very much.  You may not need to define as many as you do.

Section 4.2: I don’t feel as though I have sufficient background knowledge to judge this section.

L461: Why shouldn’t they be included?