the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
30 Nov 2021
30 Nov 2021
Assessment of NAAPS-RA performance in Maritime Southeast Asia during CAMP2Ex
- 1Department of Chemical and Environmental Engineering, University of Arizona, Tucson, AZ, 85721, USA
- 2Marine Meteorology Division, U. S. Naval Research Laboratory, Monterey, CA, 93943, USA
- 3NASA Langley Research Center, Hampton, VA, 23681, USA
- 4Science Systems and Applications, Inc., Hampton, VA, 23666, USA
- 5Department of Atmospheric Science, Colorado State University, Fort Collins, CO, 80523, USA
- 6SPEC Inc., Boulder, CO, 80301, USA
- 7Department of Hydrology and Atmospheric Sciences, University of Arizona, Tucson, AZ, 85721, USA
- 1Department of Chemical and Environmental Engineering, University of Arizona, Tucson, AZ, 85721, USA
- 2Marine Meteorology Division, U. S. Naval Research Laboratory, Monterey, CA, 93943, USA
- 3NASA Langley Research Center, Hampton, VA, 23681, USA
- 4Science Systems and Applications, Inc., Hampton, VA, 23666, USA
- 5Department of Atmospheric Science, Colorado State University, Fort Collins, CO, 80523, USA
- 6SPEC Inc., Boulder, CO, 80301, USA
- 7Department of Hydrology and Atmospheric Sciences, University of Arizona, Tucson, AZ, 85721, USA
Abstract. Monitoring and modeling aerosol particle lifecycle in Southeast Asia (SEA) is challenged by high cloud cover, complex meteorology, and the wide range of aerosol species, sources, and transformations found throughout the region. Satellite observations are limited, and there are few in situ observations of aerosol extinction profiles, aerosol properties, and environmental conditions. Therefore, accurate aerosol model outputs are crucial for the region. This work evaluates the Navy Aerosol Analysis and Prediction System Reanalysis (NAAPS-RA) aerosol optical thickness (AOT) and light extinction products using airborne aerosol and meteorological measurements from the Cloud, Aerosol, and Monsoon Processes Philippines Experiment (CAMP2Ex) in SEA. Modeled AOTs and extinction coefficients were compared to those retrieved with a High Spectral Resolution Lidar (HSRL-2). Correlations were highest for AOT in the mixed layer (AOTML; R2 = 0.83, bias = 0.00, root mean square error [RMSE] = 0.03) compared to total AOT (R2 = 0.68, bias = 0.01, RMSE = 0.14), although the correlations between the observations and 1° × 1° degree NAAPS-RA outputs were weaker in regions with strong gradients in aerosol properties, such as near areas of active convection. Correlations between simulated and retrieved aerosol extinction coefficients were highest from 145–500 m (R2 = 0.75, bias = 0.01 km−1, RMSE = 0.08 km−1) and decreased with increasing altitude (R2 = 0.69 and 0.26, bias = 0.00 and 0.00 km−1, RMSE = 0.09 and 0.00 km−1 for 500–1500 m and > 1500 m, respectively), which was likely a result of the use of bulk cloud mixing parameterizations. We also investigated the role of possible relative humidity (RH) errors in extinction simulations. Despite negative biases in modeled RH (−4.9, −7.7, and −2.3 % for altitudes < 500 m, 500–1500 m, and > 1500 m, respectively), AOT and extinction agreement with the HSRL-2 did not change significantly at any altitude when RHs from dropsondes were substituted into the model. Improvements may have been stunted due to errors in how NAAPS-RA modeled physics of particle hygroscopic growth, dry particle mass concentrations, and/or dry mass extinction efficiencies, especially when combined with AOT corrections from data assimilation. Specifically, the model overestimated the hygroscopicity of (i) smoke particles from biomass burning in the Maritime Continent (MC), and (ii) anthropogenic emissions transported from East Asia. This work provides insight into how certain environmental and microphysical properties influence AOT and extinction simulations, which can then be interpreted in the context of modeling global concentrations of particle mass and cloud condensation nuclei (CCN).
- Preprint
(2902 KB) -
Supplement
(3711 KB) - BibTeX
- EndNote
Eva-Lou Edwards et al.
Status: final response (author comments only)
-
RC1: 'Comment on acp-2021-870', Anonymous Referee #1, 24 Dec 2021
This paper evaluates the ability of the U.S. Navy’s NAAPS-RA model in reproducing observations of extinction profiles and AOT in the vicinity of the Philippines during a recent field campaign. While both anthropogenic and biomass burning aerosols are transported into this region, the high frequency of cloudiness makes using satellite measurements of AOD problematic. The low frequency of AOD retrievals also means that NAAPS-RA has little information to constrain the simulation of aerosols in this region via data assimilation. Therefore, it is useful to take advantage of airborne HSRL-2 measurements to evaluate the performance of the model, in addition to other measurements that can be used to understand the factors affecting extinction profiles. Evaluation of extinction is complicated since errors can arise from many sources as pointed out by the authors. In general, the paper is well written, but the discussion of the evaluation methodology and the interpretation of the results need improvement.
General Comments:
1) The introduction provides some motivation of NAAPS-RA, which includes aerosol-cloud relationships. However, it is not clear how that can be accomplished with an offline model. The authors do not describe how the model results are used to compute CCN and/or aerosol-cloud-radiation interactions. The paper focuses on aerosol optical properties which are important for aerosol-radiation calculations and also indirectly affecting clouds by modifying heating at the surface and heating profiles. It does not make a connection between extinction and aerosol-cloud interactions. The paper does a nice job at quantifying the errors in simulated extinction and AOT, but the evaluation seems disconnected from the introductory material. In addition, are the errors significant for other potential uses? It is not clear “how good is good enough”. It would be useful to describe in the conclusion/summary the implications of this work for NAAPS-RA applications.
2) Apparently other sources of uncertainty in simulated extinction, such as aerosol mass, is left to a subsequent study. I am torn about that approach taken by the authors. While adding that component to this paper would increase its length and complexity, the paper seems incomplete without it. After reading the paper, the main outcome is a straight-forward evaluation of the simulated aerosol optical properties (extinction and AOT) from NAAPS-RS for CampEx. Performing the sensitivity calculations with RH only partially explores the possible sources of uncertainty. Therefore, the reader is left with the perception that there are many uncertainties and conundrums (mass concentrations higher than observed while extinction is too low) that are unresolved. There is some comparison/evaluation with observed aerosol mass, but apparently there is not sufficient analysis in the present study since there are a lot of statements sprinkled in the results sections about material left to future studies. If the authors wish to leave the manuscript in its present form, they should more clearly articulate in the purpose of the present study versus a subsequent study.
There are also some aspects of the model assumptions that are not commented on. For example. The mass fractions shown in the supplemental material show relatively large mass fractions of sea-salt far above the surface. Does this seem realistic? There are lots of aerosol measurements from other aircraft campaigns that could be tapped into to at least in general confirm whether or not that seems to be a reasonable assumption. The authors mention in a couple of places that data assimilation may introduce some uncertainty in the mass fractions.
3) The measurement and modeling comparison is rather complicated. I suggest the authors look over Section 2.4 – 2.9 to explain as best as possible the methodology and possible consequences on the results. Part of this is organization. Section 2.4 just provides a broad description of the strategy, which is talked about in more detail in Sections 2.5 – 2.9 and those sections really should be sub-sections within 2.4. The evaluation strategy complication seems to arise from the flight paths and sampling strategy used. Ideally two aircraft are needed: one to obtain the HSRL-2 extinction profile sampling while the other aircraft sampling aerosol mass that coincides with the HRSL-2 measurements as was done during TCAP (Berg et al. JGR, 2016).
4) One of the conclusions of the paper is that simulated errors in RH were not the primary source of uncertainty in simulated extinction. This is not backed up with sufficient evidence. Other sources of errors seem to be left to another study and assumptions used in the aerosol water uptake calculations (which may be minimizing the sensitivity to RH) are not fully explored.
Specific Comments:
Line 124: Reid et al. (2021) has not been submitted. Papers that are in preparation or in submission stage should not be cited. If the paper is published by the time this paper is accepted, it could be included. There seems to be sufficient discussion on the field campaign measurements used in this study.
Line 151: Please include the time period of the campaign here. It is included later in lines 159-160, but it would be useful to include it up front.
Lines 151-159: The interests of the campaign could be applied to many regions of the world. What is needed here are some specifics as to the value of data collected around the Philippines. Perhaps some of the material in lines 168-170 could be moved here to provide a better motivation of the campaign for the reader.
Lines 175-177: MLH and aerosol classifications are derived products from the HSRL-2 measurements, but here they are put at the same level as the primary measurements (it measures backscatter to that is not mentioned). Most readers will not know that, so putting all this information at the same level is a bit deceiving. Perhaps including citations for HSRL-2 and the other products should be included here.
Lines 173-187: Where black carbon measurements available, i.e. from SP2? It would seem that those measurements would be useful in identifying anthropogenic and BB plumes, as well as aging of BB plumes (via coating of BC particles).
Line 186: Perhaps change “particles” to “cloud droplets and aerosol”. Just saying particles might imply to readers that only aerosols are measured.
Line 211: Does NAVGEM include feedbacks between aerosols and meteorology via radiation and clouds? In areas of high aerosol concentrations, such as the biomass burning plumes examined in this study, aerosols can affect the meteorology which would then be used to drive NAAPS-RA.
Line 219: Does the phrase “species-dependent mass scattering” mean that the model treats aerosols as an external mixture? If so, it might be useful to explicitly say that so that the reader better understands the assumptions in NAAPS-RA. Atmospheric particles are often a complex mixture of different species. Some models treat aerosol optical properties as internal mixtures which is the other extreme. In reality, aerosol populations often complex in that some regions may be more externally mixed and more internally mixed in other regions.
Lines 237-239: Aerosol water significantly affects extinction in regions of relatively high RH, but is not included as a specie in NAAPS-RA. Instead, it seems that aerosol water is diagnosed when computing extinction. The issue I have is how MODIS AOT is used for assimilation. If the data assimilation process adjusts the four species to be close to the observed AOT when neglecting aerosol water, then the NAAPS-RA should always exceed the observed AOT once water uptake is accounted for. I am probably missing something important here that is not described.
Line 338-340: It would seem that a more appropriate comparison is to average the 15-m HSRL-2 range gates within the model vertical grid cell rather than just take the points closest to the mid-point of the model grid. I assume the model is assuming an average within its cell, so a coarse grid spacing will not resolve large gradients in extinction. So averaging the HSRL-2 data would seem to be a better approach, but it may not change the overall conclusions of this study. This can also be applied to the dropsonde comparison described starting on line 344.
Line 254: After reading Section 2.4, not using the other wavelengths from HSRL-2 seems to be a missed opportunity. Instead, the evaluation focuses only on 550 nm. Is that because NAAPS-RA does not account for aerosol size distribution? Atmospheric models that can account for aerosols in their radiation calculations simulate their effect on all wavelengths, not just at one. Or is AOD at 550 nm the primary purpose of NAAPS-RA? Some discussion would be useful to describe why the evaluation focusses only on one wavelength.
Line 366: I assume that only FCDP measurements outside of clouds are used.
Line 397: This may be an overly broad statement. There are a wide range of aerosol models and the degree to which aerosols are parameterized varies. NAAPS-RA does have a simple treatment, since it only predicts bulk aerosols for four species. Other aerosol models are more explicit in predicting size resolved mass and number for a larger number of species.
Line 401: I understand trying to make the connection between the evaluation and CCN; however, the use of “representative” is misleading in this context. The authors are evaluating extinction, but CCN depends on aerosol size (which is neglected in NAAPS-RA) and hygroscopicity (via relative mass specie contributions). As noted by the authors in other places of the manuscript, extinction and AOT can have compensating errors – so how extinction alone relates to CCN is problematic.
Line 400-403: This text is confusing. First they state that the performance focuses on the ML (even though earlier they note 3 layers that are used for the evaluation), then they say evaluation of the performance in the PBL is the subject of another paper. Is there a difference in the ML and PBL, since these terms are often treated interchangeably? This seems to be the second area (in addition to aerosol mass?) that is left to another study?
Line 405: The authors mention one HSRL-2 profile is used in the 1 x 1 deg box. Why not horizontally average the extinction profiles within the 1 x 1 deg box? The authors do not show any time-height profiles of HSRL-2 extinction to know whether there are large spatial gradients or not.
Line 406-407: I am confused by this statement. It sounds like the nephelometer, AMS, and FCDP measurements were usually not available in the 1 x 1 box. Is this because that box is chosen because of the dropsonde location which is made at high altitude? So you are using that data at lower altitudes (which may be in a different 1 x 1 box) for comparisons? This is obviously not ideal but one has to deal with the aircraft measurements you get. Ideally would be useful for the aircraft to also obtain an aerosol profile in the same column as the dropsonde – but I assume this rarely happened. It would be useful to reiterate the assumptions here. The comparison methodology is getting quite complex at this point.
Line 465: The way this sentence is phrased implies the observed MLH is biased, but I assume the authors compared the model MLH to each of the three dropsonde methods and the HSRL-2 and the bias refers to the model. If this is not a comparison with the model, what does the bias in Table S2 refer to?
Lines 493-499: What is missing in this paragraph is noting that while the correlation is reasonable, there is still a lot of scatter in Fig 2 with some differences as large as two orders of magnitude.
Line 514: Does NAAPS-RS include wet-scavenging? I do not recall that being mentioned in the model description. 1 x 1 deg grid spacing is coarse, but I assume the parent meteorological model would simulate clouds and precipitation in some way that could be used for wet scavenging?
Line 523: Doesn’t Table S2 contain the model bias in MLH?
Lines 523-529: Seems that another explanation might be the assimilation of MODIS AOD and how that is handled in the vertical. Has past evaluations of NAAPS-RS provided any guidance on that? Although there are not many retrievals in this area, presumably aerosols from other regions (which would be subject to assimilation) would be advected over the Philippines.
Line 537: Does this statement mean vertical variability between 145 and 500 m or horizontal variability of extinction I that layer. Not clear.
Line 552: Change “Biases” to “Flight-averaged biases”. Figure 2 has the biases for each profile, but it looks like Fig. 3 averages them for each flight.
Lines 570-578: I wonder if Figures 5 and 4 can be combined in some way to highlight the differences which are difficult to see currently. Is it important to differentiate the flights in these plots? If not, the two figures could be combined showing the simulated AOTs using the model vs observed RH as different colors. Then the original figures could be moved to the supplemental information.
Line 576: I agree that other parameters in the model might be contributing to uncertainties in the simulated AOT, however, I would have expected changing RH would have had a much larger effect. Figure 4 indicates there were some cases in which observed RH was 20% higher than simulated below 500 m – and the differences could be larger at higher altitudes. Since NAAPS-RA is using simplified techniques to represent aerosols – how good is its method of computing aerosol water uptake? There are aerosol box models available with complex thermodynamical representations that could be used to estimate aerosol water uptake and compare those results with the methodology in NAAPS-RS.
Line 620-627: I appreciate this discussion on the simulated mass concentrations with relation to observations. In line 626 the authors say that mass concentrations need to be increased, but the fine mode mass is similar to observed and coarse mode mass is higher than observed. So the authors are saying they would have to create another error to fix a current error in extinction. There is a mystery here, and it seems that another study would be needed to understand the true source of error(s) in the extinction calculation. I wonder if the source of uncertainties are the assumptions used in the simple treatment of hygroscopicity and/or aerosol water.
Figure 6 - 8: It would be better to plot a) and b) in the same panel to better see the differences between the simulated and observed RH. After reading the figure caption multiple times, I still do not understand what the vertical black line and gray shading is.
Line 657: The authors mention the possibility of aerosol mass increasing with height. Why not use the AMS measurements confirm this? Are some of the contradictions (i.e. simulated aerosol mass larger than observed when simulated extinction is slightly lower than observed) due to the different boxes (Fig. S2) where aerosol mass and extinction profiles are compared? With smoke plumes, there could be large spatial gradients.
Line 677: Again it would seem that the AMS could be used to evaluate the ABF and smoke species in NAAPS-RS. At the end of the paragraph, they state that more work is needed – so I presume this will be the subject of a future paper?
Lines 781-782: I felt that only the authors only presented some preliminary speculation as to what the possible errors may be. There were no concrete conclusions here regarding what the specific errors for specific cases, so no tangible understanding is provided in this paper.
Line 802: The conclusions are probably not applicable to the entire modeling community. It seems that the uncertainties are largely applicable to NAAPS-RA, and perhaps to other similar classes of aerosol models such as GOCART.
Lines 805-807: Are there other studies evaluating NAAPS-RA in other locations with other field campaign observations that might have urban and biomass burning sources? If so, it would be useful to compare the present work with results from those locales.
-
RC2: 'Comment on acp-2021-870', Anonymous Referee #2, 17 Jan 2022
The manuscript describes an evaluation of the performance of the NAAPS Aerosol Reanalysis over the Philippines region based on airborne measurements conducted in the framework of CAMP2Ex. The introduction is suitable to motivate the importance of information from aerosol reanalysis models in regions such as the Philippines where cloudiness poses a huge obstacle for regular satellite observations. The subsequent presentation of the work leaves much to be desired, though. Overall, the paper is lengthy and rather unfocussed. The authors focus on presenting all they have done in the manuscript and an additional 25 pages of supplement, rather than fitting it into an easy-to-follow story for the readers. This reviewer therefore suggest major revisions before the work could be considered for publication in ACP. Some specific issues are listed below:
- The manuscript is written mostly in past tense. Please note that everything that still holds today, i.e. results show, should be in present tense.
- The Introduction is very broad and long given that the work deals with a relatively straightforward comparison of AOT and extinction coefficient from modelling and measurements and the effect of relative humidity and hygroscopic growth on the model output. Could the text be sharpened towards what is presented in the paper?
- Section 2 is too fragmented for my taste. First, it should really be “Data and Methods”, Second, the description of the measurement data, the model and its output, and the comparison methodology should be clearly structures along the lines of, e.g., (i) measurement campaign, (ii) airborne in-situ measurements, (iii) airborne lidar measurements, (iv) NAAPS-RA model description (the relevant part that is needed for this study, and (v) comparison approach and model refinement.
- Table 2: is there any source for these values?
- Mixed layer heights: Is there any conclusion on which method is used in the final comparison? I read about an abundance of methods, but later in the presentation of the findings, there’s just the parameter MLH.
- Section 2.6: The method to replace land-contaminated grid cells with the nearest neighbouring grid cell over open water is problematic as it leads to comparing apples and oranges. Better omit these data points rather than introducing comparisons that complicate the entire procedure. It might be better to instead relax the criterion that lidar profiles need to be in the vicinity of a dropsonde release to increase the number of comparison pairs. The findings later show that the shift to dropsonde RH has little effect on the modelled aerosol-optical properties.
- Later in the text the authors refer to 1 degree grid. It is not clear of this is a reference to an individual grid cell or a sub-grid within the cell.
- Section 2.8: It doesn’t make any sense to me that the authors compare lidar extinction coefficients at altitudes closest to the mid-point of a model pressure layer to the modelled value. Lidar profiles can be very noisy and picking a value at just a single height risks to introduce all this noise of real-world data into the comparison. I’d suggest to work with a mean lidar extinction coefficient averaged over the width of the height layer covered by the corresponding pressure layer.
- Figure 2: please consider a different presentation of the data, e.g. as 2d histograms. It is really hard to extract useful information from the point clouds that are currently presented. The information regarding the research flight is not necessary here, as it is presented also in the next figure. In the context of this figure, I wonder if there is a minimum AOT that NAAPS-RA can represent?
- Figure 3: please revise into a format that allows to extract the information; or present the findings as a table?
- Figure 4: see comments on Figure 2. All I can see is noise.
- Figure 5: Omit or move to supplement as there is almost no change compared to Figure 2.
- Case studies: I understand that it is very interesting to assess the performance of the model under very different aerosol conditions. However, the current presentation of the case studies leaves much to be desired. Rather than shedding light on the model’s performance, they are raising more questions than they answer. It is not sufficient to simply leave the deeper investigation of the issues raised by the case studies for later publications. The authors should at least formulate a solid hypothesis as to the nature of the inconsistencies.
- Table 3 and approach to adjust gamma: Is it possible to revise the table for easier access to the findings; maybe by use of colour coding? Also, does it make sense to just adjust the modelled gamma to the value provided by the in-situ measurements? I reckon that the modelled value is the result of some mixing of the numbers in Table 2. Does this allow for a deeper view into the contribution of the individual aerosol types to gamma by adjusting the individual values that go into the mixing rule rather than exchanging the output? This might also lead to more consistent adjustments? Or are you doing it like this already?
- The modelled aerosol optical properties seems to be dominated by the fine mode. It is surprising to me that a modelled coarse-mode mass that is one order of magnitude larger than the measurements is supposed to have no effect. Or is this effect systematic as the same difference seems to be found for all case studies? I think that this topic should be explored deeper in the paper.
- The work ends with a short list of conclusions that don’t seem to warrant to amount of material presented in the manuscript and the supplement. I recommend that the authors either streamline the presentation into a short paper or make an effort in exploring all the questions raised by the case studies.
Eva-Lou Edwards et al.
Eva-Lou Edwards et al.
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 209 | 52 | 8 | 269 | 24 | 6 | 4 |
- HTML: 209
- PDF: 52
- XML: 8
- Total: 269
- Supplement: 24
- BibTeX: 6
- EndNote: 4
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1