the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
New particle formation event detection with Mask R-CNN
Peifeng Su
Jorma Joutsensaari
Lubna Dada
Martha Arbayani Zaidan
Tuomo Nieminen
Xinyang Li
Yusheng Wu
Stefano Decesari
Sasu Tarkoma
Tuukka Petäjä
Markku Kulmala
Petri Pellikka
Download
- Final revised paper (published on 25 Jan 2022)
- Preprint (discussion started on 13 Sep 2021)
Interactive discussion
Status: closed
-
RC1: 'Comment on acp-2021-771', Anonymous Referee #1, 23 Oct 2021
The manuscript by Su et al focuses on automatic identification of new particle formation (NPF) events with particle number size distribution surface plots from filed observations. A deep learning model, Mask R-CNN, is introduced to identify NPF events in this work. Compared to traditional manual classification, the Mask R-CNN model shows much higher efficiency and prevents bias from subjectivity. In addition, the model can determine the growth rates and start & end times for NPF events and the statistical characteristics of identified NPF events also are analyzed. The paper is within the scope of ACP and is recommended for publication after the following suggestions/comments are addressed.
Major:
- In the section of 2.3 Mask R-CNN, description of the model is rather brief. The readability will be enhanced if authors provide a general description of convolutional neural networks and more details of the model, especially for ones not familiar with deep learning.
- The Mask R-CNN model in this work was tuned with fixed training-validation ratio (300/58). Is testing set not necessary for the evolution of the model? Besides, the reason for choosing the training-validation ratio (300/58) and image size (256 × 256 pixels) should be explained, although it is mentioned that the Mask R-CNN model is insensitive to the sizes and aspect ratios of the input NPF images. (line 234-235)
- As stated in line 123, there seems no distinct boundary between â ¡-type NPF events and the Undef types, i.e. the overlapping between different types may occur. Hence, the uncertainty due to the overlapping should be discussed in the main text.
- In line 142, the value of objectiveness score is limited within the range of [0, 1]. However, how the value of objectiveness score corresponds to the exact classification type is not clear in the main text. This would originate from the characteristics of Mask R-CNN, and the authors should give an explanation.
- As shown in Table 1 & 2, the accuracy/performance of the model is dependent on the threshold of the objectiveness score, and the threshold of objectiveness score could vary dramatically when applying the model to other datasets. Can we simply set the threshold 0 to get the maximal accuracy? The authors should discuss the general criterion to choose the threshold of the objectiveness score.
- The Conclusion section is too plain. The authors may want to summarize the novelty of this work here by comparing with previous works or point out the implications for future research.
Minor:
- The title is too closed to a previous study (Atmos. Chem. Phys., 18, 9597–9615, 2018) and suggested to be reorganized.
- In line 170, one may not identify the misclassification of NPF types directly from the Tables.
- According to the description in line 195-201, there may be some errors in the first panel of Figure 7.
REFERENCES
Joutsensaari, J., Ozon, M., Nieminen, T., et al., 2018. Identification of new particle formation events with deep learning. Atmos. Chem. Phys. 2018, 9597–9615.
Citation: https://doi.org/10.5194/acp-2021-771-RC1 -
RC2: 'Comment on acp-2021-771', Anonymous Referee #2, 06 Nov 2021
I found the manuscript interesting, and the method has high potential value, and the topic is appropriate for ACP. However, I have three major concerns:
- As the number of different ways of analyzing and identifying these events grows, so does the confusion of which one should be used. When presenting a new method, therefore, it would be good to have a comparison to other automated methods. Some methods are cited, but no comparisons are given, and therefore it is impossible to know which method to choose. I would very much like to see a an intercomparison between other similar automated methods. If this is not presented, this paper will just be another in a list of methods, and the user has no information on which one to use. Also, one method that is missing is Heintzenberg et al., 2007, doi: https://doi.org/10.1111/j.1600-0889.2007.00249.x There may also be others.
- The actual phenomenon that is occurring is particle formation and growth in the atmosphere over a period over several hours, which is detectable by using a specific instrumentation and plotting this in a specific way. For example, the ‘banana’ images are a result of plotting the data in a specific way, which includes using a specific colormap with the logarithm of the log-normalized concentration density, which seems to be also hard-capped at some concentrations (1e4 in this case, based on the figures). For some situations, for example, the number concentration function value may well exceed the capping value, which changes the figure a lot. (Examples could be polluted megacities where even background concentrations can exceed 1e Also, some authors in the literature have used linear number distribution function values for plotting, and in these cases different features are highlighted. If, as I understand, this paper is very much focusing on analysing images instead of the actual data, then the various choices made in the image processing should be justified, and a section analysing the sensitivity of the model to this should be added. The authors note that the model is not sensitive to some image features such as aspect ratios, but from the text it is impossible to see whether other transformations (changes in colorscales, log/linear plotting, etc) affect the outcome. Of course, it would be highly interesting to do a double study, where the effect of such choices is studied on the human decision-making – here I think the automatic method could actually shine. But this might be outside of the scope of the study.
- There is no discussion on whether the growth rates given by either method are related to the actual growth rate of the particles. There are different method for determining the growth rate (e.g. mode method, appearance time method etc.) as noted by the authors. These have specific physical meanings and their biases are at least to some extent known. The GR given by the present method seems to be just the maximum concentration method, which has some problems (for example, an aerosol that has the maximum inside a rather wide instrumental size bin will appear not to grow), I think that the authors should add discussion on the applicability of the results to estimate the actual physical parameters.
I really think that these aspects should be addressed in the revision of the manuscript, and this will require additional work by the authors. With these revisions I would recommend publication in ACP. Otherwise, maybe a journal focusing on computer codes could be more appropriate.
In addition to these major comments, I have some specific minor comments:
- Last sentence of the abstract: What is meant by more consistent results? How is this defined? Consistent with what? If the results are consistent with manual results, how can they then be more consistent? This should be defined and clarified.
- line 212: “However, the histograms of the start times and end times determined by different methods show similar shapes (Fig. 8), illustrating the validity of the automatic method.” Is the similarity of a histogram enough to validate the method? I think direct intercomparison of earlier data , and looking at the point-by-point difference would give a more robust way of looking at whether there is a bias or other error.
- Figure 6: As one comparison is between traditional and automatic methods, could the GR given by the traditional methods be added to the figure too?
Citation: https://doi.org/10.5194/acp-2021-771-RC2 - AC1: 'Final response on acp-2021-771', Peifeng Su, 14 Dec 2021