Review "Quantification of the modelling uncertainties in atmospheric release source assessment and application to the reconstruction of the autumn 2017 Ruthenium 106 source"

The authors present a technical and modelling study of Bayesian inversion applied to the airborne Ru-106 observations made in Europe in 2017. The technical part of the paper focuses on three aspects. First, the choice of the likelihood function: three likelihood functions were proposed based on four proposed criteria which a likelihood function should ideally fulfill. Second, an alternative formulation for the error covariance matrix is proposed, in particular with the purpose to discriminate between informative and non-informative observations; the latter leads to artificially low inferred errors, which is remediated by the alternative error covariance matrix. Third, the observation operator is replaced by a linear combination using an ensemble, of which the weights are sampled by the Bayesian inference. In the second part of the paper, these techniques are illustrated by applying Bayesian inference using airborne observations of Ru-106 observations. The results show an (impressive) agreement with the most likely source location (the Mayak institute in Russia).

The topics discussed in the paper are interesting and relevant. However, I have some comments and concerns about some of the methodology. Also, at times, specific information is lacking, or the formulation of ideas is done with insufficient care, leaving the reader with confusion. I hope the authors find my comments below useful. I recommend major revisions before the manuscript can be accepted for publication.

Major comments

1/ I suggest to reformulate parts of Section 1.2:

- "An efficient way to use these forecasts to better estimate uncertainties is to combine them": it is not clear to me what the authors mean here. Please be more specific. (If you would combine the members of an ensemble into a best estimate of the true state of the atmosphere, and use only that best estimate, you would loose the uncertainty information.)

- "This approach is known as multimodel ensemble forecasting (Zhou and Du, 2010).": It is not clear to me what "This approach" is. If you use the same model with perturbed input parameters and / or perturbed physics, I would not call it multimodel ensemble forecasting.

- What is "sequential aggregation"? Do you mean sequential in time (= an ensemble of deterministic forecasts run at different starting times)?

- "An aggregated forecast is then formed by the weighted linear combination of the forecasts of the ensemble": I wonder why the authors want to create a best realization, rather than extracting the uncertainty from the ensemble. I suggest to add some discussion to explain the reasoning for this.

2/ While the discussion on the choice of the likelihood function is valuable and interesting, I'm not sure if all arguments made in the paper are valid.

- I'm not sure if I can agree with the discussion in Lines 111 - 117. There, you ignore the fact that the uncertainties will be larger for the higher observation-prediction couple; both observation-prediction couples could have the same penalty if they have the same relative uncertainty, which is not unlikely considering the error from the atmospheric transport and dispersion model. Therefore, I would rather say that the problem that you mention is a result of the oversimplified error covariance and not because of the Gaussian likelihood.

- Similarly, you cannot make a statement about relativity in line 125 without considering the uncertainty.

- Note also that some authors use Gaussian likelihood, but work with ln(y) and ln(Hx)

- In Section 3.3.3, the posterior is shown for the different likelihood functions. It can be seen that the posteriors don't overlap too much for the longitude and latitude parameters, but they do overlap for the Total Retrieved Released Activity. While it is not explicitly stated in the paper, the operator H was calculated on a grid with grid spacings of 1°? That would imply that the likelihood for the location is extremely sharp, thereby hinting to an unphysically small uncertainty in the location for all considered likelihood functions. I would expect that, if the uncertainties would be larger for all likelihood functions, then they would overlap much more, as is already the case for the TRRA. The conclusion would then be that the likelihood has some impact on the posterior shape, but not too much, which is what I would expect a priori.

3/ Threshold values

Line 157: "As a consequence, it can be deduced that a "good" threshold for the log-normal distribution in a case involving important quantities released should lie between 0.5 mBq.m −3 and 3 mBq.m −3 ." Could you give some explanation on how the values were deduced? I have a concern that the thresholds that are mentioned here are large compared to instrumental detection thresholds, which might explain why many observations in Central Europe are non-informative (r = 0.09) in Fig 3. As an alternative, De Meutter and Hoffman (2020) formulated likelihood functions that explicitly consider detections, non-detections, false alarms and misses.

4/

Line 179: "Indeed, the error is a function of time and space and is obviously not common for every observation-prediction couple.". I wonder why the authors do not prescribe the uncertainty on the observation and the prediction, and make it observation-specific? In De Meutter et al. (2021), the observation uncertainties are combined with the prediction uncertainties, which were obtained from an ensemble. As a result, the uncertainty on the input is no longer a parameter that needs to be inferred. Ideally, the distribution of these uncertainties should also be consistent with the likelihood function, which could be mentioned in Section 2.1.

5/ There is limited discussion on the results using the spatial clustering (Lines 379-383). Could you provide some discussion, for instance whether you would recommend it or not, and why? And what is the effect of changing the threshold (please see also my comment 3)?

6/ Enhanced ensemble

It is not surprising that a pointwise comparison will give the result in Figure 6a: a ten-member global ensemble can only represent the uncertainty on large spatio-temporal scales (which is of interest here, since you do a long-range atmospheric transport and dispersion calculation). Also, it seems strange to suggest to compensate underdispersiveness in the weather data by perturbing the atmospheric transport and dispersion model. The latter has its own uncertainties which should ideally be taken into account. In Lines 419 - 428, the discussion is inconsistent with the (incorrect) motivation for perturbing ldX.

7/ In the conclusions, it is stated: "Moreover, we provided a method to add meteorological and dispersion uncertainties to the reconstruction of the distributions of a source, improving its evaluation." However, no improvement is mentioned or discussed in Section 3.3.4.

Minor comments

Line 6: Firstly,... Secondly, ..., Finally, ...

Line 55: "modelling choices": it is not clear what is meant with this. Is it the atmospheric transport and dispersion model, or does it also include the likelihood and error covariance?

Line 56: (see also the above comment): "The objective of this study is to investigate the various sources of uncertainties compounding the problem of source reconstruction": but the title suggested modelling uncertainties, which I would associate to the atmospheric transport and dispersion modelling.

Line 58: "The quantification of the uncertainties largely depends on the definition of the likelihood and its components." Could you clarify this?

Section 1.4: the section numbering is confusing. I suggest to use more sections, for instance a new section for "Summary and Conclusions".

Line 108: "... the likelihood part of the cost should be zero and it should increase when the difference between the observation and the prediction values grows.": you mean the cost part of the likelihood.

Page 5, criteria for the likelihood function: there is a contradiction between the first and the fourth criterion. The likelihood should indeed measure the difference between observations and predictions (fourth criterion), so that the positive support requirement becomes invalid (first criterion). If you consider the differences, I would rather suggest that it should be symmetric around its maximum, which should be at 0 (zero difference between observation and prediction).

Lines 232 - 238: I suggest to omit this.

Table 1: the spatial resolution, vertical resolution and time resolution: this is for ldX and not for ERA5? Furthermore, ldX was run forward in time? With one simulation for each day and each grid point? And this grid had grid spacings of 1°, while the output grid spacings are 0.28125°? I suggest to make this information more explicit.

Line 299-300: units are missing for the variances.

Line 301: "When the algorithm to discriminate pertinent observations presented in section 2.2 is used, ..." What are "pertinent" observations? Previously, you used the terms "discriminant" and "non-discriminant"?

Line 333-334: units are missing for the error variances.

Line 341-342: same as above.

L 368: Figure 4c should be Figure 4b.

L 379: Figure 4b should be Figure 4c.

Line 405: "... and the standard deviation (std) of the joint multi-model TRRA is therefore far more important than the std of the joint HRES TRRA." What is the meaning of standard deviation here? And what do you mean with "important"?

References

De Meutter, P., & Hoffman, I. (2020). Bayesian source reconstruction of an anomalous Selenium-75 release at a nuclear research institute. Journal of environmental radioactivity, 218, 106225.

De Meutter, P., Hoffman, I., & Ungar, K. (2021). On the model uncertainties in Bayesian source reconstruction using an ensemble of weather predictions, the emission inverse modelling system FREARtool v1.0 and the Lagrangian transport and dispersion model Flexpart v9.0.2. Accepted for publication in Geoscientific Model Development, 1-23.

Review of the article: “Quantification of thr modelling uncertainties in atmospheric release source assessment and application to the reconstruction of the autumn 2017 Ruthenium 106 source” by Dumont Le Brazidec et al.

The manuscripts presents an evaluation of the impact on assumptions surrounding statistical model selection on posterior estimates of source properties of (unknown) radiological releases. This manuscript will be informative to researchers and operational users. I have tried to avoid repetition if the previously posted comment. One major drawback of the manuscript is that much of the motivation is based on a straw-man argument, i.e. the original Gaussian set-up is designed in the manuscript such that it will fail. Below are a number of suggestions for revisions to improve the manuscript, followed by technical comments.

1. Paragraph starting line 110: This is a straw-man argument. The assumption from the start is that the error is larger for higher measurements. This may not always be valid, e.g. an incorrectly dispersed wide plume with a very high concentration. It may be that the error is much larger for the smaller measurements than the larger measurements. There are other approaches to improve the validity of Gaussian (or any) likelihoods through transformation of variables. For example, using a non-linear forward model. Caveats, justification and ‘typical errors’ needs explaining, preferably at the start of section 2.
2. Much of the arguments surround having independent and identically distributed (iid) model-measurement error in the covariance. Many of the arguments throughout can be countered by the use of non-iid covariances, e.g. tR, where the diagonal of R is the measurement value and t is a scaling – equivalent to having e.g. a 10% model-measurement error. This needs further discussions and better justification for the arguments proposed (e.g. non-negativity).
3. Section 3.2.3: It would be useful for many readers to provide a brief conceptual introduction to MCMC methods (i.e. asymptotically exact methods not reliant on closed form solutions or conjugacy).
4. Section 3.4: This section needs expanding considerably. It is a paper with lot of content. At current, the summary provides an overview of the approach of the paper but no summary of the finding. The summary should summarise the results and findings to adequately inform the lazy reader on the paper’s content.
5. I understand the following would require a lot of extra work, and so do not mandate it for publication. It would however, much improve the paper. Seeing as an ensemble is used, it would seem sensible to me to use a simulated dataset (a simulation using an ensemble member) to draw conclusions from the various experiment. It is not perfect, but would be useful to have a ‘truth’.
6. It would be useful in the analysis and plot to also show the original case of a Gaussian likelihood. This is needed to prove the worth of using a non-Gaussian likelihood.

Technical comments:

Title: “modelling uncertainties” is ambiguous as it can refer to a statistical model or a transport model. The title also is not grammatically correct. A suggested improvement is “Quantification of uncertainties in atmospheric release source assessment applied to the autumn 2017 Ruthenium 106 source”.

Abstract, line 5: “improve on these distributions” is vague. ‘Better quantify’ or ‘improve estimates of these distributions’ would be better.

Line 7: A space is not needed before a colon in English.

Line 8: Clarify ‘model errors’ (I assume transport errors?)

Line 10: ‘several suited distributions for the errors are advised’ the passive voice reads as though you are advising the distributions. Better “we suggest several suitable distributions for the errors” or “are suggested” if sticking with the passive.

Line 17: sources or a source; remove ‘many’; ‘Therefore’ doesn’t follow – delete.

Line 31: ‘finds it origin in’ to ‘originates from’

Line 38: This is an oversimplification of the weighting strategy. See, for example, importance resampling or Ensemble Kalman filter methods.

Line 46: Why index vector x elements with x1, x2 and then ln(q)? Perhaps x3=ln(q) would be clearer.

Line 47: R is better described as the covariance matrix containing the model-measurement errors.

Line 52: Introduce for the reader what the prior is (i.e. the probability distribution of prior knowledge before considering data).

Line 57: reconstructed posterior distribution

Line 60: ‘transformation’ is better than ‘parameterisation’

Line 61: ‘are the results’ would be better as ‘are the results of a simulation’

Line 62: ‘and are therefore depending’ to ‘and depend on’

Line 70: This isn’t an expansion.

Line 103: A cost function is a non-probabilistic concept and so better to refer to as simply the negative log-likelihood.

Equation 4: There should be no divide by 2 in the first term.

Line 112: A space not full stop is needed between units

Line 115: Capital G on Gaussian.

Line 120: Unless there has been a transform (e.g. ln(y)). Square bracket is facing the wrong way.

Line 123: Space between units.

Line 156: ‘Large multiple’

Line 178: ‘as this paper’

Line 278: What are the upper/lower bounds of the uniform distribution?

Line 280: What are the shape parameters of the log-gamma distribution?

Line 348: ‘Harmful’ is an incorrect choice of work here. You can just delete it;

Line 348-350: This sentence isn’t correct. Observations don’t have a high likelihood. Please rephrase.

Line 351: Change ‘totally legitimate’ to ‘valid’

Line 353-355: This sentence does not make sense. I’m unsure of its meaning, please revise.

Line 368: 4c and 5a

Equation A1 and A2: Second term is incorrect, not divide by 2 but multiplied by 2.

The manuscript presents interesting study based on estimation of atmospheric release from ambient concentration measurement coupled with atmospheric model. Few prior models of a release are presented, discussed and evaluated on Ruthenium 106 case from 2017. Here, there is consensus on release location and approximate release time-profile which makes this case very interesting and a playground for model testing. The manuscript is nicely written and clear to understand. What I lack is clarification and verification of some statements. I also recommend to extend conclusion (or discussion) by some suggestions and recommendations for future cases, see specific comments bellow. In sum, I would recommend the paper for publication after these clarifications. 

Specific comments:

p. 5, line 115: Although I understand the importance of lower values in measurements, there might be a good reason for high significance of higher values since they may bring more confident infromation wiht lower uncertainty, especially in case with spatialy and temporaly long transport.

Figure 2: I am curious whether similar results are obtained for latitude. Considering the dominant direction of the atmospheric transport is probably in longitude axis, it is maybe different in latitude axis. Please, comment.

p. 14, line 375: Regarding temporal profiles of the estimated release, what is exactly the time-resution of the posterior, is it one day? Is it possible to plot the release profiles somehow, e.g. using medians or similar? Did you estimate some significant activity also in other days except 25th and 26th September?

p. 17, line 398: Could you please clarify the choice of reference source in [60, 55] while Ozyorsk (near where the plant is located) is located at 60°43' E 55°45' N and modelled spatial resolution is 0.5 degree? Shouldnt it be closer point [61.5 55.5]? Or maybe you have different numbering, please, clarify.

In general, I lack discussion and recommendation what settings should one choose when situation similar to the Ru-106 case occur in the future and, let say, one location and one total of the release need to be reported. Also, are your findings rather general, or case specific?

Technical corrections:

p. 3, line 63: $S$ should be defined in term $\mathbf{y}_S$, probably $S$th obsevation

Eq. (2): the $\mathbf{x}$ is used as the source term previously while $\mathbf{q}$ it is used here. Please, clarify whether they are the same of have different meanings.

Eq. (5a-5c): norm with two indexes in subscript should be defined (although clear for many readers, for many may not).

p. 5, line 141: "Secondly, a location term appears..." I am not sure what you mean by this statement, please, clarify.

p. 6, line 163: there sould be (5c) instead of (6), probably.

Figure 8: there are missing labels (a) - (f) in subplots.

This paper does exactly what it says it will do: source term estimation for an emission of Ruthenium using observations, an atmospheric model, and Bayesian inversion. It excels at explaining the concepts involved, making it especially accessible to someone who has not done this exact type of problem before. However, many corrections are needed to the wording, particularly in section 2, and the organization, particularly in section 3.

Comments:

37. In the climate modeling community we would refer to an initial conditions ensemble of the same atmospheric model as a “single-model ensemble” rather than a “multi-model ensemble.”

39. Missing “a”

60-63. Comment: I like this concise explanation of how a model, source, observations, and likelihood fit together.  

89. Suggested “These three sources of uncertainty are explored in an application of source term estimation for the ¹⁰⁶Ru release…”

90. state of the art of

89-94. Can you rephrase this so that it flows monotonically, i.e. reference section 2 before section 3?

103. The math is correct but the wording is not quite right. I think you mean that r is a positive coefficient and R (and rI) is a positive diagonal matrix; r itself is not a “positive diagonal coefficient.”
     
     
     
     
104. Of the problem

111-112. Suggest something like…choosing Gaussian likelihood penalizes the largest errors to an extent that smaller errors are negligible.

115. Rephrase. Consecutive sentences starting with “in other words.”

116. I think you should delete the sentence starting with ``Every “ as the wording is confusing. Your example (100,120) vs (10,12) has already made this point.

120. Bracket typo.

126. What do you mean by mitigated here? I think you can say “should be 1” or “should be close to 1.”

131. Missing a word here, which obscures the meaning of the sentence.

144. It may be helpful for the reader if you reference the section in which the threshold is discussed.

190. If the observation sorting algorithm is the division into r and r_nd, then you should not start a new paragraph for sentence 191.

268, 274. “22^nd”

305. This summary section should be clarified if possible. For uniformity, I recommend starting each bullet point with a section number, e.g.

• Section 3.3.2 is an application of the observation sorting algorithm…;
• Section 3.3.3 is an application of the different likelihood functions and spatial clustering …;
• Section 3.3.4 is an application of the perturbed dispersion parameters and enhanced ensemble…

Secondly, the section heading “Summary” section seems out of place, especially since you have a summary section later. I would suggest renaming 3.3 “Results” and renaming 3.3.1 “Overview.”

345. “Probable sources”

355. “which is not justifiable.”

440. Explain when and where this accident took place, and maybe ad some thoughts about how this might compare to what you just did.

436-440. I think more discussion would be helpful for the reader. Remember, many readers skim the paper until they get to the conclusions.