25 Jan 2021
25 Jan 2021
Quantification of the modelling uncertainties in atmospheric release source assessment and application to the reconstruction of the autumn 2017 Ruthenium 106 source
 ^{1}IRSN, PSESANTE, SESUC, BMCA, FontenayauxRoses, France
 ^{2}CEREA, Joint laboratory École des Ponts ParisTech and EDF R&D, Université ParisEst, MarnelaVallée, France
 ^{1}IRSN, PSESANTE, SESUC, BMCA, FontenayauxRoses, France
 ^{2}CEREA, Joint laboratory École des Ponts ParisTech and EDF R&D, Université ParisEst, MarnelaVallée, France
Abstract. Using a Bayesian framework in the inverse problem of estimating the source of an atmospheric release of a pollutant has proven fruitful in recent years. Through Markov chain Monte Carlo (MCMC) algorithms, the statistical distribution of the release parameters such as the location, the duration, and the magnitude as well as the likelihood covariances can be sampled so as to get a complete characterisation of the source. In this study, several approaches are described and applied to improve on these distributions, and therefore to get a better representation of the uncertainties. First, a method based on ensemble forecasting is proposed: physical parameters of both the meteorological fields and the transport model are perturbed to create an enhanced ensemble. In order to account for model errors, the importance of ensemble members are represented by weights and sampled together with the other variables of the source. Secondly, the choice of the statistical likelihood is shown to alter the nuclear source assessment, and several suited distributions for the errors are advised. Finally, two advanced designs of the covariance matrix associated to the observation error are proposed. These methods are applied to the case of the detection of Ruthenium 106 of unknown origin in Europe in autumn 2017. A posteriori distributions meant to identify the origin of the release, to assess the source term, to quantify the uncertainties associated to the observations and the model, as well as densities of the weights of the perturbed ensemble, are presented.
Joffrey Dumont Le Brazidec et al.
Status: open (until 22 Mar 2021)

RC1: 'Comment on acp20201129', Pieter De Meutter, 15 Feb 2021
reply
Review "Quantification of the modelling uncertainties in atmospheric release source assessment and application to the reconstruction of the autumn 2017 Ruthenium 106 source"
The authors present a technical and modelling study of Bayesian inversion applied to the airborne Ru106 observations made in Europe in 2017. The technical part of the paper focuses on three aspects. First, the choice of the likelihood function: three likelihood functions were proposed based on four proposed criteria which a likelihood function should ideally fulfill. Second, an alternative formulation for the error covariance matrix is proposed, in particular with the purpose to discriminate between informative and noninformative observations; the latter leads to artificially low inferred errors, which is remediated by the alternative error covariance matrix. Third, the observation operator is replaced by a linear combination using an ensemble, of which the weights are sampled by the Bayesian inference. In the second part of the paper, these techniques are illustrated by applying Bayesian inference using airborne observations of Ru106 observations. The results show an (impressive) agreement with the most likely source location (the Mayak institute in Russia).
The topics discussed in the paper are interesting and relevant. However, I have some comments and concerns about some of the methodology. Also, at times, specific information is lacking, or the formulation of ideas is done with insufficient care, leaving the reader with confusion. I hope the authors find my comments below useful. I recommend major revisions before the manuscript can be accepted for publication.
Major comments
1/ I suggest to reformulate parts of Section 1.2:
 "An efficient way to use these forecasts to better estimate uncertainties is to combine them": it is not clear to me what the authors mean here. Please be more specific. (If you would combine the members of an ensemble into a best estimate of the true state of the atmosphere, and use only that best estimate, you would loose the uncertainty information.)
 "This approach is known as multimodel ensemble forecasting (Zhou and Du, 2010).": It is not clear to me what "This approach" is. If you use the same model with perturbed input parameters and / or perturbed physics, I would not call it multimodel ensemble forecasting.
 What is "sequential aggregation"? Do you mean sequential in time (= an ensemble of deterministic forecasts run at different starting times)?
 "An aggregated forecast is then formed by the weighted linear combination of the forecasts of the ensemble": I wonder why the authors want to create a best realization, rather than extracting the uncertainty from the ensemble. I suggest to add some discussion to explain the reasoning for this.
2/ While the discussion on the choice of the likelihood function is valuable and interesting, I'm not sure if all arguments made in the paper are valid.
 I'm not sure if I can agree with the discussion in Lines 111  117. There, you ignore the fact that the uncertainties will be larger for the higher observationprediction couple; both observationprediction couples could have the same penalty if they have the same relative uncertainty, which is not unlikely considering the error from the atmospheric transport and dispersion model. Therefore, I would rather say that the problem that you mention is a result of the oversimplified error covariance and not because of the Gaussian likelihood.
 Similarly, you cannot make a statement about relativity in line 125 without considering the uncertainty.
 Note also that some authors use Gaussian likelihood, but work with ln(y) and ln(Hx)
 In Section 3.3.3, the posterior is shown for the different likelihood functions. It can be seen that the posteriors don't overlap too much for the longitude and latitude parameters, but they do overlap for the Total Retrieved Released Activity. While it is not explicitly stated in the paper, the operator H was calculated on a grid with grid spacings of 1°? That would imply that the likelihood for the location is extremely sharp, thereby hinting to an unphysically small uncertainty in the location for all considered likelihood functions. I would expect that, if the uncertainties would be larger for all likelihood functions, then they would overlap much more, as is already the case for the TRRA. The conclusion would then be that the likelihood has some impact on the posterior shape, but not too much, which is what I would expect a priori.
3/ Threshold values
Line 157: "As a consequence, it can be deduced that a "good" threshold for the lognormal distribution in a case involving important quantities released should lie between 0.5 mBq.m −3 and 3 mBq.m −3 ." Could you give some explanation on how the values were deduced? I have a concern that the thresholds that are mentioned here are large compared to instrumental detection thresholds, which might explain why many observations in Central Europe are noninformative (r = 0.09) in Fig 3. As an alternative, De Meutter and Hoffman (2020) formulated likelihood functions that explicitly consider detections, nondetections, false alarms and misses.
4/
Line 179: "Indeed, the error is a function of time and space and is obviously not common for every observationprediction couple.". I wonder why the authors do not prescribe the uncertainty on the observation and the prediction, and make it observationspecific? In De Meutter et al. (2021), the observation uncertainties are combined with the prediction uncertainties, which were obtained from an ensemble. As a result, the uncertainty on the input is no longer a parameter that needs to be inferred. Ideally, the distribution of these uncertainties should also be consistent with the likelihood function, which could be mentioned in Section 2.1.
5/ There is limited discussion on the results using the spatial clustering (Lines 379383). Could you provide some discussion, for instance whether you would recommend it or not, and why? And what is the effect of changing the threshold (please see also my comment 3)?
6/ Enhanced ensemble
It is not surprising that a pointwise comparison will give the result in Figure 6a: a tenmember global ensemble can only represent the uncertainty on large spatiotemporal scales (which is of interest here, since you do a longrange atmospheric transport and dispersion calculation). Also, it seems strange to suggest to compensate underdispersiveness in the weather data by perturbing the atmospheric transport and dispersion model. The latter has its own uncertainties which should ideally be taken into account. In Lines 419  428, the discussion is inconsistent with the (incorrect) motivation for perturbing ldX.
7/ In the conclusions, it is stated: "Moreover, we provided a method to add meteorological and dispersion uncertainties to the reconstruction of the distributions of a source, improving its evaluation." However, no improvement is mentioned or discussed in Section 3.3.4.
Minor comments
Line 6: Firstly,... Secondly, ..., Finally, ...
Line 55: "modelling choices": it is not clear what is meant with this. Is it the atmospheric transport and dispersion model, or does it also include the likelihood and error covariance?
Line 56: (see also the above comment): "The objective of this study is to investigate the various sources of uncertainties compounding the problem of source reconstruction": but the title suggested modelling uncertainties, which I would associate to the atmospheric transport and dispersion modelling.
Line 58: "The quantification of the uncertainties largely depends on the definition of the likelihood and its components." Could you clarify this?
Section 1.4: the section numbering is confusing. I suggest to use more sections, for instance a new section for "Summary and Conclusions".
Line 108: "... the likelihood part of the cost should be zero and it should increase when the difference between the observation and the prediction values grows.": you mean the cost part of the likelihood.
Page 5, criteria for the likelihood function: there is a contradiction between the first and the fourth criterion. The likelihood should indeed measure the difference between observations and predictions (fourth criterion), so that the positive support requirement becomes invalid (first criterion). If you consider the differences, I would rather suggest that it should be symmetric around its maximum, which should be at 0 (zero difference between observation and prediction).
Lines 232  238: I suggest to omit this.
Table 1: the spatial resolution, vertical resolution and time resolution: this is for ldX and not for ERA5? Furthermore, ldX was run forward in time? With one simulation for each day and each grid point? And this grid had grid spacings of 1°, while the output grid spacings are 0.28125°? I suggest to make this information more explicit.
Line 299300: units are missing for the variances.
Line 301: "When the algorithm to discriminate pertinent observations presented in section 2.2 is used, ..." What are "pertinent" observations? Previously, you used the terms "discriminant" and "nondiscriminant"?
Line 333334: units are missing for the error variances.
Line 341342: same as above.
L 368: Figure 4c should be Figure 4b.
L 379: Figure 4b should be Figure 4c.
Line 405: "... and the standard deviation (std) of the joint multimodel TRRA is therefore far more important than the std of the joint HRES TRRA." What is the meaning of standard deviation here? And what do you mean with "important"?
References
De Meutter, P., & Hoffman, I. (2020). Bayesian source reconstruction of an anomalous Selenium75 release at a nuclear research institute. Journal of environmental radioactivity, 218, 106225.
De Meutter, P., Hoffman, I., & Ungar, K. (2021). On the model uncertainties in Bayesian source reconstruction using an ensemble of weather predictions, the emission inverse modelling system FREARtool v1.0 and the Lagrangian transport and dispersion model Flexpart v9.0.2. Accepted for publication in Geoscientific Model Development, 123.

RC2: 'Comment on acp20201129', Anonymous Referee #1, 19 Feb 2021
reply
Review of the article: “Quantification of thr modelling uncertainties in atmospheric release source assessment and application to the reconstruction of the autumn 2017 Ruthenium 106 source” by Dumont Le Brazidec et al.
The manuscripts presents an evaluation of the impact on assumptions surrounding statistical model selection on posterior estimates of source properties of (unknown) radiological releases. This manuscript will be informative to researchers and operational users. I have tried to avoid repetition if the previously posted comment. One major drawback of the manuscript is that much of the motivation is based on a strawman argument, i.e. the original Gaussian setup is designed in the manuscript such that it will fail. Below are a number of suggestions for revisions to improve the manuscript, followed by technical comments.
 Paragraph starting line 110: This is a strawman argument. The assumption from the start is that the error is larger for higher measurements. This may not always be valid, e.g. an incorrectly dispersed wide plume with a very high concentration. It may be that the error is much larger for the smaller measurements than the larger measurements. There are other approaches to improve the validity of Gaussian (or any) likelihoods through transformation of variables. For example, using a nonlinear forward model. Caveats, justification and ‘typical errors’ needs explaining, preferably at the start of section 2.
 Much of the arguments surround having independent and identically distributed (iid) modelmeasurement error in the covariance. Many of the arguments throughout can be countered by the use of noniid covariances, e.g. tR, where the diagonal of R is the measurement value and t is a scaling – equivalent to having e.g. a 10% modelmeasurement error. This needs further discussions and better justification for the arguments proposed (e.g. nonnegativity).
 Section 3.2.3: It would be useful for many readers to provide a brief conceptual introduction to MCMC methods (i.e. asymptotically exact methods not reliant on closed form solutions or conjugacy).
 Section 3.4: This section needs expanding considerably. It is a paper with lot of content. At current, the summary provides an overview of the approach of the paper but no summary of the finding. The summary should summarise the results and findings to adequately inform the lazy reader on the paper’s content.
 I understand the following would require a lot of extra work, and so do not mandate it for publication. It would however, much improve the paper. Seeing as an ensemble is used, it would seem sensible to me to use a simulated dataset (a simulation using an ensemble member) to draw conclusions from the various experiment. It is not perfect, but would be useful to have a ‘truth’.
 It would be useful in the analysis and plot to also show the original case of a Gaussian likelihood. This is needed to prove the worth of using a nonGaussian likelihood.
Technical comments:
Title: “modelling uncertainties” is ambiguous as it can refer to a statistical model or a transport model. The title also is not grammatically correct. A suggested improvement is “Quantification of uncertainties in atmospheric release source assessment applied to the autumn 2017 Ruthenium 106 source”.
Abstract, line 5: “improve on these distributions” is vague. ‘Better quantify’ or ‘improve estimates of these distributions’ would be better.
Line 7: A space is not needed before a colon in English.
Line 8: Clarify ‘model errors’ (I assume transport errors?)
Line 10: ‘several suited distributions for the errors are advised’ the passive voice reads as though you are advising the distributions. Better “we suggest several suitable distributions for the errors” or “are suggested” if sticking with the passive.
Line 17: sources or a source; remove ‘many’; ‘Therefore’ doesn’t follow – delete.
Line 31: ‘finds it origin in’ to ‘originates from’
Line 38: This is an oversimplification of the weighting strategy. See, for example, importance resampling or Ensemble Kalman filter methods.
Line 46: Why index vector x elements with x1, x2 and then ln(q)? Perhaps x3=ln(q) would be clearer.
Line 47: R is better described as the covariance matrix containing the modelmeasurement errors.
Line 52: Introduce for the reader what the prior is (i.e. the probability distribution of prior knowledge before considering data).
Line 57: reconstructed posterior distribution
Line 60: ‘transformation’ is better than ‘parameterisation’
Line 61: ‘are the results’ would be better as ‘are the results of a simulation’
Line 62: ‘and are therefore depending’ to ‘and depend on’
Line 70: This isn’t an expansion.
Line 103: A cost function is a nonprobabilistic concept and so better to refer to as simply the negative loglikelihood.
Equation 4: There should be no divide by 2 in the first term.
Line 112: A space not full stop is needed between units
Line 115: Capital G on Gaussian.
Line 120: Unless there has been a transform (e.g. ln(y)). Square bracket is facing the wrong way.
Line 123: Space between units.
Line 156: ‘Large multiple’
Line 178: ‘as this paper’
Line 278: What are the upper/lower bounds of the uniform distribution?
Line 280: What are the shape parameters of the loggamma distribution?
Line 348: ‘Harmful’ is an incorrect choice of work here. You can just delete it;
Line 348350: This sentence isn’t correct. Observations don’t have a high likelihood. Please rephrase.
Line 351: Change ‘totally legitimate’ to ‘valid’
Line 353355: This sentence does not make sense. I’m unsure of its meaning, please revise.
Line 368: 4c and 5a
Equation A1 and A2: Second term is incorrect, not divide by 2 but multiplied by 2.

RC5: 'Reply on RC2', Anonymous Referee #1, 23 Feb 2021
reply

RC3: 'Comment on acp20201129', Ondřej Tichý, 22 Feb 2021
reply
The manuscript presents interesting study based on estimation of atmospheric release from ambient concentration measurement coupled with atmospheric model. Few prior models of a release are presented, discussed and evaluated on Ruthenium 106 case from 2017. Here, there is consensus on release location and approximate release timeprofile which makes this case very interesting and a playground for model testing. The manuscript is nicely written and clear to understand. What I lack is clarification and verification of some statements. I also recommend to extend conclusion (or discussion) by some suggestions and recommendations for future cases, see specific comments bellow. In sum, I would recommend the paper for publication after these clarifications.
Specific comments:p. 5, line 115: Although I understand the importance of lower values in measurements, there might be a good reason for high significance of higher values since they may bring more confident infromation wiht lower uncertainty, especially in case with spatialy and temporaly long transport.
Figure 2: I am curious whether similar results are obtained for latitude. Considering the dominant direction of the atmospheric transport is probably in longitude axis, it is maybe different in latitude axis. Please, comment.
p. 14, line 375: Regarding temporal profiles of the estimated release, what is exactly the timeresution of the posterior, is it one day? Is it possible to plot the release profiles somehow, e.g. using medians or similar? Did you estimate some significant activity also in other days except 25th and 26th September?
p. 17, line 398: Could you please clarify the choice of reference source in [60, 55] while Ozyorsk (near where the plant is located) is located at 60°43' E 55°45' N and modelled spatial resolution is 0.5 degree? Shouldnt it be closer point [61.5 55.5]? Or maybe you have different numbering, please, clarify.
In general, I lack discussion and recommendation what settings should one choose when situation similar to the Ru106 case occur in the future and, let say, one location and one total of the release need to be reported. Also, are your findings rather general, or case specific?
Technical corrections:p. 3, line 63: $S$ should be defined in term $\mathbf{y}_S$, probably $S$th obsevation
Eq. (2): the $\mathbf{x}$ is used as the source term previously while $\mathbf{q}$ it is used here. Please, clarify whether they are the same of have different meanings.
Eq. (5a5c): norm with two indexes in subscript should be defined (although clear for many readers, for many may not).
p. 5, line 141: "Secondly, a location term appears..." I am not sure what you mean by this statement, please, clarify.
p. 6, line 163: there sould be (5c) instead of (6), probably.
Figure 8: there are missing labels (a)  (f) in subplots.

RC4: 'Comment on acp20201129', Anonymous Referee #3, 23 Feb 2021
reply
This paper does exactly what it says it will do: source term estimation for an emission of Ruthenium using observations, an atmospheric model, and Bayesian inversion. It excels at explaining the concepts involved, making it especially accessible to someone who has not done this exact type of problem before. However, many corrections are needed to the wording, particularly in section 2, and the organization, particularly in section 3.
Comments:
 In the climate modeling community we would refer to an initial conditions ensemble of the same atmospheric model as a “singlemodel ensemble” rather than a “multimodel ensemble.”
 Missing “a”
6063. Comment: I like this concise explanation of how a model, source, observations, and likelihood fit together.
 Suggested “These three sources of uncertainty are explored in an application of source term estimation for the ^{106}Ru release…”
 state of the art of
8994. Can you rephrase this so that it flows monotonically, i.e. reference section 2 before section 3?
 The math is correct but the wording is not quite right. I think you mean that r is a positive coefficient and R (and rI) is a positive diagonal matrix; r itself is not a “positive diagonal coefficient.”
 Of the problem
111112. Suggest something like…choosing Gaussian likelihood penalizes the largest errors to an extent that smaller errors are negligible.
 Rephrase. Consecutive sentences starting with “in other words.”
 I think you should delete the sentence starting with ``Every “ as the wording is confusing. Your example (100,120) vs (10,12) has already made this point.
 Bracket typo.
 What do you mean by mitigated here? I think you can say “should be 1” or “should be close to 1.”
 Missing a word here, which obscures the meaning of the sentence.
 It may be helpful for the reader if you reference the section in which the threshold is discussed.
 If the observation sorting algorithm is the division into r and r_{nd}, then you should not start a new paragraph for sentence 191.
268, 274. “22^{nd}”
 This summary section should be clarified if possible. For uniformity, I recommend starting each bullet point with a section number, e.g.
 Section 3.3.2 is an application of the observation sorting algorithm…;
 Section 3.3.3 is an application of the different likelihood functions and spatial clustering …;
 Section 3.3.4 is an application of the perturbed dispersion parameters and enhanced ensemble…
Secondly, the section heading “Summary” section seems out of place, especially since you have a summary section later. I would suggest renaming 3.3 “Results” and renaming 3.3.1 “Overview.”
 “Probable sources”
 “which is not justifiable.”
 Explain when and where this accident took place, and maybe ad some thoughts about how this might compare to what you just did.
436440. I think more discussion would be helpful for the reader. Remember, many readers skim the paper until they get to the conclusions.
Joffrey Dumont Le Brazidec et al.
Joffrey Dumont Le Brazidec et al.
Viewed
HTML  XML  Total  BibTeX  EndNote  

201  52  10  263  1  0 
 HTML: 201
 PDF: 52
 XML: 10
 Total: 263
 BibTeX: 1
 EndNote: 0
Viewed (geographical distribution)
Country  #  Views  % 

Total:  0 
HTML:  0 
PDF:  0 
XML:  0 
 1