This is a very good study, written by experts at EPA and it will be a welcome addition to the literature. I find the research question to be important and the conclusions are clearly supported by the analysis.  I don’t have any concerns regarding the analysis or conclusions, my only recommendation is that the authors update their terminology regarding the reporting of statistical findings, to be more consistent with the Tropospheric Ozone Assessment Report and current thinking regarding the limitations of the expression “statistically significant”, as described below.

Regarding the use of the Theil-Sen/Mann-Kendall method for calculating trends, the authors state that they chose this method because of the small sample sizes and because it does not require assumptions about the distribution of the residuals.  Another reason that is often given for the choice of this method is that it is resistant to outliers. The problem is that in order to remove the impact from outliers, this method automatically ignores up to 29% of the data points in a sample (see Section 2 of Chang et al., 2021).  This would be fine if the analyst believed that the outliers are due to instrument errors, but there is no reason to throw out data if they are believed to be reliable.  In your case there is no reason to believe that your samples contain erroneous data points that should be ignored.  For this reason, the Tropospheric Ozone Assessment Report has abandoned the Theil-Sen/Mann-Kendall method that was used in the first phase of TOAR (2014-2019).  A further problem with the Theil-Sen method is that it produces unrealistically narrow 95% confidence intervals.  This is shown in Figure 1 of the TOAR-II Recommendations for Statistical Analyses (available at https://igacproject.org/activities/TOAR/TOAR-II).  Figure 1 compares the trend and 95% confidence interval calculated by 10 different methods for the ozone time series at Mace Head, Ireland.  The Theil-Sen method has the narrowest 95% confidence interval by far, and the reason is that this method ignores 29% of the data; by throwing out all of the extreme values the sample has very little variability and therefore a straight line can be fit through the remaining data within a very narrow range.  The second phase of TOAR-II is now recommending the use of quantile regression, as described in the TOAR-II Recommendations for Statistical Analyses.  Quantile regression was used to good effect in the very nice paper by co-author B. Wells (Wells et al., 2021), and it could easily be applied to your current analysis.

Throughout the paper the authors use the expression “statistically significant”, however this expression is now recognized as being problematic and it should be abandoned and replaced by the more useful method of reporting all trends (with uncertainty) and all p-values, followed by a discussion of the trends and the author’s opinion regarding their confidence in the trend values.  This advice comes from a highly influential paper by Wasserstein et al. (2019), published in the journal, The American Statistician, that has already been cited over 1300 times (according to Web of Science).  This advice was adopted by the first phase of TOAR (Tarasick et al., 2019) and will also be used by TOAR-II.  Some other recent papers on ozone trends that have taken this advice are:  Chang et al., 2020; Cooper et al., 2020; Gaudel et al., 2020; Chang et al., 2022; Wang et al., 2022; Mousavinezhad et al., 2023.  Because these papers report all trend values, uncertainties, and all p-values, and also discuss the trend results, there is no confusion regarding the findings, and one does not even notice that the term “statistically significant” is not used at all.

The authors describe a trend as “no trend” when the p-value is greater than 0.05.  There are two problems with this approach:  

1) as described above the expression “statistically significant” which is tied to the p-value of 0.05 should be abandoned.  Just because a trend has a p-value of 0.06, it does not mean that there is absolutely no trend, it just means that there is a gray area and the trend is not as robust as one that has a p-value of 0.02.  Chang et al. (2017) provide a nice demonstration of the useful information that can be gleaned from a trend with a p-value greater than 0.05 (see their Figure 13). They calculated a regional ozone trend for the eastern USA using all available ozone monitors (in summer the trend was strongly negative for the period 2000-2014). They then conducted an exercise to see what would happen to the regional trend if they threw out all time series with a p-value less than 0.05.  The result was almost the same because the time series with p-values greater than 0.05 still reflected the overall regional decrease of ozone. 

2) The authors are using the Theil-Sen method to calculate trends and p-values.  As described above the 95% confidence intervals are unrealistically narrow using this method, and therefore the p-values are also too low.  This means that too many sites are classified as having a real trend, according to the 0.05 p-value threshold.  If the authors use another method for calculating trends (like quantile regression) the p-values will increase and they would then have to classify more sites as having “no trend”.  Given the gray area around p-values, and given that trends with p-values greater than 0.05 can still be reliable, there is no justification for dichotomizing ozone time series as “trend” or “no trend” based on a p-value.

When I look at the maps in Figure 4 and 5 I am left wondering about the non-attainment regions labeled as “no trend”. Is there really no trend here, i.e. a flat line, or is there still a decrease, but it just doesn’t reach the arbitrary threshold of p<0.05?  A good example is Tuscan Buttes. Table S-1 shows the observed and modelled trend is the same (0.14) but because the model has a p-value of 0.02 this trend is considered to be real, while the observations have a p-value of 0.06 and are classified as “no trend”.  The TOAR papers report all trends and all p-values and the trend values in their map plots are colored according to p-value (Fleming et al., 2018). This allow the reader to see if a trend is still notable (e.g. a p-value between 0.05 and 0.10) or if there really and truly is no trend (e.g. a p-value > 0.33). It would be very helpful to the reader if the authors can color their maps according to p-value, in a manner similar to TOAR. 

References:

Chang, K-L, et al 2017 Regional trend analysis of surface ozone observations from monitoring networks in eastern North America, Europe and East Asia. Elem Sci Anth, 5: 50, DOI: https://doi.org/10.1525/elementa.243

Chang, K.-L., et al. (2020), Statistical regularization for trend detection: An integrated approach for detecting long-term trends from sparse tropospheric ozone profiles, Atmos. Chem. Phys., 20, 9915–9938, https://doi.org/10.5194/acp-20-9915-2020

Chang, K-L, et al. 2021. Trend detection of atmospheric time series: Incorporating appropriate uncertainty estimates and handling extreme events. Elem Sci Anth, 9: 1. DOI: https://doi.org/10.1525/elementa.2021.00035

Chang, K.-L., (2022), Impact of the COVID-19 economic downturn on tropospheric ozone trends: an uncertainty weighted data synthesis for quantifying regional anomalies above western North America and Europe, AGU Advances, 3, e2021AV000542. https://doi.org/10.1029/2021AV000542

Cooper, et al. 2020. Multi-decadal surface ozone trends at globally distributed remote locations. Elem Sci Anth, 8: 23. DOI: https://doi.org/10.1525/elementa.420

Fleming, Z. L., R. M.  Doherty, E. von Schneidemesser, C. S. Malley, O. R. Cooper et al. (2018), Tropospheric Ozone Assessment Report: Present-day ozone distribution and trends relevant to human health, Elem Sci Anth, 6(1):12, DOI: https://doi.org/10.1525/elementa.273

Gaudel, A., et al.  (2020), Aircraft observations since the 1990s reveal increases of tropospheric ozone at multiple locations across the Northern Hemisphere. Sci. Adv. 6, eaba8272, DOI: 10.1126/sciadv.aba8272

Mousavinezhad, S., Ghahremanloo, M., Choi, Y., Pouyaei, A., Khorshidian, N. and Sadeghi, B., 2023. Surface ozone trends and related mortality across the climate regions of the contiguous United States during the most recent climate period, 1991–2020. Atmospheric Environment, p.119693.

Tarasick, D. W. et al. (2019), Tropospheric Ozone Assessment Report: Tropospheric ozone from 1877 to 2016, observed levels, trends and uncertainties. Elem Sci Anth, 7(1), DOI: http://doi.org/10.1525/elementa.376

Wang, H., et al. (2022), Global tropospheric ozone trends, attributions, and radiative impacts in 1995–2017: an integrated analysis using aircraft (IAGOS) observations, ozonesonde, and multi-decadal chemical model simulations, Atmos. Chem. Phys., 22, 13753–13782, https://doi.org/10.5194/acp-22-13753-2022

Wasserstein, R. L., Schirm, A. L., and Lazar, N. A.: Moving to a world beyond p < 0:05, Am. Stat., 73, 1–29,

https://doi.org/10.1080/00031305.2019.1583913, 2019.

Review of “Disappearing day-of-week ozone patterns in US nonattainment areas” by Simon et al.

In this analysis the authors evaluate day of week patterns in average O₃ and O₃ exceedance days for the 2002-2019 timeframe at ~50 different high O₃ sites in the U.S.   The authors identify several different patterns including disappearing weekend-weekday differences and others.   The analysis mixes observations and model results, which I find to be somewhat problematic.    In addition I have some statistical concerns that would need to be corrected or clarified before this could be published.

Major points:  

1. The authors mix observed patterns with model patterns in a way that I believe is misleading. I think its incumbent on the authors to first clearly document what the observations show.  Then we can ask how well the model reproduces the observations and what we can learn from the model where it is consistent with the observations, or if not consistent, then why.  Certainly there are plenty of NOx observations that could have been used for this work (see Jaffe et al 2020), so I am not sure what is gained by showing and using only the modeled NOx.  But NOx is not that big of a concern.  Its formaldehyde that I find much more problematic.  For formaldehyde, we have much poorer understanding of emissions and chemistry, both of which are essential to understanding the concentrations.   Without any evaluation of the modeled formaldehyde, these results should be removed.    In other places the authors quote both modeled trends and observed trends and appear to put equal weight on these.   That is incorrect, in my opinion, for the reasons stated above.
2. I believe the authors may over-state some of the statistical significance due to auto-correlation. This could be true for both the t-tests for individual years and the trends, which use 5-year running means.    
3. Finally I note that this has a lot of overlap with our earlier analysis (Jaffe et al 2020). We used data for 1995-2020.  This analysis uses data for 2002-2019.   There are some modest differences, but overall the results are quite similar.  I think its essential that the authors clearly describe what is new and/or whether these results are consistent with the earlier analysis.   One area that is different is use of probability of exceedance vs mean concentration.   The authors seem to want to discount any differences as being due to random variability, but I am not sure that is true.  One focuses on the highest days and the other approach focuses on all days in the O₃   Do these days have the same VOC-NOx sensitivity?

Abstract:  It is important in abstract to describe the scope:  All US O3, all US urban areas or all US non-attainment areas.  What years?  How many regions considered?   In addition, I am unclear what it means if you have a “disappearing weekday” effect.  The information here is contained in the relative O₃ and NOx behavior between weekday and weekend.   So the terms “disappearing weekday” is confusing.  

Line 25-26:  “both datasets” ?

Line 27: The abstract uses area names that are consistent (I think) with EPA designations, but are often rather non-intuitive.   For example,  Los Angeles –  San Bernardino County vs Los Angeles – South Coast.   The San Bernardino monitors are in Riverside CBSA, so aren’t these two locations essentially same region.   It would be more interesting to include a site closer to downtown LA like Azusa, where we might expect a different pattern.

Line 33:   It is not clear what model evaluation for this work.  As near as I can tell, nothing was shown about the models ability to capture year-year variations.   The model does seem to capture the trend in weekend-weekday differences.

132-135:  While I understand why you excluded 3 out of 7 days, does this change the results?

144:  Not clear how t-tests were done.  I think you took every weekday and weekend day in one year and compared the means and treated each day as an independent observation.  If this is right, then I don’t think autocorrelation was taken into account.   In any case please clarify how the t-tests were performed.

169:   Given the 5 year running means, these values will have sig autocorrelation.    Was this taken into account in the results?

173-174:   Unclear meaning.

193-194:    Unclear meaning.

Figure 1:  Please clarify meaning of P values in top right plot.  I think these are for each individual year, correct?   Given that the NOx and CH2O plots are for all years, not sure what is the value in showing these.    There are major differences (for NOx) between the early and later part of the data record.

234:   As noted above, the terminology “disappearing weekend effect” is very misleading.   Its really about the difference between weekend and weekday values.

258-259:   So how do we interpret these model obs differences?  You may say the random variations impact the obs more than the model, but aren’t these variations important?

283:  I don’t think the probability approach is inherently noisier, especially when averaged over several years as you have done.   I think this is an interesting spot to do a deeper dive.

Conclusions:  As noted above it would be good to understand what is new here.   Please add some discussion to clarify, perhaps focusing on the differences between the prob of an exceedance approach and mean O₃ approach.  

Finally, I note that the regression information in the right plots of figures 1-9 (not 5) is almost impossible to read.