Confidence in model estimates of pollutant distributions is established through direct comparisons of modeled concentrations with corresponding observations made at discrete locations for retrospective cases. Pinder et al. (2008) discussed the reducible (i.e., structural and parametric) uncertainties that are attributable to the errors in model input data (e.g., meteorology, emissions, and initial and boundary conditions) as well as our incomplete or inadequate understanding of the relevant atmospheric processes (e.g., chemical transformation, planetary boundary layer evolution, transport and dispersion, deposition, rain, and clouds). Inherent or irreducible (random or unsystematic) uncertainties stem from our inability to properly characterize the stochastic nature of the atmosphere (Wilmott, 1981; Wilmott et al., 1985; Fox, 1984; Rao et al., 1985, 2011a, b; Dennis et al., 2010) and from the incommensurability associated with comparing the volume-averaged model estimates with point measurements (e.g., McNair et al., 1996; Swall and Foley, 2009). Also, without completely knowing the three-dimensional initial physical and chemical state of the atmosphere, its future state cannot be simulated accurately (Lamb, 1984; Lamb and Hati, 1987; Lewellen and Sykes, 1989; Pielke, 1998; Gilliam et al., 2015). Given the presence of the irreducible uncertainties, precise replication of observed concentrations or their changes by the models cannot be expected (Dennis et al., 2010; Rao et al., 2011a; Porter et al., 2015; Astitha et al., 2017).

Whereas an air quality model's prediction represents some time-/space-averaged concentrations, an observation at any given time at a monitoring location reflects an individual event or specific realization out of a population that will almost always differ from the model estimate even if the model and its input data were perfect (Rao et al., 1985). Consequently, comparisons of modeled and observed concentrations paired in space and time indicate biases and errors in simulating absolute levels of pollutant concentrations at individual monitoring sites (Porter et al., 2015). The scientific discussion on modeling uncertainty goes back more than 3 decades with the current practice including data assimilation, ensemble modeling, and model performance evaluation (e.g., Fox, 1981, 1984; Lamb, 1984; Demerjian, 1985; Oreskes et al., 1994; Pielke, 1998; Lewellen and Sykes, 1989; Lee et al., 1997; Carmichael et al., 2008; Hogrefe et al., 2001a, b; Biswas and Rao, 2001; Grell and Baklanov, 2011; Gilliam et al., 2006; Herwehe et al., 2011; Baklanov et al., 2014; Bocquet et al., 2015; Solazzo and Galmarini, 2015a; Ying and Zhang, 2018; McNider and Pour-Biazar, 2020; Stockwell et al., 2020). While ever-improving process knowledge and increasing computational power will continue to help reduce the structural and parametric uncertainties in air quality models, the inherent uncertainty associated with our inability to properly characterize the stochastic nature of the atmosphere will always result in some mismatch between the model results and measurements; this could lead to speculation on the inferred accuracy of the future states simulated by the regional-scale air quality models (Dennis et al., 2010; Rao et al., 2011a; Porter et al., 2015; Astitha et al., 2017; Luo et al., 2019).

The sensitivity of model results to meteorology, chemical mechanisms, and emissions has been examined in numerous studies (e.g., Vautard et al., 2012; Sarwar et al., 2013; Pierce et al., 2010; Napelenok et al., 2011; Kang et al., 2013). Herwehe et al. (2011) attributed the differences in ground-level ozone predictions between the Weather Research and Forecasting (WRF) model coupled with Chemistry (WRF-Chem) and the modeling system consisting of WRF and the Community Multiscale Air Quality (CMAQ) model (WRF-QMAC) to the way meteorology and chemistry interactions are handled within these two modeling systems. Thomas et al. (2019) examined the ozone predictions in the mid-Atlantic region of the United States during June 2016 through a series of simulations with WRF-Chem, focusing on the sensitivity to the meteorological initial/boundary conditions (IC/BCs), emissions inventory (EI), and planetary boundary layer (PBL) scheme. Ying and Zhang (2018) discussed the use of satellite-based observations for improving the predictability of multiscale tropical weather and equatorial waves. Ensemble modeling is being advocated for quantifying the uncertainty in model predictions; however, the spread in the model estimates for the variable of interest reflects the impact of our incomplete or inadequate knowledge of the physical and chemical processes (i.e., the reducible errors stemming from structural and parametric uncertainty) occurring in the atmosphere (Solazzo and Galmarini, 2015b; Thomas et al., 2019; Stockwell et al., 2020). McNider and Pour-Biazar (2020) reviewed the many issues in predicting the prevailing meteorology for regional air quality simulations and indicated that errors in the specification of the physical atmosphere such as temperature, winds, and mixing heights can affect the air quality predictions. Stockwell et al. (2020) discussed the problems relating to the atmospheric chemical mechanisms currently being used for simulating air quality. The current generation of regional models consider only the mean values of a meteorological variable for a given timescale and the average rate constant derived from gas chamber experiments for chemical reactions and does not include their fluctuations in solving the equations of motion for each time step. Further, the current operational regional-scale meteorological and air quality models do not explicitly simulate the stochastic nature of the atmosphere and, as such, typically miss the extreme values at both the low and high ends of the concentration distribution function.

In most applications of regional-scale air quality models, statistical metrics such as bias, the root mean square error (RMSE), correlation, and the index of agreement are used to judge the quality of model predictions and determine if the model is suitable for forecasting or regulatory purposes (e.g., Fox, 1981, 1984; Solazzo et al., 2012; Appel et al., 2012; Simon et al., 2012; Foley et al., 2014; Ryan, 2016; Emery et al., 2016; Zhang et al., 2016; U.S. Environmental Protection Agency, 2018). While significant improvements in the formulation, physical and chemical parameterizations, and numerical techniques have been implemented in atmospheric models over the past 3 decades, it is not clear if the improvement claimed in the model's performance relative to the routine network measurements is statistically significant based on these metrics (Hogrefe et al., 2008). Also, no assessments have been made to date on the errors that are to be expected even from “perfect” regional-scale air quality modeling systems. To estimate such irreducible model errors due to atmospheric stochasticity (which we consider to be the errors that are expected even from a perfect model – devoid of structural and parametric uncertainties – with perfect – error-free – inputs), we analyzed the observed daily maximum 8 h (DM8HR) ozone time series data at monitoring locations across the contiguous United States (CONUS) during the 1981–2014 time period and present the results of this analysis in Sect. 3.1. In Sect. 3.2, we illustrate how this information could be used in guiding model development specifically aimed at addressing reducible errors in the synoptic (SY) component by contrasting the results from Sect. 3.1 with analysis using the synoptic component from a 21-year simulation performed with the fully coupled WRF-CMAQ simulations covering the 1990–2010 period. Since we relied on multi-decadal historical ozone observations to assess the impact of the stochastic nature of the atmosphere, the results presented here are applicable to both forecasting and retrospective applications of current regional-scale air quality models.

Ground-level DM8HR ozone data covering CONUS during May to September in each year were obtained from the U.S. Environmental Protection Agency's (EPA) Air Quality System (AQS) (see https://www.epa.gov/aqs, last access: 3 February 2020). A valid ozone season consists of at least 80 % data coverage during May to September at each station. A total 185 monitoring stations with at least 30 valid years (to provide enough variety of synoptic conditions, denoted hereafter as 30+ in this paper) from the year 1981 to 2014 are analyzed. Also, fully coupled WRF-CMAQ model simulations over CONUS for the 1990–2010 period were utilized in this study to demonstrate a new perspective on model performance evaluation. To ensure better characterization of the prevailing meteorology (i.e., synoptic forcing) in the retrospective 21-year WRF-CMAQ simulations, four-dimensional data assimilation (FDDA) was utilized following the methodology suggested by Gilliam et al. (2012) and modified for fully coupled meteorology–chemistry model applications as described in Hogrefe et al. (2015). The model setup and performance evaluation of these historical multiyear WRF-CMAQ simulations have been published by Xing et al. (2015), Gan et al. (2015), and Astitha et al. (2017). Time-varying chemical lateral boundary conditions are nested from the 108 km hemispheric WRF-CMAQ simulation from 1990 to 2010 (Xing et al., 2015).

It has been shown that time series of the daily maximum 8 h ozone concentrations contain fluctuations operating on different timescales (e.g., intra-day forcing induced by the fast-changing emissions and atmospheric boundary layer evolution; diurnal forcing induced by the day and night differences; and synoptic forcing induced by the passage of weather systems across the country, sub-seasonal forcing due to the Madden–Julian Oscillation – MJO, and long-term forcing induced by emissions, El Niño–Southern Oscillation – ENSO, climate change, and other slow-varying processes such as seasonal and sub-seasonal variations in the atmospheric deposition and stratosphere–troposphere exchange processes) as noted by Rao et al. (1997), Vukovich (1997), Hogrefe et al. (2000), Porter et al. (2015), Astitha et al. (2017), Xing et al. (2016), and Mathur et al. (2017). Variations in the 8 h ozone can be thought of comprising of the baseline (BL) of pollution that is created by various emitting sources and modulated by the prevailing synoptic weather conditions (Rao et al., 1996, 2011b). Thus, the magnitude of the baseline concentration and the strength of the synoptic component should be viewed as the necessary and sufficient conditions for how high ozone levels can reach on a given day (Astitha et al., 2017). Scale separation can be achieved by applying filtering methods such as the empirical mode decomposition (EMD; Huang et al., 1998), elliptic filter (Poularika, 1998), Kolmogorov–Zurbenko (KZ) filter (Rao and Zurbenko, 1994), adaptive filter technique (Zurbenko et al., 1996), and wavelet (Lau and Weng, 1995). Because improved complete ensemble empirical mode decomposition with adaptive noise (Improved CEEMDAN; Colominas et al., 2014; a version of the empirical mode decomposition method) and KZ filter yielded similar results for the DM8HR time series data as shown in Figs. 1–2 discussed in the next section, only the results from the KZ filter are presented in the subsequent analysis for quantifying the impact of the stochastic nature of the atmosphere on observed and simulated ozone concentrations. Furthermore, the KZ filtering is a simple method and works well even in the presence of missing data (Hogrefe et al., 2003). In this study, we used the KZ5,5 with a window size of 5 d and five iterations on raw ozone time series [O3 (t)] in the same manner as in Luo et al. (2019), Porter et al. (2015), and Rao et al. (2011b). The size of the window and the number of iterations determine the desired scale separation. The KZ5,5 filtering process helps separate the synoptic-scale weather-induced variations embedded in the May–September DM8HR time series data (short-term component) from the long-term baseline component. 1BL(t)=KZ5,5O3(t)2SY(t)=O3(t)-KZ5,5O3(t)3O3(t)=SY(t)+BL(t) Because we are working with the daily maximum 8 h ozone data, the Nyquist interval is 2 d, indicating that the dynamical features having timescales less than 2 d (e.g., intra-day forcing from fast changing emissions and chemical transformations, boundary layer evolution, and diurnal forcing due to night vs. day differences) are not resolvable in this analysis (see Fig. 2 in Dennis et al., 2010). The 50 % cutoff frequency for the KZ5,5 is ∼24 d, and, hence, timescales less than those associated with large-scale weather fluctuations are embedded in the short-term or SY forcing. The KZ filtering is applied to both DM8HR observations and modeled DM8HR time series. Once the baseline is separated from the original DM8HR time series from all monitoring stations, then the synoptic forcing in the historical ozone time series data is used to estimate the variability in ozone concentrations that can be expected because of the chaotic/stochastic nature of the atmosphere by taking into account the relationship between the strength of synoptic forcing and mean of baseline ozone at each location over CONUS; this methodology was applied to both measured and modeled ozone concentrations (see details in Luo et al., 2019). Whereas the focus of Luo et al. (2019) was on transforming the deterministic modeling results into a probabilistic framework for assessing the efficacy of different emission control strategies in achieving compliance with the ozone standard, this paper is aimed at quantifying the model performance errors to be expected at each monitoring site over CONUS even from perfect regional-scale ozone models driven with perfect input data from the ever-present stochastic nature of the atmosphere.

Regardless of how accurate the regional air quality model is, the stochastic variations in the atmosphere cannot be consistently reproduced by the deterministic numerical models. In this study, we demonstrate how to quantify this irreproducible stochastic component by isolating the synoptic forcing imbedded in 30+ years of historical observations and assess the performance of the 36 km fully coupled WRF-CMAQ model in simulating 21 years of ozone concentrations over the contiguous US. Observation-based analysis reveals that on average, the irreducible error attributable to the stochastic nature of the atmosphere ranges from ∼2 ppb at the 50th percentile to ∼5 ppb at the 95th percentile. To improve regional-scale ozone air quality models, attention should be paid to accurately simulate the baseline concentration by focusing on the quality of the emission inventory and the model's treatment for the boundary conditions and slow-changing (operating on sub-seasonal, seasonal, and longer-term timescales) atmospheric processes. Also, errors in reproducing the synoptic forcing can possibly be reduced with high-resolution meteorological modeling using appropriate data assimilation techniques. Nonetheless, these results demonstrate the presence of large variability in the upper tail of the DM8HR O3 concentration cumulative distribution even with perfect models using perfect input data. Having this quantitative estimation of practical limits for a model's accuracy helps in objectively assessing the current state of regional-scale air quality models, measuring progress in their evolution, and providing meaningful and firm targets for improvements in their accuracy relative to measurements from routine networks.