The MACC global products for reactive gases consist of a reanalysis performed for the years 2003–2012 (Inness et al., 2013) and the near-real-time analysis and forecast, largely based on the same assimilation and forecasting system, but targeting different user groups (operational air quality forecasting and regional climate modelling, respectively). The Model for OZone And Related chemical Tracers (MOZART) chemical transport model (CTM) is coupled to the integrated forecast system (IFS) of the European Centre for Medium-Range Weather Forecast (ECMWF), which together represent the MOZART–IFS model system (Flemming et al., 2009; Stein et al., 2012). An alternative analysis system has been set up based on the global chemistry transport model version 5 (TM5; see also Huijnen et al., 2010). Details of the MOZART version used in the MACC global products can be found in Kinnison et al. (2007) and Stein et al. (2011, 2012). In our simulation, the IFS and the MOZART model run in parallel and exchange several two- and three-dimensional fields every model hour using the Ocean Atmosphere Sea Ice Soil version 4 (OASIS4) coupling software (Valcke and Redler, 2006), thereby producing three-dimensional IFS fields for O3, CO, SO2, NOx, HCHO, sea salt aerosol, desert dust, black carbon, organic matter, and total aerosol. The IFS provides meteorological data to MOZART. Data assimilation and transport of the MACC species takes place in the IFS, while the whole chemical reaction system is calculated in the MOZART model.

The MACC_osuite (operational suite) is the global near-real-time MACC model production run for aerosol and reactive gases. Here, we have investigated only the MACC analysis. In contrast to the reanalysis, the MACC_osuite is a near-real-time run, which implies that it is only run once in near-real-time and may thus contain inconsistencies in, e.g., the assimilated data. The MACC_osuite was based on the IFS cycle CY36R1 with IFS model resolution of approximately 100 km by 100 km at 60 levels (T159L60) from September 2009 to July 2012. The gas-phase chemistry module in this cycle is based on MOZART version 3.0 (Kinnison et al., 2007). The model has been upgraded, following updates of the ECMWF meteorological model and MACC-specific updates, i.e. in chemical data assimilation and with respect to the chemical model itself. Thus, from July 2012 onwards, the MACC_osuite has run with a change of the meteorological model to a new IFS cycle (version CY37R3), with an IFS model resolution of approximately 80 km at 60 levels (T255L60) and an upgrade of the MOZART version 3.5 (Kinnison et al., 2007; Emmons et al., 2011; Stein et al., 2013). This includes, amongst others, updated velocity fields for the dry deposition of O3 over ice, as described in Stein et al. (2013). A detailed documentation of system changes can be found at http://atmosphere.copernicus.eu/user-support/operational-info.

In this study, we have tended to use the same evaluation data sets as during the MACC near-real-time validation exercise. This implies some discontinuities in the evaluations, e.g. the substitution of SCIAMACHY data with GOME-2 data after the loss of the Envisat sensor or an exclusion of MOPITT satellite data after the start of its assimilation into the model. The continuous process of updating and complementation of data sets in databases requires the selection and definition of a validation data set at some point. The comparatively small inconsistencies between our data sets are considered to have a negligible impact on the overall evaluation results.

A comprehensive model validation requires the selection of validation metrics that provide complementary aspects of model performance. The following metrics have been used in the validation: Modifiednormalisedmeanbias(MNMB)MNMB=2N∑ifi-oifi+oiRootmeansquareerror(RMSE)RMSE=1N∑i(fi-oi)2 CorrelationcoefficientR=1N∑ifi-f¯oi-o¯σfσo, where N is the number of observations, f the modelled analysis and o the observed values, f¯ and o¯ are the mean values of the analysis and observed values and σf and σo the corresponding standard deviations.

The validation metrics above have been chosen to provide complementary aspects of model performance. The modified normalised mean bias is a normalisation based on the mean of the observed and forecast value (e.g. Elguindi et al., 2010). It ranges between -2 and 2 and when multiplied by 100 %, it can be interpreted as a percentage bias.

We chose to use the modified normalised mean bias (MNMB) in our evaluations because verifying chemical species concentration values significantly differs from verifying standard meteorological fields. For example, spatial or temporal variations can be much greater and the differences between model and observed values (“model errors”) are frequently much larger in magnitude. Most importantly, typical concentrations can vary quite widely between different pollutant types (e.g. O3 and CO) and regions (e.g. Europe vs. Antarctica), and a given bias or error value can have a quite different significance. It is useful, therefore, to consider bias and error metrics that are normalised with respect to observed concentrations and hence can provide a consistent scale regardless of pollutant type (see e.g. Elguindi et al., 2010, or Savage et al., 2013). Moreover, the MNMB is robust to outliers and converges to the normal bias for biases approaching zero, while taking into account the representativeness issue when comparing coarse-resolved global models versus site-specific station observations. Though GAW stations prove regionally representative in general, the experience is that local effects cannot always be ruled out reliably in long worldwide data sets, because each of the different species has its individual scale of transport and chemical processes, which in one case may exceed and in the other case fall bellow the model resolution. Referencing to the model/observation mean again constitutes a pragmatic solution to avoid misleading bias tendencies, particularly in sensitive regions with sparse data coverage. Within MACC, the MNMB is used as an important standard score. It is used in the MACC quarterly validation reports and it appears in many recent publications, e.g. Cuevas et al. (2015), Eskes et al. (2015), Sheel et al. (2014).

The MNMB varies symmetrically with respect to under- and overestimation. However, when calculated over longer time periods, a balance in model error, with model over- and underestimation compensating each other, can lead to a small MNMB for the overall period. For this reason, it is important to additionally consider an absolute measure, such as the root mean square error (RMSE). However, it has to be noted that the RMSE is strongly influenced by larger values and outliers, due to squaring. The correlation coefficient R can vary between 1 (perfect correlation) and -1 (perfect negative correlation) and is an important measure for checking the linearity between model and observations.

The evaluation of the MACC_osuite run with O3 from GAW surface observations (described in Sect. 2.2.1) demonstrates good agreement in absolute values and seasonality for most regions. Figure 2 shows maps with MNMB (see Sect. 2.3) evaluations for 50 GAW stations globally (top) and in Europe (bottom). Figure 3 presents selected time series plots representing the results for high latitudes, low latitudes and Europe. Large negative MNMBs over the whole period September 2009 to December 2012 (-30 to -82 %) are observed for stations located in Antarctica (Neumayer (NEU), South Pole (SPO), Syowa (SYO) and Concordia (CON)) whereby O3 surface mixing ratios are strongly underestimated by the model. For stations located at high latitudes in the Northern Hemisphere (Barrow (BAR), Alaska, and Summit (SUM), Denmark), the MACC_osuite exhibits similar underestimated values of up to -35 % for the whole evaluation period. The time series plots for Arctic and Antarctic stations (e.g. SUM, NEU and SPO) in Fig. 3 show that an underestimation seen in these regions appears to be remedied and model performance improved with an updated dry deposition parameterisation over ice, which has been introduced with the new model cycle in July 2012 (see Sect. 2.1).

Large positive MNMBs (up to 50 to 70 %, Fig. 2) are observed for stations that are located in or nearby cities and thus exposed to regional sources of contamination (Iskrba (ISK), Slovenia; Tsukuba (TSU), Japan; Cairo (CAI), Egypt). In tropical and subtropical regions, O3 surface mixing ratios are systematically overestimated (by about 20 % on average) during the evaluation period. The time series plots for tropical and subtropical stations (e.g. for Ragged Point (RAG), Barbados, and Cape Verde Observatory, Cape Verde (CVO), Fig. 3) reveal a slight systematic positive offset throughout the year, however with high correlation coefficients (0.6 on average).

For GAW stations in Europe, the evaluation of the MACC_osuite for the whole period shows MNMBs between -80 and 67 %. Large biases appear only for two GAW stations located in Europe: Rigi (RIG), Switzerland (-80 %), located near mountainous terrain and ISK, Slovenia (67 %). For the rest of the stations MNMBs lie between 22 and -30 %. RMSEs (see Sect. 2.3) range between 7 and 35 ppb (15 ppb on average). Again, results for ISK and RIG show the largest errors. All other stations show RMSEs between 7 and 20 ppb. Correlation coefficients here range between 0.1 and 0.7 (with 0.5 on average). Table 4 summarises the results for all stations individually.

Monthly MNMBs (see Fig. 4) show a seasonally varying bias, with positive MNMBs occurring during the northern summer months (with global average ranging between 5 and 29 % during the months June and October), and negative MNMBs during the northern winter months (between -2 and -33 % during the months December to March). These deviations partly cancel each other out in MNMB for the whole evaluation period. For the RMSEs (Fig. 5) maximum values also occur during the northern summer months with the global average ranging between 11 and 16 ppb for June to September. The smallest errors appear during the northern hemispheric winter months (global average falling between 8 and 10 ppb for December and January). The correlation does not show a distinct seasonal behaviour (see Fig. 6).

The time series plots in Fig. 3 show that the seasonal cycle of O3 mixing ratios with maximum concentrations during the summer months and minimum values occurring during winter times for European stations (e.g. Monte Cimone (MCI), Italy; Kosetice (KOS), Czech Republic; and Kovk (KOV), Slovenia), could be well reproduced by the model, although there is some overestimation in summer resulting mostly from observed minimum concentrations that are not captured correctly by the MACC_ osuite (KOS, Czech Republic, and KOV, Slovenia).

The validation with EMEP surface ozone observations (described in Sect. 2.2.2) in three different regions in Europe for the period September 2009 to December 2012 likewise confirms the behaviour of the model to overestimate O3 mixing ratios during the warm period and underestimate O3 concentrations during the cold period of the year (see Fig. 7). The mostly positive bias (May–November) is between -9 and 56 % for northern Europe and central Europe and between 8 and 48 % for southern Europe. Negative MNMBs appear, in accordance with GAW validation results, during the winter–spring period (December–April) ranging between -48 and -7 % for EMEP stations in northern Europe (exception: December 2012 with 25 %), between -1 and -39 % in central Europe (exception: December 2012 with 31 %), whereas in southern Europe, deviations are smaller and remain mostly positive (between -8 and 9 %) in winter (exception: December 2012 with 37 %). The different behaviour for December 2012 likely results from the limited availability of observations towards the end of the validation period. The separate evaluation of day and night-time O3 mixing ratios (Fig. 8) shows that for northern Europe night-time biases exceed daytime biases during all seasons. For central Europe and southern Europe night-time biases are larger (negative MNMBs) during cold periods (December–April), whereas during warm periods (May–November) larger biases (positive MNMBs) appear during daytime.

The validation of the MACC_osuite with surface observations of 29 GAW stations (described in Sect. 2.2.1) shows that over the whole period September 2009 to December 2012, CO mixing ratios could be reproduced with an average MNMB of -10 %. The MNMBs for all stations range between -50 and +30 %. Results are listed in Table 5; a selection of time series plots shows the results for stations in Europe, Asia and Canada in Fig. 9. MNMBs exceeding ±30 % appear for stations that are either located in or nearby cities and thus exposed to regional sources of contamination (KOS, Czech Republic) or are located in or near complex mountainous terrain (RIG, Switzerland, and BEO Moussala (BEO), Bulgaria) which is not resolved by the topography of the global model. The RMSEs fall between 12 and 143 ppb (on average 48 ppb) for all stations during the validation period, but for only four stations (RIG, KOS, Payerne (PAY), Switzerland and BEO, all located in Europe) do the RMSEs exceed 70 ppb. Correlation coefficients from the comparison with GAW station data calculated over the whole time period range between 0 and 0.8 (on average 0.4), with only four stations showing values smaller than 0.2 (RIG, BEO, East Trout Lake (ETL) and Lac la Biche (LAC); the latter two located in Canada).

Considering the global monthly MNMBs and RMSEs, it can be seen that during the northern hemispheric summer months, June to September, both are small (absolute differences less than 5 %); see Figs. 10 and 11. Negative MNMBs (up to -35 %) and larger RMSEs (up to 72 ppb) appear during the northern hemispheric winter months, November to March, when anthropogenic emissions are at a highest level, especially for the US, northern latitudes and Europe. Monthly correlation coefficients are between 0.1 and 0.5 and do not show a distinct seasonal behaviour (see Fig. 12), the low values of 0.1 during the period January to October 2011 result from the reading error in the fire emissions (see Sect. “Emission inventories and assimilated data sets”). The generally only moderate correlation coefficient is related to mismatches in the strong short-term variability seen in both the model and the measurements.

The time series plots for stations in Europe, Asia and Canada in Fig. 9 demonstrate that the annual CO cycle could to a large degree be reproduced correctly by the model with maximum values occurring during the winter period and minimum values appearing during the summer season. However, the model shows a negative offset during the winter period. Seasonal air mass transport patterns that lead to regular annual re-occurring CO variations could be reproduced for GAW stations in East Asia: the time series plots for Yonagunijima (YON) and Minamitorishima (MNM) station, Japan (Fig. 9), show that the drop of CO, associated with the air mass change from continental to cleaner marine air masses after the onset of the monsoon season during the early summer months, is captured by the MACC_osuite. Deterioration in all scores is visible during December 2010 in the time series plots of several stations (e.g. Jungfraujoch (JFJ), and Sonnblick (SBL), Fig. 9). This is likely a result of changes in the processing of the L2 IASI data and a temporary blacklisting of IASI data (to avoid model failure) in the assimilation.

The comparison with MOPITT satellite CO total columns between October 2009 and June 2012 (described in Sect. 2.2.3) shows a good qualitative agreement of spatial patterns and seasonality; see Table 6. The MNMBs for eight regions are listed in Fig. 13 and range between -22 and 14 %. The seasonality of the satellite observations is captured well by the MACC_osuite over Asia and Africa, with MNMBs between -6 and 9 % (North Africa), -12 and 8 % (southern Africa), -11 and 12 % (East Asia) and -3 and 14 % (South Asia). The largest negative MNMBs appear during the winter periods, especially from December 2010 to May 2011 and from September 2011 to April 2012, for Alaska and Siberia and for the US and Europe (MNMBs up to -22 %), which coincides with large differences between MOPITT and IASI satellite data (see Fig. 14). On the global scale the average difference between the IASI and MOPITT total columns is less than 10 % (George et al., 2009), and there is a close agreement of MOPITT and IASI for South Asia and Africa (see Fig. 14). However, larger differences between MOPITT and IASI data appear during the northern winter months over Alaska, Siberia, Europe and the US, which result in lower CO concentrations in the model, due to the assimilation of IASI CO data in the MACC_osuite. The differences between MOPITT and IASI data can be mainly explained by the use of different a priori assumptions in the IASI and MOPITT retrieval algorithms (George et al., 2015). The Fast Optimal Retrievals on Layers for IASI (FORLI; Hurtmans et al., 2012) software uses a single a priori CO profile (with an associated variance-covariance matrix) whereas the MOPITT retrieval algorithm uses a variable a priori, depending on time and location. George et al. (2015) showed that differences above Europe and the US in January and December (for a 6 year study) decrease by a factor of 2 when comparing IASI with a modified MOPITT product using the IASI single a priori. Between January and October 2011 there has also been a reading error in the fire emissions that contributes to larger MNMBs during this period (see Sect. “Emission inventories and assimilated data sets”).

Figure 15 shows global maps of daily tropospheric NO2 VCD averaged from September 2009 to March 2012. Overall, the spatial distribution and magnitude of tropospheric NO2 observed by SCIAMACHY are well reproduced by the model. This indicates that emission patterns and NOx photochemistry are reasonably well represented by the model. However, the model underestimates tropospheric NO2 VCDs over industrial areas in Europe, East China, Russia and Southeast Africa compared to satellite data. This could imply that anthropogenic emissions from RETRO–REAS are too low in these regions, or that the lifetime in the model is too short. The model simulates larger NO2 VCD maxima over central Africa, which mainly originate from wild fires. It remains unclear if GFEDv2/GFAS fire emissions are too high here or if NO2 fire plumes closer to the ground cannot be seen by SCIAMACHY due to light scattering by biomass burning aerosols (Leitão et al., 2010). In the Northern Hemisphere, background values of NO2 VCD over the ocean are lower in the simulations than in the satellite data. The same is true for the South Atlantic Ocean to the west of Africa (see Fig. 15). This might suggest a model underestimation of NO2 export from continental sources towards the ocean or too rapid conversion of NO2 into its reservoirs. However, as the NO2 columns over the oceans are close to the uncertainties in the satellite data, care needs to be taken when interpreting these differences.

Time series of daily tropospheric NO2 VCD averaged over different regions and corresponding monthly means are presented in Figs. 16 and 17, respectively. Time series of the MNMB and RMSE are shown in Figs. 18 and 19, respectively. Table 7 summarises the statistical values derived over the whole time period. High anthropogenic emissions occur over the US, Europe, South Asia and East Asia compared to other regions on the globe (e.g. Richter et al., 2005). In principle, the MACC_osuite catches the pattern of satellite NO2 VCD over these regions. However, the model tends to underestimate NO2 VCDs throughout the whole time period investigated here. The negative bias is most pronounced over East Asia with a modelled mean NO2 VCD for September 2009 to December 2012 of about 3.8 × 1015 molec cm-2 lower than that derived from satellite measurements (see Table 7).

Considering monthly values, the MACC_osuite strongly underestimates magnitude and seasonal variation of satellite NO2 VCD over East Asia (MNMBs between about -40 and -110 % and RMSE between 1×1015 and 14×1015 molec cm-2 throughout the whole time period). A change in the modelled NO2 values is apparent in July 2012 when the emission inventories changed and the agreement with the satellite data improved for South and East Asia but deteriorated for the US and Europe. This results in a drop of MNMBs (Fig. 18) for Europe and the US with values approaching around -70 % by the end of 2012. Nevertheless, correlations between daily satellite and model data derived for the whole time period (see Table 7) are high for East Asia (0.8), South Asia (0.8), Europe (0.8), and lower, but still rather high, for the US (0.6).

The North African and southern African regions are strongly affected by biomass burning (Schreier et al., 2014). Magnitude and seasonality of daily and monthly tropospheric NO2 VCDs (Figs. 16 and 17, respectively) are rather well represented by the model, apart from January to October 2011, due to difficulties in reading fire emissions for this time period (see Sect. “Emission inventories and assimilated data sets”). The latter results in large absolute values of the MNMB (Fig. 18) and large RMSEs (Fig. 19) between January and October 2011 compared to the rest of the time period. As for other regions investigated in this section, mean values of simulated daily tropospheric NO2 VCDs over North Africa and southern Africa between September 2009 and December 2012 tend to be lower than the corresponding satellite mean values (see Table 7). The correlation between daily model and satellite data over the whole time period is about 0.6 for southern Africa and 0.5 for North Africa. Whether this difference in model performance for the African regions is due to meteorology, chemistry or emissions needs to be investigated, but this is outside the scope of this paper.

The evaluation of modelled NO2 with GAW surface data for six European stations accordingly shows that NO2 is generally underestimated at the surface. The MNMBs are typically in the range of -26 and -45 %, larger MNMBs appear only for two stations in complex mountainous terrain (RIG 68 % and SBL -160 %). The RMSEs are between 0.3 and 9 ppb, and the correlation coefficients between 0.1 and 0.6 for the period between September 2009 and December 2012 (Table 8). The annual cycle of NO2 with maximum concentrations during the winter period is in principle captured by the model, shown in the time series plots in Fig. 20. As is observed for the satellite VCDs, NO2 surface concentrations decrease in the model with the introduction of the updated model version and emission inventories. For stations located in complex terrain (e.g. Rigi, Fig. 20), results improve after the model update, likely also due to the higher model resolution. Monthly values of MNMB, R and correlation coefficient are shown in Figs. 21 to 23.