Production sites within the 50×50 km2 domain are randomly placed on the 1.3×1.3 km2 WRF model grid, with at most one site per grid cell. Emission statistics for the sites are based on observations from the Barnett Shale Coordinated Campaign (Lyon et al., 2015). For each scenario we randomly assign a production size category to each site with 23 % of the sites as small, 62 % as medium, and 15 % as large (Rella et al., 2015). We then assign an emission rate for each site by randomly sampling the bimodal probability density functions (PDFs) describing low-mode emissions and high-mode emissions for each size category (Lan et al., 2015; Rella et al., 2015; Yacovitch et al., 2015). We assume no other sources in the domain.

Figure 1 shows the PDFs of methane emissions for each production site size category. We flag production sites to be in the high-emission mode if they exceed an emission threshold of 40 kg h-1 (axis break in Fig. 1), which corresponds on average to 5 % of all the sites. High-mode emissions from small facilities are much lower, centered around 24 kg h-1, and would be difficult to distinguish from the normal (low) emission mode. Thus we do not attempt to detect them as high-mode emitters.

Figure 2 shows a sample realization of the oil/gas field with 24 small production sites, 67 medium sites, and 9 large sites (100 total) within the 50×50 km2 domain. In this realization there are five sites in the high-emission mode. We generate 500 emission scenarios in the same fashion as Fig. 2 by randomly assigning size categories for each site (small, medium, large) and randomly sampling the emission PDFs from Fig. 1.

We use the meteorological simulation previously generated by Turner et al. (2018) for a 1-week period (19–25 October 2013) in the Barnett Shale. This simulation applied the Weather Research and Forecasting Model (WRF; Skamarock et al., 2008) at 1.3 km horizontal resolution to drive the Stochastic Time-Inverted Lagrangian Transport (STILT) model (Nehrkorn et al., 2010). STILT is a receptor-oriented Lagrangian particle dispersion model that defines the source footprints for individual atmospheric observations. Turner et al. (2018) applied it to generate 1.3×1.3 km2 hourly footprints for any daytime surface or atmospheric column observation in a 70×70 km2 domain. Footprints for each column were obtained by releasing and tracking back in time 100 particles from vertical levels centered at 28, 97, 190, and 300 m above ground, and 8 additional levels up to 14 km altitude spaced evenly on a pressure grid. The column footprints were weighted with a typical near-uniform SWIR averaging kernel for satellite observations (Worden et al., 2015). Surface observations are taken in the lowest model layer (centered at 28 m above ground) and the corresponding footprints are obtained by releasing and tracking back in time 100 particles at the observation location and time. We use the ensemble of footprints generated by Turner et al. (2018) and add to it hourly footprints for surface observations at night. The 70×70 km2 observing domain encompasses our 50×50 km2 oil/gas field plus 10 km outside the boundaries (Fig. 2) to account for plume transport.

The 70×70 km2 archive of WRF-STILT footprints allows us to immediately compute the time-dependent methane concentration field associated with any emission scenario. Figure 3 shows a sample footprint, expressing the sensitivity of atmospheric concentrations at a given location and time i to the emission field upwind. Column footprints are about an order of magnitude smaller than surface footprints because surface signal is weakened for receptors (e.g., satellites) with total column sensitivity. Taking the footprints to represent the true atmospheric transport relating emissions to atmospheric concentrations for that location and time, we can combine them with any realization of our emission field (Sect. 2.1) to generate the true time-dependent methane concentrations in the domain to be sampled by the instruments.

Satellite observations of methane column concentrations are conventionally expressed in units of dry column mean mixing ratio (ppb), which is the ratio of the vertical column density of methane to the vertical column density of dry air (Jacob et al., 2016). The footprint for location and time i is mathematically represented as hi=(∂yi/∂x)T (units: ppb µmol-1 m2 s) where yi is the methane concentration (ppb) for that location and time, and x (µmol m-2 s-1) is a vector of dimension n describing the emission field for the n emitters in the domain. The vector hi is also a vector of n dimension. The true atmospheric concentration can be immediately constructed for any emission field x as yi=hi⋅x+b, where ⋅ denotes the scalar product and b is a background assumed here to be constant.

A given methane observing configuration makes m observations of the domain over the 1-week simulation period. The true methane concentrations for that observation ensemble can be assembled as an m-dimensional vector ytrue=Hx+b, where H=∂ytrue/∂x is the m×n Jacobian matrix of footprints with rows hiT. The pseudo-observations are then generated as y=ytrue+σε, where σ is the instrument precision (1 standard deviation) and the vector ε is a random realization of Gaussian noise with mean value of zero and standard deviation of unity for each vector element. SWIR instruments may also suffer from systematic errors but we do not account for those here in the absence of information. The largest source of systematic error on our scale would likely be the inhomogeneity in surface reflectivity (Pfister et al., 2005).

The mean daytime 10 m horizontal wind speed inside the observing domain during the simulated week is 5.4 m s-1. Stronger winds could further dilute plumes within an observing domain, making the ability for satellite detection of emitters more difficult; on the other hand, the model transport error is less for stronger winds (Varon et al., 2018).

Table 1 describes the different satellite observing configurations evaluated in this work including TROPOMI, GeoCARB with 2 or 4 return times per day, and an aspirational next-generation geostationary instrument with 1.3×1.3 km2 pixel resolution, 1 ppb precision, and hourly return frequency between 08:00 and 17:00 LT (local time). Successful methane retrievals from satellites require a clear sky. The probability of clear sky in a partly cloudy domain depends greatly on pixel size (Remer et al., 2012). Results for a partly cloudy condition would depend on the particular cloud configuration and would be difficult to generalize. Here we assume clear-sky conditions to avoid this complication, but the detection probability for high-mode emitters should then be viewed as an upper limit. In particular, it should be recognized that no detection from satellite is possible for a cloudy domain.

We also wish to determine the benefit of a well-positioned surface air monitoring network for supplementing the satellite observations. Assuming that we have M fixed monitoring instruments to deploy measuring surface air methane concentrations in situ. We want to place them in a configuration that maximizes the information that they would provide, assuming an isotropic wind for generality. A trivial solution would be to place an instrument at each production site, in which case the monitoring problem would be fully solved, but this solution may not be practical for a large number of production sites. Given a known spatial distribution of emitters (the locations of the production sites), we use the k-means spatial clustering approach (Hartigan and Wong, 1979) to select monitoring site locations minimizing the distances to emitter locations. Figure 2 shows the selected locations for five surface monitoring sites. We assume that these sites report hourly data with 1 ppb precision and that the background concentration in surface air is constant, consistent with the assumption made for satellite observations. A variable background would complicate the problem but could be retrieved as part of the inversion (Wecht et al., 2014b).

An important consideration in the interpretation of satellite observations is that methane column enhancements from individual point sources are typically small relative to instrument precision, even in the high-emitting mode (Jacob et al., 2016; Varon et al., 2018). Figure 4 shows the pixel-resolved distribution of atmospheric methane column enhancements above the background for a single pass of the different satellite instruments sampling the emission field of Fig. 2. The enhancements are less than 1 ppb even for 1.3×1.3 km2 pixels and are weaker at coarser pixel resolution. This is less than the single-scene precision of the satellite instruments (Table 1). Successful detection of high-mode emitters thus requires the sampling of many pixels, across the plume and/or through repeated sampling, to reduce the noise. This is less of an issue for surface air measurements, where methane enhancements are an order of magnitude higher (Fig. 3). On the other hand, surface monitoring sites are spatially sparse. For both satellite and surface air observations, a formal inverse analysis of the ensemble of atmospheric observations accounting for plume transport is required for detection of the high-mode emitters.

Given a set of observations y and Jacobian matrix H, we need an inverse method to determine the best solution x^ of the emission field x at predetermined locations. We use the same matrix H for both pseudo-observation construction and the inversion. The inversion should be able to detect the small fraction of sources in the high-emitting mode, with detection being more important than quantification. This is known as a sparse-solution problem, where most elements of the emission field x are very small (for which an optimized value of zero would be acceptable), and a few of the elements are relatively large. We use regularized least squares regression (e.g., Hansen, 2010), also known as Tikhonov regularization, where the solution is found by minimizing the cost function J(x), J(x)=Hx-yTR-1Hx-y+λxLp. Here the first term on the right-hand side represents the ordinary least-squares cost function, such that the solution would minimize the residuals between the prediction Hx and the observations weighted by the observational error covariance matrix R. The second term represents an adjustable parameter λ and the L-norm of x, which is a measure of the magnitude of the vector x defined as the following: xL=Σk=1nxkLL. Adding this second term in the cost function penalizes the total magnitude of x in the solution, which reduces overfitting to noise and regularizes the solution. When L=1 and p=1, this is known as L1 regularization or the least absolute shrinkage and selection operator (LASSO; Tibshirani, 1996), and Eq. (1) takes the form J(x)=Hx-yTR-1Hx-y+λ∑k=1nxk. When L=2 and p=2 , Eq. (1) takes the form known as L2 regularization or ridge regression (Evgeniou et al., 2000): J(x)=Hx-yTR-1Hx-y+λxTx. Equation (4) is equivalent to the standard Bayesian optimization (Rodgers, 2000) assuming Gaussian distributions, a prior emission estimate of zero, and uniform prior error variance of λ-1.

The observational error covariance matrix R=(rij) adds and accounts for both instrument and model transport errors. Representation errors are negligible due to the model grid resolution being finer or the same resolution as the instrument pixels (Turner et al., 2018). The diagonal terms add the corresponding error variances in quadrature: rii=σI2+σM2, where σI is the instrument error standard deviation as given by the precision in Table 1, and σM is the model transport error standard deviation previously estimated to be 4 ppb for methane columns (Turner et al., 2018). Given the order of magnitude difference in sensitivity between satellite columns and surface measurements (Fig. 3), we assume σM to be 40 ppb for surface measurements. Off-diagonal terms account for model transport error correlation between different observations. Following Turner et al. (2018), we assume a temporal error correlation length scale (τ) of 2 h and a spatial error correlation length scale (ℓ) of 40 km: rij=σM2exp-dlexp-tτfori≠j, where d and t are the distance and elapsed time, respectively, between observations yi and yj.

Additional model transport error correlation applies when combining satellite and surface air observations in the inversion, since the footprints can be similar (Fig. 3). To quantify this error correlation, we use the work of Sheng et al. (2018b) who jointly compared column (TCCON) and surface air (NOAA) measurements of methane at Lamont, Oklahoma, with GEOS-Chem transport model simulations. By correlating the model–observation differences for coincident column (i) and surface air (j) observations we find a model transport error correlation coefficient cor(i,j)=0.65 that we apply to the corresponding off-diagonal terms: rij=cori,jσMiσMjexp-dlexp-tτ. Inverse solutions derived using L1 regularization produce sparser solutions than the L2 counterpart (Tibshirani, 1996), which is desirable for our application and has previously been shown to produce good results for constraining methane hot spots (Hase et al., 2017). Here we will perform both L1 and L2 inversions and compare the results. Minimization of J(x) in Eqs. (3) and (4) to obtain the solution x^ corresponding to dJ/dx=0 is done numerically using coordinate gradient descent (Friedman et al., 2009). The regularization parameter λ is chosen so that the mismatch between model and observations is small, but not so small that the solution x^ is over fit to random noise, which would occur when λ=0. We use the process of 5-fold cross-validation to select an optimal λ value (Arlot and Celisse, 2010). This process randomly samples H and y into a training and validation set. Minimization of J is done on the training set using an array of λ values. The process is repeated five times, and the value of λ that on average minimizes the residual error in the validation set is retained.

Figure 5 shows the distribution x^ from a single realization of emissions, GeoCARB 4 times per day (denoted as 4× day-1) pseudo-observations, and both L1 and L2 regularization. In this simulation, L1 regularization enables the retrieval of high-mode emitters while L2 regularization is more restrictive in allowing excursions from the low-mode mean.

Success in the detection of high-mode emitters from the distribution of x^ can be determined by comparison to the actual occurrence and location of these emitters as defined in Sect. 2.1 and illustrated in Fig. 2. In a real-world application we would not know the actual PDFs of emissions (Fig. 1), so we need to diagnose the occurrence of high-mode emitters on the basis of anomalies in the distribution of x^. We define high-mode elements as being more than S standard deviations from the mean of the x^ distribution, where S is varied in the 1.65–2.5 range to examine the associated sensitivity. Using anomaly detection on x^ instead of a fixed threshold (e.g., 40 kg h-1) allows for generalization to other emission fields where the mean normal and high modes may be different than the Barnett Shale. Figure 5 shows thresholds for classifying high-mode emitters using anomaly detection and a fixed value of 40 kg h-1. The L1 threshold is larger than the L2 threshold, but smaller than 40 kg h-1. Had the fixed threshold been used, some high-mode emitters (relative to x^) would not have been classified as such.

The detection of high-mode emitters by the inversion is graded into four categories: (1) true positives (TP), or the inversion correctly identifying the locations of the high-mode emitters; (2) true negatives (TN), or the inversion correctly identifying the locations of the low-mode emitters; (3) false positives (FPs), or the inversion signaling a high-mode emitter when in reality the emitter is in the low mode; and (4) false negatives (FNs), or the inversion signaling a low-mode emitter when in reality the emitter is in the high mode.

We compile these grades into three overall performance metrics (Brasseur and Jacob, 2017). The probability of detection (POD) is defined as the ratio of true positives to true positives plus false negatives: POD=ΣTPΣTP+ΣFN. This metric measures the ability to detect high-mode emitters. The false alarm ratio (FAR) is defined as the ratio of false positives to false positives plus true positives: FAR=ΣFPΣTP+ΣFP. This metric measures the reliability of high-mode emission occurrences detected by the inversion.

A perfect observing system would have a POD of 1 and a FAR of 0. Here we define a successful observing system as achieving a POD of 0.8 (80 %) and a FAR of 0.2 (20 %). These criteria, although somewhat arbitrary, allow us to succinctly summarize the success of each observing configuration.

We combine the POD and FAR metrics into one overall performance metric called the equitable threat score (ETS; Wang, 2014): ETS=ΣTP-αΣTP+ΣFP+ΣFN-α, where α is the number of TP predictions that are expected by chance: α=ΣTP+ΣFPΣTP+ΣFNΣTP+ΣFP+ΣFN+ΣTN=1NΣFPFARΣTPPOD and N=ΣTP+ΣFP+ΣFN+ΣTN. The ETS measures how well the high-mode emitters detected by the observing system correspond to the actual occurrences, beyond what could be achieved by chance. A perfect observing system has an ETS of 1, and a system performing worse than chance would have a negative ETS. An observing system with POD of 0.8 and FAR of 0.2 has an ETS of 0.65 for a field where 5 % of emitters are in the high mode. We take this as our ETS criterion for successful detection.

We begin by testing the ability of each satellite configuration of Table 1 to detect high-mode emitters from fields of 20 to 500 randomly scattered production sites within the 50×50 km2 domain. For a given number of sites, we conduct each test for 500 different realizations of the emission field randomly assigning each production site to a size category (small, medium, large) and randomly sampling the PDFs of Fig. 1. Emitter locations are fixed across all 500 realizations. Figure 6 shows the POD, FAR, and ETS results for a field of 100 emitters and compares the results of L1 and L2 regularizations. The values represent the mean results for the ensemble of 500 realizations, and the error bars represent the range of results when the high-mode detection threshold S is varied from 1.65 to 2.5. We find that L1 regularization provides better predictions for all cases. This is especially the case for the next-generation satellite, where L1 regularization produces a POD of 0.85 with a near-perfect FAR of 0.04. L2 regularization is more conducive to spreading emissions across a broader array of state vector elements. The better performance of L1 regularization is also observed for other site densities (not shown). We use L1 regularization in what follows.

Figure 6 also compares the performances of the satellite observing systems to those of an ensemble of 5–20 optimally placed (k means) surface sites. We find that the surface observing system performs comparably to GeoCARB. We explore combining satellite and surface observations into a single prediction in Sect. 3.3.

The results from Fig. 6 show that TROPOMI and GeoCARB are unsuccessful in locating high-mode emitters for a field of 100 production sites (0.04 sites km-2). We examine the sensitivity of this result to site density. Figure 7 compares the detection results for fields of 20, 50, 100, and 500 production sites within the 50×50 km2 domain. For a field of only 20 emitters, TROPOMI is successful and GeoCARB produces near-perfect results. For a field of 50 emitters, TROPOMI is no longer successful, but GeoCARB is still marginally successful due to finer pixel resolution and higher instrument precision. We find in general that GeoCARB gains little by sampling 4 times a day (4× day-1) vs. 2× day-1. This is due to the temporal model error correlation between successive GeoCARB observations. Accounting for cloud cover would show more benefit from 4× day-1 observations, since a higher frequency of observations allows for a greater chance of sampling clear-sky conditions, although the benefit depends on the cloud persistence timescale (Sheng et al., 2018a).

The ability of a satellite observing configuration to localize high-mode emitters thus depends not only on repeat time, resolution, precision, and cloud cover, but also on the density of emitters within a field. For the high-density fields of 100 and 500 production sites considered here (0.04 and 0.2 sites km-2), we find that only the next-generation satellite instrument is successful. Actual fields can be even denser but we are limited in our investigation by the 1.3×1.3 km2 resolution of the WRF simulation. Detecting individual high-mode emitters in denser fields would require geostationary satellite observations with sub-kilometer pixels but this is beyond the scope of current proposals.

The results from Fig. 7 are somewhat pessimistic regarding the ability of near-future satellite observations (TROPOMI and GeoCARB) to detect the locations of high-mode emitters in fields of 100+ wells. It may be acceptable to relax the localization criterion. If the observing system detects a false positive that is sufficiently close to the actual location of a high-mode emitter, then the detection may still have some value. In our OSSE setup, localization is effectively limited by the 1.3×1.3 km2 grid resolution of the WRF simulation. To examine the sensitivity to localization, we repeated the analysis allowing for 3–5 km tolerance of false predictions. Figure 8 shows the results for a field of 100 emitters. We find that spatial tolerance significantly improves the performance of GeoCARB but still falls short of our success criterion. The FAR decreases below 0.2 for 3 km tolerance and below 0.1 for 5 km tolerance, but the POD only improves to 0.7 and thus the ETS remains below 0.65.

We saw in Sect. 3.1 that only the next-generation satellite instrument can successfully detect high-mode emitters when the site density is high. Here we examine if a combination of satellite and surface observations can improve detection, i.e., if TROPOMI and GeoCARB could benefit from an in situ supporting surface network and vice versa. This is addressed with a joint inversion of the satellite and surface observations, taking into account the error correlation between the two as described in Sect. 2.4.

Figure 9 shows the results for a field of 100 emitters. The already successful next-generation instrument shows no benefit from added surface sites, and the uncertainty increases slightly with the number surface sites. This increase is due to imperfect accounting of correlated error between satellite and surface measurements. On the other hand, the surface sites provide greatly added value to TROPOMI and GeoCARB. Adding 10–20 surface sites enables near-successful detection of the high-mode emitters. At the same time, TROPOMI and GeoCARB data add significantly to the performance of a surface observing system alone by providing observations with more spatial coverage. We find that TROPOMI and GeoCARB perform similarly when added to surface sites, and that their main benefit is to decrease the FAR. Accounting for clouds would show more benefit for GeoCARB because the finer pixels allow for more frequent clear-sky observations (Sheng et al., 2018a).