The theory of eddy covariance is well established (e.g., Baldocchi, 2008). Average surface fluxes of sensible heat, latent heat and CO2 may be estimated over a large upwind area (Kljun et al., 2004; Kormann and Meixner, 2001) using fast response instruments by QSENS=cpρdw′θv′‾QLAT=Lhcdsw′rq′‾FCO2=cdsw′rc′‾, where QSENS is the sensible heat flux, cp is the specific heat of dry air, ρd is the mass density of dry air, e.g., w=w‾+w′ is the Reynolds decomposition of vertical wind speed into its average w‾ and turbulent w′ components, θv is potential virtual temperature, QLAT is the latent heat flux, Lh is the latent heat of vaporization, cds is the molar concentration of dry air moldrym-3, FCO2 is the CO2 flux and rq=molqmoldry-1 and rc=molcmoldry-1 are the dry mixing ratio of humidity and CO2 concentration scalars, respectively. The terms w′θv′‾, w′rq′‾ and w′rc′‾ are the covariance between turbulent fluctuations of vertical wind and turbulent fluctuations of potential virtual temperature and dry mixing ratio of humidity and CO2 concentration, respectively. For Eq. (1) to truly represent the vertical fluxes a number of assumptions should be met during field operation. Principal among these are stationarity of the observation ∂ξ∂t=0, horizontal homogeneity ∂ξ∂x1=0&∂ξ∂x2=0, mass conservation w‾=0&∂ςj∂xj=0, negligible density flux ρ′ρ≪1 and a vertical constant flux layer (e.g., dFCO2dz=0) (Foken and Wichura, 1996). Here, ρ is the air density, xj represents the three axes of observed flow, ς=u,v,w is the wind vectors, s=θv,rq,rc is the scalars of interest and ξ=ς,s is the latter two combined.

Flux estimates Eq. (1) may be decomposed into frequency-dependent contributions, called co-spectra Cowsf, between vertical wind-velocity w and the scalars of interest s, for frequencies f. Deviations of observed co-spectra from theoretical co-spectra (Kaimal et al., 1972; Moore, 1986) can be linked to a number of issues including influence of the eddy covariance system on the flow, oscillations of the tower (or ship), topographical forcing on the flow, etc., and is often used to filter out observations characterized by excessive non-turbulent influence (Novick et al., 2014).

Subsequently we may perform an ogive analysis (Desjardins et al., 1989; Foken et al., 2006; Lee et al., 2004). The analysis requires the same basic assumptions and involves the cumulative summation of co-spectral energy, starting from the highest frequencies, Ogwsf0=∫∞f0Cowsfdf. The principal use of ogives is to estimate the optimal flux averaging time as the point of convergence of cumulative co-spectral energy to an asymptote (Berger et al., 2001; Foken et al., 2006). However, low-frequency influences may result in ogives which instead either converge to an extremum or diverge, depending on the direction of low-frequency fluxes. Such conditions may arise in the absence of a distinct spectral gap during significant overlap of high-frequency (turbulent) and low-frequency flux contributions. Depending on the severity of the deviation from asymptotic behaviour, an optimal averaging time can be impossible to determine. Such cases are conventionally considered to be in-stationary and no flux estimation is possible.

Figure 1 illustrates a number of observational situations showing examples of how low-frequency influence could affect our ability to capture local fluxes. In the figure, situations are shown using both co-spectral and ogive plots.

In the ideal case (Fig. 1a), turbulent and low-frequency flux contributions are separated by a spectral gap, allowing investigators to isolate the former simply by choosing an appropriate flux averaging time T1 and using fast response instruments recording at frequency T2. Accordingly, the corresponding ogive distribution is seen to converge to a stable flux estimate within T1 (Fig. 1b). Both turbulent fluxes and the low-frequency contribution, shown in Fig. 1b as a blue region of ogive divergence relative to asymptotic convergence, may be positive or negative, though the former has been illustrated as positive here.

Given the unclear existence of a spectral gap (Lee et al., 2004), however, another more general situation is the case (Fig. 1c) of overlapping contributions from low-frequency motions, turbulence and site and instrument-specific non-white noise/dampening. One way to strike a balance between adequate inclusion of the turbulent contribution and exclusion of excessive low-frequency influence is by adjusting T1. Typically a fixed averaging time is set for an entire experiment (here 30 min is shown) and the flux errors are assumed, or tested (e.g., Novick et al., 2014), to be negligible. In the high-frequency end of the spectrum, instrument response limitations may prevent observation of the smallest scales of turbulence contribution (Here 10 Hz is shown). Furthermore, instrument-specific non-white noise and/or dampening may at times be reduced by application of site-specific co-spectral corrections, called transfer functions (Aubinet et al., 2000; De Ligne et al., 2010; Massman and Ibrom, 2008; Kaimal, 1968; Moncrieff et al., 1997; Moore, 1986; Silverman, 1968). Accordingly the ogive distribution indicates negligible influence on the flux estimates for a 30 min averaging time and an instrument response time of 20 Hz (Fig. 1d). Note that the influence of dampening and non-white noise on the ogive distribution occurs in the reverse (Fig. 1d) relative to co-spectral space (Fig. 1c).

Observations reflecting excessive low-frequency influence, relative to the turbulent contribution, (Fig. 1e) are typically discarded. This is because strong relative low-frequency influence results in non-negligible flux contribution to the overall estimate and further obstructs any efforts to separate contributions by adjusting the flux averaging time (Fig. 1f). The use of the term relative in this context refers to the fact that an identical problem can arise despite modest low-frequency influence when estimating fluxes in a low-flux environment. Flux estimation in such environments are often further complicated by a high ratio of co-spectral variance to actual turbulent flux contribution. This prohibits unambiguous evaluation of similarity between observed co-spectra and theoretical co-spectrum distributions, as well as proper estimation of the co-spectral peak (Sorensen and Larsen, 2010).

In order to fulfil the stationarity requirement described in Sect. 2.1, continuous observations are typically subdivided into averaging intervals. Averaging interval time T1 has conventionally been assumed constant, based on the requirements that T1 should be long enough to reduce random error (Berger et al., 2001; Lenschow and Stankov, 1986) and short enough to avoid low-frequency influence associated with non-stationarity (Foken and Wichura, 1996; Vickers and Mahrt, 1997). However, as noted, adjustment of T1 will not generally allow for separation of turbulent and low-frequency flux contributions. Here, we propose a method that makes no assumption regarding the presence of a spectral gap. Instead we require averaging time to be as long as necessary while ensuring stationarity of the local processes, irrespective of the temporal evolution of low-frequency contributions.

The following is an iterative scheme for developing averaging intervals based on basic data quality requirements. Data collected during a field-experiment is considered continuous with end points Tmin and Tmax, despite the presence of gaps. A range of possible subsets are preliminarily determined based on a linear series of interval midpoints Tmi within the range Tmin+T12 to Tmax-T12 in intervals of T1, and data set lengths Twi within the range 10 to 60 min in 5 min intervals. Here T1 is set according to desired temporal resolution of flux estimates. For this study we use site-specific settings of T1=5min, T1=15min and the conventional T1=30min to strike a balance between desired temporal resolution and computational cost of running the ogive optimization method. The minimum data set length is chosen to be 10 min for the ogive function to yield statistically representative estimates of the scales of turbulence-driven fluxes and the maximum data set length is chosen to be 60 min to ensure approximate stationarity of the local turbulence-driven fluxes.

Using an iterative bisectional algorithm for enhanced computational speed, combinations of Tmi and Twj are evaluated with regard to a number of basic quality assessment criteria to obtain the longest data set around Twj, which passes the assessment criteria. These include absence of instrument diagnostics errors, absence of long data gaps, favourable mean wind direction and reasonably narrow range of wind directions (≤60∘). Minor spiking (≤1%) is corrected based on the median of surrounding data points, and the data set is discarded otherwise (for spiking > 1%) according to Vickers and Mahrt (1997). Other quality assessment includes the requirement that momentum fluxes should be negative. Evidence to the contrary would imply a disconnection between the upwind surface processes and the point of observation on the tower, a condition typically associated with low wind conditions. However, momentum fluxes may also be affected by low-frequency contributions. As direction, but not strength, of the turbulent momentum flux is relevant, we may simplify by calculating an ogive distribution OgwU based on the momentum flux co-spectrum CowU, where U is the horizontal along-wind component, and simply verify that OgwU<0 at the mid-range natural frequency of fm=0.1Hz. The choice of fm reflects a spectral region least impacted by instrument-specific non-white noise, dampening and low-frequency influences. Typically around four estimates of Twj are evaluated by the iterative bisectional algorithm for each Tmi before the optimal Twj is determined.

Finally, signals with very rapid evolutions such as transient signals in dynamic systems like eddy covariance observations may undergo abrupt changes associated with observational interference e.g., electrical interference or instrument error. These are referred to as dropouts and discontinuities in Vickers and Mahrt (1997). Global transforms, like the Fourier transform, are usually not able to detect these events. In contrast, Wavelet transforms such as the Haar transform, permit a localized evolutionary spectral study of signals, thus allowing for detection of subtle signal discontinuities leading to semi-permanent changes (Lee et al., 2004; Mahrt, 1991; Vickers and Mahrt, 1997). In this study we perform the Haar analysis for the data set Twj, given by the bisectional algorithm, and observations are sorted in three categories (good, soft flag, hard flag) according to the presence and severity of signal discontinuities (Vickers and Mahrt, 1997). Conventionally these events are thought to preclude flux estimation on the basis of stationarity violations. Accordingly we discard such data sets (hard flags) when applying the conventional eddy covariance method. However, as will be shown, we find in this study that the ogive optimization method allows for convincing flux estimation in many cases of soft and hard flags. Therefore no flux estimates derived using the ogive optimization are discarded, unless visually inspected.

To evaluate the ogive optimization method, five sites reflecting different environments in terms of ecosystem, topography and flux strengths (QSENS, QLAT and FCO2) are investigated (Fig. 4).

During post-processing, a number of instrument-specific corrections are needed to adjust for instrument-bias. For the sonic anemometers (Gill R2, Gill R3, Gill Windmaster Pro and METEK USA-1) these include the following: an empirical angle of attack correction (Nakai and Shimoyama, 2012), and humidity and crosswind corrections (Liu et al., 2001; Schotanus et al., 1983). We convert all observations to mixing ratios (Burba et al., 2012) using the Webb–Pearman–Leuning correction when necessary (Sahlee et al., 2008; Webb et al., 1980) as recommended by Ibrom et al. (2007). The need for instrument heating corrections (Burba et al., 2008) associated with operation of the open path LI-7500 in a cold environment (Daneborg, POLYI and ICEI) is alleviated by using the newer LI-7500A with a “cold” setting correcting observations down to -25∘ (Burba et al., 2011). Sites featuring the LI-7500 (Abisko and RIMI) never reached sufficiently cold temperatures to warrant instrument heating corrections during this experiment. Coordinate rotation, linear de-trending and iterative de-spiking of raw data is performed according to Vickers and Mahrt (1997). Temporal offset between sensor signals is typically corrected based on a maximum cross-covariance analysis (Berger et al., 2001; Fan et al., 1990). In the limit of low absolute covariance, however, actual temporal offset may be obscured by secondary cross-covariance optima. Here, we locate the optimal cross-correlation automatically based on the incident horizontal wind flow and the specific geometry of each covariance system.

In the following, we describe several typical cases observed and the associated performance of the ogive optimization method.

Near-absence of low-frequency influence is observed leading to a strong similarity between the ogive density pattern, the 30 min linear detrended ogive and the modelled ogive. This is illustrated in Fig. 5a for a case of sensible heat flux at the Abisko site. Disregarding the high-frequency component associated with extrapolation of model results, seen here to contribute FHF≈-5Wm-2 to the overall modelled sensible heat flux (Eq. 4), the standard 30 min linear detrending approach will suffice to provide the turbulent flux estimate. The observational period in question was characterized by neutral atmospheric stability, high wind speed U‾=8.25ms-1 and winds originating from the fen area (Fig. 5a), along with fairly constant temperature 8.2 ± 0.4∘ (Fig. 5b) and a slight gradual increase in uptake (Fig. 5c).

3.

An example of ambivalence caused by bimodality in the ogive density pattern is illustrated in Fig. 7, for a case of sensible heat flux at the Abisko site. Such cases indicate that fluxes are changing within the sampling period. The ogive optimization method is seen to capture the turbulent flux contribution with the strongest data density. Had both modes been of equal ogive density, the choice of mode during subjective evaluation would be based on the quality of the model ogive optimization, and the length of the time-series responsible for the modes. If both ogive models were equally good, the choice would fall on the mode produced by the ogives which consist of shorter time-series as they represent a more instantaneous flux estimate relative to the mode produced by longer time-series. The sampling period in question was characterized by a slightly stable atmosphere zL-1=0.12, low wind speed U‾=2ms-1 and wind originating from the fen area (Fig. 7a), along with a steady decline in atmospheric temperature (Fig. 7b) and a strong variation in flux covariance (Fig. 7c).

4.

The inadequacy of applying a fixed averaging interval for flux estimation becomes apparent in Fig. 8, for a case of sensible heat flux at the Daneborg site. Here, the ogive density pattern is seen to reflect a gradual evolution in the ogive flux pattern with increasing averaging time. The standard 30 min averaging time is seen to be too long and also to increasingly reflect low-frequency interference (Fig. 8a). This is consistent with an abrupt increase in atmospheric temperature (Fig. 8b) and decrease in covariance (Fig. 8c) around 30–40 min. The ogive optimization method identifies the appropriate flux estimate (Fig. 8a), whereas the standard 30 min linear detrending method fails on account of in-stationarity. In addition, the case is a perfect example of how co-spectral evaluation of frequency-wise contributions can be misleading (Fig. 8a, inner plot). The observational period in question was characterized by a stable atmosphere zL-1=0.29, low wind speed U‾=2.6ms-1 and wind originating from the fen area (Fig. 8a).

5.

The inadequacy of applying a fixed averaging interval for flux estimation becomes apparent in Fig. 8, for a case of sensible heat flux at the Daneborg site. Here, the ogive density pattern is seen to reflect a gradual evolution in the ogive flux pattern with increasing averaging time. The standard 30 min averaging time is seen to be too long and also to increasingly reflect low-frequency interference (Fig. 8a). This is consistent with an abrupt increase in atmospheric temperature (Fig. 8b) and decrease in covariance (Fig. 8c) around 30–40 min. The ogive optimization method identifies the appropriate flux estimate (Fig. 8a), whereas the standard 30 min linear detrending method fails on account of in-stationarity. In addition, the case is a perfect example of how co-spectral evaluation of frequency-wise contributions can be misleading (Fig. 8a, inner plot). The observational period in question was characterized by a stable atmosphere zL-1=0.29, low wind speed U‾=2.6ms-1 and wind originating from the fen area (Fig. 8a).

7.

During conditions of strong high-frequency dampening caused by the use of a closed path instrument, the ogive optimization method automatically shifts the high-frequency bound on optimization towards lower frequencies to avoid influence of the dampened frequencies during optimization. This is illustrated in Fig. 10 for a case of latent heat flux at the ICEI site. Here the upper optimization bound is shifted back to 1Hz thus allowing for an accurate description of the high-frequency fluxes as well (Fig. 10a, inner plot). The observational period in question was characterized by a slightly unstable atmosphere zL-1=-0.08 and moderate wind speeds U‾=5ms-1 (Fig. 10a), along with gradual increases in both atmospheric H2O content and atmospheric temperature (Fig. 10b) and a gradual increase in flux covariance (Fig. 10c).

The difference in flux estimates of the standard 30 min linear detrending approach and the ogive optimization method is associated with both the inclusion/exclusion of low-frequency contributions, the inadequacy of the fixed averaging interval and the extrapolation of modelled ogives into un-observable high/low frequencies. The relative flux difference δ is evaluated within i=10 intervals of absolute flux estimates F30mini as the standard deviation of difference in flux estimate relative to the mean absolute ogive optimization flux estimate within respective intervals: δi=stdF30mini-FO2iF30mini‾×100%, where square brackets []i signify flux estimates native to interval i of the equivalent absolute flux estimates by the standard eddy covariance method F30mini. Estimates of relative flux difference δ are shown logarithmically in Fig. 11 for all three scalar flux types, at all five observation sites and for all 10 intervals of the respective ogive optimization flux ranges. Outliers have been excluded from the flux ranges shown in the bottom of the figure to ensure a minimum of three flux estimates within the largest absolute flux-estimate bin and resolution of the resulting δ estimates have been doubled by spline interpolation. The median relative difference is shown (red line) along with standard deviation (light gray area) and 25–75 % percentile range (dark gray area).

As hypothesized in Sect. 2.2, the average relative flux difference is seen to be very high for small absolute flux estimates, peaking at δ‾SENS=80%, δ‾LAT=23% and δ‾CO2=98% for the lowest absolute flux estimates. The variation in δ is quite high for low absolute flux estimates, with the 13.6 and 86.4 percentiles of δ reaching as much as δSENS=40-208%, δLAT=10-98% and δCO2=52-538%. For larger absolute flux estimates (QSENS> 40Wm-2, QLAT> 20Wm-2 and FCO2> 100mmolm-2d-1) the relative difference is seen for all three flux types to drop and level off to a near-stable range of δ=5–20 %. These absolute flux thresholds thus mark clear shifts between non-negligible low-frequency contributions on one side and plausibly negligible low-frequency contributions on the other.

Depending on perspective and the character of observed fluxes at a particular site the described thresholds may either serve as an indicator of a lower limit to local-scale flux resolvability by the standard 30 min linear detrending approach, or as an argument for the application of enhanced flux estimation techniques such as the presented method. For the presented observations the consequences are illustrated by the histograms of the different sites (Fig. 11). Although the location of the flux threshold is a bit unclear for latent heat flux, estimation of locally meaningful fluxes at the three sea-ice sites Daneborg, POLYI and ICEI is essentially impossible without accounting for low-frequency contributions. The same applies for sensible heat flux at the Abisko lake site, latent heat flux at the grassland site RIMI and CO2 flux at both the Abisko lake site and the RIMI site. Note that only the Abisko Fen environment showed a dynamic range in excess of FCO2=400mmolm-2d-1 and that most flux estimates from RIMI are from the morning or late evening/night, which explains the range of relatively small fluxes.

The relative flux difference was furthermore investigated in terms of atmospheric stability (Fig. 12). Though variation in δ is significant for all flux types, average δ appears to be lowest between slightly unstable zL-1≈-0.2 and neutral conditions (δ≈10–20 %). In contrast δ is significantly larger for zL-1> 0.220<δ<1000%. This is consistent with the current consensus on turbulence spectra (Kaimal et al., 1972; Olesen et al., 1984): for strongly unstable conditions zL-1<-0.2 all spectra have increased low frequency components (Hojstrup, 1982), which would have been filtered out using the ogive optimization method. For strongly stable conditions zL-1> 0.2 the turbulence spectral intensity is often small relative to low frequency variation associated with meso-scale variability (Larsen et al., 1980; Vickers and Mahrt, 2003). Exactly for neutral and slightly unstable conditions boundary layer turbulence structure is at its simplest being dominated by shear produced turbulence that is best described by the standard spectral expressions, being the background for the standard eddy-correlation flux determination methods (Kaimal et al., 1972; Olesen et al., 1984).

For many flux estimates the vertical wind speed signal or the scalar signal are non-stationary to the point of prohibiting a flux estimation using traditional methodology. Hence the ogive optimization method may also provide a greater number of flux estimates. This is shown in Table 1 to generally be true for the Abisko and Daneborg sites, both of which characterized by degraded signal quality at times. Sites RIMI and POLYI are inconclusive in this respect and the conventional method appears superior in the case of ICEI. The latter may be related to the very low fluxes observed for this site (Fig. 11) suggesting the presence of a detection limit for the ogive optimization method when using the particular instrument setup at ICEI (LI-7200 enclosed gas analyzer) within the respective ranges QSENS<25Wm-2, QLAT<3Wm-2 and FCO2<10mmolm-2d-1. No similar characteristics indicate the presence of a detection limit of the ogive optimization method at any of the other sites (open path gas analyzers), suggesting superiority of open path instruments in very low flux environments when using the ogive optimization method.

Low-frequency shifts in flux direction were found to be common in this study. To our knowledge such occurrences are not described by any existing theoretical framework, indicating a puzzling caveat to current theory. The occurrences challenge the notion that fluxes should be of same sign regardless of incident eddy scales. One explanation might be that vertical low-frequency contributions represent only one part of a net low-frequency contribution and hence is balanced by a horizontal component. Indeed the horizontal low-frequency component has been shown to be significant during certain conditions (Yi et al., 2008; Zeri et al., 2010), despite typically being assumed negligible. The finding indicates that further investigation of the interplay between low-frequency contributions, and their influence on turbulent flux estimates, is necessary.