The broad range of cloud sizes in the atmosphere poses a significant challenge to the modeling of weather and climate. Small clouds tend to be most numerous, while large clouds have more significant meteorological and climate impacts. An approximate balance means that all size classes contribute to overall cloud cover , total rainfall and the dissipation of buoyant potential energy . The commonly used “divide-and-conquer” approach to the problem isolates a particular spatial scale for study, such as mesoscale convective systems larger than ∼ 100 km , shallow clouds in the trades between 20 and 200 km or sub-kilometer cumulus . While this approach has practical benefits, it cannot easily be used to address how clouds of all scales interact.

Revealingly, independent of the spatial scale or cloud type considered, the measured horizontal dimensions of clouds tend to follow power law distributions such that the number of clouds is proportional to their size to some power . Quantities that follow power law distributions are often described as being “scale-free” or “scale-invariant”, meaning that there is no “characteristic” object scale as there would be in defining, e.g., an exponential or Gaussian distribution. Power law behavior is in fact quite general among physical, social and biological systems, applying to, e.g., meteor diameters, neuronal firing, personal incomes, city populations and forest sizes .

Quantities that exhibit scale-free behaviors, however deterministically complicated they may be, allow for an important mathematical simplification. That is, phenomena measured at any one scale shed light on the behavior at others. They also present a practical challenge, which is the unavoidable limitation that geometrically defined objects must inevitably be measured within a domain of some finite size, i.e., a domain that is not scale-free. The domain enforces a maximum scale for object measurement – the size of the domain – and this may not reflect the maximum scale that the objects can attain.

For cloud areas a, we recently argued that improper consideration of the scale of the measurement domain has contributed to wide discrepancies in the reported nature of cloud area distributions and in particular to the upper bound amax⁡ to which a power law can be claimed to apply . Because clouds cannot be arbitrarily small, there must also be a lower bound amin⁡ to the power law regime, one that has not yet been determined but that could approach the Kolmogorov microscale of ∼ 1 mm, below which turbulent circulations are damped by viscous forces. Between amin⁡ and amax⁡, the power law regime can be represented by the probability distribution 1n(a)∝a-α+1;amin⁡<a<amax⁡, where α is a constant, indicating that n(a) is linear on doubly logarithmic axes. The upper bound at amax⁡ represents a “scale break” beyond which studies generally find that the distribution is “cut off” by a regime following either a steeper power law with a larger value of α or an exponential .

Estimates of the location of the scale break at amax⁡ differ widely. This uncertainty has underappreciated implications for studies of the role of clouds in climate because the integral ∫amin⁡amax⁡an(a)da, which is the total cloud amount, is sensitive to the scale break location. For example, a cutoff regime at areas of order ∼ 10 km2, as suggested by some , would imply that clouds larger than ∼ 4000 km2 would be so rare that they would contribute negligibly to the total, while other findings suggest that such large clouds contribute approximately 50 % to the global cloud cover .

The power law exponent α for cloud areas is also highly uncertain, with similar implications for the relative roles of different cloud types. The exponent determines the relative numbers of small and large clouds. Values close to unity e.g., imply that clouds of all orders of magnitude contribute equally to the total cloud cover, in which case small clouds that are often left unresolved by models and measurements may be an important omission. Conversely, values less than unity e.g., indicate that large clouds dominate the total area and so remain a reasonable subject for more focused study.

The lack of consensus among studies on the value of α may be due to differences in the dominant cloud type that was considered or how diurnal variability affects amax⁡ . However, even if temporal and spatial variability of the size distribution exists, there remains a necessary prerequisite to measuring such variability, which is to first ensure that the size distribution is being accurately measured in the first place. To this end, recently argued that, while size distributions do show some variability, the use of inaccurate statistical methods to fit power law distributions could also partially explain the lack of consensus among prior measurements of cloud sizes. In particular, they showed that the common method of fitting a least-squares linear regression to a logarithmically transformed histogram of cloud areas can lead to biased measurements of α.

Here we argue that the choice of fitting method is less important than whether past studies properly accounted for the finite size of the study domain. A finite domain size is a general problem for measuring scale-free quantities. For example, argued that scaling properties of networks can be obscured by such finite-size effects, causing a truly scale-free network to appear non-scaling.

Similarly, cloud sizes must necessarily be measured within a non-scaling finite domain. It is easy to appreciate that the area of clouds larger than the domain size cannot be measured. A more subtle effect is that the measured numbers of clouds of a given area, even those smaller than the domain area, are highly sensitive to whether clouds that cross the domain edge are included or removed in the measured distribution (an example is shown in Fig. ). We term such clouds “truncated clouds” as they appear effectively truncated by the domain edge, with only the portion of the cloud lying within the domain available for measurement.

Whether truncated clouds are included or removed from distribution fits is an issue rarely mentioned in past studies, but those that do consider the effect tend to remove truncated clouds without applying any correction factor e.g.,. One exception is a study of one-dimensional cloud chords by , who found that the removal of chords truncated by the domain edge leads to an undercounting of large chords relative to what would be measured in a larger domain. For cloud areas, it may be hypothesized that a similar effect could explain the observed differences in measurements of amax⁡.

In this study, Sect.  reconsiders the hypothesis proposed by that discrepancies in distribution parameters can be largely explained by improper methods used to fit a power law distribution to measurements of cloud sizes. Section  then examines how the choice of either including or removing clouds that are truncated by the domain edge can change the measured cloud size distributions. We suggest that such a methodology may bias measured distribution parameters, and we offer recommendations for future studies measuring any object size distribution within a finite domain, for both clouds and any other geometrically defined objects.

The most straightforward method to fit a power law to empirical measurements of cloud areas is to bin the data into discrete bins of constant width δa, resulting in a discrete set of counts ni for each bin i. The logarithm of Eq. () is the linear equation log⁡ni=-(α+1)log⁡a+const., so a line can be fit to log⁡ni vs. log⁡a to estimate α using a least-squares linear regression.

showed that such a linear-regression-based estimate can be biased by up to 36 % relative to the known value in computer-generated power-law-distributed data. “Logarithmic binning,” with bins of exponentially increasing width or constant δlog⁡a, increases bin counts ni at large a. Increasing ni reduces the “statistical error”, which is the standard deviation of ni evaluated over many hypothetical realizations of a given experiment. This reduction in the statistical error enables more accurate estimates of α . In the case of logarithmic binning, the calculated slope of a histogram is -α rather than -(α+1) because the number of clouds in a given bin is the bin width, which is proportional to a, times the distribution in that region, which is a-(α+1) (mathematically, dn/dln⁡a=adn/da).

However, even with logarithmic binning, linear regression has been found to produce a biased estimate of α relative to a known value for empirical tests that use computer-generated power-law-distributed data . Nonetheless, linear regression – whether to linearly or logarithmically spaced bins – remains a commonly employed method in cloud studies e.g.,.

There are two other linear-regression-based approaches worth mentioning, i.e., cumulative distributions and rank-frequency plots, both of which approximate the integral ∫amin⁡an(a′)da′. Fitting a linear regression to such plots has been argued to be superior to fitting a linear regression to a histogram of counts . Such approaches work well for unbounded power law distributions with amax⁡→∞, but for a truncated power law distribution with finite amax⁡ the cumulative distribution is not linear, even with doubly logarithmic axes . The nonlinearity implies that a linear regression would be inappropriate for estimating α.

An alternative method of fitting a power law to data, maximum likelihood estimation, is argued on empirical grounds to be generally more accurate than linear-regression-based approaches . Maximum likelihood estimation employs the “likelihood function”, which estimates the probability of observing the measured data given many different possible power law distributions. The distribution that is the best fit is the one that maximizes the likelihood function. argued that some of the disagreement between prior measurements of cloud size distributions could be resolved through the use of maximum likelihood estimation rather than linear-regression-based approaches.

Evidence supporting the superiority of maximum likelihood estimation put forth by , and was obtained from numerical experiments using synthetic data generated from an unbounded power law (i.e., amax⁡→∞ in Eq. ). In this case, the likelihood function may be analytically maximized, resulting in a simple formula that can be used to estimate α. For a truncated power law with finite amax⁡, however, the likelihood function must instead be numerically maximized , introducing much more complexity and computational expense to the analysis , especially when compared to a least-squares linear regression.

In fact, because the truncation at amax⁡ removes the portion of the distribution at large a that contains the most statistical errors, it might be argued that linear-regression-based approaches are more accurate for power laws that are bounded, as they inevitably are for clouds. Indeed, found that, when linear regression was applied to only the five smallest linearly spaced bins, which effectively truncated the distribution at the upper limit of bin 5, the power law exponent was estimated accurately relative to the known value.

To evaluate the accuracy of the linear regression approach for fitting a power law with finite amax⁡, as is relevant for any physical dataset, we randomly sample values for a from a synthetic truncated power law distribution (Eq. ) with parameters α=1, amin⁡=10 and amax⁡=1000, which are close to what might be measured for cloud sizes. This is accomplished by first drawing N values ai from an unbounded power law (amax⁡→∞) using the Python package powerlaw and then removing and re-drawing values larger than amax⁡ until all N values lay within (amin⁡,amax⁡). This process is repeated until 200 “samples” were created, each with N=1000, 3000 or 10 000 values. Samples are then binned into 30, 100 or 300 logarithmically spaced bins, and a “minimum bin count” threshold is applied, which removes any bin with a count lower than a range of specified thresholds between 0 and 50. A fit to each sample is then performed only if the remaining bins span at least 1 order of magnitude in a. This requirement is necessary for any fitting method because power law distributions fundamentally describe systems spanning many scales but is less stringent than the span of 2 orders of magnitude recommended by . Because the fitting accuracy is increased for datasets spanning a larger range of values, the requirement of 1 order of magnitude used here represents a conservative threshold for the purpose of evaluating fitting methods using a known power law distribution. If power law behavior itself is in question, a larger span is required.

Estimated values of the power law exponent, denoted as α^ to avoid confusion with the specified value α, are determined by fitting a least-squares linear regression to the bins satisfying the above criteria. Statistical uncertainty ε associated with fitted values α^ is estimated using the Python package scipy as 2 standard errors on the linear regression, corresponding to a 95 % confidence interval. For each combination of sample size, number of bins and minimum bin threshold, 200 samples are generated. A “failure rate” is calculated as the fraction of estimates that do not include the true value α=1 within their 95 % uncertainty range: 2failure rate≡count of α^∉α-ε,α+ε200. We define an “accurate” estimation method as one whose failure rate is less than 5 %. Selected tabular results are listed in Appendix .

As shown in Fig. , if the minimum bin count threshold is less than 24, α cannot be accurately estimated using a linear regression technique, in agreement with what was argued by , and . However, regardless of sample size or the number of bins, if the regression is only applied to bins with counts of at least 24, estimates of α^ determined from linear regression lay outside of uncertainty bounds less than 5 % of the time, which is consistent with a 95 % confidence threshold. In this sense, they are accurate.

Applying a simple rule that least-squares linear regression only be applied to those bins with sufficiently large counts may seem obvious: estimating any statistical measure using a very small sample tends to result in error. In this particular case of statistically independent measurements, the low failure rates for high minimum bin count thresholds can be understood in terms of the central limit theorem, where the successive measurement and binning of power-law-distributed variables can be interpreted as a counting process (see Appendix ). Provided bin counts ni exceed approximately 24, statistical error in both counts and the logarithm of the counts follows a Gaussian distribution (Fig. ). In this case, general-purpose linear regression packages that assume Gaussian error at each point may be used to accurately estimate the exponent of a power law distribution.

In summary, whether binning is done linearly or logarithmically, there may be bias in previously calculated values of α for cloud area distributions that are power-law-distributed, but only if bins with fewer than 24 counts are included in the regression. A very simple fix is to omit such bins. Other studies that estimated α over a range of scales that exclusively included large bin counts e.g., may have obtained estimates of α that were as reliable as maximum-likelihood-derived estimates. In fact, estimated α for shallow cumulus using both a linear regression to logarithmically spaced bins and maximum likelihood estimation, finding that both fitting methods produced similar results.

state that, while a histogram that is linear when logarithmically transformed is not sufficient to identify a power law distribution, linearity is certainly necessary. The challenge with cloud sizes is that some portions of the cloud size distribution appear linear in some studies but are clearly nonlinear in others. For example, , and all find a scale break at ∼ 1 km2, where a power law regime transitions to an exponential or a different power law with a much larger value of α. Either case indicates a clear nonlinear portion of the doubly logarithmic histogram at or beyond the scale break. This is in disagreement with other studies that find linear power law scaling up to ∼ 10 km2 or ∼ 100 km2 and especially with the findings of power law scaling extending beyond 105 km2 . Differences in the choice of fitting method used, whether maximum likelihood estimation or regressions to linearly or logarithmically spaced bins, cannot explain these differences in measured amax⁡. While differences in meteorological conditions may contribute, meteorological influences may still be obscured by methodological problems. We next explore how the improper treatment of clouds that are truncated by the edge of the measurement domain could influence measured size distributions.

Truncated clouds, which span the domain edge (Fig. ), present a conundrum. If one wants to accurately measure the size distribution within a finite domain, should one remove them from consideration, risking undercounting clouds in some size classes, or should the clouds be included, risking inaccurate area measurements? To investigate the magnitude of this truncation effect, we explore measured size distributions for various domain sizes.

There is significant disagreement in the literature on what the appropriate choice of distribution should be to describe cloud horizontal areas. Most studies find that cloud areas follow a power law n(a)∝a-(α+1), although there is considerable disagreement on the range of scales over which the power law applies and the value of α. A recent study proposed that, while differences in local climatological characteristics contribute to variability, some of the disagreement is due to the use of inferior linear-regression-based fitting methods, arguing that maximum-likelihood-based methods are superior .

The present study shows that the choice of fitting method cannot explain the disagreement among observations, particularly for the range of scales over which a power law applies. We find that a linear regression to logarithmically spaced bins is an equally accurate fitting method for power-law-distributed data provided the simple requirement is adopted that bins with fewer than ∼ 24 counts are omitted from the regression. Linear regression also has the advantage of being computationally trivial and more conceptually straightforward than maximum-likelihood-based alternatives.

We suggest that different accounts of cloud power law behavior in the literature are best explained by treatments of clouds whose geometries are “truncated” by the edge of the measurement domain. Removal of truncated clouds from the distribution introduces an artificial “cutoff scale” beyond which clouds can be significantly undersampled, with a resulting distribution consistent with many previous findings e.g.,. If included, a local maximum in the distribution appears at areas comparable to the domain scale that does not reflect the true distribution. Even when a periodic domain is used, measured size distributions do not reproduce the size distributions that would be obtained in larger domains. In any case, a power law may still easily be measured, but the value of the power law exponent could be underestimated or overestimated by 20 % to 30 % or more.

While size distributions measured within any domain size are affected by truncation effects, they are most important only for the largest clouds. The affected scale is easily identified by counting for each bin the fraction of clouds that are truncated relative to the total in that bin. We recommend that power law fits be applied only to bins in which the fraction of these clouds is less than 50 %.

Truncation effects are not limited to power law size distributions, as exponentially distributed objects can be similarly affected. Fortunately, the 50 % truncated object criterion is applicable regardless of the underlying form of the distribution.

The issues and remedies discussed here are not specific to atmospheric clouds and can be applied to size distributions characterizing any other phenomena measured within a finite geometric domain, e.g., with ecological predator–prey models , CO2 pockets in sedimentary rocks , snowflakes , cloud droplets , aerosols and soil particles .