Total and partial cloud amount detection during summer 2005 at Westerland ( Sylt , Germany )

The detection of cloudiness is investigated by means of partial and total cloud amount estimations from pyrgeometer radiation measurements and visible all-sky imager observations. The measurements have been performed in Westerland, a seaside resort on the North Sea island of Sylt, Germany, during summer 2005. An improvement to previous studies on this subject resulting in the first time partial cloud amounts (PCAs), defined as cloud amounts without high clouds calculated from longwave downward radiation (LDR) according to the APCADA algorithm (Dürr and Philipona, 2004), are validated against both human observations from the National Meteorological ServiveDWD at the nearby airport of Sylt and digital all-sky imaging. The aim is to establish the APCADA scheme at a coastal midlatitude site for longterm observations of cloud cover and to quantify errors resulting from the different methods of detecting cloudiness. Differences between the resulting total cloud amounts (TCAs), defined as cloud amount for all-cloud situations, derived from the camera images and from human observations are within±1 octa in 72% and within±2 octa in 85% of the cases. Compared to human observations, PCA measurements, according to APCADA, underestimate the observed cloud cover in 47% of all cases and the differences are within±1 octa in 60% and±2 octa in 74% of all cases. Since high cirrus clouds can not be derived from LDR, separate comparisons for all cases without high clouds have been performed showing an agreement within ±1(2) octa in 73(90)% for PCA and also for camera-derived TCA. For this coastal mid-latitude site under investigation, we find similar Correspondence to: A. Macke (amacke@ifm-geomar.de) though slightly smaller agreements to human observations as reported by D̈urr and Philipona (2004). Though limited to daytime, the cloud cover retrievals from the sky imager are not really affected by cirrus clouds and provide a more reliable cloud climatology for all-cloud conditions than APCADA.


Introduction
Surface based observations of cloud amount and cloud type are a valuable source of information for the interpretation of the surface radiation budget and for the validation of satellite based retrievals of cloud and radiation properties (Hollmann et al., 2006).To this end, automated systems are required to monitor and archive cloudy sky information with high accuracy and reliability.Several authors introduced various "allsky cameras" (Oznovitch et al., 1994;Shields et al., 1998;Long et al., 2001;Morris, 2004;Feister et al., 2000;Pfister et al., 2003;Beaubien and Bisberg, 1999;Kalisch and Macke, 2008) to estimate cloud amounts directly from digital full-sky imaging.Unless high-cost thermal camera systems are used, this method is restricted to daylight conditions.Furthermore, the interpretation of sky images, in terms of cloud cover, is subject to a number of systematical errors which will be described below.Other cloud detection methods are based on shortwave downward radiation measurements by means of pyranometer (Long and Ackermann, 2000), which also works for daytime only.Dürr and Philipona (2004) developed the Automatic Partial Cloud Amount Detection Algorithm (APCADA) for estimating the cloud amount without high clouds, which they denote as Partial Cloud Amount (PCA), directly from longwave downward radiation (LDR), air temperature and humidity for several Alpine Surface Radiation Budget (ASRB) and Baseline Surface Radiation Network (BSRN, Ohmura et al., 1998) stations.Thus, APCADA only requires a ventilated pyrgeometer for measuring LDR and standard meteorological observations.So far, APCADA is used for testing atmospheric profiling products (Ruffieux et al., 2006, Morland andMätzler, 2007), and for identifying cloud-free situations for climate research (Sutter et al., 2006).
During the summer of 2005, the Institute for Medical Climatology at the University of Kiel and the IFM-GEOMAR have undertaken a cloud and radiation measurement campaign on the North Sea island of Sylt focusing on the cloudinduced excess solar and UV radiation at the surface (Schade et al., 2007).Because of the availability of pyrgeometer data during this campaign, it was possible to also derive cloud cover estimates from the APCADA algorithm.
The aim of the present work is to establish the APCADA scheme for a coastal midlatitude site for longterm observations of cloud cover and to quantify the random and systematic errors of cloud-cover retrieval from full-sky imagers and from APCADA compared to those from human observations.As APCADA is not sensitive to high ice clouds, the comparison is divided into situations for all clouds, to show potential errors if APCADA is not applied to conditions it was designed for and to clarify if APCADA can nevertheless provide reliable results in an observational mode, and for situations for all clouds without high clouds.Calculations of the total cloud cover from all-sky images are described in detail by Schade et al. (2007) and Schade (2005).As APCADA requires an adjustment to local cloud-free LDR conditions, the method and its application to the Sylt observations is described in Sect.2.3.The quality of both cloud cover estimations in comparison to synoptical observations from the National Meteorological Service DWD at the airport, Sylt is presented and discussed in Sect.3, followed by a summary and conclusion.

Measurements
From April to August 2005 measurements of shortwave and longwave downward radiation, visible all-sky imager, and standard meteorological data have been performed at the seaside radiation measurement station of the Institute for Medical Climatology in the north of Westerland/Sylt, Germany.This site was chosen because of the clean air conditions with little continental aerosol that could affect the measurements, the occurance of broken clouds due to land-sea wind circulations and the seaside location, where people expose themselves to the sun (Schade et al., 2007).
In addition, hourly standard synoptical observations including cloud cover and cloud type are carried out by the National Meteorological Service DWD at the Sylt airport, about 1 km east of the seaside station, on a routinely basis.Given this small distance, it is assumed that cloud observations from the DWD station are representative of the conditions at the seaside station.The observations are taken as "truth", because automated systems still have to be programmed manually on complex pattern recognition up to this time and because of the fact that DWD observers are specially trained for synoptical observations.
In total, a dataset of 1605 cloud amounts for the different methods, is compared.All cases, where the cloud amounts by camera could not be calculated properly, were excluded from the analysis, i.e. at sunset and sundown, when the bluish stain of the images prevents correct calculations, and images contaminated with "blooming", when pixels are whitened by a surplus of charge delivered from its neighbors.

All-sky camera
In this study, the same prototype visible all-sky CCD-camera as described in Schade et al. (2007) is used.Therefore, only the general method is briefly described here.After labeling each sky pixel of the all-sky images as clear or cloudy by its red/blue ratio, the total cloud amount (TCA) is simply taken as the fraction of cloudy pixels.The threshold for labeling a pixel as cloudy was selected as a red/blue ratio of above 0.75, derived from manual inspection of several cloud images taken in summer 2004 for different situations.It was also selected to minimize misinterpretations without suppressing the detection of cloudy pixels too much.No cosine weighting of the pixels with respect to the viewing zenith angle has been applied, since cloud sides at the boundary of the images would be misinterpreted as cloudy pixels, which in turn would lead to an overestimation of the TCA, especially for skies with small cloud amounts (see Schade, 2005).Therefore, the larger portion of sky seen at lower elevation angles has not been taken into account.
The total cloud amount derived from the camera is set to 0 octa for TCA<0.016 and to 8 octa for TCA≥0.981.Both thresholds were chosen from a visual inspection of sky images that have been identified as clear and overcast by the observer, since the cloud-octa scheme gives 1 octa if only a very small cloud is observed and 7 octa when just a bit of blue sky can be seen.Further details for the image processing and interpretation can be found in Schade et al. (2007).

Radiation measurements
Measurements of longwave downward irradiation (LDR) at the surface have been performed with a ventilated pyrgeometer (CG4, Kipp & Zonen, Delft, The Netherlands).The output signal was recorded by a digital voltmeter (DMM Model 2000/2000 SCAN, Keithley Instruments Inc. Cleveland, Ohio, USA).A measurement frequency of 1.0 Hz was used.The LDR data has been averaged to 10 min, since APCADA is optimized for a 10 min resolution (Dürr and Philipona, 2004).
Exact coincidence of the radiation measurements and the all-sky camera images was achieved by synchronization of the computers, using radio-controlled clocks (DCF77 radioclock PCI511, Meinberg Funkuhren, Bad Pyrmont, Germany).Daily inspections of the pyrgeometer have been carried out to ensure that no sea salt, sand, or other contaminations might have affected the measurements.

APCADA
The use of APCADA requires an adjustment of certain algorithm parameters to clear sky conditions.Therefore, the following section provides a short description of the procedures that are required to apply APCADA.
With the exception of high and cold ice clouds, cloud bases have larger and stronger fluctuating LDR than the cloud-free atmosphere (Berk et al., 2000).On this basis, Dürr and Philipona ( 2004) have developed APCADA for estimating the partial cloud amount without high clouds (PCA).
The determination of PCA according to APCADA is based on two parameters.The cloud-free index (CFI) is based on the clear-sky index (CSI) by Marty and Philipona (2000), which is used primarily to find clear-sky situations in climate research, and calculated from LDR measurements as where σ is the Stephan-Boltzmann constant, T L the air temperature in Kelvin, AC the emissivity of a cloud-free sky, as described in Brutsaert (1975), AD a constant value of 0.23, and e the water vapor pressure in Pascal, which is calculated from measurements of relative humidity and the empirical equation for saturated water vapor pressure, valid for temperatures from −30 • C to 50 • C (see Buck, 1981).k(t) and k(t) are time-dependent functions describing the diurnal course of the cloud-free sky emissivity.Therefore, both functions have to be fitted to observations of clear skies for day-and night-time separately, in order to find the maximum and minimum values, i.e. to describe the amplitude of the diurnal course.The day-fit should be performed three hours after local noon when the temperature reaches maximum values, i.e. at 16:30 local time (14:30 UTC) for the position of Sylt.For longterm measurements, the algorithm adjustment has to further distinguish between summer and winter conditions.More detailed information on the fitting process can be found in Dürr and Philipona (2004).
The LDR variability during the last hour, described by the standard deviation (STD LDR), is required to identify cloudiness by means of an increased variability compared to cloudfree (low CFI) and overcast (high CFI) situations.Figure 1 shows STD LDR and CFI as a function of cloud cover from the synoptical observations.The region of the upper and lower quartile in the data is indicated by the box.The extent of the rest of the data is given by the error bar.It can be seen that the CFI is increasing with rising cloud cover, and that 0 octa and 8 octa skies are clearly distinguished by the CFI.This is important because STD LDR is similar for both cases and cannot be used as additional information.The CFI error bars in the diagram further show that neighbored octa values can often not be distinguished from the CFI data alone.Also it can bee seen in the diagram that STD LDR error bars for 6-8 octa extent equally which in turn should lead to errors in the classification of 6-8 octa skies as will be shown below.
Three thresholds, 1+az, 1+bz and 1+cz with z= 1 AC −1, a=0.12, b=0.21, and c= 0.38, are defined to separate CFI into different sectors.Dürr and Philipona (2004) derived the factors a, b, and c at a radiation measurement station at Payerne, Switzerland, and found these factors to be appropriate for other Swiss stations, for Ny Ålesund, Spitzbergen (78.93 • N, 11.95 • E), for the Marshall Islands (8.72 • N, 167.73 • E) and several other as well.Therefore, we assume that these factors will also be appropriate for the position of Sylt.
The final APCADA scheme, according to Dürr and Philipona ( 2004) for estimating the partial cloud amount (PCA) in "octa", is shown in Table 1.

Results
To define the quality of the automated camera-based retrievals of cloud amounts (TCA) and of APCADA (PCA), both results are compared to synoptical observations.Since these observations were made on the hour, all TCAs and PCAs are taken as close as possible to the observation time.
In total, a dataset of 1605 TCAs, PCAs and observations is investigated.Although it is obvious that APCADA will always underestimate TCA in the presence of cirrus clouds, we still include APCADA results in the TCA comparison in order to quantify the resulting APCADA bias for observations where no additional information on the presence of cirrus clouds is available.
For the comparison, we choose the same Score-Index as defined by Dürr and Philipona (2004) where n (±1(2) octa) are cases with a maximum difference of 1(2) octa between TCA (PCA) and observations and n the number of cases.

Total cloud amount comparison
Figure 2 shows the frequency distribution of the differences in total cloud amount derived from the full-sky camera and the synoptical observations (upper diagram), and from APCADA and the synoptical observations (lower diagram).TCAs from the camera do not differ systematically from the observations.The deviations are nearly symmetrically distributed around zero towards larger and smaller cloud amounts.The mean difference (overall bias) is merely −0.01 octa, the mean cloud amount for the observations is 5.21 octa, for the camera's TCA 5.20 octa.Because of the insensitivity to the presence of ice clouds, APCADA underestimates the observed cloud cover (46.98% of all cases) by an overall bias of 1.01 octa, which can be directly seen from the asymmetry of the frequency distribution.The mean cloud amount from APCADA is 4.2 octa.APCADA shows much smaller overestimations (with respect to the synoptical observations) than the camera.In the camera data, clear sky regions near the sun's location often appear as cloudy due to intense aerosol scattering.
Within a tolerance of ±1(2) octa, the camera TCAs reflect the observations in 58 (72)% of all cases, the APCADA based cloud cover within 60 (74)% of all cases.
The frequency of detected cloud cover from all three data sets is shown in Fig. 3.The camera reflects the high occurances of 7 octa and 8 octa better than the APCADA data.Again, this is most likely caused by the fact that APCADA misses high clouds.The good agreement between synoptical observations and camera data at 0 and at 8 octa is partly artificial because of the correspondingly chosen clear sky and overcast thresholds used in the camera algorithm.In general, camera based cloud cover better reflects the observations than the APCADA data.Interestingly, 6 octa cases are hardly detected by APCADA.This is not caused by missing cirrus clouds as shown in the next section.
Figure 4 shows the mean diurnal course of the ±1 octa and the ±2 octa Score-Index for the camera and APCADA cloud cover estimates.Again, because of its insensitivity to cirrus clouds, APCADA yields lower scores in the range of 60 (70)% at ±1 (2) octa, whereas the camera skill is 80 (90)% at ±1 (2) octa.Both cloud cover skills show no pronounced diurnal cycle.The camera based cloud cover has the lowest skill at dawn, most likely caused by a color shift towards red in the RGB pixel during sunset and sunrise.An example case for large disagreements between all three methods, is given in Fig. 5.While the observer says 4 octa, the camera's TCA was calculated to 7 octa and APCADA's cloud cover was estimated as 2 octa because APCADA did not detect the cirrus clouds shown in the picture.The misinterpretation of the camera's TCA is most likely due to the reduced fraction of blue color at dawn.

Partial cloud amount comparison
In the present section, the cloud-cover data of all three data sets will be compared for cirrus-free conditions, which are denoted as "partial cloud amount" (PCA) by Dürr and     4, but for all cases without cirrus clouds.Philipona (2004).This provides a true validation of the AP-CADA algorithm whereas the previous pointed more at potential user errors that can result if the algorithm is not correctly applied to those atmospheric conditions for which it was designed.By manual inspection of all digital sky images, all sky cases, where cirrus clouds contributed to the total cloud, cover have been excluded.As the presence of cirrus clouds is very easily seen both from its structure and its slow advection velocity (seen in fast motion animation) we assume that most if not all cirrus-contaminated cases have been correctly removed.
Figure 6 shows again the frequency distribution of differences in cloud amount between camera results and synoptical observations (upper diagram)  to Fig. 2, the negative camera errors (i.e.cloud cover underestimation with respect to the synoptical observations) were reduced.Apparently, large errors in the camera based estimate of cloud cover occur in the presence of the semi transparent cirrus clouds, where the red-to-blue ratio of the camera pixels is close to those for clear skies.The error distribution is more skewed towards −1 octa differences, which may be due to a shift of large negative errors to smaller values.On the other side, cloud cover overestimation by the camera is not really affected when cirrus clouds are excluded.Overall, the cloud-cover estimate was improved from 72 (85)% within ±1 (2) octa at all-cloud conditions to 78 (89)% within ±1 (2) octa at no-cirrus conditions, the overall bias now is 0.02 octa, the mean cloud amount values are 5.36 octa for the observations dataset and 5.38 octa for the camera's TCA dataset.Not surprisingly, the APCADA errors are strongly reduced compared to the all-cloud comparison because of the exclusion of cloud cover underestimations in the presence of cirrus clouds.The overall bias could be reduced from −1.01 to −0.28 octa and a mean cloud amount of 5.08 octa.Cloud-cover overestimation by APCADA is not really affected.The cloud-cover estimate was improved from 60 (75)% within ±1 (2) octa at all-cloud conditions to 73 (89)% within ±1 (2) octa at no-cirrus conditions.
Figure 7 shows the frequency of detected cloud cover from all three data sets.The exclusion of cirrus clouds does not really change the octa-distribution in the synoptical observations compared to Fig. 3.The largest differences between camera and synoptical observations occur at 1 octa and at 7 octa.However, the sum of cloud cover frequencies at 6 and at 7 octa is nearly the same in both data sets with an underestimation at 6 octa compensated by an overestimation at 7 octa.A possible explanation is the frequently occurring direct sun contribution through altocumulus clouds which is often recognized as cloud by the camera algorithm.The same Cloud amount (octas) Fig. 7.As Fig. 3, but for all cases without cirrus clouds.misinterpretation is also seen in the all-cloud data comparison.A similar misclassification may also be possible at 1 and 2 octa.Here, the camera overestimates cloud cover at 1 octa and underestimates cloud cover at 2 octa.Since the lowermost 15 • of solar elevation is excluded in the camera based cloud cover algorithm to avoid cloud side contamination, cloud fields near the horizon are not detected.A typical example is land-sea-breeze induced clouds under otherwise clear sky high pressure weather conditions.
The APCADA based cloud cover distribution now fits much better to the synoptical observations compared to Fig. 3.However, APCADA still strongly underestimates 6 and 7 octa skies, and also strongly overestimates 8 octa skies.Because APCADA only uses the standard deviation of the longwave downwelling radiation to distinguish between 6, 7 and 8 octa (see Table 2.3), and because medium level altocumulus clouds have rather small LDR variability, it is most likely this specific cloud type with 6 and 7 octa that is categorized as 8 octa in APCADA.
Figure 8 again shows the mean diurnal cycle of the Score-Index for a tolerance of ±1 (upper diagram) and ±2 (lower diagram) octa for camera and APCADA based cloud-cover estimates.APCADA yields lower scores in the range of 70 to 80% at ±1 octa, whereas the camera skill is between 75 and 85%.The exclusion of cirrus clouds shows a slight improvement in the camera cloud data skill and as expected a strong improvement for the APCADA based cloud amounts.For the 1 octa tolerance the camera provides higher skills and at 2 octa tolerance both data sets have comparable skills.The generally good agreement in the skill of both data sets may be due to the fact that misclassifications of both camera algorithm and APCADA usually occur within neighbored octaclasses as discussed above.As an example for the generally good agreement between all three data sets, Fig. 9 shows a situation where observer, APCADA and camera algorithm give 5 octa cloud cover.Most important, no cirrus clouds are present that could bias APCADA results.Furthermore, the sky is very clear without atmospheric aerosol or haze contaminations, which minimizes camera misclassification.

Summary and conclusions
A cloud and radiation measurement campaign during summer 2005 on the Island of Sylt was utilized to investigate the quality of ground-based cloud-cover retrieval from a visible all-sky imager and from longwave downwelling radiation (APCADA).Synoptical observations of cloud amount at a nearby observer station of the DWD were used as validation truth.The APCADA algorithm introduced by Dürr and Philipona ( 2004) was adjusted to the clear sky conditions at this measurement site.Although APCADA is not designed for high cold cirrus clouds, a first validation for all-cloud situations (total cloud amount, TCA) was performed in order to point at potential user errors that can result if the algorithm is not correctly applied.TCAs from the camera slightly overestimate the synoptical observations with nearly symmetrical biases towards larger and smaller cloud cover values.The camera-based score-skill is 80 (90)% at ±1 (2) octa tolerance.APCADA underestimates the observed cloud cover in 47% of all cases resulting in a mean bias of −1.01 octa.Because of its insensitivity to cirrus clouds, APCADA yields lower scores in the range of 60 (70)% at ±1 (2) octa.
The validation for partial cloud amount PCA (all cases without cirrus) yields a slight improvement for the camera based cloud score-skills from 72 (85)% within ±1 (2) octa at all-cloud conditions to 78 (89)% within ±1 (2) at nocirrus conditions.As expected, APCADA strongly improves from 60 (75)% within ±1 (2) octa at all-cloud conditions to 73 (89)% within ±1 (2) octa at no-cirrus conditions.Both data sets show no dependency of their score-skills on the diurnal cycle.The investigation of cloud-cover errors by cloud cover classes shows that the high skill of both data sets may be caused by misclassifications of both camera algorithm and APCADA within neighbored octa-classes.
We conclude that an operational use of APCADA provides reliable statistics of PCA.However, to this end an additional information on the presence of cirrus clouds is required.The shortwave downwelling radiation (SWR) is a promising piece of information in this regard as it exhibits strong variability including large enhancements compared to clear sky radiation.This could also help to identify altocumulus clouds at high octa cloud amounts (Schade et al., 2007).The incorrect application of APCADA for all-cloud conditions yields unacceptable systematic errors.Our error estimates of APCADA for PCA is slightly larger than that reported by Dürr and Philipona (2004) which may be caused by the shorter time series investigated in our work, which in turn may have caused a less optimal fit of the emissivity for cloud-free conditions.
Though limited to day-time the cloud cover retrievals from the sky imager are not greatly affected by cirrus clouds and provide a more acceptable cloud climatology for all-cloud conditions.However, the exclusion of cirrus clouds also yields a small improvement in the cloud cover identification.

Fig. 4 .
Fig. 4. Mean diurnal course of Score Index for ±1 octa (top), and 2 octa (bottom) difference to the for all-sky camera and APCADA.
ample of a good agreement in cloud amount estimation for observations, APCADA, and all-sky ne 17th 2005, 21:00 local time.