Note : Time-dependent limb-darkening calibration for solar occultation instruments

Solar occultation has proven to be a reliable technique for the measurement of atmospheric constituents in the stratosphere. NASA’s Stratospheric Aerosol and Gas Experiments (SAGE, SAGE II, and SAGE III) together have provided over 25 years of quality solar occultation data, a data record which has been an important resource for the scientific exploration of atmospheric composition and climate change. Herein, we describe an improvement to the processing of SAGE data that corrects for a previously uncorrected short-term time-dependence in the calibration function. The variability relates to the apparent rotation of the scanning track with respect to the face of the sun due to the motion of the satellite. Correcting for this effect results in a decrease in the measurement noise in the Level 1 line-of-sight optical depth measurements of approximately 40% in the middle and upper stratospheric SAGE II and III observations where it has been applied. The technique is potentially useful for any scanning solar occultation instrument and suggests further improvement for future occultation measurements if a full disk imaging system can be included.

The long lifetime of the instrument has permitted an evolving understanding of retrieval methods that has led to substantial improvements of key data products like ozone relative to early versions of the data sets (e.g.Cunnold et al., 1996;Steinbrecht, 2006;Terao and Logan, 2007).Herein, we describe an effort to further enhance the quality of SAGE data products by recognizing and reducing a previously uncorrected source of noise, namely the apparent rotation of the scan track across the face of the sun during the course of a measurement event.The rotation impacts the normalization process which is essential in the inference of the vertical dependence of transmission.In the frame of reference fixed to the satellite, the sun appears to rotate slightly as it rises or sets, while the scanning continues to be vertical (in the satellite frame) throughout the observation.This implies a mismatch between the limb darkening function in effect during the calibration phase and the atmospheric measurement phase of each observation event.The self-calibration of solar occultation data and the assumption of constant limb darkening will be described in more detail in the next two subsections.
In the remaining subsections, we detail the development of a technique to reduce this error associated with rotation of the scan track.We describe the analysis used to identify and ameliorate the calibration error, and thereby to reduce the variance in the transmission measurements.Comparisons of processed Level 1 SAGE II data with and without this new correction are presented.The reduction in variance in the middle to upper stratosphere and above, where the correction Published by Copernicus Publications on behalf of the European Geosciences Union.
is easily applicable, amounts to approximately 40%.The technique outlined here is potentially valuable to any solar occultation measurement that scans across the sun, not just the SAGE experiments.Furthermore, it suggests that a fulldisk imaging camera on future solar occultation instruments would be highly useful, for it would allow precise calibration with the appropriate limb darkening function regardless of the rotation of the instrument relative to the Sun.

SAGE measurement technique
The SAGE II mission was terminated in August 2005, after more than 21 years of solar occultation measurements.The SAGE II data set consists of aerosol extinction measurements at four wavelengths, 1020 nm, 525 nm, 452 nm, and 386 nm; ozone and NO 2 number density; and water vapor mixing ratio.SAGE II was preceded by the original SAGE (now often referred to as SAGE I), which provided ozone, NO 2 and aerosol extinction at 1000 nm and 450 nm.The SAGE III instrument (2002)(2003)(2004)(2005)(2006) added to the suite of data products with temperature and pressure measurements, an expanded set of nine wavelengths for aerosol extinction profiles, and nighttime lunar occultation profiles of ozone, NO 2 , NO 3 , and OClO.Descriptions of instrument specifications are given by Mauldin et al. (1985) and SAGE III ATBD (2002).
The three SAGE instruments made measurements using solar occultation, recording the attenuation of sunlight at multiple wavelengths through the Earth's atmospheric limb during each satellite-observed sunrise and sunset.Solar occultation measurements are insensitive to calibration, in that transmittance measurements are formed by normalizing the observations through the atmospheric limb by observations during the same event along paths that do not intersect the atmosphere.Multiple transmittance measurements are obtained at each altitude by repeatedly scanning across the face of the sun as it rises or sets (Chu et al., 1989;SAGE III ATDB, 2002).Figure 1 illustrates an example of the SAGE data stream as a function of time for a sunrise event.In Level 1 processing, the measurements are combined to form a mean profile of transmission versus altitude at each wavelength, plus an uncertainty estimate, which is the input for Level 2 processing, where the contributions by various atmospheric constituents are separated and the slant path measurements are inverted into vertical profiles (Chu et al., 1989;SPARC, 2006;SAGE III ATDB, 2002).The uncertainty in the transmission data primarily consists of altitude registration errors for individual data points and normalization mismatch errors in registering the corresponding position of in-atmosphere data in the exoatmospheric scans.For SAGE II, electronic noise is generally well below the count level and not readily measurable.
As already mentioned, multiple transmittance measurements are obtained at each altitude as the instrument repeatedly scans the sun.There are generally several measurements of transmission within each small altitude region, Fig. 1.The time history of sunrise SAGE event at a single wavelength (1020 nm in this example).The sinusoid-like structure is due to the vertical scanning across and off the Sun.The alternating wide and narrow "waves" are the result of scanning either with or against the effective direction of the Sun's motion.For sunrise events, the narrow waves are downward (toward the Earth) scans whereas the wide waves are upward scans.
since consecutive scans overlap by about 50-100%, depending on altitude, the angle between the spacecraft direction orbit plane and the Sun (the beta angle), and the direction of the scan (with the motion of the sun or against it).This redundancy allows for better characterization of the atmospheric attenuation and permits the estimation of the uncertainty in the averaged transmission profiles.The current project aims to reduce the errors in the normalization step so as to decrease the variance in the transmission measurements and, concomitantly, reduce the reported measurement uncertainties for the Level 1 products.Ultimately, the primary goal is to improve the precision of the Level 2 data products particularly at the upper ranges of where it is possible to measure them (e.g.ozone between 40 and 60 km).

Assumption of constant limb-darkening
A key part of the processing of solar occultation observations is the calibration of each vertical scan using a limb-darkening curve, or I 0 function, as in the following equation: where T is the transmittance, I is the measured counts, and I 0 is the calibration function.I 0 is a function of position on the Sun, which is defined by convention in SAGE data processing as ranging from 0 (at the "top" of the Sun or the point furthest from the limb of the Earth from the satellite point-of-view) to 2 (the "bottom" of the Sun).The I 0 function is derived from exoatmospheric scans obtained at the beginning of each sunset and at the end of each sunrise event.It is primarily due to this self-calibration that the asset of a multi-decadal record from a single instrument was possible (21 years from SAGE II).However, the calibration function (I 0 ) is simply a function of just one dimension, while the measurements of the limb-darkening are weakly two-dimensional.By "weakly two-dimensional" we mean that while we can successfully represent the measurements as varying only with vertical position along the scan track they are not actually confined to a single track.Rather there are a family of tracks in which each scan is rotated a fraction of a degree relative to the previous scan throughout an event.The apparent rotation of the image of the Sun in the field of view is the result of the orbital motion of the satellite while scanning is always vertically up and down in the satellite reference frame.The full rotation, depending on a variety of factors, can be up to ten degrees during the entire course of an event, a time interval of one to three minutes.
Figure 2 shows a schematic of the SAGE measurement process that includes the field of view (FOV), scan direction and apparent solar rotation.In our approach, we treat the effect of this rotation as a small, time-dependent perturbation of the one-dimensional calibration function.
Rewriting Eq. ( 1) with the dependent variables specified and with the addition of this time-dependent perturbation produces the following: where z represents the minimum altitude above the Earth's surface (or "tangent" height) of the path the sunlight follows through the atmosphere to the instrument; p is the position on the face of the sun, measured along the scanning track from the top of the sun; and t is time.to 2 (bottom of sun).The data shown here are for slant paths entirely above the atmosphere (minimum altitude is 100 km).At position approximately 0.9, there is a signature of a large sunspot, which has not been filtered for this demonstration.Variations in transmission away from unity are due to errors in the limb-darkening curve used for calibration, as well as instrument noise.Calibration errors are correlated from scan to scan and are hypothesized to be due to rotation of the scan plane.
The assumption of a one-dimensional, time-independent calibration function, or I 0 function, has been used for 26 years in SAGE processing and has produced excellent results.However, further improvement can be obtained by recognizing and correcting for the rotation of the scan track.The rotation of the scan track results in deviations of the measured transmittance that are not due to atmospheric attenuation, but rather are due to a slight mismatch between the limb-darkening function obtained from exoatmospheric observations and the solar image sampled along the slightly different scan track of a given in-atmosphere scan.This is especially evident and easy to visualize if the scan track skims the edge of a sunspot, which will be seen in the following example.A spot can be observed in the exoatmospheric calibration data but missing in the in-atmosphere scans, or vice versa.In addition, small variations can be observed even away from sunspots due to granulation, faculae or other features of the solar photosphere.A mechanism for identifying the presence of sunspots has been a part of SAGE processing for many years but it is not possible to eliminate variations due to other forms of solar inhomogeneity.
The effects of the mismatch can be seen in Fig. 3. Here, normalized transmission values are shown for tangent altitudes between 100 and 230 km over the course of www.atmos-chem-phys.net/10/1/2010/approximately one minute.For the purpose of this demonstration, a sunspot has been included that would be filtered in normal processing.In the left panel, the data are shown as a function of altitude.The transmission values cluster around unity because all the observations were taken outside the atmosphere.Perfect normalization should result in a nearly uniform line of transmission with value of unity, with some random scatter due to instrument noise.The overall scale of the variance is small at all altitudes; however the normalization is noticeably imperfect.For instance, there is a visibly smaller variance towards the middle of the selected altitude range.
In the right panel of Fig. 3, the same normalized transmission values are shown as a function of the position on the sun, or rather, the linear distance along the vertical scan track.The first thing to notice is that the points with greatest deviation from unity have been resolved to the same sun position: these points originate from the unfiltered sunspot.Furthermore, even far from the sunspot, this choice of abscissa highlights the differences in calibration from scan to scan, since points at the same sun position are normalized by the same calibration value.In this display, points originating in the same scan are connected by a colored line.This color coding reveals the correlation in the apparent noise from scan to scan, and hints at a time-dependence in the variance.The noise is not random; instead, each scan appears to be a slight perturbation of the previous one.This correlation also explains the pattern in the variation in the left panel.The normalization function is an average over all the exoatmospheric scans.The average appears to be more accurate in the middle of the range because there is a continuous, not random, variation, between measurements at higher and lower altitudes, or rather at early and late times.
It is proposed here that the apparent time-dependence and correlation with position on the sun observed in Fig. 3 and other exoatmospheric SAGE measurements is due to a growing mismatch between each scan and the limb-darkening function that was derived from measurements up to a minute earlier or later.We hypothesize that the observed timedependence stems primarily from the rotation of the scan plane due to satellite orbital motion.
In the next section, a model simulation is presented to support the hypothesis that scan plane rotation produces apparent noise in the transmission measurement.It's followed in Sect.2.3 and 2.4 by the description of an algorithm to correct for the effect in SAGE or SAGE-like solar occultation data.Data are presented which show the effect of the correction on SAGE II and SAGE III observations.

Model simulation of time dependent normalization function
In order to qualitatively confirm that the proposed mechanism could be responsible for the observed correlation in the transmission uncertainties, we have modeled the orbital geometry and resulting scan patterns.The model combines actual spacecraft and solar ephemeris data from a SAGE II event with a CCD image of the sun obtained from the Gas and Aerosol Measurement Sensor (GAMS) during the second SAGE-III Ozone Loss Validation Experiment (SOLVE-II) (Pitts et al., 2006).Figure 4 shows a solar image from the GAMS.
Modeling the SAGE transmission starts with rotating the image by varying degrees, determined by the SAGE II ephemeris.Meanwhile the scanning of the SAGE instrument is approximated by taking a series of vertical slices through the rotated images.Integration is performed on all pixels which fall in the calculated SAGE II field of view to produce a simulated record of radiance data for each scan.An I 0 function is produced by combining six of these scans at nearby angles, reflecting the fact that SAGE II processing in Version 6.2 and earlier versions used six scans across the face of the sun to create the I 0 function.This composite function is then used to normalize the generated scan data for the other slices to produce transmission values for each scan.Any deviations away from 1.0 in the transmission values can be attributed solely to rotation of the scan plane, since there is no atmosphere in this simple model and the same base image is used for all scans.Large errors near the edges of the sun result from the rotation of a refracted (and therefore not completely circular) solar image.These errors are not relevant to this study.The calculated transmissions are shown in Fig. 5.Although the simulation described here is a rough experiment and there are notable differences between the GAMS imaging and the measurement environment of SAGE instrument, the visual similarity of the transmission variations here to those of actual SAGE II events is striking.This supports the idea that the rotation of the scan track is a likely cause of a significant proportion of the observed variance.

A first-order correction method
In the previous section we noted that the noise pattern in unbinned SAGE transmissions is consistent with rotation of the scan track with respect to features on the face of the Sun.Time series showing the transmission for selected positions along the scan track are shown in Fig. 6.The transmission shown here is normalized by the I 0 function as usual and then normalized again by a "best guess" transmission profile derived by the same averaging and smoothing process as in the standard production process.This renormalization isolates the time-dependent part of the transmission measurements, thereby approximating the time-dependent function f (actually 1/f ) introduced in Eq. ( 2).The derived error function is shown in Fig. 6 as a function of scan number for a few selected sun position values.Each scan does not sample exactly the same positions along the scan track, so the data were splined onto a grid in sun position to create Fig. 6.Because of the difficulty in creating an accurate transmission vs. altitude profile in the lower stratosphere and troposphere, the error function values are only shown down to approximately 40 km.For this altitude range, the variation in the error function is well captured by quadratic functions, which are also plotted in Fig. 6.The next section will use an alternative way to approach the isolation of the effects of the variation in the scan track from the actual atmospheric transmission, which is to represent the time-dependent function as a residual, that is, an additive function rather than a multiplicative function.Accordingly, where T true represents the true transmission; e (t,p) is an "error" variable; and the other variables are as in Eqs.
(1) and (2).The "error" variable is the time dependent part, and is the difference between the true transmission and the calculated transmission, that is, measured counts divided by the onedimensional calibration function (I /I 0 ).Combining Eq. (3) with Eq. ( 2) shows the relationship between the error variable and the time dependent I-zero correction, f (t,p) www.atmos-chem-phys.net/10/1/2010/Atmos.Chem.Phys., 10, 1-8, 2010 In either case, we approximate the function, e or f, by using a best guess transmission in place of T true (z).Simple fits to the error function are then substituted back in to produce the time-dependent correction.In the simple method, the correction is applied by multiplying T by f , although in a full solution the correction is applied specifically to the I 0 part.Results using the quadratic fits described above are shown in Fig. 7 for a test event from 1 May 1989.Comparison of the left and right panels in this figure show a conspicuous decrease in the variance in the unbinned transmission, especially in the 386 nm channel (bottom panels).This relatively simple approach demonstrates the feasibility of correcting for I 0 for time-dependent changes.However, some of the simplifications are not optimal for use in the SAGE data processing scheme.For instance, we felt that splining onto a grid in sun position and the use of a global quadratic fit introduce unnecessary error.A more robust modification of this method has been implemented in routine data production and is described in the next section.

A second-order method
The following algorithm was implemented for SAGE III version 4.0 data processing (e.g.Thomason et al., 2009) and for a planned SAGE II version 6.3.Our goal was to remove as much variation correlated with position on the Sun as possible while minimizing the risk of being unable to separate the effects of the rotating scan track from real atmospheric variability.Accordingly, we moved the cutoff down to 25 km, and thereby used more measurements in each fit.After subtraction of the altitude based transmission profile from the individual transmission measurements to produce the error function, represented in Eq. ( 3), local fits were calculated for each measurement point based on nearest neighbors in a combined time and sun-position space.That is, for each point in each scan, a set of neighbors is defined which are the error function estimates for points that are close together in time and linear distance along the scan (sun position).A local fit, cubic in both variables with cross terms, is calculated on the neighbors of each point in each scan and applied only to that single point.By applying the fits only locally, we minimize systematic errors which might otherwise result from using a small-order fit.Fitting in the two-dimensional space, using the sun position variable in addition to the time variable, dispenses with the necessity of splining the measurements onto a grid, so no loss of accuracy is introduced in that way.
The time dependent I 0 correction for each measurement, f (t,p), is derived from the fits using Eq. ( 4), and then applied to the calculated I 0 function as in Eq. ( 2).With the new calibration function, Level 1 processing is iterated.Figure 8 shows the effect of the full correction on an event that occurred on 1 May 2005.The final transmission is shown just before the data are binned to produce a profile.The standard deviation is 30-50% smaller in most bins above 50 km.This is visible in the figure as a clear reduction in the scatter in the corrected case.Below about 50 km, where there begins to be a measurable signal, the uncorrected variance is already smaller, but the corrected data is still noticeably smoother.
Figure 9 shows the reduction in the uncertainty in the SAGE III short wavelength operational transmission product between Version 3 and Version 4. Average profiles for all sunset events from October 2003, or about 400 events, are shown here.The approximately 40% reduction in the noise is due to a number of factors of which the variable I 0 is a key but not sole component reduction, but varies from channel to channel, due to variation in the relative importance of solar granularity at different wavelengths.Other factors include improved refraction calculations (at low altitudes) and improved scan registration.The final ozone product is also improved in Version 4 due to reduction in the noise.We find that the useable extent of the SAGE III ozone profiles increases by nearly a scale height from 55 km in Version 3 to above 60 km in Version 4 as the overall ozone measurement uncertainty was reduced by about a factor of 2. The calibration correction described here is responsible for an approximately 30% reduction in the ozone uncertainty.
Because of the better match between each measurement and the new calibration function, the time dependent I 0 correction improves other parts of the Level 1 processing including the algorithms responsible for aligning scans and locating the edges of the solar disk.In this way, the time dependent correction also affects measurements at altitudes lower than the 25 km cutoff.However, it is above the cutoff where the correction works to greatest effect.Below this level, we use the last corrected I 0 available from above 25 km.
Although the correction was possible and seemed unbiased in some tests penetrating down to just a few kilometers above the tropopause, the more conservative cutoff of 25 km has been adopted for SAGE III Version 4 and the future SAGE II version 6.3 to more confidently avoid the introduction of systematic error.We have retained the cutoff for two reasons.Firstly, although the error function is defined as the deviation between the calculated and true transmission, in practice the "true" transmission must be approximated.Specifically, we use a "best guess" profile, constructed by smoothing the data using a boxcar average and a running median, the same algorithm that is used to generate the final profile from the transmission scatter data.It is possible that errors in the best guess transmission profile could lead to errors in the time dependent correction, and these errors are more likely where the atmosphere has a high degree of natural variability.There is also some risk of falsely "correcting" or attempting to correct real atmospheric variability by the method described here.It is possible that future developments will allow the algorithm to distinguish between real atmospheric variability and time dependent calibration errors.

Concluding remarks
In this work we have presented evidence that a previously unaddressed source of measurement uncertainty exists in the solar occultation technique.This is the time dependent error in the calibration function due to the small (<10 • ) rotation of the scan plane about the center of the sun over the course of a measurement event.Correcting for this effect in SAGE II and SAGE III observations produces a decrease of 20-50% in the variance of the transmission measurements (before the measurements are combined into a single profile), at high altitudes where the correction is most straightforward to implement.While the improvement is smaller and more difficult to apply in the lower stratosphere and troposphere, we feel that the correction technique and, perhaps more importantly, the realization that this error source exists, holds promise for the further improvement of future solar occultation experiments.Specifically, any solar occultation instrument that is deployed in tandem with a full sun imaging device can reap great benefits in that the correct calibration function for every scan in any orientation will be able to be precisely determined.
Edited by: M. Van Roozendael

Fig. 2 .
Fig. 2. A schematic of how the Sun appears to a SAGE instrument including to-scale representations of the field of view (FOV) for SAGE II and SAGE III.The scan direction is centered along the center axis of the Sun normal to the Earth's surface.Dark ellipses represent sunspots.In this coordinate system, the Sun appears to rotate such that solar features may either rotate into or out of the instrument's FOV.

Fig. 3 .
Fig. 3. Transmission is shown as a function of altitude (left) and apparent position on the face of the sun (right) for a SAGE II measurement event on 21 July 2000.Measurements from the 386 nm wavelength channel are displayed.The vertical axis in the right panel is position along a vertical line bisecting the solar image, represented in arbitrary units from 0 (top of the sun as seen from the spacecraft)to 2 (bottom of sun).The data shown here are for slant paths entirely above the atmosphere (minimum altitude is 100 km).At position approximately 0.9, there is a signature of a large sunspot, which has not been filtered for this demonstration.Variations in transmission away from unity are due to errors in the limb-darkening curve used for calibration, as well as instrument noise.Calibration errors are correlated from scan to scan and are hypothesized to be due to rotation of the scan plane.

Fig. 4 .
Fig. 4.An 865-nm image of the solar photosphere from the Gas and Aerosol Measurement Sensor, which flew aboard the NASA DC-8 aircraft during the SOLVE II campaign on 21 January 2003.This image was taken at a solar zenith angle of approximately 83.7 • .

Fig. 5 .
Fig. 5. Simulated transmissions are shown as a function of position on the face of the sun.The biases in transmission near the edges of the sun result from faulty edge matching, due to the image of the sun not being round.Since all scan data are generated from the same image, deviations from one (except at the edges) are indicative of the amount of variation that can be expected from rotation of the scan plane alone.

Fig. 6 .
Fig. 6.Transmission, calculated by normalizing the SAGE II measurements against the exoatmospheric measurements, and then renormalized for this experiment by dividing by an average transmission profile, is shown as a function of scan number.The measurement event shown here occurred on 1 May 1989.The panels show the transmission within a narrow range of Sun position for channels 1 and 7. Specifically, frames (a)-(c) show channel 1 (1020 nm)for Sun positions 0.5, 1.0, and 1.5, respectively while frames (d) through (f) show channel 7 (386 nm) for the same Sun positions.Broad scale variation, highlighted by overlaid quadratic fits, is attributed to apparent changes in the solar structure (granulation, solar faculae) due to movement of the scan track over the course of the event.

Fig. 7 .
Fig. 7. Unbinned transmission is shown as a function of altitude between 40 and 60 km for SAGE II channels 1 (1020 nm) and channel 7 (386 nm).Frames (a) and (b) show uncorrected and corrected channel 1 while frames (c) and (d) show uncorrected and corrected channel 7. Left and right panels depict transmission before and after the correction, respectively.The specific SAGE II measurement event shown here occurred on 1 May 1989.

Fig. 8 .
Fig. 8. Unbinned transmission is shown in the ozone channel (600 nm) between 40 and 60 km after all other transmission processing.In frame (a), no time-dependent I 0 correction was made, while data in frame (b) employs the correction as described in Sect.3.4.The measurement event depicted here occurred on 1 May 2005 at (36.0 • N, 58.8 • E).

Fig. 9 .
Fig. 9. Mean short wavelength transmission error for approximately 400 sunset events in October 2003 for Version 4 (black) and Version 3 (grey).The uncertainty is reduced by approximately 40%.The calibration correction described in this paper accounts for about half of the overall reduction.