The Principal Component Analysis (PCA) is a method of dimensionality reduction of a dataset. It is commonly used to reduce high (N)-dimensional data to its core components and finds application in many natural sciences, such as biology and atmospheric science. Depending on the field it is also known under different terms, such as Karhunen–Loève transform in signal processing, and is strongly related to singular value decomposition in mathematics. The same method was already used to suppress noise in calibration measurements for the GLORIA instrument . The N-dimensional principal components ci of a (N,k)-dimensional dataset (containing k observations of N features) X are the pairwise orthogonal vectors along which the internal variance of the data projected to that principal component is maximal such that var(X⋅ci)>var(X⋅ci+1). Along the first principal component the data variance is thus maximal, with the second component having the second largest variance and so on. Components, along which the data variance is small, convey little information, and can thus be neglected. In particular, statistical noise conveys little to no information and is usually prevalent in the lowest principal components. Choosing an arbitrary cut-off n, the PCA provides an (N-n)-dimensional approximation of the data X still containing k observations which retains most of the original data's structure. The cut-off is usually chosen such that 80 %–90 % of the data variance is retained, following considerations described in . Since the PCA is a linear transformation, there exists a unique inverse transformation, albeit not without loss of a little bit of information due to the cut-off n.

A statistical mixture model is a probabilistic model which describes a statistical population as the combination of many, statistically distinct sub-populations. In simple terms, given an N-dimensional dataset X, a mixture model assumes that the dataset X was generated by sampling from k random variables with N-dimensional probability densities D0,D1,…,Dk with different (mixing) weights w0,…,wk . The statistical properties of each component distribution Dk are inferred directly from the dataset X. A Gaussian mixture model assumes each component distribution to be a Gaussian distribution N. The mixture distribution from which X was generated is then given by: 2D(X|μ1,…,μk,Σ1,…,Σk,w1,…,wk)=∑i=0kwi⋅(2π)-N2⋅deti(Σi)-12⋅exp⁡-12(X-μi)TΣi-1(X-μi)=∑i=0kwiN(X|μi,Σi) where μi,Σi,wi are the mean vector, covariance matrix and mixing weight of each component, with ∑i=0kwi=1. The number of components k is the free parameter of the statistical mixture model. In Sect.  a method to estimate this parameter from the data itself is presented. In contrast to a regular Gaussian mixture model, which determines its parameters using expectation maximization , a Bayesian Gaussian mixture model (BGMM) determines its parameters via variational inference. The necessary a priori values for this Bayesian approach can be provided by a Dirichlet process. A trained BGMM provides probabilities for each data point x according to Eq. (). Since it is assumed that a point necessarily must have been sampled from any of the distributions, these probabilities can be normalized: 3pi(x)=wiN(x|μi,Σi)∑j=0kwjN(x|μj,Σj) A BGMM can also be used as a classifier by assigning each point x a label according to the most likely component the point was sampled from. The label is assigned according to: 4l(x)=argmax(log⁡(wk)+log⁡(N(x|μk,Σk)) The maximum value of Eq. () then corresponds to the probability that the point was accurately labeled. For disjoint distributions Eq. () typically takes values between 1 (high confidence) and 0.5 (ambiguous labeling between two classes). In the following we refer to this quantity as the class alignment.

The classifier works well for all points which have suitably high probabilities to be sampled from any one of the component distributions and thus are well represented by the model. If points are not well represented by the model, each probability Ni vanishes and Eq. () is dominated by its second term. Labels are then determined by which distribution Ni vanishes the slowest, regardless of the mixture weights wi, which leads to sudden changes in labeling on the edge of phase space. Since each Gaussian distribution is compact in phase space the labels should be compact similarly. To account for this behaviour we determine labels using Eq. () only for points with at least one sufficiently high probability and determine labels for all other points using a nearest-neighbor classifier. Only few points require such a correction and the resulting labels are compact in phase space.

The majority of clustering algorithms have free parameters p which are chosen by the data analyst, thus introducing human judgement. A Gaussian mixture model requires the number of components as such a free parameter. A way to a more objective selection and to deriving an estimate of these parameters from the data itself is to iterate a classifier over various possible choices and compare the resulting classification scores. The estimate itself should not explicitly depend on the particular classification score. A suitable choice for many classifiers is the silhouette score , which describes how dissimilar a data point within one class is to the points outside of that class compared to the data points within the class. The silhouette score becomes maximal if the points within each class are most similar to each other while being most dissimilar to points outside of that class, e.g. if the classification reflects the data structure well. Increasing the number of classes does not necessarily increase the silhouette score, as splitting a class of similar points is penalized due to reducing dissimilarity to points in other classes. The parameters p for which the silhouette is maximal indicate the highest agreement between the data and the underlying model of the classifier.

The classification analysis described in Sect.  is applied to the retrieved trace gas mixing ratios of H2O, PAN, O3, HNO3 and CFC-12. Due to the aforementioned issues of models to resolve the finer structures/mixing we use only the retrieved values in the first step of the classification. Spatial information such as longitude, latitude or potential temperature was not considered in the classification. The standardization parameters used for each dataset and the resulting principal components are given in Tables  and , respectively. We estimate that there are five distinct features in the retrieval data using the method described in Sect. . The BGMM is thus trained assuming five mixture components. Labels are determined using the method described in Sect. .

The resulting classification for both hexagon retrievals are shown in Fig. .

The choice of 5 classes is well reflected in the transformed retrieval data with five distinct clusters of points visible in Fig. a. Shown are the cluster centers of each class, which correspond to the average chemical composition of each class. Each class lies well separated from each other and the labelling is unambiguous for the majority of points, with approximately 1 % of points requiring label adjustment. The classification of hexagon 2 is shown in Fig. c). For better visualization, the first two principal components of each hexagon PCA are shown in Fig. b and d. It should be noted that clusters may overlap in this visualization due to perspective. Given that hexagon 2 was measured in a dynamically similar, but temporally later situation, the PCA deviates from hexagon 1 in several ways. The blue and gray clusters are much less distinct from each other. An argument can be made to consider them merged into a new cluster, which we will only do visually for consistency reasons. The green and orange clusters also converge in hexagon 2, implying that mixing resulting in similar air masses is occurring. The effect is most visible for those points having positive values for the principal components. Unfortunately, no air parcels lying in this region were revisited and consequently these changes were not observed directly. It shall be noted that while the orange cluster in hexagon 1 shares a large border with the violet cluster and might be considered an extension of the later, the local correlations in both clusters are quite different, meaning key chemical components may be similar, but show distinct differences in their proportions.

For each class the cluster centers (shown as white dots in Fig. b, d) represent the mean values of points in each class. These cluster centers are basically the mean values of each class and can be considered representative for the whole class, which will be further explored in the following. Similar to these representative points for each class, representative points for the contribution of each Asian monsoon source region can be constructed by taking the weighted mean of the observation weighted by the surface-origin tracers (see Sect. ). These values are then transformed and shown within the PCA in Fig. b and d as red markers. The positions where these transformed points are located within the PCA show which source region at the Earth's surface has the largest impact to the respective classes.

To express this classification in physically relevant terms, the inverse transformation of the PCA is applied to each cluster's center as the most representative point to transform the point back into regular tracer composition. The resulting characteristic chemical composition of each class is shown in Fig.  in percentiles of the total distribution.

When considering both the information of these characteristic compositions of each class and the information about the regions of origin aligned with each class, a physical interpretation of each class can be made:

Continental tropospheric class: this typically tropospheric air (blue label) shows the by far highest mixing ratios for each tropospheric tracer and the by far lowest values for each stratospheric tracer. High water vapor and high PAN/CFC-12 indicate a deep tropospheric character, possibly from within the ASM region, which the surface-origin tracers link to continental regions such as the Eastern China region. We refer to this type of air as continental tropospheric air (CT).

The classification was derived from the chemical composition only and did not have direct information of the three-dimensional spatial distribution of the different trace gases/classes within the hexagon. If the classification and its interpretation are meaningful, the spatial distribution of the classes is expected to be consistent with this interpretation.

The spatial structure of each class is illustrated in Figs.  and .

Panels (a) to (c) in Fig.  show the class distributions of hexagon 1 along the same cross-sections used in Fig. . Their position within the hexagon is shown in Fig. a and b. The background in between measurements in these cross-sections is interpolated on the retrieved values and positions. These interpolated values are then transformed and classified as described before. A high probability for the class assignment is shown in bright colors, lower probabilities (corresponding to labeling ambiguity) are shown in pastels or white. Regions which contain few measurements are most influenced by interpolation, but generally reflect the class structure well due to the compactness of the PCA. Air parcels revisited between the two flights are indicated by black outlines. Figure  illustrates the full spatial structure of each class. In particular, each class does form a compact region not only in phase space, but also in real space (in the atmosphere), despite no spatial information was used.

The tropospheric classes CT and MT are at the lowest altitudes and are mostly located below the 2-PV isoline, which is commonly used as an upper limit for the troposphere. In hexagon 2 this behaviour is most consistent. In hexagon 2 both tropospheric classes lie intertwined, suggesting mixing of maritime and continental tropospheric air occurs. This mixing (merging) of those air types was already suggested by the classification in Fig. d. In the following both tropospheric classes are considered functionally the same (and referred to simply as the tropopheric class (T)) and are indicated in blue color. Conversely, the stratospheric class is located in the highest altitudes. Due to the tropopause fold in hexagon 1 the vertical structure is distorted, which is correctly captured in the classification, in which the CT class and the S class lie diametrically opposed. Comparing the location of the ASMO class to Fig. , the ASMO class coincides with the location of the filament in both hexagons, further justifying the identification of the class with ASMO air. In hexagon 1 an intrusion of ASMO air into the stratosphere is seen (panel b). This suggests inmixing of ASMO air directly into the lower stratosphere. The blended class is located bordering the ASM air filament between the troposphere and the stratosphere, mostly at the tip of the stratospheric intrusion. Visually it appears as if the blended class acts as a partially dissolved extension of the intrusion. This characterization would be in line with its chemical composition. As stated beforehand, this classification is not necessarily unique, and the blended class could also be the mixing of ASMO air with stratospheric air. Neither point can be substantiated further from this analysis alone.

Active mixing between classes is directly indicated by reduced probabilities of class assignment, as the transition from one class into another crosses the regions of low class assignment probability. Considering these probabilities as shown in bright and dark colors in Fig. , mixing is occurring near the edges of most classes in either hexagon. Edges with still high probabilities imply only recent contact between those air masses, such that mixing has not yet sufficiently occurred. The strongest ongoing mixing is occurring in hexagon 1 between the CT class and the A class/B class. In hexagon 2, however, the strongest ongoing mixing however is between the blended class and its bordering classes. For those air parcels which were observed in both hexagons the mixing processes can be derived explicitly by comparing their chemical composition in both hexagons.

Mixing in the form of changes in classification are summarized in the confusion matrix in Fig. .

A confusion matrix summarizes each possible combination of two instances of labeling, here the labels in hexagon 1 and 2. The column i indicates the label in hexagon 1 (X), the row j indicates the label in hexagon 2 (Y). The number of transitions of that kind are then given by the matrix element i,j, which in the following is referred to as “X-Y”. Each class is abbreviated by their first letter as defined above. A transition X-Y would indicate air of type X is becoming of type Y. Air type Y is not necessarily the air type X is mixing with, but simply the resulting air type. If the mixing is relative slow air type Y is likely the air type X is mixing with. In general not every possible combination is observed. Therefore, most elements are zero. Elements with very few transitions can indicate issues with the trajectories or with the classification and are not considered significant. It is however not to say that these transitions are not real. From the confusion matrix qualitative insights into the mixing processes can be derived:

A large share of blended air is created via the transitions T-M and A-B. The transition B-B is observed, but rarely. ASMO air also transitions into stratospheric air via A-S in large quantities. There are no significant transitions X-A, where X is any other class, which lead to the creation of ASMO air, which again supports the characterization as ASMO air, as Asian Summer monsoon should be the product of the monsoon and not a product of mixing. The transition “T-S” sticks out as a direct transition of tropospheric air into stratospheric air without observed intermediate state, which is likely related to a classification issue, as no tropospheric air lies directly bordering the stratospheric air. The lack of this transition would also reflect the transport barrier between the troposphere and the stratosphere in the extra-tropics Otherwise most tropospheric air and stratospheric air do not undergo significant changes due to mixing (T-T and S-S).

The transitions T-B and A-B stand out because when considering the average chemical composition of the classes T, A and B (see Fig. ), neither T air nor the A air contain sufficiently much stratospheric tracers such that B air could have been created due to mixing from any combination of T, A and B air. The deficit of stratospheric tracers requires at least some S air to make up this deficit. A possible explanation would be the transition A-S, which has consistent increases in both O3 and HNO3. The chemical budget would add up if A air would partially mix with S air, and afterwards mixes with T air creating B air. How these transitions would happen in detail, e.g. which intermediate states occur, cannot be derived from these observations. Otherwise, an additional source of O3 and HNO3 would be required, which is investigated in Sect. .

If more than two mixing partners are involved in such a process, these intermediate states would no longer be mixing lines, as is traditionally considered, but rather mixing curves of more complex shapes. This would explain why the phase space in Fig. a somewhat resembles a traditional mixing curve (in 3-D), but few to no explicit mixing “lines” are contained.

Due to the 3-D tomographic retrievals the spatial structure of the changes in chemical composition, which implies the mixing, is known as well. The spatial structure of each transition is shown in Fig. .

The mixing processes are structured as follows: the transition T-T is located in the lowermost altitudes purely within the troposphere in both hexagons, which extends up to 9.5 and 11.2 km, respectively. This represents the transport of air within the troposphere. Above the troposphere lies a layer of blended air from around 9.5/10.5 to 11/11.2 km. The blended layer itself consists of two individual layers, which meet at around 9.7 km in hexagon 1. The lower layer is created from the transition T-B, the upper layer is created from the transition A-B. In hexagon 2 the lower blended layer is stacked similarly, but the upper blended layer is much more interwoven with the stratospheric air originating from A-S.

Above the blended layer lies a layer of ASMO air A, which transitions into stratospheric air and extends to the stratosphere at around 12/13.5 km. Upwards lies the stratosphere with transport within itself. Each transition again forms coherent regions, with the transition B-B being a small exception. Despite the spatial distortion observed in hexagon 1, each transition follows a stratified vertical structure. The exchange between the troposphere and the stratosphere appears to be still mostly vertical, despite the perturbation caused by the tropopause fold most likely inducing the mixing being mostly horizontal. The stratification of the mixing can be explained by the dynamical properties of the atmosphere, which counteracts the distortion. The mixing of tropospheric air and stratospheric air via ASMO air occurs over a larger vertical range, which could imply that said distortion is indeed necessary to initiate this tracer exchange.

So far the mixing processes have been explained by considering changes in complex correlations via PCA. Since for each matched point there are measurements in both hexagon 1 and 2, the direct changes for each observed tracer can be considered as well.

Quantitative insights into the mixing processes can be derived from the autocorrelations in Fig. a–g.

The potential temperature (panel g) shows the isentropic nature of the mixing. Since the measurements were derived from a retrieval we estimate their uncertainties by applying a rigorous Monte-Carlo approach to determine the influence of different factors on the retrieval result. A detailed overview over the exact methodology can be found in and . The uncertainties are generally small compared to the variance in the measurements. Investigating the retrieval diagnostic reveals that the results are dominated by observations, with little influence of the a priori (not shown). Since H2O can condense out, it is generally not a well conserved quantity and therefore not very descriptive for the mixing processes. Such effects appear to be visible for the tropospheric water vapor, which sees significant reductions in between flights. The changes in PAN divide the matched points into two groups, the tropospheric air and everything else. It stands out that the stratospheric air, (in hexagon 2) which was created from the ASMO air (in hexagon 1) contains significantly more PAN than the other stratospheric air, further substantiating the transport of tropospheric tracers into the lower stratosphere via filaments of ASMO air. O3 shows no such structures and basically reflects the vertical structure. The potential vorticity decreases for the transition A-S and remains fairly constant otherwise. HNO3 shows significant increases within the troposphere, with the largest increases in the transition T-B, which is to be expected considering the change in class. Such increases are utterly unexpected and considering the other stratospheric tracer O3 cannot be explained by directly mixing with stratospheric air. While HNO3 shows the largest uncertainties of all retrieved species, such uncertainties cannot explain this increase.

A potential source for HNO3 within the atmosphere are lightning storms, which may have occurred in between flights. state that lightning events can lead to increases of up to 300 pptV in HNO3 compared to the background, which lies in the order of magnitude of observed increases. Observations by the GOES-18 satellites allow to link possible lightning activity in the region to the observed increases in HNO3. Figure  in Appendix B shows the cumulative lightning events between the 10 and 11 September 2023. Each white dot indicates a lightning signal picked up by the satellites. The hexagons and the trajectories of the outflow are indicated similarly to Fig. . Those trajectories, which see these significant increases in HNO3, are marked in orange. There are multiple lightning events that come close to or intersect with the trajectories. Such lighting activity may very well be the reason for the increases in HNO3. Given that such changes in HNO3 are not due to mixing, a reevaluation of these transitions assuming constant HNO3 seems reasonable. We find that most transitions remain the same, the strongest effect is in the transitions T-B, which shows 30 air parcels in this transition that would remain tropospheric air, and the transition A-B, in which 18 air parcels would remain ASMO air. The transition A-S appears largely unaffected by this. However, we cannot isolate the effect of lightning activity from changes due to mixing and therefore cannot draw any additional conclusions with certainty.