Let {zi} denote a prior ensemble of NE latent-space vectors, with zi∈RK, and let yobs∈RM denote the vector of M observations. The latent-space ensemble is mapped to the physical space through the VAE decoder D, 5xi=D(zi) with xi∈RN. The corresponding observation-space ensemble is then obtained by applying the observation operator, 6yi=H(xi)=H(D(zi)), where H is the operator mapping the physical state to the observation space. Thus, although the analysis is performed in the K-dimensional latent space, the observations are related to the latent variables through the composition of the VAE decoder and the observation operator.

The prior ensemble mean in latent space is defined as 7z¯=1NE∑i=1NEzi We define the background anomaly matrix Z′∈RK×NE, whose ith column represents the deviation of the corresponding ensemble member from the ensemble mean, zi-z¯. Similarly, we define the observation-space anomaly matrix Y′∈RM×NE, whose columns represent the deviations of the observation-space ensemble members from the corresponding ensemble mean, yi-y¯.

The Ensemble transform Kalman filter (ETKF), introduced by , is employed in this work following the formulation adopted by . The analysis ensemble is obtained by updating both the prior ensemble mean (z¯→z¯a) and the prior ensemble anomalies (Z′→Z′a). The analysis ensemble mean z¯a is expressed as a linear combination of the background ensemble anomalies, 8z¯a=z¯+Z′w, where w∈RNE is a vector of weights to be determined. In turn, the analysis anomalies Z′a are obtained by applying a transformation matrix W to the background anomalies, 9Z′a=Z′W. The weight vector w in Eq. () is obtained by solving the linear system 10Aw=Y′⊺R-1d, where R is the observation-error covariance matrix, d=yobs-y¯ is the innovation vector, and the matrix A is defined as 11A=(k-1)I+Y′⊺R-1Y′. The analysis covariance in the latent space is given by 12Pa=Z′A-1Z′⊺ In order to obtain an ensemble with this covariance, the transformation matrix W in Eq. () is constructed as a symmetric square root of (NE-1)A-1. Specifically, using the eigendecomposition A=VΛV⊺ the transformation matrix W is given by 13W=NE-1VΛ-1/2V⊺ This transformation ensures that the sample covariance of the analysis ensemble is consistent with Pa. The posterior latent ensemble members are obtained from Eqs. () and () by adding the transformed anomalies to the analysis mean. Finally, the resulting set of vectors za are then be passed through the VAE decoder to obtain the corresponding physical-space ensemble resulting from the analysis step.

Particle filtering has been successfully applied to low-dimensional models, e.g. in non-linear tracking problems , but direct application of the basic PF to high-dimensional systems is prone to weight collapse, as discussed previously. As a result, several variants have been proposed to enable the application of particle filters in geophysical systems . Since the dimensionality of the latent space is substantially lower than that of the original physical space, we explore whether particle filtering becomes a viable option for data assimilation in this reduced representation. In this work, a basic bootstrap particle filter with Gaussian kernel regularization is tested for latent-space data assimilation. The particle weights are computed from the observation likelihood, 14wi∝pyobs∣yi, and normalized to sum to one. The particles are then resampled according to their weights using systematic resampling , producing an equally weighted ensemble of size NE. When the particle weights are highly concentrated on a small number of particles, resampling can substantially reduce the diversity of the ensemble. To restore particle diversity, the resampled particles, zir, are subsequently perturbed using a Gaussian kernel, 15zia=zir+ϵi,ϵi∼N(0,Σreg), where Σreg=h2P, with P the sample covariance of the prior latent ensemble and h a prescribed regularization factor. This kernel perturbation provides a simple regularization aimed at increasing the diversity among the resampled particles.

The latest release (version 9.1) of the open-source code FALL3D was used to produce an ensemble of numerical simulations for the two experiments. The resulting datasets were then used for VAE training, testing and evaluation. FALL3D is an Eulerian model for atmospheric passive transport and deposition of different atmospheric species (e.g., volcanic ash and gases, mineral dust and radionuclides) based on the so-called Advection-Diffusion-Sedimentation (ADS) equation . The numerical model is GPU-accelerated using the OpenACC parallel programming model and has been designed to support increasingly larger scientific workloads in preparation for the transition to extreme-scale computing systems . The FALL3D model has been deployed and optimised on several EuroHPC supercomputing systems: LUMI (Finland), Leonardo (Italy), and MareNostrum-5 (Spain), covering both NVIDIA and AMD GPU architectures. FALL3D is designed to run ensemble simulations as a single, massively parallel job, allowing to maximize computational efficiency and scalability . Broadly speaking, ensemble modelling allows to characterise and quantify model uncertainties due to poorly constrained input parameters and errors in the numerical model, physical parameterisations or underlying model-driving meteorological data. In addition, the ability to generate ensemble runs makes it possible to improve forecasts by incorporating observations using different ensemble-based data assimilation techniques. The configuration of the FALL3D model for the two experiments is summarised in Table .

To define the first experiment, we used model data from the recent EU-MODEX Tenerife 2025 exercise , organised under the European Civil Protection Mechanism. This exercise represented the first large-scale volcanic eruption drill in Spain to practice the response to a potential volcanic eruption from Garachico volcano (Tenerife, Canary Islands) and test the evacuation, the social assistance and communication protocols during a volcanic crisis. On 26 September 2025, we provided real-time ash-dispersion forecasts for a hypothetical eruption scenario to support emergency decision-making. Here we use the resulting simulations to characterize the statistical properties of an augmented ensemble generated by the VAE in a controlled experiment. In this context, the drill scenario refers to a fictitious eruption defined by given emission source parameters (eruption column height, mass eruption rate, and timing), rather than being constrained by real-time observations as would happen during a real event. Since our study focuses on generating ensembles from numerical simulations and approximating their statistical distribution, the distinction between real and hypothetical eruption scenarios does not affect the methodology or conclusions. This case was deliberately selected because the ensemble simulations produce qualitatively different plume transport patterns, providing a suitable test bed to inspect the ability of the VAE to reproduce the variability and statistical characteristics of the simulated ensemble.

Three ensembles were simulated with FALL3D to produce the training, validation, and test datasets. The training ensemble, consisting of 256 members, was used to train the VAE and the validation ensemble, consisting of 64 members, was used during the training step to monitor the performance and tuning of the neural network hyper-parameters. Finally, the larger test ensemble, consisting of 2048 members, was used only at the end as a purely physics-based reference to evaluate the VAE model, providing a fair (unbiased) report of model accuracy. All numerical simulations were performed on the LUMI supercomputing system, hosted at the Finish CSC. In particular, the execution of the test ensemble required substantial HPC resources and was executed on the LUMI-G partition allocating 256 computing nodes and a total of 2048 GPUs (one per ensemble member). In all cases, a 24 h time window forecast was performed assuming a continuous ash emission starting at 09:00 UTC (see Table ), a local-scale computational domain with a horizontal resolution of 0.1° and 120 × 100 × 25 grid cells, and the FALL3D dispersal model was driven by the Global Forecast System (GFS) weather forecast from the National Centers for Environmental Prediction (NCEP). Each ensemble was simulated by perturbing Eruption Source Parameters (ESPs) and wind components, which are widely recognized as among the most sensitive parameters/inputs for volcanic ash transport models . The column height was sampled around a central value of 9 km above the vent level. Further details on the perturbed parameters and sampling ranges are provided in the Supplement.

The second experiment explores the feasibility of the latent-space DA using a VAE-generated prior ensemble for the 24 December 2018 Etna eruption . In this case, the FALL3D model was driven by the Copernicus regional reanalysis for Europe CERRA;, with 106 vertical model levels, and initialized at 10:00 UTC. An 8 h time window forecast was performed using a 128-member ensemble (model prior). An additional 48-member ensemble was simulated to be used as a validation dataset during the training process (see Table ). Each ensemble was simulated by perturbing the eruptive column-top height and wind components by 20 % (see Supplement for further details).

The 128-member model prior was used to train a new VAE, which was subsequently used to sample a 4096-members ensemble (VAE prior) for the analysis step. As a proof-of-concept, we used two-dimensional ash column-mass fields rather than the full three-dimensional concentration field, both to reduce the dimensionality of the problem and to maintain a consistent neural-network architecture throughout the study. The input training data, defined on a 256 × 128 grid (N= 131 072), was encoded into 32-dimensional latent space vectors. Before normalization, a log⁡(1+x) transformation was applied to the ash column mass (x) to reduce the strong positive skewness of its distribution, with x expressed in mg m⁻².

A single analysis step at 18:00 UTC was performed by assimilating Spinning Enhanced Visible and Infra-Red Imager (SEVIRI) volcanic ash retrievals considering two ensemble-based approaches, i.e., the ETKF (Sect. ) and the PF (Sect. ). Only positive retrievals were considered, resulting in 2678 valid observations of ash column mass. A diagonal observation-error covariance matrix was assumed, with the observation-error standard deviation set to 50 % of the retrieved ash column mass, subject to a minimum value of 100 mg m⁻², representative of a typical instrument detection limit.

A VAE was implemented in the PyTorch framework  using an architecture based on two-dimensional convolutional layers with kernel size of 3 × 3. The VAE configuration is defined by the hyperparameters reported in Table . The resulting number of trainable parameters was 698 849 and 1 783 489 for the synthetic and the Etna experiments respectively. The neural network was trained using the Adam optimizer during 400 epochs using mini-batches of 16 samples. The reconstruction loss and KL divergence (averaged over batches) were monitored at each epoch for both the training and validation datasets to prevent overfitting. Training was performed using the simulated ensembles of two-dimensional volcanic ash column-mass fields at a single forecast time. As outlined in Fig. , the encoder network maps the input data x through successive convolutional layers, followed by a flattening operation to two distinct latent space parameter vectors μ (mean) and log⁡σ2 (log-variance), with dimensions K=16 and K=32 for the synthetic and Etna experiments, respectively. The decoder then reconstructs the input from the sampled latent vector z using a fully connected layer followed by successive upsampling and convolutional operations, restoring the original spatial dimensions. For the synthetic experiment, transposed convolutional layers are used for upsampling, whereas bilinear interpolation followed by convolutional layers is used in the Etna experiment to reduce checkerboard artifacts.

A sensitivity analysis was conducted to evaluate the performance of different neural network configurations trained with the synthetic experiment datasets. Multiple VAE configurations were trained by exploring a range of values for two key hyper-parameters: the latent space dimension K and the factor β in Eq. (), while fixing the remaining hyper-parameters with the values specified in Table . For each trained VAE, an ensemble was generated by independently sampling latent vectors from the standard normal distribution and decoding them to recover the corresponding physical-space variables. In this section, each generated ensemble consisted of 2048 members.

The performance was evaluated in terms of the RMSE using Eq. (), computed between the ensemble mean of the test dataset (taken as the reference) and the means of the VAE-generated ensembles. To account for the stochastic variability of the VAE generation, 20 ensembles were generated for each set of hyper-parameters tested. The resulting RMSE values are shown as box plots in Fig. . In all cases, the network was trained for 400 epochs, except for configurations with latent dimensions of K=4 and K=2, where early stopping was required to avoid overfitting. According to the trend suggested by Fig. , the best performance is achieved for latent dimensions K≳6 and β≳6. Remarkably, performance degrades slightly for large latent space dimensions (Fig. a). There are two possible explanations for this anomaly. First, large latent spaces often produce less stable training dynamics and worse generalization: enlarging latent spaces introduces more degrees of freedom than the data can realistically support. When the dimensionality grows beyond what is needed to represent the underlying factors, the model may spread information across latent variables in an inefficient way. This leads to poor compression, weaker regularization and a latent representation that becomes harder to structure during training, limiting the extraction of compact and meaningful representations. As a result, the decoder receives a less informative latent code, reducing the reconstruction quality. On the other hand, increasing the latent dimensionality can slow convergence: since the model must explore a much larger parameter space before reaching a stable configuration, it requires more training iterations and makes optimisation more difficult.

In addition, Fig.  shows also the impact of the β parameter, which controls the strength of the KL regularization in the loss function and, therefore, the balance between reconstruction fidelity and latent-space structure. Higher values force the model to impose a tighter, more organised latent space, at the expense of losing some fine-scale details in the reconstructions. In our case, β≈6 suggests that a strongly regularised latent representation is more beneficial for the task than capturing every small feature in the input data. This indicates that the model gains more from enforcing a well-structured latent space than from preserving high-resolution details during reconstruction.

This is further supported by the structure of the latent space shown in Fig. . The nearly diagonal covariance matrix (Fig. a) indicates weak correlation between the latent components, consistent with a well-regularized latent representation. Figure b shows a two-dimensional visualization of the latent space based on the Uniform Manifold Approximation and Projection (UMAP) algorithm , a technique for non-linear dimensionality reduction specifically designed to preserve the local structure of the data. The UMAP projection shows that the test dataset (circular markers) occupies the same region of latent space than the training data (plus-shaped markers), indicating that the learned representation generalizes to unseen data, rather than only organizing the training samples. In addition, it is also worth showing how the encoding can discriminate between qualitatively different model solutions (ensemble members) by grouping them in different regions of the latent space. This is illustrated in Fig. b by means of coloured-scale dots, where members with eruption column heights higher and lower than the reference (ensemble central member) are clearly concentrated in disjoint (i.e. non-overlapping) regions of latent space. In other words, the VAE encoding implicitly produces a latent space with a clustering-like structure. These results indicate that the network was trained correctly and behaves as expected. Based on this sensitivity analysis, a latent dimension of K=16 and β=6 were adopted for the VAE configuration used in the next section.

Once the VAE neural network is properly trained, new samples (ensemble members) can be generated at negligible computational cost. This section explores whether these VAE-generated samples can reproduce the statistical characteristics of an independent and larger ensemble of numerical simulations (test dataset from the synthetic experiment), even when the new samples have not been obtained by explicitly solving the underlying physical equations. For illustrative purposes, Fig.  shows a few VAE-generated samples of two-dimensional ash column mass. The variability in the ensemble is a consequence of the different eruptive source parameters (e.g., eruptive column height) and meteorological conditions used to simulate the training dataset. For example, samples in Fig. c,d correspond to cases with high eruptive column heights and strong emission, whereas other samples are associated with low column heights and weak emission (e.g. Fig. h). In most cases, the volcanic plume is advected eastwards from the vent (16.76° W, 28.27° N). Additionally, some cases are characterised by a secondary plume with a SE- or even W-component in the transport pattern, suggesting the presence of strong wind shear that introduces variability into the ensemble and leads to qualitatively different solutions.

This section shows the results of the latent-space data assimilation for the Etna experiment (see Sect. ). Figure a shows the ensemble mean of the model prior (NE=128), i.e. the ensemble of two-dimensional ash column mass at 18:00 UTC simulated with the dispersal model FALL3D. The corresponding SEVIRI retrievals at the same time instant are shown in Fig. b. The relatively complex spatial structure of the observed plume results from a pronounced clockwise rotation (veering) of its trajectory during the initial phase of the eruption. This change in flow direction is not captured by the driving meteorological dataset, which leads FALL3D to simulate ash transport under a largely uniform directional flow. Consequently, this case represents a challenging test for the data assimilation methodologies.

In the context of this work, the VAE is viewed as a complement to the physics-based model. We showed that, making use of an ensemble of a few hundred simulations, the VAE can learn a compact, low-dimensional representation of the complex, high-dimensional variability captured by the physics-based simulations. This representation provides an alternative space in which the structure of the ensemble can be easily analysed and manipulated.

In this first work, the VAE is trained at a single forecast time to simplify the neural network architecture and avoid the need for additional layers for time encoding. This design choice reflects the complementary role of the VAE and the physics-based model: while the numerical model provides detailed temporal evolution of volcanic ash clouds, the VAE provides a robust statistical representation of the ensemble at the forecast times required for data assimilation or ensemble augmentation. By focusing on a single forecast time, the VAE can exploit the redundancy implicitly present in the ensemble of physics-based simulations to learn a low-dimensional manifold, while retaining a relatively simple and robust neural network architecture.

Sampling in the latent space to generate an augmented ensemble is the most straightforward application of this methodology. Although it does not introduce new physical knowledge, the VAE can generate a much larger number of realizations at very low computational cost during inference, while retaining variability that is statistically consistent with that captured by the physics-based simulations. It is interesting to consider whether this methodology can be computationally efficient, accounting for both training and inference, when generating ensembles much larger than the training ensemble. However, the answer is not simple as it depends on the available computational resources, the cost of the physics-based model, the total simulation time, and the ensemble size, among other factors. As an illustrative example, let's consider the Etna experiment, which is the most computationally demanding case presented here. The 128 FALL3D simulations were run on the accelerated partition of MareNostrum5 using 128 NVIDIA Hopper GPUs, requiring approximately 1 GPU-hour in total (i.e., aggregated across all GPUs). In contrast, VAE training was performed on a laptop using a single RTX 5000 Ada Generation NVIDIA GPU and required only 5.83 × 10⁻³ GPU-hours. In this experiment, the forecast period was limited to 8 h; extending the forecast period would further increase the relative cost of the numerical simulations. Most importantly, the approach substantially reduces the computational resources required to generate large ensembles; for example, the simulations for the synthetic experiment on the LUMI supercomputer required at least 2048 GPUs. Of course, these examples do not allow us to draw general conclusions about the relative efficiency of augmenting an ensemble with a VAE versus explicitly simulating larger ensembles, as the computational cost depends on the type of model and the available computational resources. However, computational efficiency is arguably not the main advantage of the proposed approach.

The methodology presented in this work also offers potential advantages beyond ensemble augmentation. Increasing the ensemble size can reduce sampling errors and improve the estimation of statistical quantities that are particularly sensitive to small ensemble sizes, such as exceedance probabilities and other measures of uncertainty. Such estimates are particularly relevant for probabilistic hazard assessment and decision-making in crisis management. For example, as shown in Sect. , the VAE provides a better approximation to the reference distribution than the training ensemble in terms of the Jensen–Shannon (JS) divergence when the ensemble size exceeds the number of numerical simulations used for training.

Additionally, due to the generative nature of the VAE, instead of simply duplicating or resampling existing numerical simulations, as in a bootstrap approach, new realizations absent from the original dataset can be generated. This could potentially lead to a richer statistical representation of the variability captured by the physics-based model, particularly if some of the generated realizations correspond to regions of the distribution that are poorly represented in the training ensemble. This interpretation is consistent with the results presented in Sect. , where the VAE-generated ensemble shows better agreement with the observations than the model prior in terms of RMSE, RMSLE, and Pearson correlation (Table ).

Finally and most importantly, the learned representation enables different operations to be performed in a substantially lower-dimensional space, including data assimilation, clustering, perturbation, conditional sampling, and other statistical techniques. These capabilities provide an excellent opportunity for further research. In this study, we have explored two latent-space data assimilation approaches that, in the formulation adopted here, would not be directly applicable in the original physical space. The assimilation experiments demonstrate the feasibility of performing data assimilation directly in the VAE latent space. Both the ETKF and PF adjust the augmented prior towards the satellite observations, improving the agreement across all considered evaluation metrics and reducing the ensemble spread without an excessive loss of ensemble variability. Although the two assimilation approaches produce markedly different posterior distributions in latent space, their resulting posterior ensembles in physical space are relatively similar. The well-known particle-collapse problem of the PF was not fully resolved, as the posterior distribution remained concentrated around two particles, resulting in a bimodal distribution after the perturbation introduced by the kernel regularization. Applying such a naive perturbation directly in the physical space for a non-linear model would result in states that are inconsistent with the model dynamics. Instead, perturbing in latent space takes advantage of the learned mapping, allowing perturbed particles to be mapped back to physical space while retaining consistency with the physics-based simulations. This clearly illustrates one of the advantages of performing operations directly in the latent space.

Overall, the results presented in Sect.  demonstrate that latent-space data assimilation can effectively incorporate satellite observations to correct a large VAE-generated ensemble of two-dimensional volcanic ash column mass. Although the present study is centred on volcanic ash applications, the proposed methodology could be naturally extended to other atmospheric modelling and forecasting problems involving the transport and dispersion of atmospheric components. Potential applications include air-quality forecasting and, particularly, modelling of smoke transport from large wildfire events. Future research is needed to assess whether the methodology can be extended to the full three-dimensional concentration field and multiple particle classes, which will require adapting the neural network architecture to the increased dimensionality of the problem. This extension would be an important step for enabling multiple data assimilation cycles and, ultimately, supporting operational forecasting.