The ACCESS-AM2 model is an atmosphere-only configuration of the ACCESS-CM2 coupled climate model and is described in . The atmospheric component of ACCESS is the Unified Model (UM) at vn10.6, GA7.1, which is fully described in . In brief, the radiation scheme used in ACCESS-AM2 is the Suite Of Community RAdiative Transfer codes based on Edwards and Slingo (SOCRATES) and is called hourly. The prognostic cloud fraction and condensate (PC2) cloud scheme is used , which includes large-scale and convective clouds. The convective scheme, including downdraughts and momentum transport, is based on . Further details on ACCESS-CM2 and GA7.1 configurations can be found in and . In the atmosphere-only configuration, we prescribe sea surface temperature (SST) and sea ice concentration (SIC) following the CMIP6 AMIP protocol . The input fields can be found at the input4MIPs web page (https://esgf-node.llnl.gov/projects/input4mips/, last access: 4 March 2021), and the data method is described in . Solar forcing, greenhouse gases, volcanic aerosol optical depth, aerosol chemistry emissions and ozone are also prescribed according to CMIP6 and are available at the input4MIPs site.

ACCESS-AM2 includes the GLOMAP-mode (Global Model of Aerosol Processes modal version) aerosol microphysical scheme , including parameterised sulfur chemistry driven by prescribed oxidants. GLOMAP-mode is a two-moment, pseudo-modal microphysical aerosol scheme, representing four soluble aerosol modes (nucleation, Aitken, accumulation and coarse) and one insoluble aerosol mode (Aitken). Each mode is internally mixed. GLOMAP uses a scaled (1.7×, as recommended in ) dimethyl sulfide surface water monthly climatology combined with the flux parameterisation (see also ). Sea salt emissions are calculated online and occur following the parameterisation. Dust is treated separately according to , using a six-bin scheme.

Historical (pre-2014) anthropogenic aerosol emissions are provided by the Community Emissions Data System , and biomass burning emissions are provided by the Global Fire Emissions Database with small fires (GFED4s) . Post-2014, the shared socioeconomic pathway (SSP) 2-4.5, a “middle of the road emissions pathway” , is used, with emissions developed by the Integrated Assessment Modeling Consortium and which is described in .

The land surface scheme in ACCESS-AM2 is the Community Atmosphere Biosphere Land Exchange (CABLE) version 2.5 land surface model (also described in ). ACCESS-AM2 runs at a 1.25 × 1.875∘ horizontal resolution with 85 vertical levels and uses the “ENDGame” dynamical core . Further information on the general model set-up and preliminary model evaluation against standard climate fields, such as surface air temperatures, sea surface temperature, rainfall, mean sea level pressure (MSLP) and precipitation, can be found in and .

The simulation presented in this study has been nudged to the European Centre for Medium-Range Weather Forecasts (ECMWF) Reanalysis 5 (ERA5) product . Nudging occurs at every dynamical time step from reanalysis fields that are updated every 3 h for the horizontal winds and temperature in the free troposphere and stratosphere. In this study, output is created at monthly and daily mean resolution for the years 2015–2019 (chosen to overlap with recent Southern Ocean field campaigns described in ).

Simulated cloud fields are evaluated against the Moderate Resolution Imaging Spectroradiometer (MODIS) combined Aqua and Terra, Level 3 daily, 1 × 1∘ grid, Collection 6.1 COSP product (MCD06COSP_D3_MODIS) . This product has been derived specifically for use in model evaluation using the COSP outputs for CMIP6 and is described in . Available properties include the cloud optical depth (τ) for total, ice and liquid clouds; cloud top pressure (CTP); cloud mask fraction (derived from pixel-level cloudiness assessments) for total, low, middle and high clouds; cloud retrieval fraction (derived from the successful retrieval of τ) for total, ice and liquid clouds; cloud effective particle radius (Reff) for ice and liquid clouds; liquid water path (LWP); ice water path (IWP); and joint histogram for CTP and τ for cloudy and partly cloudy pixels. All cloud properties in the MCD06COSP_D3_MODIS product are for daytime-only scenes. Full descriptions of these cloud properties can be found in , and .

Collection 6 now includes partly cloudy scenes that represent heterogeneous broken cloudy or cloud edge pixels . Successfully retrieved cloudy pixels are considered to be high quality, whereas partly cloudy scenes have a higher rate of retrieval failure of 34 % and are slightly less robust . Collection 6 has shown a marked improvement upon Collection 5 in part due to the re-writing of the cloud optical property retrievals, resulting in an increase in cloud-phase classification of 10 % and a 90 % agreement in total cloud phases between MODIS and the CALIOP (Cloud-Aerosol Lidar With Orthogonal Polarization) retrievals for cloudy pixels . The largest improvement in cloud-phase detection was found for opaque clouds over ice- or snow-covered areas, whilst detection for thin cloud retrievals over warm or bright surfaces remains an issue. Improved optical retrievals have also reduced the biases in Reff; however, it has been noted that much of the evaluation performed has been for single-layered clouds, with multi-layered clouds remaining un-assessed .

Comparisons of MODIS retrievals to aircraft field campaigns over the north-eastern and south-eastern Pacific, as well as ground-based observations in Finland, have indicated that liquid Reff is overestimated, which impacts the LWP retrievals . However, a number of studies have also noted that there is good agreement in variability between liquid Reff and the LWP . The cloud retrieval fraction and cloud optical depth have been found to perform better and be strongly correlated with field observations . Ice cloud retrievals have been known to be more difficult for a number of reasons relating to crystal shape and scattering properties as well as lifetime and expanse. Evaluation of the MODIS Collection 5 ice retrievals found an overestimation compared to an infrared radiative closure method for determining ice τ . The updates to the optical retrieval method in Collection 6 have reduced the bias to a more satisfactory level. We note that the evaluations discussed above have not been performed for the specific COSP product we are using in this work but give us an idea of the MODIS satellite retrievals' overall performance.

In this work, we are using the cloud retrieval fraction, which we subsequently refer to as the cloud fraction (CF), for ice (CFI) and liquid (CFL) clouds and the joint histogram for CTP and τ for cloudy and partly cloudy pixels. Following the methods of , and , we use the combined joint histogram product (i.e. the sum of the cloudy and partly cloudy products). Very large biases that we considered unrealistic were found for the modelled Reff, and hence these fields have not been used for this work. Similar biases were found for the LWP and IWP, and hence, the COSP-derived products (described in the next section) for these fields were not used but were replaced with the raw model output. While this adds a degree of uncertainty to this work, we believe such an analysis with the derived COSP fields would not have been useful.

Furthermore, care has been taken to ensure that we are comparing only grid-box mean values of the LWP and IWP. Model outputs (both COSP and raw) are provided directly as grid-box means, while the MODIS products are provided as in-cloud mean values. We have performed the appropriate conversions where the grid-box mean is equal to the in-cloud mean multiplied by the cloud fraction. After comparison of both grid-box mean and in-cloud mean values, we have chosen to use the grid-box mean values. This choice does effect some of the results of this study, as would choosing to use the in-cloud values instead. However, we believe this choice is robust for two reasons: firstly, grid-box means are the native model output and this is a model evaluation study and, secondly, the grid-box means showed a better model performance than the in-cloud mean values, likely due to the weighting of the cloud fraction.

For this analysis we have removed all instances of clear sky from both the model and the observed histograms. We have done this by finding the instances where the CTP–τ histograms summed to zero. While this allows us to focus on instances when the model is simulating cloud, either correctly or incorrectly, it also means we are not considering instances when either the model or the satellite simulated cloud while the other simulated clear sky. In this work, we are considering grid boxes of 1.25 × 1.875∘ and daily means, meaning that there are very few instances of clear sky occurring for a full day over a large domain. On average, there are fewer than 1.5 % of grid boxes simulated as clear sky in ACCESS-AM2 over the Southern Ocean and far fewer in the satellite (in part due to the addition of the partly cloudy optical depth retrievals as well). For these reasons, we believe this choice is robust.

To evaluate the radiative bias in ACCESS-AM2, we use the Clouds and the Earth’s Radiant Energy System (CERES) SYN1deg product for evaluation at daily timescales. The bias in the outgoing top-of-atmosphere (TOA) shortwave (SW) cloud radiative effect (CRE) (SWCRETOA) has been chosen for analysis in this work based on previous findings from the literature presenting the SWCRE as the most problematic aspect of the SO energy balance. The CRE presented in this work is the difference between the clear-sky radiation and the all-sky radiation fields (for both the model and the satellite products). Throughout this paper, a predominantly positive SWCRETOA is present; e.g. the ACCESS-AM2 model is allowing too much shortwave radiation to pass through the clouds and not reflecting enough shortwave radiation out to space via clouds, corresponding to too much shortwave radiation reaching the surface.

To directly compare the ACCESS-AM2 cloud properties to those of satellite products, we use the COSP, described by , as prescribed for simulations within the CFMIP activity of CMIP6 . The MODIS products provided the best coverage for the time period of interest to this study. For this reason, no other satellite products are considered in this work. Simulated fields include the joint histograms of CTP and τ, as well as liquid and ice cloud fractions, water paths, and τ. Using COSP output allows an appropriate comparison of the model to satellite products. This method applies the assumptions and limitations of the satellite algorithms to the model output, limiting the possibility that biases are due to differences in processing. For the LWP and IWP, we are considering the grid-box mean (as opposed to the in-cloud mean). We note that in the IWP and LWP fields, significantly large and seemingly unrealistic biases between the model COSP product and the MODIS product were found. The IWP and LWP are reliant on Reff retrievals, which as discussed above, are less well captured in satellite products. For this reason, as stated above, the actual simulated (i.e. raw model output) IWP and LWP are used for this analysis instead of the COSP-derived product.

In this study, we use k-means clustering to derive cloud regimes . k-means clustering is a form of unsupervised machine learning that separates N points into k clusters by minimising the sum of squared distances within each cluster. In this case, the Euclidean distance is considered, which is equivalent to the minimisation of variance (or inertia) within each cluster. To perform this analysis, the scikit-learn and Dask-ML Python packages were used, where k-means clustering has been implemented for distributed computing.

We apply k-means clustering to 4 years (2015–2019) of daily MODIS histogram data (CTP–τ), over the entire globe. We have normalised the CTP–τ histograms to 1 (as opposed to the cloud fraction) to limit the impact of cloud fraction biases within the ACCESS-AM2 model on the clustering results. Whilst this impacts our ability to compare our results to other studies, it allows the clustering to target cloud vertical extent and thickness regardless of the total fraction and how well it is captured by the ACCESS-AM2 model.

A complication of clustering methods, including k-means clustering, is the choice in the number of clusters. While there is no right or wrong answer to the number of clusters to select, there are practical considerations and statistical metrics that can help guide this choice. Three statistical metrics were applied to this work in an attempt to aid the decision on the number of clusters to choose, including the “elbow” method , the Calinski–Harabasz (CH) index and the Davies–Bouldin (DB) index . The scikit-learn application programming interface provides a detailed explanation of each metric in addition to their advantages and disadvantages. Unfortunately, the statistical guidance provided by these metrics was not useful in cluster number selection (suggesting 2, 4 and 17 clusters respectively). After consideration of a range of choices, we selected 12 clusters.

We then used the MODIS cluster regimes to predict the cloud regimes of ACCESS-AM2, by fitting each ACCESS-AM2 data point to 1 of the 12 cluster centres. We chose this method, as opposed to a hybrid approach as taken by , so that we could apply the same cluster centres to multiple model simulations, allowing a direct comparison over a number of simulations. A similar method is described in , where they also point out that this method eliminates further subjective choices.

We have chosen to perform clustering over the entire globe, though only results for the SO (defined for this work as the broad region from 30–69∘ S) are examined in this current study. This choice will allow us to assess if any changes applied to the model focused on improving Southern Ocean clouds have unintended effects outside this region. For this reason, while the cloud regime histograms are defined globally, only those important to the SO will be examined in detail.

To explore cloud property biases, we follow the decomposed bias metrics described in , , and , where the bias (Δ: ACCESS-AM2 - satellite) of a simulated field of interest F for each cluster r can be summarised as the errors due to the relative frequency of occurrence (RFO; Frsat⋅ΔRFOr, referred to as RFO errors) plus the errors in the simulated field (ΔFr⋅RFOrsat, referred to as field errors) plus a second-order co-variation term (ΔFr⋅ΔRFOr) otherwise referred to as the cross term. 1δFr=Frsat⋅ΔRFOr+ΔFr⋅RFOrsat+ΔFr⋅ΔRFOr

The k-means clustering technique was applied to 5 years of MODIS daily-mean joint histograms over the entire globe. The 12 resulting cluster centres are shown in Fig. . We have not shown the global RFO but have chosen to show the RFO for the Southern Hemisphere only in Fig.  to focus on the Southern Ocean domain (30–69∘ S). The clusters have been approximately arranged from low to high CTP along the vertical and from thin to thick τ in the horizontal.

Three of the resulting clusters are limited to the Antarctic region and have similar (yet accordingly distinct) cloud characteristics. These clusters have come about in part as a result of our normalisation by 1, instead of the cloud fraction, and make up only a small fraction of overall cloud occurrence. Furthermore, as noted by , cloud retrievals over ice-covered regions with high albedo can be problematic. For these reasons, the three Antarctic clusters are merged into one for the remainder of this analysis. We have labelled the remaining cloud regimes according to the cloud types that best reflect these profiles; however we acknowledge that CTP–τ is often not enough to truly distinguish one cloud type from another (and that some quite different clouds may have similar CTP–τ profiles). Hence, these labels should be considered just that: a way of easily differentiating the cloud regimes in this study.

Along the top row of Figs.  and , high-level clouds are shown. The thin cirrus cluster is restricted to tropical regions, particularly over the western Pacific and eastern Indian oceans, and is characterised by clouds that are optically thin with very low CTP. Similarly, the cirrus cluster is characterised by very optically thin clouds, at a slightly lower altitude, and is found predominantly in the tropical and mid-latitude regions. Both the cirrus and thin cirrus show some thin, low-level clouds too, likely associated with shallow convective clouds. At a similar height, the convective cluster is more optically thick and more strongly associated with the ITCZ and South Pacific Convergence Zone (SPCZ), although this is not shown by this map projection. Finally, along the top row, the frontal cluster, while similar in altitude to the convective cluster, is more optically thick and is found more predominantly in the mid-latitudes, in particular around westerly storm tracks. We note that convective and frontal clouds can have similar CTP–τ profiles , and hence their location is also important to consider.

The mid-level cluster shows mid-range optical properties as well as mid-level CTPs and is without one or two clearly dominant bins as found in the other clusters (Fig. ). This mid-level cluster is most common over the SO, marine equatorial regions and terrestrial regions (in particular on the west coasts of a number of continents). These varied geographic locations represent some significantly different cloud formation processes, raising an important point for this work: while the optical and height characteristics within a cluster may be similar, each cloud's trigger mechanism is not necessarily the same. While further analysis to understand the formation processes for the cloud regimes is possible via meteorological fields such as proximity to cyclone centres or vertical motion or via physical fields such as proximity to topography and would be of interest, such analysis is out of scope for this work.

Along the bottom row, the shallow cumulus cluster is found to have low, optically thin clouds that occur mostly over marine regions in the tropical latitudes and mid-latitudes, away from any large-scale cloud formations such as in the ITCZ/SPCZ (not shown) or stratiform cloud decks. This cluster has little vertical structure. By comparison, the cloud deck cluster is optically thicker and dominant at the eastern boundary of ocean basins, regions of extended stratiform cloud, again with little vertical structure. The marine stratiform cluster is the most optically thick cloud regime, with high CTPs and little vertical structure, and is found predominantly in higher-latitude marine regions, as well as of the very eastern boundaries of the cloud deck regions. Finally, the stratocumulus cluster is characterised by mid-range optical thickness but has lower CTPs than those of the other low-cloud clusters, suggesting a greater vertical extent associated with stratocumulus clouds. This cluster is geographically widespread, although dominant in the polar region of the SO and absent in regions of deep tropical convection.

Using the cluster centres defined by the MODIS joint histograms described above, we fit the ACCESS-AM2 joint histograms to the same 12 cluster definitions (noting that the 3 Antarctic clusters were subsequently merged). Figure  shows the difference in frequency of occurrence between ACCESS-AM2 and MODIS. It is immediately clear that ACCESS-AM2 has significant problems with the mid-level and stratocumulus clouds, simulating them too frequently (by 13.4 % and 19.8 %) across all regions. The low-level cloud fields by contrast are all underestimated, including shallow cumulus (-9.9 %), cloud decks (-9.5 %) and marine stratiform clouds (-10.0 %). The higher clouds (thin cirrus, cirrus, convective and frontal) are simulated comparatively well, although also with slight underestimation of frequency. Interestingly, for the SO, most of the RFO biases are spatially consistent in sign. This result could be interpreted as a consistent bias, with consistent causes, across the latitudes. We will explore this in more detail in the subsequent sections.

We can now use the biases in the frequency of occurrence of the cloud regimes to help gain a better understanding of the biases in radiation using the traditional decomposition method described in Sect. . Note that from this point on, we only consider the broad SO region and the three sub-regions defined within. Figure  shows the cumulative components of the SWCRETOA biases over each region, for each cloud regime. The horizontal bar represents the total bias. Similarly to previous studies, including , and , the largest component of the SWCRETOA bias is due to the errors in the RFO (blue bars). As previously discussed, DJF has the largest SWCRETOA biases, though the dominance of the RFO bias is true for all seasons. For DJF, clear negative RFO errors are found for the mid-level clouds and the stratocumulus clouds, which are both overestimated in frequency. This negative RFO error has a compensating effect, as the mid-level clouds are simulated in the place of other lower-cloud regimes (e.g. marine stratiform, cloud decks and shallow cumulus), which are underestimated in frequency and hence have positive RFO errors. A much smaller component of the bias is made up by the error in the simulated field (pink bars), primarily in DJF, while this is insignificant in the other seasons. hypothesised that the dominance of the RFO errors and comparatively smaller field errors, also found here, indicate that the clouds themselves, when simulated with the correct frequency, could in fact be simulated well. This hypothesis will be tested in the next section. The cross term (green bars) appears to contribute to the radiative biases again only in summer and predominantly in the polar region. What this cross term represents is not easy to evaluate; however it is clear that the field biases and RFO biases for this time and season can only explain part of the story.

We have also applied this bias decomposition technique to other cloud properties, including biases in the LWP, IWP, and CFI and CFL (see Appendix). The results for the LWP, IWP and CFL, for all regions, show large biases across all cloud clusters and decomposed terms. Interestingly, the field error and RFO error are of opposite signs and approximately cancel out, leaving the cross term to make up the total radiative bias. For the CFI, the RFO error was dominant, similarly to the SWCRETOA, though with the field error and cross term being non-negligible, unlike for the SWCRETOA. The importance of the cross term in these results makes the respective biases much more difficult to understand, though it is clear that the decomposed errors in RFO and the fields themselves are playing a large role in the total bias.

The strength in this method is that it is time and space agnostic; i.e. it does not require the two “climates” to be directly comparable as it is using the mean RFOs and field values to make judgements about how each of them contribute to a particular bias. For this reason, this method has been popular in the literature, whose authors have often used CMIP simulations as the basis of their work. However, a limitation of this method is that it cannot compare like-for-like instances in cloud properties to gain a better understanding of what particular conditions are leading to large SWCRETOA biases. This work suggests that for all seasons and all regions, a negative bias occurs because the frequency of mid-level and stratocumulus clouds are overestimated, compensating for the underestimation of the lower-level clouds. We ask at what cost, specifically, were the mid-level and stratocumulus clouds overestimated? Is the compensating error due to the model predicting clouds that are too frequent, optically thick, high or with too much liquid? Applying the same technique (the bias decomposition) to such cloud fields was unable to satisfactorily give us these answers. Additionally, we would like to know whether the model ever actually does a good job, with the correct cloud type and a small (or wishfully no) radiative bias.

One strength of comparing a daily, nudged simulation to daily MODIS fields is the ability to make direct comparisons in time and space. The synoptic meteorology is considered to be the same in the model and the observed conditions, due to the nudging of the model. We therefore expect that the model microphysics, if accurate, would generate the same cloud types that the large-scale dynamics prescribes. With this assumption, we are able to isolate instances (exact points in time and space) where the model simulates the same cloud type as MODIS, which we define as “correctly” simulating the cloud type. Similarly, we can also define the instances where the model simulates a different cloud type, which we consider an “incorrect” cloud type assignment by the model. We can then use these instances to generate a statistical understanding of the conditions during these times and places. We demonstrate these definitions in Fig. a and b.

In the previous section, we saw that the model generally tends to simulate the cloud type RFOs incorrectly and that this bias in RFOs dominates the radiative bias. This result demonstrates the limit of the decomposition method as we are not able to gain any further information about how the microphysical biases in cloud types affect the radiative properties or about the nuances of which cloud types are being simulated in place of others. In this section, we take advantage of the model nudging and explore in detail the instances when ACCESS-AM2 correctly and incorrectly simulates the MODIS-identified cloud regimes and begin to understand the associated radiative and cloud biases in much greater detail than has previously been achieved.

Each panel of Fig.  shows, for each MODIS cloud type (y axis), the percent of time that each cloud type is assigned by ACCESS-AM2 (x axis). The total number of instances that cloud type is observed by MODIS for each region is shown on the right of each panel. If ACCESS-AM2 simulated every cloud type to be the same as what the MODIS product did, we would expect a diagonal line through each panel in Fig.  of 100 %, with zeros elsewhere. What is shown, however, reflects clearly the biases of the RFOs, where ACCESS incorrectly assigns the stratocumulus and mid-level clusters over most other clusters. For the low-level cloud regimes (shallow cumulus, cloud decks and marine stratiform), we see that a large number (between 48 % and 59 %) of points have been wrongly assigned to the stratocumulus cluster, while the high-level clouds (convective and frontal, as well as the less important cirrus and thin cirrus) tend to be assigned as the mid-level cluster (between 38 % and 67 %). Only 12 %, 11 % and 5 % of the marine stratiform points have been correctly assigned by ACCESS-AM2 for each region (mid-latitudes, sub-polar and polar). This statistic is worse for some other less important SO cloud types. In contrast, 63 %, 68 % and 68 % of mid-level clouds and 60 %, 66 % and 56 % of stratocumulus clouds are simulated in the correct time and place for the three SO regions (mid-latitudes, sub-polar and polar).

Having identified exactly when ACCESS-AM2 correctly or incorrectly simulates cloud regimes, we can now begin to evaluate what the biases in radiation and other cloud fields are in these instances. Figures  (mid-latitude), (sub-polar) and (polar) show seasonal mean biases when the ACCESS-AM2 correctly simulates each cloud cluster (coloured bars) or incorrectly simulates each cloud cluster (transparent bars with black outlines). An example of how one of these subsets (polar mid-level clouds) is derived is shown in Fig. . We note that these results are not weighted by the frequency of occurrence, unlike in the previous figure, as we are interested in understanding the microphysical properties of each cloud type, even if they occur infrequently. Figure  can provide an indication of the frequency of each cloud type and how often it is correctly or incorrectly identified. Furthermore, we emphasise that these plots show the impact on the radiation and cloud fields from the perspective of the right cluster being assigned wrongly, unlike in Fig. , which shows the radiative biases from the perspective of the accumulative wrong clusters. For this reason, some of the values between the figures have opposite signs but support the same overall finding.