Opinion: New directions in atmospheric research offered by research infrastructures combined with open and data-intensive science

The acquisition and dissemination of essential information for understanding global biogeochemical interactions between the atmosphere and ecosystems and how climate–ecosystem feedback loops may change atmospheric composition in the future comprise a fundamental prerequisite for societal resilience in the face of climate change. In particular, the detection of trends and seasonality in the abundance of greenhouse gases and short-lived climate-active atmospheric constituents is an important aspect of climate science. Therefore, easy and fast access to reliable, long-term, and high-quality observational environmental data is recognised as fundamental to research and the development of environmental forecasting and assessment services. In our opinion article, we discuss the potential role that environmental research infrastructures in Europe (ENVRI RIs) can play in the context of an integrated global observation system. In particular, we focus on the role of the atmosphere-centred research infrastructures ACTRIS (Aerosol, Clouds and Trace Gases Research Infrastructure), IAGOS (In-service Aircraft for a Global Observing System), and ICOS (Integrated Carbon Observation System), also referred to as ATMO-RIs, with their capabilities for standardised collection and provision of long-term and high-quality observational data, complemented by rich metadata. The ATMO-RIs provide data through open access and offer data interoperability across different research fields including all fields of environmental sciences and beyond. As a result of these capabilities in data collection and provision, we elaborate on the novel research opportunities in atmospheric sciences which arise from the combination of open-access and interoperable observational data, tools, and technologies offered by data-intensive science and the emerging collaboration platform ENVRI-Hub, hosted by the European Open Science Cloud (EOSC).

1 Rationale

Beyond doubt, we have entered the Anthropocene (Steffen et al., 2011) and are now in a period where humankind is the main determinant in the fate of the planet Earth. Natural as well as anthropogenic factors lead to environmental changes on all scales from local to global. Documenting and quantifying these changes are necessary requirements for advancing our scientific understanding of Earth and its environment, including its complex feedback mechanisms. Such documentation and quantification are also needed for developing mitigation and adaptation strategies, for fact-based decision-making, and for the development of environment-friendly innovations. Furthermore, reliable predictions of environmental change must be based on trustworthy, well-documented, observational data which capture the entire complexity of the Earth system and the manifold interactions between the atmosphere, the land, and the ocean and impacts from and on life in all its forms. Ultimately, observational data from all segments of the planet Earth and its environment provide the scientific basis for analysing the physical, biological, and economic processes on our planet which are affecting all sectors of society as well as wildlife and biodiversity (IPCC, 2021a, 2023).

The requirements for a global climate observing system are now widely discussed. The Global Climate Observing System (GCOS, 2010, 2016, 2022b) international programme, co-sponsored by the World Meteorological Organization (WMO), the Intergovernmental Oceanographic Commission of the United Nations Educational, Scientific and Cultural Organization (IOC-UNESCO), the United Nations Environment Programme (UN Environment), and the International Science Council (ISC), introduced “essential climate variables” (ECVs) to provide a science-based framework for coordinated climate observations; see more on that topic in Sect. A2 in the Appendix. Founded on this framework, Weatherhead et al. (2018) proposed one potential approach towards the observing system of the future and noted that for each grand challenge identified by the World Climate Research Programme (WCRP, 2024), observations are needed for long-term monitoring, process studies, and forecasting capabilities. The proposed observing system should include satellite-borne instruments and ground-based and airborne in situ observations as well as potentially new, unidentified observational approaches.

In recent contributions, Carmichael et al. (2023) from the WMO Global Atmosphere Watch (WMO GAW) programme and Kulmala and co-workers (Kulmala, 2018; Kulmala et al., 2023a, b) discuss critically the requirements for a long-term sustained and research-driven global observation infrastructure for atmospheric composition. Carmichael and co-authors identify an “urgent need to move from opportunities-driven one-component observations to more systematic, planned multifunctional infrastructure, where the observational data flow is coupled with Earth system models to serve both operational and research purposes”. Despite their different foci, all articles conclude that comprehensive and open data are essential to meet environmental grand challenges.

To cope with the scale and complexity of the threats posed to our planet by climate change, environmental sciences need to integrate excellent curiosity-driven science in the traditional fields of Earth system sciences with science focusing on human interactions and societal grand challenges. The task is huge, but the means are there. The Strategy Working Group on Environment of the European Strategy Forum on Research Infrastructures (ESFRI) has identified in their recent landscape analysis of the environment domain (ESFRI, 2021a) that European environmental research infrastructures (ENVRI RIs) in atmospheric sciences, jointly with the other branches of environmental sciences, are well positioned to tackle the challenges of climate change at all levels.

In our opinion article, we discuss the potential role that atmosphere-centred ENVRI RIs in Europe can play with their capabilities for standardised collection of observational data complemented by rich metadata and their provision through open access. As a result of these capabilities in data collection and provision of data and services, we elaborate on the novel research opportunities and methods that arise from the combination of open-access and interoperable environmental data and the tools and technologies offered by data-intensive science. It should be noted that the focus of our opinion article is on atmospheric sciences and the atmosphere-centred research infrastructures (ATMO-RIs) providing continuous data on atmospheric composition from a European-wide network of surface-based observation stations (ACTRIS, ICOS) and a fleet of passenger aircraft equipped with scientific instrumentation on board (IAGOS). However, most of the conclusions drawn apply to other areas of environmental research as well.

2 Setting the stage

2.1 Atmosphere-centred research infrastructures in Europe

Recognising that the in-depth investigation of the effects of climate change and their impacts on human health, food security, and other aspects of the United Nations Sustainable Development Goals (SDG, 2024) requires a critical mass in terms of scientific capacities and resources, the European Commission has initiated the development and implementation of environmental research infrastructures at the European level. In the early 2000s, ESFRI started a roadmap process to establish a comprehensive landscape of pan-European environmental research infrastructures (ESFRI, 2021b).

The atmosphere-centred ESFRI environmental research infrastructures include the Aerosol, Clouds and Trace Gases Research Infrastructure (ACTRIS); the In-service Aircraft for a Global Observing System (IAGOS); and the Integrated Carbon Observation System (ICOS) Atmosphere Thematic Centre. As is illustrated in Fig. 1, the ATMO-RIs cover the main contributors to the observed warming (IPCC, 2021a). In the Appendix to this opinion article, we describe in detail the atmosphere-centred RIs and their achievements; also visit the website of the community of environmental research infrastructures (ENVRI, 2024) for further information.

Figure 1 Coverage of the main contributors to the observed warming; only the variable “Land-use reflectance and irrigation” is not covered by the ATMO-RIs; the figure is adapted from Fig. SPM.2 in “Summary for Policymakers” (IPCC, 2021b).

The operational ATMO-RIs are comprehensively prepared to assume their role in this important field of cross-disciplinary research, jointly with life sciences and social sciences. The currently ongoing European research projects entitled Research Infrastructures Services Reinforcing Air Quality Monitoring Capacities in European Urban & Industrial AreaS (RI-URBANS, 2024) and ICOS Cities (ICOS-Cities, 2024) are driving the development and implementation of air quality and greenhouse gas monitoring systems that will allow European cities as well as health administrations and agencies to effectively mitigate the impact of poor air quality on human health and to support climate action.

2.2 Impact of cross-disciplinary research supported by ATMO-RIs

The scientific power of this kind of production and use of environmental data is impressively demonstrated by the series of reports published by the Intergovernmental Panel on Climate Change (IPCC); see the Sixth Assessment Report (IPCC, 2021a) as its latest product. The urgency of further integrating scientific, political and societal efforts in view of the rapidly advancing climate change is presented in the most recent IPCC Synthesis Report (IPCC, 2023).

The scientific importance of long-term greenhouse gas observations as they have been conducted for many years by ICOS and, more recently, by IAGOS has been confirmed in the scientific background document on the 2021 Nobel Prize in Physics, which went to Klaus Hasselmann, Syukuro Wanabe, and Giorgio Parisi “For groundbreaking contributions to our understanding of complex physical systems”. In Chap. IV, the Nobel Committee writes, “From the perspective of laboratory science, using experimental measurements to test theory is such a self-evident step in the scientific method that it goes without saying. However physical cosmology and physical climatology are observational sciences – practitioners observe that which nature allows”.

Moreover, failure to reduce greenhouse gas emissions and concentrations over the next 2 decades is perhaps the greatest threat to the future health of human society and ecosystems. The role of the ATMO-RIs is critical to achieving efficient, effective, and verifiable global emission reductions, particularly for the control of emission reductions required as part of the COP21 Paris Agreement of 2015. The mitigation measures and the speed of their implementation need to be checked by independent methods and closely monitored, while the influence of natural feedback due to ongoing climate change will require attention, as this may force a change in the speed of implementation of mitigation measures and adaptation. The contribution of ATMO-RIs to the validation exercise of emission reductions is described by Vermeulen et al. (2020).

Another success story of the cross-discipline use of atmospheric observational data is the Copernicus Atmosphere Monitoring Service (CAMS; Peuch et al., 2022). CAMS delivers numerous global and regional information products about air quality, inventory-based emissions, and observation-based surface fluxes of greenhouse gases and about biomass burning, solar energy, ozone and UV radiation, and climate forcings, with the atmosphere-centred research infrastructures being one of its prime data and service contributors.

Besides research on climate change, the scientific field of atmospheric chemistry with its links to climate change, as well as the resulting interaction of air quality and human health, constitutes another highly interdisciplinary research area in the current agenda of atmospheric sciences (von Schneidemesser et al., 2015; The Lancet Planetary, 2020; Romanello et al., 2022). A number of recent post-COVID-19 studies on the effects of reduced anthropogenic emissions on atmospheric chemical composition (e.g. Clark et al., 2021; Evangeliou et al., 2021; Gkatzelis et al., 2021; Sokhi et al., 2021) impressively illustrate the growing need for, and the great potential of, scientific approaches that cross the traditional boundaries of scientific disciplines and methods. The studies utilise long-term and high-quality observations of atmospheric composition on a global scale, combined with statistical approaches, for identifying patterns in atmospheric composition change which can be traced back to modified emission behaviours in different parts of the world.

3 Entering the era of data-intensive science by integrating atmospheric sciences, data science, and open science

3.1 Challenges of data-intensive science

The production of scientific knowledge has developed from experimental sciences, by conducting field observations and experiments, and through theoretical approaches using models and mathematical generalisations to simulate phenomena which are too complex for analytical solutions (Gray, 2009). Now we have entered the age of the fourth paradigm of data-intensive science, which is described as collaborative, networked, and data-driven (Bell et al., 2009; Lynch, 2009).

European environmental research infrastructures are perfectly prepared for the challenges of data-intensive science, since they combine research-driven observations with comprehensive capabilities for sustainable data management and stewardship. Astronomy and high-energy physics can serve as role models for this kind of infrastructure (Bell et al., 2009) but with the essential difference that in the majority of cases environmental observations require the operation of widely distributed measurement networks together with the handling of complex data streams from highly diverse sources and with a need for short-term data availability.

The integration of data-intensive sciences and environmental sciences is critical for improving our current understanding of the major challenges to our planet, our ability to respond to and predict natural hazards, and our understanding and prevention of ecosystem loss. This effort requires the unimpeded use of multidisciplinary data from the different spheres of the Earth system, with findability, accessibility, interoperability, and reusability of the digital (meta)data (FAIR principles; Wilkinson et al., 2016) as an absolute prerequisite, embedded in a culture of open scientific cooperation as stated by the European Commission in their European Open Science Cloud (EOSC) Declaration (EC, 2017) and outlined by the Organisation for Economic Co-operation and Development (OECD; Dai et al., 2018).

We would like to recall that the research problems behind environmental and societal challenges such as climate change, food security, and natural disasters are intrinsically interdisciplinary. Therefore, the integration of data-intensive sciences and environmental sciences is crucial to improve our skills to create multidisciplinary and cross-domain scientific knowledge which is needed to act on the grand challenges facing our planet and our societies. Modelling these processes individually is difficult enough, but modelling their interactions is complex by another order of magnitude; see, for example, Bauer et al. (2021) for a discussion of the challenges and opportunities of the digital revolution of Earth system science, using the example of reliable weather forecast and climate predictions.

Among the indispensable prerequisites for data interoperability is the organisation of data management systems at each RI in compliance with the FAIR principles. The entire ENVRI cluster has organised the implementation of the respective FAIR enabling resources along the FAIR pyramid proposed by Bailo (2019); see Fig. 2 for details. As can be seen in Fig. 2, huge and comprehensive efforts are required to accomplish FAIR scientific data management. Most of the FAIR enabling resources need to be implemented at the level of data and metadata management to comply with the respective FAIR principles. Once this foundation has been set up properly, access to science-based services and the use of services can be realised. The RIs of the ENVRI cluster, including the contributing ATMO-RIs, walked this arduous road during the European Commission cluster project ENVRI-FAIR (2019–2023). The reward at the end of the road was a previously unattainable level of data interoperability among the ATMO-RIs which now allows the development and provision of cross-RI scientific services for the scientific community.

Figure 2 The four-stage FAIR roadmap – FAIR pyramid; adapted from Bailo (2019).

In today's atmospheric sciences and other research areas, scientists are challenged to collaborate across conventional disciplinary boundaries. However, first they must discover and extract data that are dispersed across many different sources and in many different formats. For this step, a data management system based on the FAIR principles will play a crucial role. In addition to this, effective research support environments are needed for various user-centralised research activities, from formulating research problems to designing experiments, discovering data and services, executing workflows, and analysing and finally publishing the results (Zhao and Hellström, 2020). Particularly, the concept of scientific workflows has developed into a powerful tool over the past decade (Atkinson et al., 2017). Scientific workflows provide a systematic way of describing data analysis with the analytical activities, methods, and data it needs (Cerezo et al., 2013). They consist of a series of activities with input and output data and are directed to the description of scientific experiments and data analysis processes. Today, data-intensive workflows are capable of exploiting rich and diverse data sources in distributed computing platforms and can be composed for reusing elements from other analytical processes in new applications; see also Chap. 5 in Bouwer et al. (2022).

FAIR-based data management and scientific workflows are key elements of the full research data life cycle for ENVRI RIs as shown in Fig. 3. The research data life cycle illustrates all elements to be contained in such research support environments, including the publication of data as a critical step for open science. Establishing those effective research support environments is best achieved through a collaborative culture of environmental scientists and computer engineers built on a co-design approach and the implementation of mixed teams consisting of team members from both professions (Schulthess, 2015; zu Castell et al., 2022).

Figure 3 The research data life cycle of ENVRI; adapted from Hidalga et al. (2020). QAQC denotes quality assurance and quality control.

To illustrate with more detail the necessary steps towards interoperable and harmonised ECV data as essential input for research support environments, we concentrate on the most relevant ECVs for climate change, which are temperature and greenhouse gas atmospheric abundance. Numerous in situ stations and satellite-borne systems deliver observational data on these ECVs. These data need to be calibrated, processed (data reduction), interpolated, re-gridded, and subject to quality control in complex workflows to generate higher-level high-quality data sets. These data sets can then be used in model systems to derive other ECV data that answer the relevant science or policy questions. Traceability, transparency, and information on the data quality through the whole value chain are essential for using these data in relevant societal evaluations and decision support systems. The ENVRI RIs have started implementing the required technologies at their RI level and developing the first scientific demonstrators for ECV data interoperability of the ATMO-RIs. Finally, ECV data can be used by interdisciplinary research groups for data assimilation, data fusion, and model development.

Access to FAIR environmental data provided by the ATMO-RIs and to research support environments will be made available to the atmospheric and environmental research communities through the ENVRI-Hub described in the Appendix.

3.2 New technology approaches supporting scientific knowledge production

The availability of effective data and research support environments is complemented by the development of new approaches to scientific knowledge production. Currently, considerable knowledge is recorded in scientific articles. This narrative text-document-based expression of research work is designed for human expert consumption, whereas the knowledge recorded in scientific articles is difficult for machines to access. As a result, machine support in scientific knowledge processing is still inadequate and research communities routinely invest enormous human resources to conduct research synthesis, e.g. in systematic reviews, as information needs to be manually extracted from articles and manually organised to support synthesis. In the coming decade, scientific knowledge infrastructures (Stocker, 2017) must further advance their digital infrastructures so that scientific knowledge is produced to be FAIR by design and therefore more efficiently reusable with advanced machine support. Initiatives and services such as the Open Research Knowledge Graph (Stocker et al., 2023) are demonstrating advanced technologies like knowledge graphs, natural language processing, terminology services, and semantic resources. They also include approaches that integrate the production of FAIR scientific knowledge into data analysis and thus enable the reuse of FAIR scientific knowledge in data science. Recent developments in artificial intelligence are also enabling large language models to extract and synthesise significant information from published scientific narratives. Such initiatives can fundamentally transform digital scholarship and the way digital infrastructures support scientific knowledge production.

Knowledge graphs and, more generally, artificial intelligence technologies, virtual research environment (VRE) services, and FAIR scientific data management are key instruments supporting the production of new scientific knowledge. Data discovery services, such as the ENVRI Knowledge Base search engine (Farshidi et al., 2021), can considerably improve the search quality of research assets. This improvement is achieved by allowing search services to effectively expand search queries with a broader set of relevant and similar keywords and by improving the similarity inference of search results using the additional semantic information provided by the concepts and their relationships in knowledge graphs (Farshidi and Zhao, 2022). Semantic information provided by knowledge graphs can help annotate a scientific experiment or observation strategy with rich meta-information for describing its domain problem, data sets, software tools, and experiment steps. This annotation then allows for effective sharing and reuse of experimental workflows across user communities. As new knowledge is derived from scientific experiments or observations, the publication of scientific papers, research software and data sets will be further linked in knowledge graphs.

4 Opening up new research opportunities

4.1 Rethinking the scientific approach

Performing atmospheric research through the scientific data and services provided by the ATMO-RIs and making use of the opportunities offered by open science require a fundamental change in the mindset of individual scientists that will open up new ways of tackling scientific questions of increasing complexity. Particularly, adaptation to the approaches and methods of data-intensive science applied to open data and services from long-term operating research infrastructures calls for rethinking at multiple levels.

At the level of data provision, the creation, curation, and processing of observational data are disconnected from data analysis and scientific exploration, and data provision is realised by the ATMO-RIs through the ENVRI-Hub or other open-access data sources, supported by rich metadata. At the level of scientific exploration, large data sets will remain at the source while data analysis is brought to the data by means of science-based composable workflows running in analytical frameworks such as Jupyter notebooks. Open-source programming environments like GitHub foster the collaborative development and application of data analytical tools and methods in science teams which are ideally composed of both domain scientists and software developers and are restricted neither institutionally nor regionally because of the technologies for remote work offered by the worldwide web.

On the conceptual side, the database accessible to scientists is significantly broadened by the implementation of the science-based framework of ECVs for the provision of ATMO-RI data, combined with the transparent reporting of provenance information in accordance with the FAIR principles and the discoverability of data via enhanced search capabilities though standardised keywords and vocabularies. Making use of open-source programming environments linked with harmonised standard operating procedures for instruments measuring ECVs in accordance with the GCOS-defined requirements (GCOS, 2022a) for data accuracy and precision promotes the collaborative development of data quality assurance and control procedures across the ATMO-RIs, culminating in an increasingly consistent data set of ECVs. In analytical frameworks made available by the ATMO-RIs as well as other ENVRI-RIs through the ENVRI-Hub, ECV-based scientific workflow templates will provide the best data handling, processing, analysis, and visualisation experience. Uses include data cleaning and transformation, numerical simulation, statistical modelling, data visualisation, and machine learning.

New ways of scientific knowledge creation are in reach when leaving the traditional path of working with self-generated data or data from a scientific collaboration and analysing the data within the involved research teams. Recalling that data-intensive science is characterised as collaborative, networked, and driven by immense quantities of data, the access to open data, open software, and resources of previously unknown computational power, combined with a positive attitude towards knowledge sharing, indeed offers a new scientific approach towards complex scientific questions which was not possible before. However, one of the most important bottlenecks in this context is the training of current and future scientists working in our field to familiarise them with the new concepts, tools, and methods.

4.2 Research infrastructures as key facilitators of the new scientific approach

The environmental research infrastructures organised in the ENVRI cluster have covered a considerable part of the way towards a harmonised global climate observation system complemented by open-access and interoperable reliable data, as requested by WMO (Weatherhead et al., 2018; Carmichael et al., 2023) or by the initiative taken by Kulmala (2018) and Kulmala et al. (2023a, b). Making data and scientific products available through free and open access further broadens the use and reuse of the scientific information provided by ACTRIS, IAGOS, and ICOS and finally of the scientific knowledge gained from the continuous operation of the ATMO-RIs. Already today, the atmosphere-centred environmental research infrastructures play a central role in the evolving global climate observing system through the provision of scientific data and knowledge to institutional users like the Copernicus Atmosphere Monitoring Service or the Global Atmosphere Watch programme of the World Meteorological Organization. Through the efforts undertaken by the ATMO-RIs jointly with the other RIs of the ENVRI cluster towards the provision of FAIR data with open access and supported by analytical frameworks, the benefits for the global scientific community in climate sciences and beyond will continuously increase and offer new research options, particularly to a new generation of scientists familiar with data-intensive tools and methods.

Continuing on the outlined path to FAIR and open data in the environmental sciences will not only open the door to broader use of data and scientific products by the global scientific community, especially across traditional scientific fields, but also foster the deeper integration of environmental, life, and social sciences, which is needed to respond to the grand societal challenges we face. The emerging facilities provided by EOSC and the ENVRI-Hub as an EOSC-hosted platform for the collaboration and co-design of the ENVRI RIs will play a central role in this integration of research infrastructures towards interoperability of data and services.

It must be noted, though, that during this integration process, the integrity and functionality of the independent research infrastructures need to be maintained. Their operational requirements are very different and call for tailored solutions for infrastructure design and implementation. Integration then takes place at the level of providing scientific data and services, as well as access to the individual infrastructures. The sustainable success model therefore consists of highly specialised, unique, and independent research infrastructures that integrate as much as possible in the provision and interoperability of data, services, and access to research infrastructure digital and physical assets of any kind.

Appendix A: Sustainable infrastructures for monitoring atmospheric composition

A1 Environmental research infrastructures in atmospheric sciences

A1.1 The community of environmental research infrastructures

By definition in Article 2 (6) of Regulation (EU) No 1291/2013 of 11 December 2013 (EC, 2013), “research infrastructures” mean facilities, resources, and services that are used by research communities to conduct research and foster innovation in their fields. Where relevant, they may be used beyond research, for example for education or public services. They include major scientific equipment or sets of instruments; knowledge-based resources such as collections, archives, or scientific data; e-infrastructures such as data and computing systems and communication networks; and any other infrastructure of a unique nature essential to achieving excellence in research and innovation.

The science cluster of ESFRI-listed environmental research infrastructures (ENVRI) in Europe was formed as a strong community of principal producers and providers of environmental research data and research services in Europe. For most of the ESFRI-listed RIs, access to their services is not simply limited to digital (data) services. Moreover, physical and remote access to the different types of research facilities (i.e. central laboratories, reference instruments, research platforms) is another integral part of their service portfolio and fulfilment of their research and innovation strategies. The science cluster of environmental RIs is one out of five science clusters of the ESFRI list of RIs. The other clusters focus on energy, health and food, physical sciences and engineering, and social and cultural innovation; see the latest ESFRI Roadmap report for more information (ESFRI, 2021b).

In the past decade, through the European Commission cluster projects ENVRI (2011–2014), ENVRIplus (2015–2019), and ENVRI-FAIR (2019–2023), the ENVRI RIs have created a robust conceptual and technical framework based on the FAIR principles that empowers the ENVRI cluster to provide discipline- or domain-oriented (e.g. atmosphere, marine, land system) or thematically/methodologically oriented (e.g. carbon cycle) services in support of climate change research in all its facets (Petzold et al., 2019). This framework includes common standards and policies for the data life cycle, including cataloguing, curation, provenance, and service provision within the ENVRI cluster, with specific consideration of the FAIR principles including interoperability solutions and open standards, e.g. on metadata description. Open access to data sets generated, hosted, and curated by the ENVRI RIs is another integrating element of this framework, which is currently realised at the RI level but will be lifted to cluster level once cross-RI authentication and authorisation infrastructures (AAIs) are in place. The numerous aspects tackled during these joint efforts are described in detail by Zhao and Hellström (2020).

A1.2 Atmosphere-centred research infrastructures

The ENVRI landscape of atmosphere-centred environmental research infrastructures (ATMO-RIs) is illustrated in Table A1, whereas Table A2 compiles brief descriptions of the ATMO-RIs, with the Aerosol, Clouds and Trace Gases Research Infrastructure (ACTRIS); the In-service Aircraft for a Global Observing System (IAGOS); and the Integrated Carbon Observation System (ICOS) Atmosphere Thematic Centre performing research on the chemical composition of the atmosphere in a changing climate. The Svalbard Integrated Arctic Earth Observing System (SIOS) organises multi-domain observation platforms on the Svalbard archipelago in the Arctic region, with atmosphere-related research data largely provided via ACTRIS and ICOS, while EISCAT_3D concentrates on the dynamics of the mesosphere. Thus, the ATMO-RI landscape consists of three RIs targeting solely atmospheric research topics (ACTRIS, EISCAT, IAGOS), and two multi-domain RIs with an atmospheric component (ICOS, SIOS).

Table A1 The landscape of environmental research infrastructures in Europe (ENVRI) in atmospheric sciences in a nutshell.

Table A2 ESFRI environmental research infrastructures of the atmospheric sciences.

Harmonising and securing the landscape of atmospheric observations

Historically, the ATMO-RIs have developed from independent research projects which started their integration processes with the emerging ESFRI Roadmap process in the early 2000s. All predecessor projects focused on different research targets but in similar research fields. ACTRIS brought together ground-based cloud remote sensing measurements (CloudNET), the European Supersites for Atmospheric Aerosol Research (EUSAAR), the European Aerosol Research Lidar Network (EARLINET), the European atmospheric simulation chambers (EUROCHAMP), and the European component of a number of additional monitoring networks (Laj et al., 2024). IAGOS merged two projects on the measurements of ozone, water vapour, and other trace species by a fleet of in-service Airbus aircraft (MOZAIC; Marenco et al., 1998) or by means of one instrumented airfreight container (CARIBIC; Brenninkmeijer et al., 1999) into one infrastructure on atmospheric observation by civil passenger aircraft (Petzold et al., 2015; Thouret et al., 2022). ICOS was built around the scientific target of continuous, long-term observations of the carbon cycle, including the quantification of greenhouse gas emissions and sinks and their impacts on Earth systems (Heiskanen et al., 2022). Predecessor projects are numerous (> 25), and Fig. 1 in Franz et al. (2018) illustrates the fragmentation of observation networks in Europe which then developed into ICOS. While ACTRIS, IAGOS, and ICOS infrastructures are characterised by strongly distributed observation networks, EISCAT and SIOS have emerged from single-sited activities.

Looking at the evolution of the ATMO-RIs, one important achievement of the research infrastructure implementation process becomes immediately apparent, namely the integration of a formerly scattered landscape of multiple observation networks into a few research infrastructures. The direct benefits of this consolidation process were joining resources of all kinds, reducing duplication of work, and offering data of previously unknown quality and complexity. Before the consolidation phase, data had been locked in silos and had been poorly standardised over decades, hampering scientific progress severely. The first harmonisation efforts were undertaken by the Global Atmosphere Watch programme of WMO, but during the consolidation process, the ATMO-RIs developed cutting-edge standardisation for their observations and implemented highly efficient data life cycle management systems, leading to timely, open, and FAIR data access by scientists all over the world. Tackling the next step, the ATMO-RIs joined forces in the European Union's Horizon 2020 project “Sustainable Access to Atmospheric Research Facilities” (ATMO-ACCESS, 2024) for establishing a comprehensive and sustainable framework for access to distributed atmospheric RIs, ensuring integrated access to and optimised use of the services they provide.

Another major achievement of the ATMO-RIs which deserves explicit mention is the conservation of long-term atmospheric observation stations and time series which were in danger during the transition process from research projects to research infrastructures. The detection of trends and seasonality in the presence of greenhouse gases and short-lived climate-active atmospheric constituents is an important aspect of climate science. An accurate description of trends relies heavily on the ability to place the measurements of all kinds of climate-active atmospheric constituents in a historical context, i.e. to compare measurements against measurements from the same location in preceding years and decades. From a scientific perspective, longer time series reduce the uncertainties in the interpretation of current measurements. The length of time series which are produced by the different ATMO-RIs is therefore a good indicator of how well these data meet the requirements of climate scientists. It is important to note that the use of long time series of observations for trend analysis requires a measurement infrastructure that guarantees inter-comparability in terms of observation precision and accuracy over the period of interest. Such inter-comparability generally requires constant vigilance and support, which operational RIs secure.

To give examples, we may first refer to the long-term in situ observations of ozone and water vapour in the upper troposphere and lowermost stratosphere by passenger aircraft equipped with MOZAIC instrumentation (Marenco et al., 1998), starting in 1994. These long-term data not only could be secured but also were subjected to the most modern data quality assurance standards, and the collection of data successfully continued after the transition of MOZAIC to IAGOS was completed. Today these data sets cover the climatologically relevant length of 30 years and are widely used, e.g. in the IPCC reports, in the Tropospheric Ozone Assessment Report (Schultz et al., 2017; Tarasick et al., 2019), and in climatological studies (Petetin et al., 2016; Gaudel et al., 2018, 2020; Petzold et al., 2020). The representativeness of the IAGOS airborne measurements in the lower troposphere was explicitly investigated to ensure the applicability of these airborne data sets in the context of air quality studies (Petetin et al., 2018). Without the implementation of IAGOS as a research infrastructure, the historical data would not be available and the continuation of observations would not have been possible.

The second example focuses on the measurements of greenhouse gases and the carbon cycle that comprise a core element in the scientific fields of ecology and evolution, environmental sciences, atmospheric sciences, forestry, and agronomy. ICOS provides long-term, high-quality observations that follow (and cooperatively set) the global standards for the best possible quality of data on the atmospheric composition of greenhouse gases, greenhouse gas exchange fluxes measured by eddy covariance, and CO₂ partial pressure at water surfaces. Besides these main data products, ICOS also produces observations of many ancillary variables, using the same highest-quality standards. The ICOS measurement infrastructure is in large part built from existing measurement stations which have been updated to provide measurements that meet ICOS standards. Data which were collected before ICOS measurement protocols were put in place have been secured physically and with respect to data quality. They now contribute to the ICOS record of long-term data sets and to the World Data Centre for Greenhouse Gases run by the Japan Meteorological Agency as an extremely important contribution to the global greenhouse gas community in the context of data archiving and dissemination.

Of ICOS' 134 measurement stations, 101 stations provided data on the length of time series held by them. Data from these 101 stations describe how long they have been operational or, in some cases, how long the station has been collecting measurements that are relevant to ICOS. The average length of time series across all domains is 11 years, and this is evenly spread between the ecosystem domain (average 11 years), atmosphere domain (12 years), and ocean domain (11 years); see Fig. A1. Time series from atmosphere stations tend to be the longest, which likely reflects the overlap between historical atmosphere measurements and current variables being measured by ICOS. Details are given in the latest ICOS Impact Assessment Report (Belle et al., 2018).

Figure A1 Number of measurement stations and length of time series held by ICOS measurement stations in 2018, for the atmosphere (ATM), ecosystem (ECO), and ocean domains (data source: Belle et al., 2018).

The third example is the aerosol ECV “aerosol single-scattering albedo” retrieved from ACTRIS measurements. ACTRIS covers observations of short-lived constituents of the atmosphere, using in situ and remote sensing methods located on the Earth surface. The series of ACTRIS projects, leading to the establishment of the ACTRIS ERIC, started in 2011, but predecessor projects date back to the year 2000, with single time series of observations having started already in 1995. Figure A2 shows an example of a long time series of aerosol particle scattering and the absorption coefficient (upper and middle panel respectively) observed at the Ispra station in northern Italy. From data with both high quality and high time resolution, the aerosol particle single-scattering albedo can be calculated, an ECV necessary for assessing the direct aerosol climate effect. The single-scattering albedo is shown in the lower panel over the period 2009–2023.

Overall, long-term observations of climate-relevant aerosol properties by ACTRIS became an indispensable ingredient of the global climate observation system and have trigged multiple scientific studies on the long-term aerosol impact on climate (e.g. Zanatta et al., 2016; Collaud Coen et al., 2020; Laj et al., 2020). Trends are detectable in the data, which is possible only due to the collection of data with consistent quality and operating procedures over long time periods.

Figure A2 Time series of aerosol particle scattering (a; Putaud and Martins dos Santos, 2022b) and absorption coefficients (b; Putaud and Martins dos Santos, 2022a), observed at the Ispra station in northern Italy associated with ACTRIS and ACTRIS pre-projects, all at 670 nm wavelength. The data displayed in (a) and (b) form the basis for calculating the aerosol particle single-scattering albedo, which is declared an essential climate variable (ECV).

Technical achievements

Technically, the ATMO-RIs have definitely pushed the borders of what can be observed. The measurements are highly standardised, connected to unified calibration systems, partially available in near real time, and well distributed over the European continent or even globally. Measurement methods deployed by the ATMO-RIs follow published common specifications and protocols and use synergies among the RIs as efficiently as possible. The data are subject to quality control and processed at dedicated data centres, using open and published processing chains. The RI-specific calibration laboratories provide the stations with working standards and analyse flask samples for trace gas concentrations. All data from raw data up to the final quality-controlled (averaged) data are openly accessible from the RI data portals (ACTRIS and IAGOS data portals, ICOS Carbon Portal).

Today, ICOS provides the densest and most integrated observational network on greenhouse gases worldwide; IAGOS is a globally unique observation infrastructure for atmospheric trace gases, aerosol, and cloud particles which reaches parts of the atmosphere not accessible for in situ observations by other platforms; and ACTRIS constitutes a research infrastructure on short-lived atmospheric constituents like reactive trace gases, aerosols, and clouds, which investigates the processes leading to the variability in these constituents in the natural environment and in controlled atmospheres of unprecedented complexity. The strategic role of the European complementary RIs in the atmospheric sciences (the ATMO-RIs) has been highlighted in a recent IPCC report (IPCC, 2021a; see Sect. 1.5.1.1, “Major expansions of observational capacity”), noting the key contribution of ICOS, IAGOS, and ACTRIS. These three ATMO-RIs are also essential data providers to the Copernicus services in the areas of atmosphere monitoring (CAMS; Wagner et al., 2021; Peuch et al., 2022) and climate change (C3S; Thepaut et al., 2018) as well as to the WMO Global Atmosphere Watch Programme (GAW; Carmichael et al., 2023) and play leading roles in the evolving Global Greenhouse Gas Watch programme.

A2 The science-based framework of essential climate variables

The Earth atmosphere is a complex medium with thousands of chemical species interacting with their environment. Consequently, monitoring the atmospheric composition comprehensively in the long term is impossible. To transform the challenging task of integrated Earth observation into a concept closer to a global climate observation system, GCOS (co-sponsored by WMO) has introduced the concept of essential climate variables (ECVs). ECVs for atmospheric composition are defined as a physical or chemical variable or a group of linked variables that critically contributes to the characterisation of the atmospheric composition (Bojinski et al., 2014). This set of variables includes not only the long-living greenhouse gas CO₂, but also many of the short-lived climate forcers such as ozone, methane, and aerosol particles and meteorological parameters like air temperature and humidity (water vapour volume mixing ratio, specific humidity, relative humidity). The observables to be recorded for climate monitoring are defined by WMO through the process of the Rolling Review of Requirements in the Observing Systems Capability Analysis and Review Tool (WMO-OSCAR, 2024). WMO has also defined a subset of the existing set of ECVs to provide the most essential parameters representing the state of the climate system. These indicators include global mean surface temperature, global ocean heat content, state of ocean acidification, glacier mass balance, Arctic and Antarctic sea ice extent, global CO₂ mole fraction, and global mean sea level (Trewin et al., 2021).

Among the nine atmospheric application areas in the process of the Rolling Review of Requirements, two are critically relevant to the work of atmospheric research infrastructures.

Atmospheric climate monitoring defines the requirements for climate observations that are essential to detect, model, and assess climate change and its impact; support adaptation to climate change; monitor the effectiveness of policies for mitigating climate change; and develop climate information services. ECVs for the atmosphere, the oceans, and terrestrial systems (Bojinski et al., 2014) are identified for this application area. ECV data sets provide the empirical evidence needed to understand and predict the evolution of climate, to guide mitigation and adaptation measures, to assess risks and enable attribution of climatic events to the underlying causes, and to underpin climate services. The current list of ECVs is specified by the Global Climate Observing System (GCOS, 2010). Updated requirements for the observation of ECVs were recently published (GCOS, 2022a).

The area of atmospheric composition forecasting and monitoring defines requirements related to evaluating and analysing changes (temporally and spatially) in atmospheric composition regionally and globally to support treaty monitoring, climatologies, and re-analyses, assessing trends in composition and emissions/fluxes, and better understanding of processes, using data of controlled quality.

The recommended essential climate variables as defined by GCOS in 2022 are listed in Table A3. A complete list of variables and requirements for atmospheric applications can be found through the OSCAR tool (WMO-OSCAR, 2024). Table A3 also lists the ATMO-RIs covering the respective parameters. ECVs based on total column densities of specific molecules are covered by a variety of satellite-borne instruments and by ground-based remote sensing observational networks such as the Total Carbon Column Observing Network (TCCON; Wunch et al., 2011), the Network for the Detection of Atmospheric Composition Change (NDACC; De Mazière et al., 2018), the federated network for aerosol characterisation AERONET (Holben et al., 1998), or the global ozonesonde network (Tarasick et al., 2021).

Concluding from the latest landscape analysis of the environment domain (ESFRI, 2021a), ENVRI RIs and ATMO-RIs in particular have been indispensable for monitoring and understanding changes in ECVs over several decades. Given the great variety and complexity of observational data provided by all ENVRI RIs, the science-based framework of ECVs offers a concept and guidelines for the implementation of tools and standards on the data and metadata level that concentrate on key variables covered by multiple ENVRI RIs and allow searchability, accessibility, and interoperability of these key ECV data, with the final goal of enabling cross-RI exploitation of data.

Table A3 Essential climate variables for the atmospheric climate monitoring application area as defined in the 2022 GCOS Implementation Plan (GCOS, 2022b) and the ATMO-RIs providing the observations.

Abbreviations and symbols: Trop., troposphere; Strat., stratosphere; AOD, aerosol optical depth; σ_ext, aerosol light extinction coefficient. Note: ACTRIS and ICOS measurements are in any case surface-based and originate either from in situ or from remote sensing techniques; IAGOS measurements originate from in situ profile observations from the boundary layer to the lowermost stratosphere.

A3 The ENVRI-Hub

The ENVRI cluster responded to the numerous challenges of data-intensive science by establishing a “Data for Science” theme in their collaborative project ENVRIplus and then by starting the development and implementation of the identified research support environments in the successor project ENVRI-FAIR. These efforts finally evolved into the ENVRI-Hub as a platform for environmental services, data, knowledge, and training to support and increase collaboration and the sharing of information beyond traditional scientific communities (Petzold et al., 2019; Zhao and Hellström, 2020). Since open science is at the core of the ENVRI RIs, the ENVRI-Hub also serves as a robust conceptual and technical framework that will facilitate the integration of the environmental science community into the European Open Science Cloud (EOSC). Details of the ENVRI-Hub architecture, its design criteria, and technical solutions implemented can be found in the ENVRI-Hub white paper (Petzold et al., 2023).

Currently, the ENVRI-Hub is accessible as a demonstrator version (ENVRI-Hub, 2024) with envisaged functionalities not yet fully developed. In its matured stage, the ENVRI-Hub will provide researchers, from, for example, the atmospheric science community, with the necessary services to practise open science through using data and services published in the ENVRI Catalogue of Services. It will provide research products and services to EOSC to enhance researcher, citizen, and industry involvement in open science. ECV workflows, data sets, services, training material, and publications will be made available to EOSC when assets and services reach the adequate maturity level. To ensure the reproducibility of results, e.g. the training and documentation of services, metadata or analytical workflows will be provided and made available through the training platform, assisted by training events. Science demonstrators will be enabled by analytical frameworks, where ECV-based workflow templates will provide the best data handling and visualisation experiences. Applications will include data cleaning and transformation, numerical simulation, statistical modelling, data visualisation, and machine learning. Already today, the ENVRI Community in Zenodo (ENVRI-Community, 2024) contains documentation, reports, and research papers with articles having open-access licences with minimal restrictions. The release of the ENVRI-Hub and the workflows developed on the ENVRI-Hub will be open-source and managed on the Git repository on EGI Infrastructure. The mature releases will also be published in Zenodo.

A4 Science demonstrators as examples for the interoperability of observation infrastructures

Atmospheric scientists entering the ENVRI-Hub though its portal on EOSC will reach graphical user interfaces for accessing data and services, such as map-based portals and lists of catalogued data sets, with semantic-driven search functionalities that enable them to search for appropriate data sets or services using controlled vocabularies of discipline-related keywords. They can use the research support environments in different ways, as illustrated in Fig. A3 for the discovery and use of data (top) or data products and services (bottom), and build their own analytical workflows from components developed beforehand by other scientists and published through the ENVRI-Hub.

Figure A3 (a) Advanced data discovery and use; (b) advanced service discovery and use with the data in, for example, Jupyter Notebook across RIs (Petzold et al., 2023).

To give a brief description of data search and discovery (Fig. A3, top panel), the scientist browses the metadata catalogue with the help of the vocabulary, discovers data of potential interest, then browses the metadata catalogue for information about the access to the RI of interest with the help of the provided access documentation, and repeats this chain until all data of interest have been collected. Finally, the whole process can be put into a workflow which can be stored for reuse and combination with other workflows. A similar procedure is possible for the advanced service discovery and use (Fig. A3, bottom panel).

The ground for offering the advanced services illustrated in Fig. A3 for atmosphere-centred research questions was prepared through the considerable improvement of cross-RI atmospheric services and interactions achieved during ENVRI-FAIR. As a result, it is now possible to implement cross-RI services relying on the convergence between RIs on important topics like vocabularies, unique identifiers, and licences. The vocabulary within the ATMO-RIs was very diverse, RI specific, and developed over a long time from various user communities and applications. Presently, there is a controlled vocabulary extending beyond our RIs that has enabled better integration with other RIs and thematic domains.

The implementation of unique identifiers was crucial for cross-RI synergies and has led to the improved exchange of services, procedures, and data products. Also, an atmospheric community standard has evolved on the licence for data use. This has led to the reliable reuse of data. With a common use of licence now considered more a “community standard”, there is also possibility of producing new high-level cross-RI data products that utilise services and data across the atmosphere domain. The convergence of solutions across the atmosphere-centred RIs has led to new products that are useful for all users across the RIs and outside the atmosphere domain, e.g. satellite-borne instrument data extraction and colocation through machine-to-machine interoperable services relevant to the full atmosphere domain and beyond.

Science-based services are designed to facilitate data discoverability and accessibility for complex research questions and thus to support researchers focusing on scientific questions instead of searching for data. The following examples from the service portfolio of the ATMO-RIs are already accessible through the current demonstrator version of the ENVRI-Hub:

1. The atmospheric colocation service is a suite of tools to identify and retrieve space-borne observations that match ground-based measurements, given a list of ground-based stations and satellites together with user-specified colocation criteria. The service builds on the implementation of the ENVRI Python library for the dynamic retrieval of platform metadata for ACTRIS, ICOS, and SIOS stations and, in the case of IAGOS, for airports for which vertical profiles of atmospheric parameters are provided. A hierarchical catalogue structure for satellite-borne instrument data based on the AERIS/ICARE information system and enhanced with specific metadata is also implemented.

2. The ICOS Carbon Portal footprint tool provides atmospheric surface emission influence calculations (a.k.a., footprints) for any 3D location or specific station in the model domain and any period for which meteorological data are available. When ready, the results can be viewed in the tool and directly accessed through our Jupyter Notebook service and Python library, to be combined with emission data for concentration calculations at the receptor(s) and comparison with observations retrieved from the RI data repositories.

3. The FAIR ENVRI Atmospheric Data Demonstrator aims to shorten time responses in the case of extreme events like fires, emissions from volcanic eruption, and desert dust intrusion by providing scientific analyses as well as harmonised data sets and tools. It will offer a search tool for the ATMO-RIs' data availability for the period and area of interest with a focus on ECVs, fetch the selected data including automatic previews of the data sets, and provide statistical analysis on the downloaded time series. In addition, access to data from ATMO-RIs is offered as a bundle with automated compilation of provenance information, and a colocation service with satellite data is available.

4. The Dashboard for the State of the Environment is designed to be completely user configurable so that the users can select from a list of indicators to be shown and their order. Providers can add, remove, and edit indicators through a standard RESTful API that allows transferring all parameters, including the configuration of the indicators and how to provide data values and thumbnail interaction. The dashboard builds on EOSC services like AAI, cloud services, and data storage, and the workflows that provide the indicators will also build on the EOSC and research infrastructure computing integration. As a proof of concept, a limited list of indicators is currently available, and we foresee that the participating RIs will provide many more indicator options in the near future. The dashboard service is completely open-source, and, like the whole concept, it is designed to be flexible and expansible.

The science demonstrators are practical examples of the provision of interoperable services from the ATMO-RIs to the scientific community, enabling cross-RI data discovery and use. The software behind the science demonstrators can be accessed in GitHub repositories for further use and composition with users' own workflows. In that respect, the science demonstrators are showcases for new research opportunities evolving from a combination of RI data infrastructures with tools and methods of data-driven science.