<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing with OASIS Tables v3.0 20080202//EN" "journalpub-oasis3.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:oasis="http://docs.oasis-open.org/ns/oasis-exchange/table" xml:lang="en" dtd-version="3.0">
  <front>
    <journal-meta><journal-id journal-id-type="publisher">ACP</journal-id><journal-title-group>
    <journal-title>Atmospheric Chemistry and Physics</journal-title>
    <abbrev-journal-title abbrev-type="publisher">ACP</abbrev-journal-title><abbrev-journal-title abbrev-type="nlm-ta">Atmos. Chem. Phys.</abbrev-journal-title>
  </journal-title-group><issn pub-type="epub">1680-7324</issn><publisher>
    <publisher-name>Copernicus Publications</publisher-name>
    <publisher-loc>Göttingen, Germany</publisher-loc>
  </publisher></journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.5194/acp-20-2303-2020</article-id><title-group><article-title>Technical note: Deep learning for creating surrogate models of
precipitation in Earth system models</article-title><alt-title>Surrogate Earth system models</alt-title>
      </title-group><?xmltex \runningtitle{Surrogate Earth system models}?><?xmltex \runningauthor{T.~Weber et al.}?>
      <contrib-group>
        <contrib contrib-type="author" corresp="no" rid="aff1">
          <name><surname>Weber</surname><given-names>Theodore</given-names></name>
          
        </contrib>
        <contrib contrib-type="author" corresp="no" rid="aff1">
          <name><surname>Corotan</surname><given-names>Austin</given-names></name>
          
        </contrib>
        <contrib contrib-type="author" corresp="yes" rid="aff1 aff2">
          <name><surname>Hutchinson</surname><given-names>Brian</given-names></name>
          <email>brian.hutchinson@wwu.edu</email>
        <ext-link>https://orcid.org/0000-0002-5537-008X</ext-link></contrib>
        <contrib contrib-type="author" corresp="no" rid="aff3 aff4">
          <name><surname>Kravitz</surname><given-names>Ben</given-names></name>
          
        <ext-link>https://orcid.org/0000-0001-6318-1150</ext-link></contrib>
        <contrib contrib-type="author" corresp="no" rid="aff5">
          <name><surname>Link</surname><given-names>Robert</given-names></name>
          
        <ext-link>https://orcid.org/0000-0002-7071-248X</ext-link></contrib>
        <aff id="aff1"><label>1</label><institution>Computer Science Department, Western Washington University,
Bellingham, WA, USA</institution>
        </aff>
        <aff id="aff2"><label>2</label><institution>Computing and Analytics Division, Pacific Northwest
National Laboratory, Seattle, WA, USA</institution>
        </aff>
        <aff id="aff3"><label>3</label><institution>Department of Earth and Atmospheric Sciences, Indiana University,
Bloomington, IN, USA</institution>
        </aff>
        <aff id="aff4"><label>4</label><institution>Atmospheric Sciences and Global Change Division, Pacific Northwest
National Laboratory, Richland, WA, USA</institution>
        </aff>
        <aff id="aff5"><label>5</label><institution>Joint Global Change Research
Institute, Pacific Northwest National Laboratory, College Park, MD, USA</institution>
        </aff>
      </contrib-group>
      <author-notes><corresp id="corr1">Brian Hutchinson (brian.hutchinson@wwu.edu)</corresp></author-notes><pub-date><day>26</day><month>February</month><year>2020</year></pub-date>
      
      <volume>20</volume>
      <issue>4</issue>
      <fpage>2303</fpage><lpage>2317</lpage>
      <history>
        <date date-type="received"><day>28</day><month>January</month><year>2019</year></date>
           <date date-type="rev-request"><day>11</day><month>April</month><year>2019</year></date>
           <date date-type="rev-recd"><day>2</day><month>January</month><year>2020</year></date>
           <date date-type="accepted"><day>27</day><month>January</month><year>2020</year></date>
      </history>
      <permissions>
        <copyright-statement>Copyright: © 2020 Theodore Weber et al.</copyright-statement>
        <copyright-year>2020</copyright-year>
      <license license-type="open-access"><license-p>This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this licence, visit <ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/">https://creativecommons.org/licenses/by/4.0/</ext-link></license-p></license></permissions><self-uri xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020.html">This article is available from https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020.html</self-uri><self-uri xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020.pdf">The full text article is available as a PDF file from https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020.pdf</self-uri>
      <abstract><title>Abstract</title>
    <p id="d1e144">We investigate techniques for using deep neural networks to produce surrogate
models for short-term climate forecasts.  A convolutional neural network is
trained on 97 years of monthly precipitation output from the 1pctCO2 run (the
<inline-formula><mml:math id="M1" display="inline"><mml:mrow class="chem"><mml:msub><mml:mi mathvariant="normal">CO</mml:mi><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:mrow></mml:math></inline-formula> concentration increases by 1 % per year) simulated by the second-generation Canadian Earth System Model (CanESM2).  The neural network clearly outperforms a persistence forecast and
does not show substantially degraded performance even when the forecast length
is extended to 120 months.  The model is prone to underpredicting precipitation
in areas characterized by intense precipitation events.  Scheduled sampling
(forcing the model to gradually use its own past predictions rather than ground
truth) is essential for avoiding amplification of early forecasting errors.
However, the use of scheduled sampling also necessitates preforecasting
(generating forecasts prior to the first forecast date) to obtain adequate
performance for the first few prediction time steps.  We document the training
procedures and hyperparameter optimization process for researchers who wish to
extend the use of neural networks in developing surrogate models.</p>
  </abstract>
    </article-meta>
  </front>
<body>
      

<sec id="Ch1.S1" sec-type="intro">
  <label>1</label><title>Introduction</title>
      <p id="d1e167">Climate prediction is a cornerstone in numerous scientific investigations and
decision-making processes <xref ref-type="bibr" rid="bib1.bibx56 bib1.bibx29" id="paren.1"><named-content content-type="pre">e.g.,</named-content></xref>.  In the long term
(decades to centuries), different possible climate outcomes pose very different
hazards, risks, and societal challenges, such as building and maintaining
infrastructure <xref ref-type="bibr" rid="bib1.bibx45" id="paren.2"><named-content content-type="pre">e.g.,</named-content></xref>.  On decadal timescales, predictability
of major modes of variability (like the El Niño–Southern Oscillation) is an
important driver of extreme events, such as flooding and drought
<xref ref-type="bibr" rid="bib1.bibx62" id="paren.3"><named-content content-type="pre">e.g.,</named-content></xref>.  On similar timescales, disappearing Arctic sea ice
has been implicated in changes in midlatitude winter storm patterns
<xref ref-type="bibr" rid="bib1.bibx11" id="paren.4"/>.  On shorter timescales (weeks to months), also sometimes
called the subseasonal-to-seasonal (S2S) regime, climate forecasts can be
critical for agriculture, water resource management, flooding and drought
mitigation, and military force mobilization <xref ref-type="bibr" rid="bib1.bibx50" id="paren.5"/>.</p>
      <p id="d1e191">Improvements in climate predictability in some of these regimes are slow to be
realized.  Decadal predictability studies have found that predictability skill
is greatly influenced by proper initialization of hindcasts to ensure that any
modeled changes due to internal variability are in phase with observations
<xref ref-type="bibr" rid="bib1.bibx3" id="paren.6"/>.  This highlights the importance of climate memory in
predictive skill in that the response to processes can be lagged and the
responses themselves can depend upon the model state.  <xref ref-type="bibr" rid="bib1.bibx63" id="text.7"/> found
the existence of processes with a relatively high portion of response that can
be explained by memory on all timescales, from monthly through multidecadal
lengths.  As an example, <xref ref-type="bibr" rid="bib1.bibx22" id="text.8"/> found that properly initialized
hindcasts were able to predict the global warming slowdown of the early 2000s
<xref ref-type="bibr" rid="bib1.bibx15" id="paren.9"/> up to 5 years ahead.</p>
      <?pagebreak page2304?><p id="d1e206">The best technique for climate prediction is to run an Earth system model (ESM),
as these models capture the state of the art in our knowledge of climate
dynamics.  These models, however, are difficult and costly to run, and for many
researchers access to ESM output is limited to a handful of runs deposited into
public archives.  Therefore, there has been a great deal of interest in
<italic>surrogate models</italic> that can produce results similar to ESMs but are more
accessible to the broader research community due to ease of use and lower
computational expense.  These surrogates are advantageous for numerous applications, such as exploring wide ranges of scenarios; such efforts are quite costly in ESMs.</p>
      <p id="d1e212">Building surrogates of ESMs can take numerous forms.  The most basic is pattern
scaling <xref ref-type="bibr" rid="bib1.bibx52" id="paren.10"/>, involving scaling a time-invariant pattern of change
by global mean temperature <xref ref-type="bibr" rid="bib1.bibx44 bib1.bibx40" id="paren.11"><named-content content-type="pre">e.g.,</named-content></xref>.  Other
methods include statistical emulation based on a set of precomputed ESM runs
<xref ref-type="bibr" rid="bib1.bibx9" id="paren.12"/>, linear time-invariant approaches
<xref ref-type="bibr" rid="bib1.bibx41" id="paren.13"/>, or dimension reduction via empirical orthogonal
functions <xref ref-type="bibr" rid="bib1.bibx25" id="paren.14"><named-content content-type="pre">e.g.,</named-content></xref>.  While all of these methods have shown
some degree of success, they inherently neither incorporate information about the
internal model state <xref ref-type="bibr" rid="bib1.bibx18" id="paren.15"/> nor capture highly nonlinear
behavior.</p>
      <p id="d1e239">Various methods of incorporating state information into surrogate models have
been studied.  <xref ref-type="bibr" rid="bib1.bibx35" id="text.16"/> explored using a linear combination of
autoregressive processes with different lag timescales in explaining global mean
temperature change.  However, such reduced-order modeling approaches, which
explicitly capture certain physical processes, will invariably have limited
structure; this can result in inaccurate predictions when evaluating variables
like precipitation, which have fine temporal and spatial features, important
nonlinear components in their responses to forcing, and decidedly non-normal
distributions of intensity and frequency. Many studies have focused on
initializing the internal model state of Earth system models (or models of
similar complexity) to capture low frequency variability; this has been found to
add additional skill beyond external forcing alone <xref ref-type="bibr" rid="bib1.bibx18" id="paren.17"/>.
However, the required computational time to create a decadal prediction ensemble
is rather expensive.</p>
      <p id="d1e248">Machine learning methods offer the possibility of overcoming these limitations
without imposing an insurmountable computational burden. The field of machine
learning studies algorithms that learn patterns (e.g., regression or
classification) from data. The subfield of deep learning focuses on models,
typically neural-network variants, that involve multiple layers of nonlinear
transformation of the input to generate predictions. Although many of the core
deep-learning techniques were developed decades ago, work in deep learning has
recently exploded, achieving state-of-the-art results on a wide range of
prediction tasks. This progress has been fueled by increases in both computing
power and available training data. While training deep-learning models is
computationally intensive, trained models can make accurate predictions quickly
(often in a fraction of a second).</p>
      <p id="d1e251">The use of deep learning in climate science is relatively new.  Most of the
applications have used convolutional neural networks (see
Sect. <xref ref-type="sec" rid="Ch1.S3"/> for further definitions and details) to detect and
track features, including extreme events <xref ref-type="bibr" rid="bib1.bibx38 bib1.bibx48 bib1.bibx12 bib1.bibx27" id="paren.18"><named-content content-type="pre">tropical cyclones, atmospheric rivers,
and weather fronts; e.g.,</named-content></xref> or
cloud types <xref ref-type="bibr" rid="bib1.bibx43" id="paren.19"/>.  Other promising applications of deep learning
are to build new parameterizations of multiscale processes in models
<xref ref-type="bibr" rid="bib1.bibx30 bib1.bibx49" id="paren.20"/> or to build surrogates of entire model components
<xref ref-type="bibr" rid="bib1.bibx39" id="paren.21"/>. More generally, <xref ref-type="bibr" rid="bib1.bibx42" id="text.22"/> have explored using deep
neural networks in nonlinear dynamical spatio-temporal environmental models, of
which climate models are an example, and <xref ref-type="bibr" rid="bib1.bibx47" id="text.23"/> applied
<italic>echo state networks</italic> to monthly rainfall prediction. These studies have
clearly demonstrated the power that deep learning can bring to climate science
research and the new insights it can provide.  However, to the best of our
knowledge, there have been few attempts to assess the ability of deep learning
to improve predictability, in particular the ability to incorporate short-term
memory.  Several deep-learning architectures (described below) are particularly
well suited for this sort of application.</p>
      <p id="d1e280">Here we explore techniques for using deep learning to produce accurate surrogate
models for predicting the forced response of precipitation in Earth system models.  Key to this approach is training the model on past precipitation data
(as described later, a sliding 5-year window), allowing it to capture relevant
information about the state space that may be important for predictability in
the presence of important modes of variability. The surrogate models will be
trained on climate model output of precipitation under a scenario of increasing
<inline-formula><mml:math id="M2" display="inline"><mml:mrow class="chem"><mml:msub><mml:mi mathvariant="normal">CO</mml:mi><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:mrow></mml:math></inline-formula> concentration and then used to project precipitation outcomes into a
period beyond the training period. That forecast will then be compared to the
actual climate model output for the same period to quantify performance.
Several model designs will be compared to evaluate the effectiveness of various
deep-learning techniques in this application. The performance of the deep-learning surrogates will be compared to other methods of forecasting, such as
persistence and autoregressive processes (described later).  A key area of
investigation will be the prediction horizon (how far out is the predictive
skill of the surrogate model better than naive extrapolation methods) for the
chosen window size (how much past information is used to condition the
surrogate's predictions).</p>
</sec>
<sec id="Ch1.S2">
  <label>2</label><title>Study description</title>
      <?pagebreak page2305?><p id="d1e302">The dataset used for this study was idealized precipitation output from the
second-generation Canadian Earth System Model <xref ref-type="bibr" rid="bib1.bibx2" id="paren.24"><named-content content-type="pre">CanESM2;</named-content></xref>.  The atmospheric model has a
horizontal resolution of approximately 2.8<inline-formula><mml:math id="M3" display="inline"><mml:msup><mml:mi/><mml:mo>∘</mml:mo></mml:msup></mml:math></inline-formula> with 35 vertical layers,
extending up to 1 hPa.  The atmosphere is fully coupled to the land surface
model CLASS <xref ref-type="bibr" rid="bib1.bibx1" id="paren.25"/> and an ocean model with approximately
1<inline-formula><mml:math id="M4" display="inline"><mml:msup><mml:mi/><mml:mo>∘</mml:mo></mml:msup></mml:math></inline-formula> horizontal resolution.  The model output used corresponds to the
1pctCO2 simulation, in which, starting from the preindustrial era, the carbon
dioxide concentration increases by 1 % per year for 140 years to approximately
quadruple the original concentration.  This idealized simulation was chosen to
reduce potential complicating factors resulting from precipitation responses to
multiple forcings (carbon dioxide, methane, sulfate aerosols, black-carbon
aerosols, dust, etc.) that might occur under more comprehensive scenarios, such
as the Representative Concentration Pathways <xref ref-type="bibr" rid="bib1.bibx58" id="paren.26"/>.  For this
study, only monthly average precipitation output was used; results for daily
average precipitation are the subject of future work.</p>
      <p id="d1e334">We divided the model output into three time periods.  The <italic>training</italic> set
consists of the period 1850–1947 and is used to train the surrogate model.
The <italic>development</italic> set (sometimes called the validation set) consists of the
period 1948–1968 and is used to evaluate the performance of the trained
surrogate model to guide further tuning of the model's hyperparameters (i.e.,
configurations external to the model that are not estimated during training).
The <italic>test</italic> set consists of the period 1969–1989 and is used only in
computing the end results, which are reported below in
Sect. <xref ref-type="sec" rid="Ch1.S4"/>.</p>
</sec>
<sec id="Ch1.S3">
  <label>3</label><title>Deep learning methodologies for improving predictability</title>
      <p id="d1e356">Deep learning is a subfield of machine learning that has
achieved widespread success in the past decade in numerous science and
technology tasks, including speech recognition
<xref ref-type="bibr" rid="bib1.bibx26 bib1.bibx10" id="paren.27"/>, image classification
<xref ref-type="bibr" rid="bib1.bibx34 bib1.bibx54 bib1.bibx24" id="paren.28"/>, and drug design
<xref ref-type="bibr" rid="bib1.bibx16" id="paren.29"/>. Its success is often attributed to a few key
characteristics.  First, rather than operating on predetermined (by humans)
features, deep learning is typically applied to raw inputs (e.g., pixels), and
all of the processing of those inputs is handled in several steps (<italic>layers</italic>)
of nonlinear transformation; the addition of these layers increases the
<italic>depth</italic> of the network.  This allows the model to learn to extract
discriminative, nonlinear features for the task at hand.  Second, all stages of
the deep-learning models, including the  training objectives (defined by a loss
function), are designed to ensure differentiability with respect to model
parameters, allowing models to be trained efficiently with stochastic gradient
descent techniques.  With enough training data and computational power, deep-learning models can learn complex, highly nonlinear input–output mappings, such
as those found in Earth system models.
<?xmltex \hack{\newpage}?></p>
<sec id="Ch1.S3.SS1">
  <label>3.1</label><title>Architectures</title>
      <p id="d1e382">In this work we consider convolutional neural networks
<xref ref-type="bibr" rid="bib1.bibx36" id="paren.30"><named-content content-type="pre">CNNs;</named-content></xref> to model the spatial precipitation patterns
over time.  CNNs are able to process data with a known grid-like topology and
have been demonstrated as effective models for understanding image content
<xref ref-type="bibr" rid="bib1.bibx34 bib1.bibx33 bib1.bibx24" id="paren.31"/>. Unlike standard
fully connected neural networks, CNNs employ a convolution operation in place of
a general matrix multiplication in at least one of their layers. In a
convolutional layer, an <inline-formula><mml:math id="M5" display="inline"><mml:mrow><mml:mi>m</mml:mi><mml:mo>×</mml:mo><mml:mi>n</mml:mi><mml:mo>×</mml:mo><mml:mi>p</mml:mi></mml:mrow></mml:math></inline-formula> input tensor is convolved with a
set of <inline-formula><mml:math id="M6" display="inline"><mml:mi>k</mml:mi></mml:math></inline-formula> <inline-formula><mml:math id="M7" display="inline"><mml:mrow><mml:mi>i</mml:mi><mml:mo>×</mml:mo><mml:mi>j</mml:mi><mml:mo>×</mml:mo><mml:mi>p</mml:mi></mml:mrow></mml:math></inline-formula> kernels to output <inline-formula><mml:math id="M8" display="inline"><mml:mi>k</mml:mi></mml:math></inline-formula> feature maps that serve
as inputs for the next layer. In this setting, <inline-formula><mml:math id="M9" display="inline"><mml:mi>m</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math id="M10" display="inline"><mml:mi>n</mml:mi></mml:math></inline-formula> correspond to the
width and height of the input, and <inline-formula><mml:math id="M11" display="inline"><mml:mi>p</mml:mi></mml:math></inline-formula> corresponds to the depth (i.e., number of
channels).  Similarly, <inline-formula><mml:math id="M12" display="inline"><mml:mi>i</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math id="M13" display="inline"><mml:mi>j</mml:mi></mml:math></inline-formula> correspond to the width and height of the
kernel, and <inline-formula><mml:math id="M14" display="inline"><mml:mi>p</mml:mi></mml:math></inline-formula> corresponds to the depth, which is equal to that of the input
tensor. In practice we choose a kernel where <inline-formula><mml:math id="M15" display="inline"><mml:mrow><mml:mi>i</mml:mi><mml:mo>≪</mml:mo><mml:mi>m</mml:mi></mml:mrow></mml:math></inline-formula> and <inline-formula><mml:math id="M16" display="inline"><mml:mrow><mml:mi>j</mml:mi><mml:mo>≪</mml:mo><mml:mi>n</mml:mi></mml:mrow></mml:math></inline-formula> (e.g.,
<inline-formula><mml:math id="M17" display="inline"><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula>). By using a small kernel, we limit the number of parameters required by
the model while maintaining the ability to detect small, meaningful features in
a large and complex input space.  This both reduces memory requirements of the
model and improves statistical efficiency
<xref ref-type="bibr" rid="bib1.bibx34 bib1.bibx21" id="paren.32"/>. The learning capacity of
the model can also be adjusted by varying the width (i.e., number of kernels)
and the depth (i.e., number of layers) of the network.</p>
      <p id="d1e526">The spatial structure of each time step of the precipitation field bears a
resemblance to the structure of image data, making CNNs a promising candidate
model. However, to produce accurate forecasts, the model must also incorporate
temporal evolution of the precipitation field. To address long-term and
short-term trends we implement a sliding-window approach where our input tensor
is built using precipitation outcomes from the most recent <inline-formula><mml:math id="M18" display="inline"><mml:mi>K</mml:mi></mml:math></inline-formula> time steps as
input channels. Our model predicts the global precipitation outcome at next time
step.  Then this procedure is iterated, using output from the previous model forecast as input into the next one, allowing for arbitrarily long prediction horizons.</p>
      <?pagebreak page2306?><p id="d1e536">We consider adding depth to our network because
many studies in image
classification have achieved leading results using very deep models
<xref ref-type="bibr" rid="bib1.bibx54 bib1.bibx57 bib1.bibx23" id="paren.33"/>. Many of the fields of
interest in climate science are effectively images, making image classification an appropriate analogue for our aims.
In deep residual
networks <xref ref-type="bibr" rid="bib1.bibx24" id="paren.34"/>, rather than train each <inline-formula><mml:math id="M19" display="inline"><mml:mi>i</mml:mi></mml:math></inline-formula> layer to directly produce a new hidden
representation <inline-formula><mml:math id="M20" display="inline"><mml:mrow><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> given the hidden representation <inline-formula><mml:math id="M21" display="inline"><mml:mrow><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> it was
provided, we instead train the layer to produce a residual map <inline-formula><mml:math id="M22" display="inline"><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:msub><mml:mo>-</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula>, which is then summed with the <inline-formula><mml:math id="M23" display="inline"><mml:mrow><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> to give the
output for the layer.  This way, each layer explicitly refines the previous one.
The residual modeling approach is motivated by the idea that it is easier to
optimize residual mapping than to optimize the original, unreferenced mapping.
Architecturally, outputting <inline-formula><mml:math id="M24" display="inline"><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>h</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> at each layer is accomplished
by introducing <italic>shortcut</italic> connections <xref ref-type="bibr" rid="bib1.bibx24" id="paren.35"><named-content content-type="pre">e.g.,</named-content></xref> that skip one or
more layers in the network. In our CNN, a shortcut connection spans every few
consecutive layers. This identity mapping is then summed with the residual
mapping <inline-formula><mml:math id="M25" display="inline"><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> produced by the series of stacked layers encompassed by the
shortcut connection.  While developing our model, we also explored using plain
CNNs (without shortcut connections).  The best-performing model is described in
the following section.  All of the neural-network architectures considered in this study are summarized in Table <xref ref-type="table" rid="Ch1.T1"/>.</p>

<table-wrap id="Ch1.T1" specific-use="star"><?xmltex \currentcnt{1}?><label>Table 1</label><caption><p id="d1e714">Description of all of the neural-network architectures used in this study, including the number of layers, whether the network is a residual or plain network (plain networks do not have shortcut connections), whether scheduled sampling was used, whether preforecasting was used, and the figures of the paper in which each network is used.  In addition, these results are compared with persistence forecasting and an autoregressive model.  Table <xref ref-type="table" rid="App1.Ch1.S1.T2"/> provides information about the hyperparameters used for each configuration, and Table <xref ref-type="table" rid="App1.Ch1.S1.T3"/> describes the hyperparameter space used in model training.</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="6">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="right"/>
     <oasis:colspec colnum="3" colname="col3" align="left"/>
     <oasis:colspec colnum="4" colname="col4" align="left"/>
     <oasis:colspec colnum="5" colname="col5" align="left"/>
     <oasis:colspec colnum="6" colname="col6" align="left"/>
     <oasis:thead>
       <oasis:row rowsep="1">
         <oasis:entry colname="col1">Name</oasis:entry>
         <oasis:entry colname="col2">No. layers</oasis:entry>
         <oasis:entry colname="col3">Residual?</oasis:entry>
         <oasis:entry colname="col4">Sched. samp.?</oasis:entry>
         <oasis:entry colname="col5">Preforecasting?</oasis:entry>
         <oasis:entry colname="col6">Figures</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>
         <oasis:entry colname="col1">18-layer residual</oasis:entry>
         <oasis:entry colname="col2">18</oasis:entry>
         <oasis:entry colname="col3">Residual</oasis:entry>
         <oasis:entry colname="col4">Yes</oasis:entry>
         <oasis:entry colname="col5">Yes</oasis:entry>
         <oasis:entry colname="col6">1, 2, 3, 4, 5, 6</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">18-layer residual (no scheduled sampling)</oasis:entry>
         <oasis:entry colname="col2">18</oasis:entry>
         <oasis:entry colname="col3">Residual</oasis:entry>
         <oasis:entry colname="col4">No</oasis:entry>
         <oasis:entry colname="col5">Yes</oasis:entry>
         <oasis:entry colname="col6">4, 5</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">18-layer residual (no preforecasting)</oasis:entry>
         <oasis:entry colname="col2">18</oasis:entry>
         <oasis:entry colname="col3">Residual</oasis:entry>
         <oasis:entry colname="col4">Yes</oasis:entry>
         <oasis:entry colname="col5">No</oasis:entry>
         <oasis:entry colname="col6">6</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">18-layer plain</oasis:entry>
         <oasis:entry colname="col2">18</oasis:entry>
         <oasis:entry colname="col3">Plain</oasis:entry>
         <oasis:entry colname="col4">Yes</oasis:entry>
         <oasis:entry colname="col5">Yes</oasis:entry>
         <oasis:entry colname="col6">1, 4, 5</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">5-layer plain</oasis:entry>
         <oasis:entry colname="col2">5</oasis:entry>
         <oasis:entry colname="col3">Plain</oasis:entry>
         <oasis:entry colname="col4">Yes</oasis:entry>
         <oasis:entry colname="col5">Yes</oasis:entry>
         <oasis:entry colname="col6">1, 4, 5</oasis:entry>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table></table-wrap>

</sec>
<sec id="Ch1.S3.SS2">
  <label>3.2</label><title>Implementation</title>

      <?xmltex \floatpos{t}?><fig id="Ch1.F1"><?xmltex \currentcnt{1}?><label>Figure 1</label><caption><p id="d1e887">Deep architectures for precipitation forecasting: <bold>(a)</bold>
18-layer residual network, <bold>(b)</bold> 18-layer plain network (no shortcut
connections), and <bold>(c)</bold> 5-layer plain network (no shortcut connections).</p></caption>
          <?xmltex \igopts{width=236.157874pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f01.png"/>

        </fig>

      <p id="d1e905">Our residual network implementation follows the work in <xref ref-type="bibr" rid="bib1.bibx24" id="text.36"/>. We
construct a 18-layer residual network with shortcut connections encompassing
every two consecutive convolutional layers (see
Fig. <xref ref-type="fig" rid="Ch1.F1"/>). Each convolutional layer uses a <inline-formula><mml:math id="M26" display="inline"><mml:mrow><mml:mn mathvariant="normal">3</mml:mn><mml:mo>×</mml:mo><mml:mn mathvariant="normal">3</mml:mn></mml:mrow></mml:math></inline-formula> kernel with a stride of 1, and zero padding is applied to preserve the spatial
dimensionality of the input throughout the network. Directly after the
convolution we apply batch normalization following practices in
<xref ref-type="bibr" rid="bib1.bibx28" id="text.37"/> and then the rectified linear unit (ReLU) <xref ref-type="bibr" rid="bib1.bibx46" id="paren.38"/> as the
nonlinear activation function. Every four layers we double the number of kernels
to increase the learning capacity of our model. When this occurs we zero pad our
identity mapping in the shortcut connection to match dimensions.</p>
      <p id="d1e931">The total number of parameters in the 18-layer deep neural network is 34 578,
relatively small
compared to the size of each input:  <inline-formula><mml:math id="M27" display="inline"><mml:mrow><mml:mn mathvariant="normal">60</mml:mn><mml:mo>⋅</mml:mo><mml:mn mathvariant="normal">128</mml:mn><mml:mo>⋅</mml:mo><mml:mn mathvariant="normal">64</mml:mn><mml:mo>=</mml:mo><mml:mn mathvariant="normal">491</mml:mn><mml:mspace width="0.125em" linebreak="nobreak"/><mml:mn mathvariant="normal">520</mml:mn></mml:mrow></mml:math></inline-formula>.  The length of the
training set is 1116 time steps.  For a fixed set of hyperparameters, training
each model takes 1–3 h (depending upon the number of training epochs) on a
single NVIDIA 980 Ti GPU (graphics processing unit).  The computational cost is effectively all in the
training; predictions take a matter of seconds, even on CPU (central processing unit) architectures.  For comparison, a 5-year simulation in an ESM can take roughly 1 d even on hundreds of CPUs.</p>
      <p id="d1e958">We chose to initialize our network parameters by drawing from a Gaussian
distribution as suggested in <xref ref-type="bibr" rid="bib1.bibx23" id="text.39"/>.  We use stochastic gradient
descent to optimize our loss function <inline-formula><mml:math id="M28" display="inline"><mml:mrow><mml:mi>L</mml:mi><mml:mo>(</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula>, which is defined as the mean
squared area-weighted difference, calculated as
            <disp-formula id="Ch1.E1" content-type="numbered"><label>1</label><mml:math id="M29" display="block"><mml:mrow><mml:mi>L</mml:mi><mml:mo>(</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac style="display"><mml:mn mathvariant="normal">1</mml:mn><mml:mi>Z</mml:mi></mml:mfrac></mml:mstyle><mml:munder><mml:mo movablelimits="false">∑</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:msup><mml:mfenced close="]" open="["><mml:mrow><mml:mfenced open="(" close=")"><mml:mrow><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>-</mml:mo><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mfenced><mml:mo>⋅</mml:mo><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mfenced><mml:mn mathvariant="normal">2</mml:mn></mml:msup><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
          where <inline-formula><mml:math id="M30" display="inline"><mml:mi>x</mml:mi></mml:math></inline-formula> iterates over the <inline-formula><mml:math id="M31" display="inline"><mml:mi>Z</mml:mi></mml:math></inline-formula> distinct spatial positions, <inline-formula><mml:math id="M32" display="inline"><mml:mrow><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the ground truth
outcome, <inline-formula><mml:math id="M33" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the CNN reconstruction (CNN with parameters
<inline-formula><mml:math id="M34" display="inline"><mml:mi mathvariant="italic">θ</mml:mi></mml:math></inline-formula>), and <inline-formula><mml:math id="M35" display="inline"><mml:mrow><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the cosine of latitude (for area weighting).</p>
      <p id="d1e1120">Our plain CNN baselines follow a similar structure as the 18-layer residual
network, but all shortcut connections are removed (see
Fig. <xref ref-type="fig" rid="Ch1.F1"/>). By evaluating the performance of the
18-layer plain network, we investigate the benefits of using shortcut
connections. We also experiment with a 5-layer plain network to understand how
depth affects the performance and trainability of our models.  Finally, we train
the 18-layer residual network without scheduled sampling (described below) to
determine if this training approach actually improves forecasting ability.</p>
</sec>
<sec id="Ch1.S3.SS3">
  <label>3.3</label><title>Training</title>
      <?pagebreak page2307?><p id="d1e1133">The distribution of our training data (aggregated over the entire <inline-formula><mml:math id="M36" display="inline"><mml:mrow><mml:mi>m</mml:mi><mml:mo>⋅</mml:mo><mml:mi>n</mml:mi><mml:mo>⋅</mml:mo><mml:mi>p</mml:mi></mml:mrow></mml:math></inline-formula> space) was heavy-tailed and positively skewed (Fisher kurtosis <inline-formula><mml:math id="M37" display="inline"><mml:mrow><mml:mo>≈</mml:mo><mml:mn mathvariant="normal">11.3</mml:mn></mml:mrow></mml:math></inline-formula>; Fischer–Pearson coefficient of skewness <inline-formula><mml:math id="M38" display="inline"><mml:mrow><mml:mo>≈</mml:mo><mml:mn mathvariant="normal">2.72</mml:mn></mml:mrow></mml:math></inline-formula>). Performance of deep architectures tends to improve when training
with Gaussian-like input features <xref ref-type="bibr" rid="bib1.bibx5" id="paren.40"/>, so we apply a log
transformation on our dataset to reduce skewness. In addition, we scale our
input values between <inline-formula><mml:math id="M39" display="inline"><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula> and 1, bringing the mean over the training set closer to
0. Scaling has been shown to balance out the rate at which parameters connected
to the inputs nodes learn, and having a mean closer to zero tends to speed up
learning by reducing the bias to update parameters in a particular direction
<xref ref-type="bibr" rid="bib1.bibx37" id="paren.41"/>.</p>
      <p id="d1e1189">Our model makes fixed-window forecasts: it requires <inline-formula><mml:math id="M40" display="inline"><mml:mi>K</mml:mi></mml:math></inline-formula> previous (ground truth
or predicted) precipitation outcomes to generate a forecast for the subsequent
time step.
During training, since ground truth is available for all time steps, one
typically feeds in ground truth data for all <inline-formula><mml:math id="M41" display="inline"><mml:mi>K</mml:mi></mml:math></inline-formula> input time steps.
After training, when used in the inference mode on new data without ground truth,
some or all of the <inline-formula><mml:math id="M42" display="inline"><mml:mi>K</mml:mi></mml:math></inline-formula> inputs will need to be previous model predictions.
Without care, this mismatch can lead to poor extrapolation:
the model is not used to consuming its own predictions as input, and any
mistakes made in early forecasts will be amplified in later forecasts.
Scheduled
sampling <xref ref-type="bibr" rid="bib1.bibx4" id="paren.42"/> alleviates this issue by gradually forcing
the model to use its own outputs at training time (despite the availability of
ground truth).  This is realized by a
sampling mechanism that randomly chooses to use the ground truth outcome with a
probability <inline-formula><mml:math id="M43" display="inline"><mml:mi mathvariant="italic">ϵ</mml:mi></mml:math></inline-formula>, or the model-generated outcome with probability <inline-formula><mml:math id="M44" display="inline"><mml:mrow><mml:mn mathvariant="normal">1</mml:mn><mml:mo>-</mml:mo><mml:mi mathvariant="italic">ϵ</mml:mi></mml:mrow></mml:math></inline-formula>, when constructing its input. In other words, if <inline-formula><mml:math id="M45" display="inline"><mml:mrow><mml:mi mathvariant="italic">ϵ</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula> the
model always uses the ground truth outcome from the previous time step, and when
<inline-formula><mml:math id="M46" display="inline"><mml:mrow><mml:mi mathvariant="italic">ϵ</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">0</mml:mn></mml:mrow></mml:math></inline-formula> the model is trained in the same setting as inference (prediction). As we
continue to train the model we gradually decrease <inline-formula><mml:math id="M47" display="inline"><mml:mi mathvariant="italic">ϵ</mml:mi></mml:math></inline-formula> from <inline-formula><mml:math id="M48" display="inline"><mml:mn mathvariant="normal">1</mml:mn></mml:math></inline-formula> to <inline-formula><mml:math id="M49" display="inline"><mml:mn mathvariant="normal">0</mml:mn></mml:math></inline-formula>
according to a linear decay function.  Practically speaking, this has the effect of explicitly degrading our initial states at training time; we discuss the implications for our results below.
To improve the forecasting ability of our
models, we employed scheduled sampling during training.  Scheduled sampling
requires a predetermined number of epochs (to decay properly).  For our results
that do not use scheduled sampling, we use early stopping <xref ref-type="bibr" rid="bib1.bibx60" id="paren.43"/>
as a regularization technique.</p>
      <p id="d1e1285">Each model has its own set of hyperparameters with the exception of window size, as modifying window size would result in each model being conditioned on different numbers of priors, making them difficult to compare fairly.
The most significant hyperparameters in our models are the learning rate,
input depth, and number of training epochs. For each model, we tuned these
hyperparameters using a random search <xref ref-type="bibr" rid="bib1.bibx6" id="paren.44"/> for 60
iterations each. Our best residual network used a learning rate of <inline-formula><mml:math id="M50" display="inline"><mml:mrow><mml:mo>∼</mml:mo><mml:mn mathvariant="normal">0.07</mml:mn></mml:mrow></mml:math></inline-formula>,
has a window size of 60, and was trained for 90 epochs with scheduled sampling. (See Appendix A for a discussion of window size.)  We
trained our baseline CNNs using the same window size and similar ranges for the
learning rate and number of training epochs in an attempt to generate comparable
models. Each model was trained on a single GPU.</p>
</sec>
</sec>
<sec id="Ch1.S4">
  <label>4</label><title>Predictability and performance</title>

      <?xmltex \floatpos{t}?><fig id="Ch1.F2"><?xmltex \currentcnt{2}?><label>Figure 2</label><caption><p id="d1e1312">Precipitation outcomes (mm d<inline-formula><mml:math id="M51" display="inline"><mml:msup><mml:mi/><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:msup></mml:math></inline-formula>) for
the period 1969–1989. Top shows the average output of the
CanESM2 over that period. Bottom shows the average output of a 252-month forecast over the same time period using the 18-layer
residual network with a window size of 60. Both models show qualitatively
similar features.</p></caption>
        <?xmltex \igopts{width=241.848425pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f02.png"/>

      </fig>

      <?xmltex \floatpos{t}?><fig id="Ch1.F3"><?xmltex \currentcnt{3}?><label>Figure 3</label><caption><p id="d1e1335">Comparison of the average precipitation outputs (mm
d<inline-formula><mml:math id="M52" display="inline"><mml:msup><mml:mi/><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:msup></mml:math></inline-formula>) of the CanESM2 (<inline-formula><mml:math id="M53" display="inline"><mml:mi>B</mml:mi></mml:math></inline-formula>) and the 18-layer residual network (<inline-formula><mml:math id="M54" display="inline"><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo mathvariant="normal" stretchy="false">^</mml:mo></mml:mover></mml:math></inline-formula>) using a
window size of 60 over the years 1969–1989. Top shows <inline-formula><mml:math id="M55" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>-</mml:mo><mml:mi>B</mml:mi></mml:mrow></mml:math></inline-formula>. Bottom
shows the percent error between <inline-formula><mml:math id="M56" display="inline"><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo mathvariant="normal" stretchy="false">^</mml:mo></mml:mover></mml:math></inline-formula> and <inline-formula><mml:math id="M57" display="inline"><mml:mi>B</mml:mi></mml:math></inline-formula>. The residual network
tends to underpredict near the Equator, midlatitude storm tracks, and areas
associated with monsoon precipitation.</p></caption>
        <?xmltex \igopts{width=241.848425pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f03.png"/>

      </fig>

      <?xmltex \floatpos{p}?><fig id="Ch1.F4" specific-use="star"><?xmltex \currentcnt{4}?><label>Figure 4</label><caption><p id="d1e1409">RMSE for decadal precipitation forecasts for six regions of interest.
Both the plain and residual CNNs used a window size of 60. Vertical bars denote
the standard error over all possible starting dates in the test set. The deep
residual network with scheduled sampling outperforms all models in all regions
and achieves consistent error over time.  Removing any of these features from
the network results in substantially lower performance.</p></caption>
        <?xmltex \igopts{width=483.69685pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f04.png"/>

      </fig>

      <?xmltex \floatpos{p}?><fig id="Ch1.F5" specific-use="star"><?xmltex \currentcnt{5}?><label>Figure 5</label><caption><p id="d1e1420">Anomaly correlation coefficient for decadal precipitation forecasts
for six regions of interest. Both the plain and residual CNNs used a window
size of 60. Vertical bars denote the standard error over all possible starting
dates in the test set. As in Fig. <xref ref-type="fig" rid="Ch1.F4"/>, the deep
residual network with scheduled sampling outperforms all models in all regions,
consistently exhibiting a positive correlation with the ground truth outcomes.</p></caption>
        <?xmltex \igopts{width=483.69685pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f05.png"/>

      </fig>

      <p id="d1e1431">We evaluate the forecasting ability of our models using the CanESM2 output over
the period 1969–1989 as ground truth. We also compare the performance of our
best models to two naive methods of forecasting.  The lowest bar, persistence forecasting,  extrapolates the most recent
precipitation outcome (i.e., the  last outcome in the development set), assuming that the
climatology remains unchanged over time.  In perhaps a closer comparison, we also use a first-order autoregressive model in each grid box, abbreviated AR(1), in which the most recent precipitation outcome depends upon a linear combination of the previous value and a stochastic (white noise) term:
          <disp-formula id="Ch1.E2" content-type="numbered"><label>2</label><mml:math id="M58" display="block"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo mathvariant="normal" stretchy="false">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mi>c</mml:mi><mml:mo>+</mml:mo><mml:mi mathvariant="italic">ϕ</mml:mi><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>)</mml:mo><mml:mo>+</mml:mo><mml:mi mathvariant="italic">ϵ</mml:mi><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
        where <inline-formula><mml:math id="M59" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the surrogate prediction at time <inline-formula><mml:math id="M60" display="inline"><mml:mi>i</mml:mi></mml:math></inline-formula>, <inline-formula><mml:math id="M61" display="inline"><mml:mi>c</mml:mi></mml:math></inline-formula> is a constant, <inline-formula><mml:math id="M62" display="inline"><mml:mi mathvariant="italic">ϵ</mml:mi></mml:math></inline-formula> represents white noise, and <inline-formula><mml:math id="M63" display="inline"><mml:mi mathvariant="italic">ϕ</mml:mi></mml:math></inline-formula> is a parameter that is tuned or trained such that the model has accurate predictions over the development set.</p>
      <?pagebreak page2308?><p id="d1e1552">To quantify our
generalization error, we compute the root-mean square over the area-weighted
difference <inline-formula><mml:math id="M64" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo mathvariant="normal" stretchy="false">^</mml:mo></mml:mover><mml:mo>-</mml:mo><mml:mi>B</mml:mi></mml:mrow></mml:math></inline-formula> for six different spatial regions – polar,
midlatitude, and tropics over both land and ocean. This is calculated as
          <disp-formula id="Ch1.E3" content-type="numbered"><label>3</label><mml:math id="M65" display="block"><mml:mrow><mml:mtext>RMSE</mml:mtext><mml:mo>=</mml:mo><mml:msqrt><mml:mstyle displaystyle="true"><mml:mfrac style="display"><mml:mrow><mml:msub><mml:mo>∑</mml:mo><mml:mi>x</mml:mi></mml:msub><mml:mo>[</mml:mo><mml:mo>(</mml:mo><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo><mml:mo>-</mml:mo><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>)</mml:mo><mml:mo>⋅</mml:mo><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>⋅</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:msup><mml:mo>]</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:msup></mml:mrow><mml:mrow><mml:msub><mml:mo>∑</mml:mo><mml:mi>x</mml:mi></mml:msub><mml:mo>[</mml:mo><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>⋅</mml:mo><mml:msub><mml:mi>M</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:msup><mml:mo>]</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:msup></mml:mrow></mml:mfrac></mml:mstyle></mml:msqrt><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
        where <inline-formula><mml:math id="M66" display="inline"><mml:mi>x</mml:mi></mml:math></inline-formula> iterates over the spatial positions, <inline-formula><mml:math id="M67" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>;</mml:mo><mml:mi mathvariant="italic">θ</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the
surrogate prediction,
<inline-formula><mml:math id="M68" display="inline"><mml:mrow><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the ground truth outcome, <inline-formula><mml:math id="M69" display="inline"><mml:mrow><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the cosine of latitude
weights, and <inline-formula><mml:math id="M70" display="inline"><mml:mrow><mml:msub><mml:mi>M</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is the region mask (for <inline-formula><mml:math id="M71" display="inline"><mml:mrow><mml:mi>k</mml:mi><mml:mo>∈</mml:mo><mml:mo mathvariant="italic">{</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>,</mml:mo><mml:mn mathvariant="normal">2</mml:mn><mml:mo>,</mml:mo><mml:mi mathvariant="normal">…</mml:mi><mml:mo>,</mml:mo><mml:mn mathvariant="normal">6</mml:mn><mml:mo mathvariant="italic">}</mml:mo></mml:mrow></mml:math></inline-formula>).</p>
      <p id="d1e1789">In addition to the root-mean-square error (RMSE), we also compute the anomaly correlation coefficient (ACC), a commonly used metric for quantifying differences in spatial fields in forecasting <xref ref-type="bibr" rid="bib1.bibx32" id="paren.45"/>.  The ACC is defined as <xref ref-type="bibr" rid="bib1.bibx31" id="paren.46"/>
          <disp-formula id="Ch1.E4" content-type="numbered"><label>4</label><mml:math id="M72" display="block"><mml:mrow><mml:mtext>ACC</mml:mtext><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac style="display"><mml:mrow><mml:msubsup><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:msubsup><mml:mo>(</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover><mml:mo>)</mml:mo><mml:mo>(</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mover accent="true"><mml:mi>a</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover><mml:mo>)</mml:mo></mml:mrow><mml:msqrt><mml:mrow><mml:msubsup><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:msubsup><mml:mo>(</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover><mml:msup><mml:mo>)</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:msup><mml:msubsup><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:msubsup><mml:mo>(</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mover accent="true"><mml:mi>a</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover><mml:msup><mml:mo>)</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:msup></mml:mrow></mml:msqrt></mml:mfrac></mml:mstyle><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>
        where <inline-formula><mml:math id="M73" display="inline"><mml:mi>n</mml:mi></mml:math></inline-formula> is the number of samples,  <inline-formula><mml:math id="M74" display="inline"><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is the difference between forecast
and reference, and <inline-formula><mml:math id="M75" display="inline"><mml:mrow><mml:msub><mml:mi>a</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is the difference between some verifying value and the
reference. We use the average precipitation over the period 1938–1968 as our
reference (the 30 years preceding the test set period).  <inline-formula><mml:math id="M76" display="inline"><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover></mml:math></inline-formula> and <inline-formula><mml:math id="M77" display="inline"><mml:mover accent="true"><mml:mi>a</mml:mi><mml:mo mathvariant="normal">¯</mml:mo></mml:mover></mml:math></inline-formula> indicate area-weighted averages over the number of samples.  ACC can take values in <inline-formula><mml:math id="M78" display="inline"><mml:mrow><mml:mo>[</mml:mo><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>,</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula>, where an ACC of 1 indicates that the anomalies of the forecast match the anomalies of the verifying value, and an ACC of <inline-formula><mml:math id="M79" display="inline"><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:math></inline-formula> indicates a reversal of the variation pattern.  Figure <xref ref-type="fig" rid="Ch1.F5"/> shows ACC values for the different models considered in this study.  The message is similar to that of Fig. <xref ref-type="fig" rid="Ch1.F4"/>, with the 18-layer residual network showing the greatest skill (ACC exceeding 0.5 in all six regions), the other neural networks showing little skill, and the persistence forecast showing variable skill, depending upon the forecast length.  Although it is difficult to make exact quantitative comparisons, the 18-layer residual network has higher values of ACC than the Community Earth System Model Decadal Prediction Large Ensemble (CESM-DPLE) in all six regions <xref ref-type="bibr" rid="bib1.bibx61" id="paren.47"/>.  Performance is similar over the Sahel, indicating some ability of the residual network to capture relevant precipitation dynamics.</p>
      <?pagebreak page2310?><p id="d1e2009">Figure <xref ref-type="fig" rid="Ch1.F2"/> shows the average precipitation  for  a 252-month forecast over the period 1969–1989 in
the 18-layer residual network (CNN with lowest forecasting error) and the average precipitation over the same period in the CanESM2 under a 1pctCO2 simulation.
Both models show qualitatively similar features, indicating that the residual
network was capable of reproducing Earth system model outputs reasonably well.
Figure <xref ref-type="fig" rid="Ch1.F3"/> shows the area-weighted difference <inline-formula><mml:math id="M80" display="inline"><mml:mrow><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo stretchy="false" mathvariant="normal">^</mml:mo></mml:mover><mml:mo>-</mml:mo><mml:mi>B</mml:mi></mml:mrow></mml:math></inline-formula> as well as the area-weighted percent error given by</p>
      <p id="d1e2031"><disp-formula id="Ch1.E5" content-type="numbered"><label>5</label><mml:math id="M81" display="block"><mml:mrow><mml:mi mathvariant="normal">pct</mml:mi><mml:mi mathvariant="normal">_</mml:mi><mml:mi mathvariant="normal">err</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:mfrac style="display"><mml:mrow><mml:mo>(</mml:mo><mml:mover accent="true"><mml:mi>B</mml:mi><mml:mo mathvariant="normal" stretchy="false">^</mml:mo></mml:mover><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>-</mml:mo><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>)</mml:mo><mml:mo>⋅</mml:mo><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mi>B</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo><mml:mo>⋅</mml:mo><mml:mi>A</mml:mi><mml:mo>(</mml:mo><mml:mi>x</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mfrac></mml:mstyle><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p>
      <p id="d1e2106">The residual model is prone to underpredict near the Equator, in the midlatitude
storm tracks, and in areas associated with monsoon precipitation.  All of these
regions experience intense precipitation events (storms), which constitute the
right tail of the precipitation distribution.  The mean-square error loss
function is less robust to outliers <xref ref-type="bibr" rid="bib1.bibx14" id="paren.48"><named-content content-type="post">chap. 10</named-content></xref>,
which are far more common in these regions than others, potentially explaining
why the residual network tends to be biased low in these regions.  On average,
our model achieves reasonably low error on the test set, with a mean
precipitation difference of <inline-formula><mml:math id="M82" display="inline"><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">0.237</mml:mn></mml:mrow></mml:math></inline-formula> mm d<inline-formula><mml:math id="M83" display="inline"><mml:msup><mml:mi/><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">1</mml:mn></mml:mrow></mml:msup></mml:math></inline-formula> and mean percent
error of <inline-formula><mml:math id="M84" display="inline"><mml:mrow><mml:mo>-</mml:mo><mml:mn mathvariant="normal">13.22</mml:mn></mml:mrow></mml:math></inline-formula> %.</p>
      <p id="d1e2146">Figure <xref ref-type="fig" rid="Ch1.F4"/> shows the forecasting performance of each model
on a decadal timescale. The 18-layer residual network outperforms all models in
all regions and exhibits relatively consistent error over time. The AR(1) model generally performs second best in all cases except some seasons in the tropics when persistence tends to perform better.  Our plain CNNs
have less consistent error  over time and performed worse than our residual
networks overall. These networks proved more difficult to optimize and would
often learn to predict similar values at each pixel regardless of spatial
location. The 5-layer network showed lower generalization error than the
18-layer network, which was expected behavior, as plain networks become more
difficult to train with increased depth <xref ref-type="bibr" rid="bib1.bibx24" id="paren.49"/>. This challenge is
well addressed in our 18-layer residual network, however, as it achieves good
accuracy with significant depth.</p>
      <p id="d1e2154">To assess the benefits of scheduled sampling, we evaluated the performance of an
identical residual network architecture trained without scheduled sampling (see
“18-layer residual (no scheduled sampling)” in
Fig. <xref ref-type="fig" rid="Ch1.F4"/>). For this model we observe the RMSE quickly
increasing during the first few forecasts, indicating that it is not accustomed
to making forecasts conditioned on past predictions. Surprisingly, this model
also had significantly higher RMSE for the 1-month forecast, which is entirely
conditioned on ground truth outcomes. We would expect a model trained without
scheduled sampling to perform well in this case, as the input contains no model-generated data. However, as there are sufficient differences in the training
setting (i.e., the use of early stopping), it is likely that these models
converged to disparate minima. We hypothesize that additional hyperparameter
tuning could decrease the RMSE for 1-month forecasts in these models.</p>

      <?xmltex \floatpos{t}?><fig id="Ch1.F6" specific-use="star"><?xmltex \currentcnt{6}?><label>Figure 6</label><caption><p id="d1e2162">Comparing the RMSE for decadal precipitation forecasts with and without
preforecasting. The model was trained using a window size of 60, and 30 preforecasts
were generated. Vertical bars denote the standard error over all possible
starting dates in the test set. Preforecasting significantly reduces the RMSE
for short-range forecasts without degrading long-range forecasts.</p></caption>
        <?xmltex \igopts{width=483.69685pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f06.png"/>

      </fig>

      <p id="d1e2171">Importantly, for the model using scheduled sampling, the skill of the forecast does not change appreciably with lead time, whereas one might expect the model to have some initial-value predictability <xref ref-type="bibr" rid="bib1.bibx8" id="paren.50"><named-content content-type="pre">e.g.,</named-content></xref> and thus more skillful predictability in the near term.  In early forecasts, the input tensors primarily consist of ground truth precipitation
outcomes. A model trained using scheduled sampling performs worse in this
setting because it is accustomed to seeing inputs that contain model-generated
data – that is, the initial states are explicitly degraded.  This was a choice in terms of the problem we attempted to solve (reducing initial-value predictability in favor of longer forecasts), and scheduled sampling may not be an appropriate choice for other applications.
To address this poor early forecast skill, we explored <italic>preforecasting</italic> – generating
forecasts preceding the first forecast date to prime the model for inference (prediction). By
taking this approach we ensure that the input tensor for the first forecast will
contain at least some portion of model-generated data. To employ preforecasting,
the number of preceding outcomes generated must be in the range <inline-formula><mml:math id="M85" display="inline"><mml:mrow><mml:mo>[</mml:mo><mml:mn mathvariant="normal">1</mml:mn><mml:mo>,</mml:mo><mml:mi mathvariant="normal">…</mml:mi><mml:mo>,</mml:mo><mml:mtext>window</mml:mtext><mml:mi mathvariant="italic">_</mml:mi><mml:mtext>size</mml:mtext><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula> and should be chosen relative to the sampling decay function
used during training. We suggest generating <inline-formula><mml:math id="M86" display="inline"><mml:mrow><mml:mtext>window</mml:mtext><mml:mi mathvariant="italic">_</mml:mi><mml:mtext>size</mml:mtext><mml:mo>/</mml:mo><mml:mn mathvariant="normal">2</mml:mn></mml:mrow></mml:math></inline-formula> preforecasts for a
model trained using a linear decay function. We take this approach in
Fig. <xref ref-type="fig" rid="Ch1.F6"/> and find that it adequately reduces the
RMSE for early forecasts while still maintaining low error for longer forecasts.</p>
</sec>
<sec id="Ch1.S5" sec-type="conclusions">
  <label>5</label><title>Discussion and conclusions</title>
      <p id="d1e2233">This study explored the application of deep-learning techniques to create
surrogate models for precipitation fields in Earth system models under <inline-formula><mml:math id="M87" display="inline"><mml:mrow class="chem"><mml:msub><mml:mi mathvariant="normal">CO</mml:mi><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:mrow></mml:math></inline-formula> forcing. From our
experiments we found that a CNN architecture was effective for modeling
spatio-temporal precipitation patterns. We also observed increased accuracy with
deeper networks, which could be adequately trained using a residual learning
approach. Finally, we found that scheduled sampling (supplemented with
preforecasting) significantly improved long-term forecasting ability, improving upon the commonly used autoregressive model (although we admit that we could have more thoroughly explored the span of different linear methods, such as higher-order AR or ARIMA models).</p>
      <p id="d1e2247">It might be expected that the forecast model skill asymptotically approaches persistence as the predictions move farther from the initial state.  We argue three reasons for why our neural network continues to have good skill and low error:
<list list-type="order"><list-item>
      <p id="d1e2252">Scheduled sampling helps the model extrapolate from its own predictions, reducing errors in later forecasts.</p></list-item><list-item>
      <p id="d1e2256">Because the model is trained on numerous time periods in the 1pctCO2 experiment, it learns some inherent properties of precipitation response to <inline-formula><mml:math id="M88" display="inline"><mml:mrow class="chem"><mml:msub><mml:mi mathvariant="normal">CO</mml:mi><mml:mn mathvariant="normal">2</mml:mn></mml:msub></mml:mrow></mml:math></inline-formula> forcing.</p></list-item><list-item>
      <p id="d1e2271">We condition each prediction on 5 years worth of data, so it is likely easier for our model to retain signals coming from the initial conditions.</p></list-item></list>
Appendix A provides a comparison between using window sizes of 6 and 60 months, with the former showing steadily decreasing predictive skill due to its inability to learn the forced response.  This is another point of verification for a conclusion well known in the decadal predictability community:  although the initial state is important for decadal predictions, in forced runs, a great deal of skill is due to the underlying climate signals <xref ref-type="bibr" rid="bib1.bibx7" id="paren.51"/>.</p>
      <p id="d1e2278">Based on
these results we can identify several ways to enhance our current surrogate
models, as well as a few promising deep-learning architectures applicable to
this work, with the overall goal of understanding which deep-learning techniques
may work well for creating surrogates of climate models.</p>
      <p id="d1e2281"><xref ref-type="bibr" rid="bib1.bibx4" id="text.52"/> proposed three scheduled sampling decay functions:
linear, exponential, and inverse sigmoid. Determining the optimal decay schedule
for our problem could have significant effects on model predictability. Weight
initialization has also been proven to affect model convergence and gradient
propagation <xref ref-type="bibr" rid="bib1.bibx23 bib1.bibx17" id="paren.53"/>; therefore this must
be investigated further. Window size was a fixed hyperparameter during tuning,
but we cannot rule out its potential impact on forecasting (see Appendix A). Tuning these
existing hyperparameters would be the first step in improving results.</p>
      <p id="d1e2290">Incorporating additional features, such as data from Earth system models with
different forcings, global mean temperature, and daily average precipitation,
would provide more relevant information and likely improve predictability.
These could be incorporated by modifying our input tensor to include these as
additional channels.  Such augmentations would be an important step toward
designing practical, effective surrogate models in the future.</p>
      <p id="d1e2293">Two architectural features that we may consider adding are dropout and replay
learning. <xref ref-type="bibr" rid="bib1.bibx55" id="text.54"/> showed that adding dropout with
convolutional layers may lead to a performance increase and prevent overfitting.
Replay learning is widely used in deep reinforcement learning and was shown to
be successful in challenging domains <xref ref-type="bibr" rid="bib1.bibx64" id="paren.55"/>. We believe that we
can apply a similar concept to our architecture, where we train our network on
random past input–output pairs so it can “remember” what it learned
previously. This technique could aid in alleviating any bias from more recent
training data and therefore boost performance.</p>
      <p id="d1e2302">Convolutional long short-term memory networks (LSTMs) have had great success in
video prediction <xref ref-type="bibr" rid="bib1.bibx13" id="paren.56"/> and even precipitation nowcasting
<xref ref-type="bibr" rid="bib1.bibx53" id="paren.57"/>.  They offer an alternate strategy for modeling both the
spatial and temporal aspects present in our dataset: whereas our model treats
the time dimension as channels, and acts upon a fixed length input window,
convolutional LSTMs use  recurrent connections to consume the input one
time step at a time with no fixed length restriction on the input. This
increases the effective depth between information contained in
the distant past and the prediction, which may prove beneficial or harmful for
our task; we leave it to future work to evaluate their suitability.
One could also draw inspiration from the spatial downsampling,
upsampling, and specific skip connectivity techniques often used work in semantic
segmentation <xref ref-type="bibr" rid="bib1.bibx51" id="paren.58"><named-content content-type="pre">e.g., U-Nets, introduced for biomedical image
segmentation;</named-content></xref>.</p>
      <?pagebreak page2312?><p id="d1e2316">Generative adversarial networks <xref ref-type="bibr" rid="bib1.bibx20" id="paren.59"><named-content content-type="pre">GANs;</named-content></xref> have
proven to offer impressive generative modeling of grid-structured data. GANs are
commonly used with image data for tasks such as super-resolution, image-to-image
translation, image generation, and representation learning. The effectiveness of
GANs to generate realistic data and ability to be conditioned on other variables
<xref ref-type="bibr" rid="bib1.bibx19" id="paren.60"/> make them quite appealing for spatio-temporal
forecasting.</p>
      <p id="d1e2327"><?xmltex \hack{\newpage}?>The results presented here show significant potential for deep learning to
improve climate predictability. Applications of this work extend beyond
providing a CNN forecasting library, as there are several enhancements that
could yield a more practical alternative to traditional Earth system models.
The ability to emulate alternate climate scenarios, for example, is desirable.
Regardless, using an approach that can incorporate the internal model state
appears to have promise in increasing prediction accuracy.</p><?xmltex \hack{\clearpage}?>
</sec>

      
      </body>
    <back><app-group>

<?pagebreak page2313?><app id="App1.Ch1.S1">
  <?xmltex \currentcnt{A}?><label>Appendix A</label><title>Effects of window size</title>
      <p id="d1e2343">We briefly explored the effects of different window sizes on predictability for the 18-layer residual network (without scheduled sampling).  Figure <xref ref-type="fig" rid="App1.Ch1.S1.F7"/> shows a  comparison between a window size of 6 months versus the 60-month window that we used in our best-performing model.  With a smaller window size, the forecast model is unable to learn enough of the forced response to improve predictive skill, so the forecast's model skill approaches that of persistence.</p>

      <?xmltex \floatpos{h!}?><fig id="App1.Ch1.S1.F7"><?xmltex \currentcnt{A1}?><label>Figure A1</label><caption><p id="d1e2350">Comparing the RMSE for decadal precipitation forecasts in the 18-layer
residual network with a 6-month window (top) and a 60-month window (bottom).
Both networks were trained using scheduled sampling, and preforecasting was not
employed when generating predictions.</p></caption>
        <?xmltex \hack{\hsize\textwidth}?>
        <?xmltex \igopts{width=483.69685pt}?><graphic xlink:href="https://acp.copernicus.org/articles/20/2303/2020/acp-20-2303-2020-f07.png"/>

      </fig>

<table-wrap id="App1.Ch1.S1.T2" specific-use="star"><?xmltex \currentcnt{A1}?><label>Table A1</label><caption><p id="d1e2363">Hyperparameters for each network architecture used in this study (see
Table <xref ref-type="table" rid="Ch1.T1"/>).  For each architecture, we tuned the learning rate,
standard deviation used for weight initialization (sampling from a truncated
normal distribution), and number of training epochs (unless training without
scheduled sampling, in which case we used early stopping with a patience
threshold of 10).  For each architecture, the optimal hyperparameter
configuration was selected after 60 iterations of random search.</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="6">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="left"/>
     <oasis:colspec colnum="3" colname="col3" align="right"/>
     <oasis:colspec colnum="4" colname="col4" align="right"/>
     <oasis:colspec colnum="5" colname="col5" align="right"/>
     <oasis:colspec colnum="6" colname="col6" align="left"/>
     <oasis:thead>
       <oasis:row>
         <oasis:entry colname="col1">Architecture</oasis:entry>
         <oasis:entry colname="col2">Decay</oasis:entry>
         <oasis:entry colname="col3">Window</oasis:entry>
         <oasis:entry colname="col4">Learning</oasis:entry>
         <oasis:entry colname="col5">Standard deviation</oasis:entry>
         <oasis:entry colname="col6">Epochs</oasis:entry>
       </oasis:row>
       <oasis:row rowsep="1">
         <oasis:entry colname="col1"/>
         <oasis:entry colname="col2">function</oasis:entry>
         <oasis:entry colname="col3">size</oasis:entry>
         <oasis:entry colname="col4">rate</oasis:entry>
         <oasis:entry colname="col5">(for Xavier initialization)</oasis:entry>
         <oasis:entry colname="col6"/>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>
         <oasis:entry colname="col1">18-layer residual</oasis:entry>
         <oasis:entry colname="col2">Linear</oasis:entry>
         <oasis:entry colname="col3">60</oasis:entry>
         <oasis:entry colname="col4">0.069</oasis:entry>
         <oasis:entry colname="col5">0.016</oasis:entry>
         <oasis:entry colname="col6">90</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">18-layer residual (no scheduled sampling)</oasis:entry>
         <oasis:entry colname="col2">N/A</oasis:entry>
         <oasis:entry colname="col3">60</oasis:entry>
         <oasis:entry colname="col4">0.095</oasis:entry>
         <oasis:entry colname="col5">0.021</oasis:entry>
         <oasis:entry colname="col6">82 (early stopping)</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">18-layer plain</oasis:entry>
         <oasis:entry colname="col2">Linear</oasis:entry>
         <oasis:entry colname="col3">60</oasis:entry>
         <oasis:entry colname="col4">0.013</oasis:entry>
         <oasis:entry colname="col5">0.01</oasis:entry>
         <oasis:entry colname="col6">100</oasis:entry>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">5-layer plain</oasis:entry>
         <oasis:entry colname="col2">Linear</oasis:entry>
         <oasis:entry colname="col3">60</oasis:entry>
         <oasis:entry colname="col4">0.049</oasis:entry>
         <oasis:entry colname="col5">0.01</oasis:entry>
         <oasis:entry colname="col6">125</oasis:entry>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table></table-wrap>

<table-wrap id="App1.Ch1.S1.T3" specific-use="star"><?xmltex \currentcnt{A2}?><label>Table A2</label><caption><p id="d1e2525">Hyperparameter space used for training the models described in Table <xref ref-type="table" rid="Ch1.T1"/>.  For more information on how the hyperparameter space is defined, see <uri>https://github.com/hyperopt/hyperopt/wiki/FMin#21-parameter-expressions</uri> (last access: February 2020).</p></caption><oasis:table frame="topbot"><oasis:tgroup cols="6">
     <oasis:colspec colnum="1" colname="col1" align="left"/>
     <oasis:colspec colnum="2" colname="col2" align="right"/>
     <oasis:colspec colnum="3" colname="col3" align="right"/>
     <oasis:colspec colnum="4" colname="col4" align="right"/>
     <oasis:colspec colnum="5" colname="col5" align="left"/>
     <oasis:colspec colnum="6" colname="col6" align="left"/>
     <oasis:thead>
       <oasis:row rowsep="1">
         <oasis:entry colname="col1">Hyperparameter</oasis:entry>
         <oasis:entry colname="col2">Min</oasis:entry>
         <oasis:entry colname="col3">Max</oasis:entry>
         <oasis:entry colname="col4">Step</oasis:entry>
         <oasis:entry colname="col5">Sampled from</oasis:entry>
         <oasis:entry colname="col6">Comment</oasis:entry>
       </oasis:row>
     </oasis:thead>
     <oasis:tbody>
       <oasis:row>
         <oasis:entry colname="col1">Epochs</oasis:entry>
         <oasis:entry colname="col2">75</oasis:entry>
         <oasis:entry colname="col3">150</oasis:entry>
         <oasis:entry colname="col4">5</oasis:entry>
         <oasis:entry colname="col5">Quantized uniform</oasis:entry>
         <oasis:entry colname="col6"/>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">Learning rate</oasis:entry>
         <oasis:entry colname="col2">0.001</oasis:entry>
         <oasis:entry colname="col3">0.01</oasis:entry>
         <oasis:entry colname="col4">n/a</oasis:entry>
         <oasis:entry colname="col5">Uniform</oasis:entry>
         <oasis:entry colname="col6"/>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">Standard deviation</oasis:entry>
         <oasis:entry colname="col2">0.001</oasis:entry>
         <oasis:entry colname="col3">0.1</oasis:entry>
         <oasis:entry colname="col4">n/a</oasis:entry>
         <oasis:entry colname="col5">Uniform</oasis:entry>
         <oasis:entry colname="col6"/>
       </oasis:row>
       <oasis:row>
         <oasis:entry colname="col1">Window size</oasis:entry>
         <oasis:entry colname="col2">6</oasis:entry>
         <oasis:entry colname="col3">120</oasis:entry>
         <oasis:entry colname="col4">n/a</oasis:entry>
         <oasis:entry colname="col5">Choice</oasis:entry>
         <oasis:entry colname="col6">Only varied during initial experiments; fixed at 60 for the study</oasis:entry>
       </oasis:row>
     </oasis:tbody>
   </oasis:tgroup></oasis:table><table-wrap-foot><p id="d1e2533">n/a means not applicable.</p></table-wrap-foot></table-wrap>

<?xmltex \hack{\clearpage}?>
</app>
  </app-group><notes notes-type="codedataavailability"><title>Code and data availability</title>

      <p id="d1e2673">All models were developed in Python using the machine learning
framework TensorFlow, developed by Google. Training and inference scripts are
available at: <uri>https://github.com/hutchresearch/deep_climate_emulator</uri> (<xref ref-type="bibr" rid="bib1.bibx59" id="altparen.61"/>). Code
used to generate the figures in this paper is available upon request.  All
climate model output used in this study is available through the Earth System
Grid Federation.</p>
  </notes><notes notes-type="authorcontribution"><title>Author contributions</title>

      <p id="d1e2685">TW and AC developed the algorithms under the guidance of BH. TW conducted the analysis, with input from all co-authors.  TW and BK wrote the paper, with help from all co-authors.</p>
  </notes><notes notes-type="competinginterests"><title>Competing interests</title>

      <p id="d1e2691">The authors declare that they have no conflict of interest.</p>
  </notes><ack><title>Acknowledgements</title><p id="d1e2697">We thank Nathan Urban and two anonymous reviewers for helpful comments.
The authors would also
like to thank Md. Monsur Hossain and Vincent Nguyen from Western Washington
University for their contributions to model development and the Nvidia
Corporation for donating GPUs used in this research.</p></ack><notes notes-type="financialsupport"><title>Financial support</title>

      <p id="d1e2702">This research has been supported in part under the Laboratory Directed Research and Development Program at Pacific Northwest National Laboratory, a multiprogram national laboratory operated by Battelle for the US Department of Energy. The Pacific Northwest National Laboratory is operated for the US Department of Energy by the Battelle Memorial Institute under contract DE-AC05-76RL01830. Support for Ben Kravitz was provided in part by the National Science Foundation through agreement CBET-1931641, the Indiana University Environmental Resilience Institute, and the Prepared for Environmental Change Grand Challenge initiative.</p>
  </notes><notes notes-type="reviewstatement"><title>Review statement</title>

      <p id="d1e2708">This paper was edited by Martina Krämer and reviewed by Nathan Urban and two anonymous referees.</p>
  </notes><ref-list>
    <title>References</title>

      <ref id="bib1.bibx1"><label>Arora and Boer(2011)</label><?label aroraboer2010?><mixed-citation>Arora, V. K. and Boer, G. J.: Uncertainties in the 20th century carbon budget
associated with land use change, Glob. Change Biol., 16, 3327–3348,
<ext-link xlink:href="https://doi.org/10.1111/j.1365-2486.2010.02202.x" ext-link-type="DOI">10.1111/j.1365-2486.2010.02202.x</ext-link>, 2011.</mixed-citation></ref>
      <ref id="bib1.bibx2"><label>Arora et al.(2011)Arora, Scinocca, Boer, Christian, Denman, Flato,
Kharin, lee, and Merryfield</label><?label arora2011?><mixed-citation>Arora, V. K., Scinocca, J. F., Boer, G. J., Christian, J. R., Denman, K. L.,
Flato, G. M., Kharin, V. V., Lee, W. G., and Merryfield, W. J.: Carbon
emission limits required to satisfy future representative concentration
pathways of greenhouse gases, Geophys. Res. Lett., 38, L05805,
<ext-link xlink:href="https://doi.org/10.1029/2010GL046270" ext-link-type="DOI">10.1029/2010GL046270</ext-link>, 2011.</mixed-citation></ref>
      <ref id="bib1.bibx3"><label>Bellucci et al.(2015)Bellucci, Haarsma, Bellouin, Booth, Cagnazzo,
van den Hurk, Keenlyside, Koenigk, Massonnet, Materia, and
Weiss</label><?label bellucci2015?><mixed-citation>Bellucci, A., Haarsma, R., Bellouin, N., Booth, B., Cagnazzo, C., van den Hurk,
B., Keenlyside, N., Koenigk, T., Massonnet, F., Materia, S., and Weiss, M.:
Advancements in decadal climate predictability: The role of nonoceanic
drivers, Rev. Geophys., 53, 165–202, <ext-link xlink:href="https://doi.org/10.1002/2014RG000473" ext-link-type="DOI">10.1002/2014RG000473</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx4"><label>Bengio et al.(2015)Bengio, Vinyals, Jaitly, and
Shazeer</label><?label bengio2015scheduled?><mixed-citation>
Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N.: Scheduled sampling for
sequence prediction with recurrent neural networks, in: Advances in Neural
Information Processing Systems, NIPS Proceedings, 1171–1179, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx5"><label>Bengio(2012)</label><?label bengio2012practical?><mixed-citation>
Bengio, Y.: Practical recommendations for gradient-based training of deep
architectures, in: Neural networks: Tricks of the trade,  437–478,
Springer, 2012.</mixed-citation></ref>
      <ref id="bib1.bibx6"><label>Bergstra and Bengio(2012)</label><?label bergstra2012random?><mixed-citation>
Bergstra, J. and Bengio, Y.: Random search for hyper-parameter optimization,
J. Mach. Learn. Res., 13, 281–305, 2012.</mixed-citation></ref>
      <ref id="bib1.bibx7"><label>Boer et al.(2019)Boer, Kharin, and Merryfield</label><?label boer2019?><mixed-citation>Boer, G. J., Kharin, V. V., and Merryfield, W. J.: Differences in potential and
actual skill in a decadal prediction experiment, Clim. Dynam., 52,
6619–6631, <ext-link xlink:href="https://doi.org/10.1007/s00382-018-4533-4" ext-link-type="DOI">10.1007/s00382-018-4533-4</ext-link>, 2019.</mixed-citation></ref>
      <ref id="bib1.bibx8"><label>Branstator et al.(2012)Branstator, Teng, and Meehl</label><?label branstator?><mixed-citation>Branstator, G., Teng, H., and Meehl, G. A.: Systematic Estimates of
Initial-Value Decadal Predictability for Six AOGCMs, J. Climate, 25, 1827–1846,
<ext-link xlink:href="https://doi.org/10.1175/JCLI-D-11-00227.1" ext-link-type="DOI">10.1175/JCLI-D-11-00227.1</ext-link>, 2012.</mixed-citation></ref>
      <ref id="bib1.bibx9"><label>Castruccio et al.(2014)Castruccio, McInerney, Stein, Crouch, Jacob,
and Moyer</label><?label castruccio2014?><mixed-citation>Castruccio, S., McInerney, D. J., Stein, M. L., Crouch, F. L., Jacob, R. L.,
and Moyer, E. J.: Statistical Emulation of Climate Model Projections Based on
Precomputed GCM Runs, J. Climate, 27, 1829–1844,
<ext-link xlink:href="https://doi.org/10.1175/JCLI-D-13-00099.1" ext-link-type="DOI">10.1175/JCLI-D-13-00099.1</ext-link>, 2014.</mixed-citation></ref>
      <ref id="bib1.bibx10"><label>Chan et al.(2016)Chan, Jaitly, Le, and Vinyals</label><?label chan2016listen?><mixed-citation>
Chan, W., Jaitly, N., Le, Q., and Vinyals, O.: Listen, attend and spell: A
neural network for large vocabulary conversational speech recognition, in:
Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International
Conference on,  4960–4964, IEEE, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx11"><label>Cohen et al.(2014)Cohen, Screen, Furtado, Barlow, Whittleston,
Coumou, Francis, Dethloff, Entekhabi, Overland, and Jones</label><?label cohen2014?><mixed-citation>Cohen, J., Screen, J. A., Furtado, J. C., Barlow, M., Whittleston, D., Coumou,
D., Francis, J., Dethloff, K., Entekhabi, D., Overland, J., and Jones, J.:
Recent Arctic amplification and extreme mid-latitude weather, Nature
Geosci., 7, 627–637, <ext-link xlink:href="https://doi.org/10.1038/ngeo2234" ext-link-type="DOI">10.1038/ngeo2234</ext-link>, 2014.</mixed-citation></ref>
      <ref id="bib1.bibx12"><label>Deo et al.(2017)Deo, Chandra, and Sharma</label><?label deo2017?><mixed-citation>Deo, R. V., Chandra, R., and Sharma, A.: Stacked transfer learning for tropical
cyclone intensity prediction, ArXiv e-prints,
<uri>http://arxiv.org/abs/1708.06539</uri>, 2017.</mixed-citation></ref>
      <ref id="bib1.bibx13"><label>Finn et al.(2016)Finn, Goodfellow, and Levine</label><?label finn2016unsupervised?><mixed-citation>
Finn, C., Goodfellow, I., and Levine, S.: Unsupervised learning for physical
interaction through video prediction, in: Advances in neural information
processing systems, NIPS Proceedings,  64–72, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx14"><label>Friedman et al.(2001)Friedman, Hastie, and
Tibshirani</label><?label friedman2001elements?><mixed-citation>
Friedman, J., Hastie, T., and Tibshirani, R.: The elements of statistical
learning, vol. 1, Springer series in statistics New York, NY, USA, 2001.</mixed-citation></ref>
      <ref id="bib1.bibx15"><label>Fyfe et al.(2016)Fyfe, Meehl, England, Mann, Santer, Flato, Hawkins,
Gillett, Xie, Kosaka, and Swart</label><?label fyfe2016?><mixed-citation>Fyfe, J. C., Meehl, G. A., England, M. H., Mann, M. E., Santer, B. D., Flato,
G. M., Hawkins, E., Gillett, N. P., Xie, S.-P., Kosaka, Y., and Swart, N. C.:
Making sense of the early-2000s warming slowdown, Nat. Clim. Change, 6,
224–228, <ext-link xlink:href="https://doi.org/10.1038/nclimate2938" ext-link-type="DOI">10.1038/nclimate2938</ext-link>, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx16"><label>Gawehn et al.(2016)Gawehn, Hiss, and Schneider</label><?label gawehn2016?><mixed-citation>Gawehn, E., Hiss, J. A., and Schneider, G.: Deep learning in drug discovery,
Mol. Inform., 35, 3–14, <ext-link xlink:href="https://doi.org/10.1002/minf.201501008" ext-link-type="DOI">10.1002/minf.201501008</ext-link>, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx17"><label>Glorot and Bengio(2010)</label><?label glorot2010understanding?><mixed-citation>
Glorot, X. and Bengio, Y.: Understanding the difficulty of training deep
feedforward neural networks, in: Proceedings of the thirteenth international
conference on artificial intelligence and statistics, J. Mach. Learn. Res., 9,  249–256, 2010.</mixed-citation></ref>
      <ref id="bib1.bibx18"><label>Goddard et al.(2013)Goddard, Kumar, Solomon, Smith, Boer, Gonzalez,
Kharin, Merryfield, Deser, Mason, Kirtman, Msadek, Sutton, Hawkins, Fricker,
Hegerl, Ferro, Stephenson, Meehl, Stockdale, Burgman, Greene, Kushnir,
Newman, Carton, Fukumori, and Delworth</label><?label goddard2013?><mixed-citation>Goddard, L., Kumar, A., Solomon, A., Smith, D., Boer, G., Gonzalez, P., Kharin,
V., Merryfield, W., Deser, C., Mason, S. J., Kirtman, B. P., Msadek, R.,
Sutton, R<?pagebreak page2316?>., Hawkins, E., Fricker, T., Hegerl, G., Ferro, C. A. T.,
Stephenson, D. B., Meehl, G. A., Stockdale, T., Burgman, R., Greene, A. M.,
Kushnir, Y., Newman, M., Carton, J., Fukumori, I., and Delworth, T.: A
verification framework for interannual-to-decadal predictions experiments,
Clim. Dynam., 40, 245–272, <ext-link xlink:href="https://doi.org/10.1007/s00382-012-1481-2" ext-link-type="DOI">10.1007/s00382-012-1481-2</ext-link>, 2013.</mixed-citation></ref>
      <ref id="bib1.bibx19"><label>Goodfellow(2016)</label><?label goodfellow2016tutorial?><mixed-citation>Goodfellow, I.: NIPS 2016 Tutorial: Generative Adversarial Networks, available at: <uri>http://arxiv.org/abs/1701.00160</uri> (last access: 24 February 2020), 2016.</mixed-citation></ref>
      <ref id="bib1.bibx20"><label>Goodfellow et al.(2014)Goodfellow, Pouget-Abadie, Mirza, Xu,
Warde-Farley, Ozair, Courville, and Bengio</label><?label goodfellow2014gans?><mixed-citation>Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair,
S., Courville, A., and Bengio, Y.: Generative Adversarial Nets, in: Advances
in Neural Information Processing Systems 27, edited by: Ghahramani, Z.,
Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q.,
2672–2680, Curran Associates, Inc., available at:
<uri>http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf</uri> (last access: 24 February 2020),
2014.</mixed-citation></ref>
      <ref id="bib1.bibx21"><label>Goodfellow et al.(2016)Goodfellow, Bengio, and
Courville</label><?label Goodfellow-et-al-2016?><mixed-citation>Goodfellow, I., Bengio, Y., and Courville, A.: Deep Learning, MIT Press, available at:
<uri>http://www.deeplearningbook.org</uri> (last access: 24 February 2020), 2016.</mixed-citation></ref>
      <ref id="bib1.bibx22"><label>Guemas et al.(2013)Guemas, Doblas-Reyes, Andreu-Burillo, and
Asif</label><?label guemas2013?><mixed-citation>Guemas, V., Doblas-Reyes, F. J., Andreu-Burillo, I., and Asif, M.:
Retrospective prediction of the global warming slowdown in the past decade,
Nat. Clim. Change, 3, 649–653, <ext-link xlink:href="https://doi.org/10.1038/nclimate1863" ext-link-type="DOI">10.1038/nclimate1863</ext-link>, 2013.</mixed-citation></ref>
      <ref id="bib1.bibx23"><label>He et al.(2015)He, Zhang, Ren, and Sun</label><?label he2015delving?><mixed-citation>
He, K., Zhang, X., Ren, S., and Sun, J.: Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification, in:
Proceedings of the IEEE international conference on computer vision,
1026–1034, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx24"><label>He et al.(2016)He, Zhang, Ren, and Sun</label><?label he2016deep?><mixed-citation>
He, K., Zhang, X., Ren, S., and Sun, J.: Deep residual learning for image
recognition, in: Proceedings of the IEEE conference on computer vision and
pattern recognition, 770–778, Computer Vision Foundation, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx25"><label>Herger et al.(2015)Herger, Sanderson, and Knutti</label><?label herger2015?><mixed-citation>Herger, N., Sanderson, B. M., and Knutti, R.: Improved pattern scaling
approaches for the use in climate impact studies, Geophys. Res. Lett., 42,
3486–3494, <ext-link xlink:href="https://doi.org/10.1002/2015GL063569" ext-link-type="DOI">10.1002/2015GL063569</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx26"><label>Hinton et al.(2012)Hinton, Deng, Yu, Dahl, Mohamed, Jaitly, Senior,
Vanhoucke, Nguyen, Sainath et al.</label><?label hinton2012deep?><mixed-citation>
Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-R., Jaitly, N., Senior,
A., Vanhoucke, V., Nguyen, P., Sainath, T. N., and Kingsbury, B.: Deep neural networks
for acoustic modeling in speech recognition: The shared views of four
research groups, IEEE Signal Processing Magazine, 29, 82–97, 2012.</mixed-citation></ref>
      <ref id="bib1.bibx27"><label>Hong et al.(2017)Hong, Kim, Joh, and Song</label><?label hong2017?><mixed-citation>Hong, S., Kim, S., Joh, M., and Song, S.-K.: GlobeNet: Convolutional Neural
Networks for Typhoon Eye Tracking from Remote Sensing Imagery, ArXiv
e-prints, <uri>http://arxiv.org/abs/1708.03417</uri>, 2017.</mixed-citation></ref>
      <ref id="bib1.bibx28"><label>Ioffe and Szegedy(2015)</label><?label ioffe2015batch?><mixed-citation>
Ioffe, S. and Szegedy, C.: Batch normalization: Accelerating deep network
training by reducing internal covariate shift, arXiv preprint
arXiv:1502.03167, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx29"><label>Jay et al.(2018)Jay, Reidmiller, Avery, Barrie, DeAngelo, Dave,
Dzaugis, Kolian, Lewis, Reeves, and Winner</label><?label nca2018?><mixed-citation>Jay, A., Reidmiller, D., Avery, C., Barrie, D., DeAngelo, B., Dave, A.,
Dzaugis, M., Kolian, M., Lewis, K., Reeves, K., and Winner, D.: Overview, in:
Impacts, Risks, and Adaptation in the United States: Fourth National Climate
Assessment, Volume II, edited by: Reidmiller, D., Avery, C., Easterling, D.,
Kunkel, K., Lewis, K., Maycock, T., and Stewart, B.,  33–71, U.S. Global
Change Research Program, Washington, DC, USA, <ext-link xlink:href="https://doi.org/10.7930/NCA4.2018.CH1" ext-link-type="DOI">10.7930/NCA4.2018.CH1</ext-link>,
2018.</mixed-citation></ref>
      <ref id="bib1.bibx30"><label>Jiang et al.(2018)Jiang, Xu, and Wei</label><?label jiang2018?><mixed-citation>Jiang, G.-Q., Xu, J., and Wei, J.: A Deep Learning Algorithm of Neural Network
for the Parameterization of Typhoon-Ocean Feedback in Typhoon Forecast
Models, Geophys. Res. Lett., 45, 3706–3716, <ext-link xlink:href="https://doi.org/10.1002/2018GL077004" ext-link-type="DOI">10.1002/2018GL077004</ext-link>,
2018.</mixed-citation></ref>
      <ref id="bib1.bibx31"><label>JMA(2019)</label><?label JMA?><mixed-citation>JMA: Verification Indices, available at:
<uri>https://www.jma.go.jp/jma/jma-eng/jma-center/nwp/outline2013-nwp/pdf/outline2013_Appendix_A.pdf</uri> (last access: February 2020),
2019.</mixed-citation></ref>
      <ref id="bib1.bibx32"><label>Joliffe and Stephenson(2003)</label><?label jolliffestephenson?><mixed-citation>
Joliffe, I. and Stephenson, D.: Forecast verification, John Wiley and Sons,
2003.</mixed-citation></ref>
      <ref id="bib1.bibx33"><label>Karpathy et al.(2014)Karpathy, Toderici, Shetty, Leung, Sukthankar,
and Fei-Fei</label><?label karpathy2014large?><mixed-citation>
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei,
L.: Large-scale video classification with convolutional neural networks, in:
Proceedings of the IEEE conference on Computer Vision and Pattern
Recognition, Computer Vision Foundation,  1725–1732, 2014.</mixed-citation></ref>
      <ref id="bib1.bibx34"><label>Krizhevsky et al.(2012)Krizhevsky, Sutskever, and
Hinton</label><?label krizhevsky2012imagenet?><mixed-citation>
Krizhevsky, A., Sutskever, I., and Hinton, G. E.: Imagenet classification with
deep convolutional neural networks, in: Advances in neural information
processing systems, NIPS Proceedings, 1097–1105, 2012.</mixed-citation></ref>
      <ref id="bib1.bibx35"><label>Lean and Rind(2009)</label><?label leanrind2009?><mixed-citation>Lean, J. L. and Rind, D. H.: How will Earth's surface temperature change in
future decades?, Geophys. Res. Lett., 36, L15708,
<ext-link xlink:href="https://doi.org/10.1029/2009GL038923" ext-link-type="DOI">10.1029/2009GL038923</ext-link>, 2009.</mixed-citation></ref>
      <ref id="bib1.bibx36"><label>LeCun et al.(1998)LeCun, Bottou, Bengio, and
Haffner</label><?label lecun1998gradient?><mixed-citation>
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.: Gradient-based learning
applied to document recognition, Proceedings of the IEEE, 86, 2278–2324,
1998.</mixed-citation></ref>
      <ref id="bib1.bibx37"><?xmltex \def\ref@label{{LeCun et~al.(2012)LeCun, Bottou, Orr, and
M{\"{u}}ller}}?><label>LeCun et al.(2012)LeCun, Bottou, Orr, and
Müller</label><?label lecun2012efficient?><mixed-citation>
LeCun, Y. A., Bottou, L., Orr, G. B., and Müller, K.-R.: Efficient
backprop, in: Neural networks: Tricks of the trade,  9–48, Springer,
2012.</mixed-citation></ref>
      <ref id="bib1.bibx38"><label>Liu et al.(2016)Liu, Racah, Prabhat, Correa, Khosrowshahi, Lavers,
Kunkel, Wehner, and Collins</label><?label liu2016?><mixed-citation>Liu, Y., Racah, E., Prabhat, Correa, J., Khosrowshahi, A., Lavers, D., Kunkel,
K., Wehner, M., and Collins, W.: Application of Deep Convolutional Neural
Networks for Detecting Extreme Weather in Climate Datasets, ArXiv e-prints,
<uri>http://arxiv.org/abs/1605.01156</uri>, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx39"><label>Lu and Ricciuto(2019)</label><?label lu.ricciuto?><mixed-citation>Lu, D. and Ricciuto, D.: Efficient surrogate modeling methods for large-scale Earth system models based on machine-learning techniques, Geosci. Model Dev., 12, 1791–1807, <ext-link xlink:href="https://doi.org/10.5194/gmd-12-1791-2019" ext-link-type="DOI">10.5194/gmd-12-1791-2019</ext-link>, 2019.</mixed-citation></ref>
      <ref id="bib1.bibx40"><label>Lynch et al.(2017)Lynch, Hartin, Bond-Lamberty, and
Kravitz</label><?label lynch2017?><mixed-citation>Lynch, C., Hartin, C., Bond-Lamberty, B., and Kravitz, B.: An open-access CMIP5 pattern library for temperature and precipitation: description and methodology, Earth Syst. Sci. Data, 9, 281–292, <ext-link xlink:href="https://doi.org/10.5194/essd-9-281-2017" ext-link-type="DOI">10.5194/essd-9-281-2017</ext-link>, 2017.</mixed-citation></ref>
      <ref id="bib1.bibx41"><label>MacMartin and Kravitz(2016)</label><?label macmartinkravitz2016?><mixed-citation>MacMartin, D. G. and Kravitz, B.: Dynamic climate emulators for solar geoengineering, Atmos. Chem. Phys., 16, 15789–15799, <ext-link xlink:href="https://doi.org/10.5194/acp-16-15789-2016" ext-link-type="DOI">10.5194/acp-16-15789-2016</ext-link>, 2016.</mixed-citation></ref>
      <ref id="bib1.bibx42"><label>McDermott and Wikle(2018)</label><?label mcdermott?><mixed-citation>McDermott, P. L. and Wikle, C. K.: Deep echo state networks with uncertainty
quantification for spatio-temporal forecasting, Environmetrics, 30, e2553,
<ext-link xlink:href="https://doi.org/10.1002/env.2553" ext-link-type="DOI">10.1002/env.2553</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx43"><label>Miller et al.(2018)Miller, Nair, Ramachandran, and
Maskey</label><?label miller2018?><mixed-citation>Miller, J., Nair, U., Ramachandran, R., and Maskey, M.: Detection of transverse
cirrus bands in satellite imagery using deep learning, Comput.
Geosci., 118, 79–85, <ext-link xlink:href="https://doi.org/10.1016/j.cageo.2018.05.012" ext-link-type="DOI">10.1016/j.cageo.2018.05.012</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx44"><label>Mitchell(2003)</label><?label mitchell2003?><mixed-citation>Mitchell, T. D.: Pattern Scaling: An Examination of the Accuracy of the
Technique for Describing Future Climates, Clim. Change, 60, 217–242,
<ext-link xlink:href="https://doi.org/10.1023/A:1026035305597" ext-link-type="DOI">10.1023/A:1026035305597</ext-link>, 2003.</mixed-citation></ref>
      <ref id="bib1.bibx45"><label>Moss et al.(2017)Moss, Kravitz, Delgado, Asrar, Brandenberger,
Wigmosta, Preston, Buzan, Gremillion, Shaw, Stocker, Higuchi, Sarma, Kosmal,
Lawless, Marqusee, Lipschultz, O'Connell, Olsen, Walker, Weaver, Westley, and
Wright</label><?label moss2017?><mixed-citation>
Moss, R. H., Kravitz, B., Delgado, A., Asrar, G., Brandenberger, J., Wigmosta,
M., Preston, K., Buzan, T., Gremillion, M., Shaw, P., Stocker, K., Higuchi,
S., Sarma, A., Kosmal, A., Lawless, S., Marqusee, J., Lipschultz, F.,
O'Connell, R., Olsen, R., Walker, D., Weaver, C., Westley, M., and Wright,
R.: Nonstationary Weather Patterns and Extreme Events: Informing Design and
Planning for Long-Lived Infrastructure, Tech. rep., ESTCP, ESTCP Project
RC-201591, 2017.</mixed-citation></ref>
      <?pagebreak page2317?><ref id="bib1.bibx46"><label>Nair and Hinton(2010)</label><?label nair2010rectified?><mixed-citation>
Nair, V. and Hinton, G. E.: Rectified linear units improve restricted boltzmann
machines, in: Proceedings of the 27th international conference on machine
learning (ICML-10), Association for Computing Machinery,  807–814, 2010.</mixed-citation></ref>
      <ref id="bib1.bibx47"><label>Ouyang and Lu(2018)</label><?label ouyang?><mixed-citation>Ouyang, Q. and Lu, W.: Monthly rainfall forecasting using echo state networks coupled with data preprocessing methods, Water Resour. Mange., 32, 659–674,
<ext-link xlink:href="https://doi.org/10.1007/s11269-017-1832-1" ext-link-type="DOI">10.1007/s11269-017-1832-1</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx48"><label>Pradhan et al.(2018)Pradhan, Aygun, Maskey, Ramachandran, and
Cecil</label><?label pradhan2018?><mixed-citation>Pradhan, R., Aygun, R. S., Maskey, M., Ramachandran, R., and Cecil, D. J.:
Tropical Cyclone Intensity Estimation Using a Deep Convolutional Neural
Network, IEEE Transactions on Image Processing, 27, 692–702,
<ext-link xlink:href="https://doi.org/10.1109/TIP.2017.2766358" ext-link-type="DOI">10.1109/TIP.2017.2766358</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx49"><label>Rasp et al.(2018)Rasp, Pritchard, and Gentine</label><?label rasp2018?><mixed-citation>Rasp, S., Pritchard, M. S., and Gentine, P.: Deep learning to represent subgrid
processes in climate models, P. Natl. Acad. Sci., 115, 9684–9689,
<ext-link xlink:href="https://doi.org/10.1073/pnas.1810286115" ext-link-type="DOI">10.1073/pnas.1810286115</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx50"><label>Robertson et al.(2015)Robertson, Kumar, na, and
Vitart</label><?label robertson2015?><mixed-citation>Robertson, A. W., Kumar, A., Peña, M., and Vitart, F.: Improving and
Promoting Subseasonal to Seasonal Prediction, B. Am. Meteor. Soc., 96,
ES49–ES53, <ext-link xlink:href="https://doi.org/10.1175/BAMS-D-14-00139.1" ext-link-type="DOI">10.1175/BAMS-D-14-00139.1</ext-link>, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx51"><label>Ronneberger et al.(2015)Ronneberger, P.Fischer, and
Brox</label><?label ronneberger2015unet?><mixed-citation>Ronneberger, O., Fischer, P., and Brox, T.: U-Net: Convolutional Networks for
Biomedical Image Segmentation, in: Medical Image Computing and
Computer-Assisted Intervention (MICCAI), vol. 9351 of <italic>LNCS</italic>,
234–241, Springer, available at:
<uri>http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a</uri> (last access: February 2020),
(available on arXiv:1505.04597 [cs.CV]), 2015.</mixed-citation></ref>
      <ref id="bib1.bibx52"><label>Santer et al.(1990)Santer, Wigley, Schlesinger, and
Mitchell</label><?label santer1990?><mixed-citation>
Santer, B., Wigley, T., Schlesinger, M., and Mitchell, J.: Developing Climate
Scenarios from Equilibrium GCM Results, Tech. rep., Hamburg, Germany,
1990.</mixed-citation></ref>
      <ref id="bib1.bibx53"><label>Shi et al.(2015)Shi, Chen, Wang, Yeung, Wong, and
WOO</label><?label shi2015precip?><mixed-citation>Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-k., and WOO, W.-c.:
Convolutional LSTM Network: A Machine Learning Approach for Precipitation
Nowcasting, in: Advances in Neural Information Processing Systems 28, edited
by: Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R.,
802–810, Curran Associates, Inc., available at:
<ext-link xlink:href="http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf">http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf</ext-link> (last access: February 2020),
2015.</mixed-citation></ref>
      <ref id="bib1.bibx54"><label>Simonyan and Zisserman(2014)</label><?label simonyan2014very?><mixed-citation>
Simonyan, K. and Zisserman, A.: Very deep convolutional networks for
large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.</mixed-citation></ref>
      <ref id="bib1.bibx55"><label>Srivastava et al.(2014)Srivastava, Hinton, Krizhevkskey, Sutskever,
and Salakhutdinov</label><?label srivastava2014dropout?><mixed-citation>
Srivastava, N., Hinton, G., Krizhevkskey, A., Sutskever, I., and Salakhutdinov,
R.: Dropout: A simple way to prevent neural networks for overfitting, J. Mach. Learn. Res., 15, 1929–1958, 2014.</mixed-citation></ref>
      <ref id="bib1.bibx56"><label>Stocker et al.(2013)</label><?label ipcc2013?><mixed-citation>
Stocker, T. F., Qin, D., Plattner, G.-K., Alexander, L. V., Allen, S. K., Bindoff, N. L., Bréon, F.-M., Church, J. A., Cubasch, U., Emori, S., Forster, P., Friedlingstein, P., Gillett, N., Gregory, J. M., Hartmann, D. L., Jansen, E., Kirtman, B., Knutti, R., Krishna Kumar, K., Lemke, P., Marotzke, J., Masson-Delmotte, V., Meehl, G. A., Mokhov, I. I., Piao, S., Ramaswamy, V., Randall, D., Rhein, M., Rojas, M., Sabine, C., Shindell, D., Talley, L. D., Vaughan, D. G., and Xie, S.-P.: Technical Summary, in: Climate Change 2013: The Physical
Science Basis. Contribution of Working Group I to the Fifth Assessment Report
of the Intergovernmental Panel on Climate Change, edited by: Stocker, T. F., Qin, D., Plattner, G.-K., Tignor, M., Allen, S. K., Boschung, J., Nauels, A., Xia, Y., Bex, V., and Midgley, P. M., Cambridge University Press, Cambridge, United Kingdom and New York,
NY, USA, 2013.</mixed-citation></ref>
      <ref id="bib1.bibx57"><label>Szegedy et al.(2015)Szegedy, Liu, Jia, Sermanet, Reed, Anguelov,
Erhan, Vanhoucke, and Rabinovich</label><?label szegedy2015going?><mixed-citation>
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D.,
Vanhoucke, V., and Rabinovich, A.: Going deeper with convolutions, in:
Proceedings of the IEEE conference on computer vision and pattern
recognition, 8–10 June 2015, Boston, Massachusetts, 1–9, 2015.</mixed-citation></ref>
      <ref id="bib1.bibx58"><label>van Vuuren et al.(2011)van Vuuren, Edmonds, Kainuma, Riahi, Thomson,
Hibbard, Hurtt, Kram, Krey, Lamarque, Masui, Meinshausen, Nakicenovic, Smith,
and Rose</label><?label vanvuuren2011?><mixed-citation>van Vuuren, D. P., Edmonds, J., Kainuma, M., Riahi, K., Thomson, A., Hibbard,
K., Hurtt, G. C., Kram, T., Krey, V., Lamarque, J.-F., Masui, T.,
Meinshausen, M., Nakicenovic, N., Smith, S. J., and Rose, S. K.: The
representative concentration pathways: An overview, Clim. Change, 109,
5–31, <ext-link xlink:href="https://doi.org/10.1007/s10584-011-0148-z" ext-link-type="DOI">10.1007/s10584-011-0148-z</ext-link>, 2011.</mixed-citation></ref>
      <ref id="bib1.bibx59"><label>Weber et al.(2020)</label><?label weber2020?><mixed-citation>Weber, T., Corotan, A., Hutchinson, B., Kravitz, B., and Link, R. P.: A Deep Neural Network approach for estimating precipitation fields in Earth System Models, available at: <uri>https://github.com/hutchresearch/deep_climate_emulator</uri>, last access: 24 February 2020.</mixed-citation></ref>
      <ref id="bib1.bibx60"><label>Yao et al.(2007)Yao, Rosasco, and Caponnetto</label><?label yao2007early?><mixed-citation>
Yao, Y., Rosasco, L., and Caponnetto, A.: On early stopping in gradient descent
learning, Constructive Approximation, 26, 289–315, 2007.</mixed-citation></ref>
      <ref id="bib1.bibx61"><label>Yeager et al.(2018)Yeager, Danabasoglu, Rosenbloom, Strand, Bates,
Meehl, Karspeck, Lindsay, Long, Teng, and Lovenduski</label><?label yeager?><mixed-citation>Yeager, S., Danabasoglu, G., Rosenbloom, N., Strand, W., Bates, S., Meehl, G.,
Karspeck, A., Lindsay, K., Long, M., Teng, H., and Lovenduski, N.: Predicting
Near-Term Changes in the Earth System: A Large Ensemble of Initialized
Decadal Prediction Simulations Using the Community Earth System Model,
B. Am. Meteor. Soc., 99, 1867–1886, <ext-link xlink:href="https://doi.org/10.1175/BAMS-D-17-0098.1" ext-link-type="DOI">10.1175/BAMS-D-17-0098.1</ext-link>,
2018.</mixed-citation></ref>
      <ref id="bib1.bibx62"><label>Yeh et al.(2018)Yeh, Cai, Min, McPhaden, Dommenget, Dewitte, Collins,
Ashok, An, Yim, and Kug</label><?label yeh2018?><mixed-citation>Yeh, S.-W., Cai, W., Min, S.-K., McPhaden, M. J., Dommenget, D., Dewitte, B.,
Collins, M., Ashok, K., An, S.-I., Yim, B.-Y., and Kug, J.-S.: ENSO
Atmospheric Teleconnections and Their Response to Greenhouse Gas Forcing,
Rev. Geophys., 56, 185–206, <ext-link xlink:href="https://doi.org/10.1002/2017RG000568" ext-link-type="DOI">10.1002/2017RG000568</ext-link>, 2018.</mixed-citation></ref>
      <ref id="bib1.bibx63"><label>Yuan et al.(2019)Yuan, Huang, Duan, Zhu, Xoplaki, and
Luterbacher</label><?label yuan2018?><mixed-citation>Yuan, N., Huang, Y., Duan, J., Zhu, C., Xoplaki, E., and Luterbacher, J.: On
climate prediction: How much can we expect from climate memory?, Clim.
Dynam., 52, 855–864, <ext-link xlink:href="https://doi.org/10.1007/s00382-018-4168-5" ext-link-type="DOI">10.1007/s00382-018-4168-5</ext-link>, 2019.</mixed-citation></ref>
      <ref id="bib1.bibx64"><label>Zhang and Sutton(2017)</label><?label zhang2017replay?><mixed-citation>
Zhang, S. and Sutton, R. S.: A Deeper Look at Experience Replay, CoRR,
abs/1712.01275, 2017.</mixed-citation></ref>

  </ref-list></back>
    <!--<article-title-html>Technical note: Deep learning for creating surrogate models of precipitation in Earth system models</article-title-html>
<abstract-html><p>We investigate techniques for using deep neural networks to produce surrogate
models for short-term climate forecasts.  A convolutional neural network is
trained on 97 years of monthly precipitation output from the 1pctCO2 run (the
CO<sub>2</sub> concentration increases by 1&thinsp;% per year) simulated by the second-generation Canadian Earth System Model (CanESM2).  The neural network clearly outperforms a persistence forecast and
does not show substantially degraded performance even when the forecast length
is extended to 120 months.  The model is prone to underpredicting precipitation
in areas characterized by intense precipitation events.  Scheduled sampling
(forcing the model to gradually use its own past predictions rather than ground
truth) is essential for avoiding amplification of early forecasting errors.
However, the use of scheduled sampling also necessitates preforecasting
(generating forecasts prior to the first forecast date) to obtain adequate
performance for the first few prediction time steps.  We document the training
procedures and hyperparameter optimization process for researchers who wish to
extend the use of neural networks in developing surrogate models.</p></abstract-html>
<ref-html id="bib1.bib1"><label>Arora and Boer(2011)</label><mixed-citation>
Arora, V. K. and Boer, G. J.: Uncertainties in the 20th century carbon budget
associated with land use change, Glob. Change Biol., 16, 3327–3348,
<a href="https://doi.org/10.1111/j.1365-2486.2010.02202.x" target="_blank">https://doi.org/10.1111/j.1365-2486.2010.02202.x</a>, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib2"><label>Arora et al.(2011)Arora, Scinocca, Boer, Christian, Denman, Flato,
Kharin, lee, and Merryfield</label><mixed-citation>
Arora, V. K., Scinocca, J. F., Boer, G. J., Christian, J. R., Denman, K. L.,
Flato, G. M., Kharin, V. V., Lee, W. G., and Merryfield, W. J.: Carbon
emission limits required to satisfy future representative concentration
pathways of greenhouse gases, Geophys. Res. Lett., 38, L05805,
<a href="https://doi.org/10.1029/2010GL046270" target="_blank">https://doi.org/10.1029/2010GL046270</a>, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib3"><label>Bellucci et al.(2015)Bellucci, Haarsma, Bellouin, Booth, Cagnazzo,
van den Hurk, Keenlyside, Koenigk, Massonnet, Materia, and
Weiss</label><mixed-citation>
Bellucci, A., Haarsma, R., Bellouin, N., Booth, B., Cagnazzo, C., van den Hurk,
B., Keenlyside, N., Koenigk, T., Massonnet, F., Materia, S., and Weiss, M.:
Advancements in decadal climate predictability: The role of nonoceanic
drivers, Rev. Geophys., 53, 165–202, <a href="https://doi.org/10.1002/2014RG000473" target="_blank">https://doi.org/10.1002/2014RG000473</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib4"><label>Bengio et al.(2015)Bengio, Vinyals, Jaitly, and
Shazeer</label><mixed-citation>
Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N.: Scheduled sampling for
sequence prediction with recurrent neural networks, in: Advances in Neural
Information Processing Systems, NIPS Proceedings, 1171–1179, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib5"><label>Bengio(2012)</label><mixed-citation>
Bengio, Y.: Practical recommendations for gradient-based training of deep
architectures, in: Neural networks: Tricks of the trade,  437–478,
Springer, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib6"><label>Bergstra and Bengio(2012)</label><mixed-citation>
Bergstra, J. and Bengio, Y.: Random search for hyper-parameter optimization,
J. Mach. Learn. Res., 13, 281–305, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib7"><label>Boer et al.(2019)Boer, Kharin, and Merryfield</label><mixed-citation>
Boer, G. J., Kharin, V. V., and Merryfield, W. J.: Differences in potential and
actual skill in a decadal prediction experiment, Clim. Dynam., 52,
6619–6631, <a href="https://doi.org/10.1007/s00382-018-4533-4" target="_blank">https://doi.org/10.1007/s00382-018-4533-4</a>, 2019.
</mixed-citation></ref-html>
<ref-html id="bib1.bib8"><label>Branstator et al.(2012)Branstator, Teng, and Meehl</label><mixed-citation>
Branstator, G., Teng, H., and Meehl, G. A.: Systematic Estimates of
Initial-Value Decadal Predictability for Six AOGCMs, J. Climate, 25, 1827–1846,
<a href="https://doi.org/10.1175/JCLI-D-11-00227.1" target="_blank">https://doi.org/10.1175/JCLI-D-11-00227.1</a>, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib9"><label>Castruccio et al.(2014)Castruccio, McInerney, Stein, Crouch, Jacob,
and Moyer</label><mixed-citation>
Castruccio, S., McInerney, D. J., Stein, M. L., Crouch, F. L., Jacob, R. L.,
and Moyer, E. J.: Statistical Emulation of Climate Model Projections Based on
Precomputed GCM Runs, J. Climate, 27, 1829–1844,
<a href="https://doi.org/10.1175/JCLI-D-13-00099.1" target="_blank">https://doi.org/10.1175/JCLI-D-13-00099.1</a>, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib10"><label>Chan et al.(2016)Chan, Jaitly, Le, and Vinyals</label><mixed-citation>
Chan, W., Jaitly, N., Le, Q., and Vinyals, O.: Listen, attend and spell: A
neural network for large vocabulary conversational speech recognition, in:
Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International
Conference on,  4960–4964, IEEE, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib11"><label>Cohen et al.(2014)Cohen, Screen, Furtado, Barlow, Whittleston,
Coumou, Francis, Dethloff, Entekhabi, Overland, and Jones</label><mixed-citation>
Cohen, J., Screen, J. A., Furtado, J. C., Barlow, M., Whittleston, D., Coumou,
D., Francis, J., Dethloff, K., Entekhabi, D., Overland, J., and Jones, J.:
Recent Arctic amplification and extreme mid-latitude weather, Nature
Geosci., 7, 627–637, <a href="https://doi.org/10.1038/ngeo2234" target="_blank">https://doi.org/10.1038/ngeo2234</a>, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib12"><label>Deo et al.(2017)Deo, Chandra, and Sharma</label><mixed-citation>
Deo, R. V., Chandra, R., and Sharma, A.: Stacked transfer learning for tropical
cyclone intensity prediction, ArXiv e-prints,
<a href="http://arxiv.org/abs/1708.06539" target="_blank"/>, 2017.
</mixed-citation></ref-html>
<ref-html id="bib1.bib13"><label>Finn et al.(2016)Finn, Goodfellow, and Levine</label><mixed-citation>
Finn, C., Goodfellow, I., and Levine, S.: Unsupervised learning for physical
interaction through video prediction, in: Advances in neural information
processing systems, NIPS Proceedings,  64–72, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib14"><label>Friedman et al.(2001)Friedman, Hastie, and
Tibshirani</label><mixed-citation>
Friedman, J., Hastie, T., and Tibshirani, R.: The elements of statistical
learning, vol. 1, Springer series in statistics New York, NY, USA, 2001.
</mixed-citation></ref-html>
<ref-html id="bib1.bib15"><label>Fyfe et al.(2016)Fyfe, Meehl, England, Mann, Santer, Flato, Hawkins,
Gillett, Xie, Kosaka, and Swart</label><mixed-citation>
Fyfe, J. C., Meehl, G. A., England, M. H., Mann, M. E., Santer, B. D., Flato,
G. M., Hawkins, E., Gillett, N. P., Xie, S.-P., Kosaka, Y., and Swart, N. C.:
Making sense of the early-2000s warming slowdown, Nat. Clim. Change, 6,
224–228, <a href="https://doi.org/10.1038/nclimate2938" target="_blank">https://doi.org/10.1038/nclimate2938</a>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib16"><label>Gawehn et al.(2016)Gawehn, Hiss, and Schneider</label><mixed-citation>
Gawehn, E., Hiss, J. A., and Schneider, G.: Deep learning in drug discovery,
Mol. Inform., 35, 3–14, <a href="https://doi.org/10.1002/minf.201501008" target="_blank">https://doi.org/10.1002/minf.201501008</a>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib17"><label>Glorot and Bengio(2010)</label><mixed-citation>
Glorot, X. and Bengio, Y.: Understanding the difficulty of training deep
feedforward neural networks, in: Proceedings of the thirteenth international
conference on artificial intelligence and statistics, J. Mach. Learn. Res., 9,  249–256, 2010.
</mixed-citation></ref-html>
<ref-html id="bib1.bib18"><label>Goddard et al.(2013)Goddard, Kumar, Solomon, Smith, Boer, Gonzalez,
Kharin, Merryfield, Deser, Mason, Kirtman, Msadek, Sutton, Hawkins, Fricker,
Hegerl, Ferro, Stephenson, Meehl, Stockdale, Burgman, Greene, Kushnir,
Newman, Carton, Fukumori, and Delworth</label><mixed-citation>
Goddard, L., Kumar, A., Solomon, A., Smith, D., Boer, G., Gonzalez, P., Kharin,
V., Merryfield, W., Deser, C., Mason, S. J., Kirtman, B. P., Msadek, R.,
Sutton, R., Hawkins, E., Fricker, T., Hegerl, G., Ferro, C. A. T.,
Stephenson, D. B., Meehl, G. A., Stockdale, T., Burgman, R., Greene, A. M.,
Kushnir, Y., Newman, M., Carton, J., Fukumori, I., and Delworth, T.: A
verification framework for interannual-to-decadal predictions experiments,
Clim. Dynam., 40, 245–272, <a href="https://doi.org/10.1007/s00382-012-1481-2" target="_blank">https://doi.org/10.1007/s00382-012-1481-2</a>, 2013.
</mixed-citation></ref-html>
<ref-html id="bib1.bib19"><label>Goodfellow(2016)</label><mixed-citation>
Goodfellow, I.: NIPS 2016 Tutorial: Generative Adversarial Networks, available at: <a href="http://arxiv.org/abs/1701.00160" target="_blank"/> (last access: 24 February 2020), 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib20"><label>Goodfellow et al.(2014)Goodfellow, Pouget-Abadie, Mirza, Xu,
Warde-Farley, Ozair, Courville, and Bengio</label><mixed-citation>
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair,
S., Courville, A., and Bengio, Y.: Generative Adversarial Nets, in: Advances
in Neural Information Processing Systems 27, edited by: Ghahramani, Z.,
Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q.,
2672–2680, Curran Associates, Inc., available at:
<a href="http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf" target="_blank"/> (last access: 24 February 2020),
2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib21"><label>Goodfellow et al.(2016)Goodfellow, Bengio, and
Courville</label><mixed-citation>
Goodfellow, I., Bengio, Y., and Courville, A.: Deep Learning, MIT Press, available at:
<a href="http://www.deeplearningbook.org" target="_blank"/> (last access: 24 February 2020), 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib22"><label>Guemas et al.(2013)Guemas, Doblas-Reyes, Andreu-Burillo, and
Asif</label><mixed-citation>
Guemas, V., Doblas-Reyes, F. J., Andreu-Burillo, I., and Asif, M.:
Retrospective prediction of the global warming slowdown in the past decade,
Nat. Clim. Change, 3, 649–653, <a href="https://doi.org/10.1038/nclimate1863" target="_blank">https://doi.org/10.1038/nclimate1863</a>, 2013.
</mixed-citation></ref-html>
<ref-html id="bib1.bib23"><label>He et al.(2015)He, Zhang, Ren, and Sun</label><mixed-citation>
He, K., Zhang, X., Ren, S., and Sun, J.: Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification, in:
Proceedings of the IEEE international conference on computer vision,
1026–1034, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib24"><label>He et al.(2016)He, Zhang, Ren, and Sun</label><mixed-citation>
He, K., Zhang, X., Ren, S., and Sun, J.: Deep residual learning for image
recognition, in: Proceedings of the IEEE conference on computer vision and
pattern recognition, 770–778, Computer Vision Foundation, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib25"><label>Herger et al.(2015)Herger, Sanderson, and Knutti</label><mixed-citation>
Herger, N., Sanderson, B. M., and Knutti, R.: Improved pattern scaling
approaches for the use in climate impact studies, Geophys. Res. Lett., 42,
3486–3494, <a href="https://doi.org/10.1002/2015GL063569" target="_blank">https://doi.org/10.1002/2015GL063569</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib26"><label>Hinton et al.(2012)Hinton, Deng, Yu, Dahl, Mohamed, Jaitly, Senior,
Vanhoucke, Nguyen, Sainath et al.</label><mixed-citation>
Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-R., Jaitly, N., Senior,
A., Vanhoucke, V., Nguyen, P., Sainath, T. N., and Kingsbury, B.: Deep neural networks
for acoustic modeling in speech recognition: The shared views of four
research groups, IEEE Signal Processing Magazine, 29, 82–97, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib27"><label>Hong et al.(2017)Hong, Kim, Joh, and Song</label><mixed-citation>
Hong, S., Kim, S., Joh, M., and Song, S.-K.: GlobeNet: Convolutional Neural
Networks for Typhoon Eye Tracking from Remote Sensing Imagery, ArXiv
e-prints, <a href="http://arxiv.org/abs/1708.03417" target="_blank"/>, 2017.
</mixed-citation></ref-html>
<ref-html id="bib1.bib28"><label>Ioffe and Szegedy(2015)</label><mixed-citation>
Ioffe, S. and Szegedy, C.: Batch normalization: Accelerating deep network
training by reducing internal covariate shift, arXiv preprint
arXiv:1502.03167, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib29"><label>Jay et al.(2018)Jay, Reidmiller, Avery, Barrie, DeAngelo, Dave,
Dzaugis, Kolian, Lewis, Reeves, and Winner</label><mixed-citation>
Jay, A., Reidmiller, D., Avery, C., Barrie, D., DeAngelo, B., Dave, A.,
Dzaugis, M., Kolian, M., Lewis, K., Reeves, K., and Winner, D.: Overview, in:
Impacts, Risks, and Adaptation in the United States: Fourth National Climate
Assessment, Volume II, edited by: Reidmiller, D., Avery, C., Easterling, D.,
Kunkel, K., Lewis, K., Maycock, T., and Stewart, B.,  33–71, U.S. Global
Change Research Program, Washington, DC, USA, <a href="https://doi.org/10.7930/NCA4.2018.CH1" target="_blank">https://doi.org/10.7930/NCA4.2018.CH1</a>,
2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib30"><label>Jiang et al.(2018)Jiang, Xu, and Wei</label><mixed-citation>
Jiang, G.-Q., Xu, J., and Wei, J.: A Deep Learning Algorithm of Neural Network
for the Parameterization of Typhoon-Ocean Feedback in Typhoon Forecast
Models, Geophys. Res. Lett., 45, 3706–3716, <a href="https://doi.org/10.1002/2018GL077004" target="_blank">https://doi.org/10.1002/2018GL077004</a>,
2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib31"><label>JMA(2019)</label><mixed-citation>
JMA: Verification Indices, available at:
<a href="https://www.jma.go.jp/jma/jma-eng/jma-center/nwp/outline2013-nwp/pdf/outline2013_Appendix_A.pdf" target="_blank"/> (last access: February 2020),
2019.
</mixed-citation></ref-html>
<ref-html id="bib1.bib32"><label>Joliffe and Stephenson(2003)</label><mixed-citation>
Joliffe, I. and Stephenson, D.: Forecast verification, John Wiley and Sons,
2003.
</mixed-citation></ref-html>
<ref-html id="bib1.bib33"><label>Karpathy et al.(2014)Karpathy, Toderici, Shetty, Leung, Sukthankar,
and Fei-Fei</label><mixed-citation>
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei,
L.: Large-scale video classification with convolutional neural networks, in:
Proceedings of the IEEE conference on Computer Vision and Pattern
Recognition, Computer Vision Foundation,  1725–1732, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib34"><label>Krizhevsky et al.(2012)Krizhevsky, Sutskever, and
Hinton</label><mixed-citation>
Krizhevsky, A., Sutskever, I., and Hinton, G. E.: Imagenet classification with
deep convolutional neural networks, in: Advances in neural information
processing systems, NIPS Proceedings, 1097–1105, 2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib35"><label>Lean and Rind(2009)</label><mixed-citation>
Lean, J. L. and Rind, D. H.: How will Earth's surface temperature change in
future decades?, Geophys. Res. Lett., 36, L15708,
<a href="https://doi.org/10.1029/2009GL038923" target="_blank">https://doi.org/10.1029/2009GL038923</a>, 2009.
</mixed-citation></ref-html>
<ref-html id="bib1.bib36"><label>LeCun et al.(1998)LeCun, Bottou, Bengio, and
Haffner</label><mixed-citation>
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.: Gradient-based learning
applied to document recognition, Proceedings of the IEEE, 86, 2278–2324,
1998.
</mixed-citation></ref-html>
<ref-html id="bib1.bib37"><label>LeCun et al.(2012)LeCun, Bottou, Orr, and
Müller</label><mixed-citation>
LeCun, Y. A., Bottou, L., Orr, G. B., and Müller, K.-R.: Efficient
backprop, in: Neural networks: Tricks of the trade,  9–48, Springer,
2012.
</mixed-citation></ref-html>
<ref-html id="bib1.bib38"><label>Liu et al.(2016)Liu, Racah, Prabhat, Correa, Khosrowshahi, Lavers,
Kunkel, Wehner, and Collins</label><mixed-citation>
Liu, Y., Racah, E., Prabhat, Correa, J., Khosrowshahi, A., Lavers, D., Kunkel,
K., Wehner, M., and Collins, W.: Application of Deep Convolutional Neural
Networks for Detecting Extreme Weather in Climate Datasets, ArXiv e-prints,
<a href="http://arxiv.org/abs/1605.01156" target="_blank"/>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib39"><label>Lu and Ricciuto(2019)</label><mixed-citation>
Lu, D. and Ricciuto, D.: Efficient surrogate modeling methods for large-scale Earth system models based on machine-learning techniques, Geosci. Model Dev., 12, 1791–1807, <a href="https://doi.org/10.5194/gmd-12-1791-2019" target="_blank">https://doi.org/10.5194/gmd-12-1791-2019</a>, 2019.
</mixed-citation></ref-html>
<ref-html id="bib1.bib40"><label>Lynch et al.(2017)Lynch, Hartin, Bond-Lamberty, and
Kravitz</label><mixed-citation>
Lynch, C., Hartin, C., Bond-Lamberty, B., and Kravitz, B.: An open-access CMIP5 pattern library for temperature and precipitation: description and methodology, Earth Syst. Sci. Data, 9, 281–292, <a href="https://doi.org/10.5194/essd-9-281-2017" target="_blank">https://doi.org/10.5194/essd-9-281-2017</a>, 2017.
</mixed-citation></ref-html>
<ref-html id="bib1.bib41"><label>MacMartin and Kravitz(2016)</label><mixed-citation>
MacMartin, D. G. and Kravitz, B.: Dynamic climate emulators for solar geoengineering, Atmos. Chem. Phys., 16, 15789–15799, <a href="https://doi.org/10.5194/acp-16-15789-2016" target="_blank">https://doi.org/10.5194/acp-16-15789-2016</a>, 2016.
</mixed-citation></ref-html>
<ref-html id="bib1.bib42"><label>McDermott and Wikle(2018)</label><mixed-citation>
McDermott, P. L. and Wikle, C. K.: Deep echo state networks with uncertainty
quantification for spatio-temporal forecasting, Environmetrics, 30, e2553,
<a href="https://doi.org/10.1002/env.2553" target="_blank">https://doi.org/10.1002/env.2553</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib43"><label>Miller et al.(2018)Miller, Nair, Ramachandran, and
Maskey</label><mixed-citation>
Miller, J., Nair, U., Ramachandran, R., and Maskey, M.: Detection of transverse
cirrus bands in satellite imagery using deep learning, Comput.
Geosci., 118, 79–85, <a href="https://doi.org/10.1016/j.cageo.2018.05.012" target="_blank">https://doi.org/10.1016/j.cageo.2018.05.012</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib44"><label>Mitchell(2003)</label><mixed-citation>
Mitchell, T. D.: Pattern Scaling: An Examination of the Accuracy of the
Technique for Describing Future Climates, Clim. Change, 60, 217–242,
<a href="https://doi.org/10.1023/A:1026035305597" target="_blank">https://doi.org/10.1023/A:1026035305597</a>, 2003.
</mixed-citation></ref-html>
<ref-html id="bib1.bib45"><label>Moss et al.(2017)Moss, Kravitz, Delgado, Asrar, Brandenberger,
Wigmosta, Preston, Buzan, Gremillion, Shaw, Stocker, Higuchi, Sarma, Kosmal,
Lawless, Marqusee, Lipschultz, O'Connell, Olsen, Walker, Weaver, Westley, and
Wright</label><mixed-citation>
Moss, R. H., Kravitz, B., Delgado, A., Asrar, G., Brandenberger, J., Wigmosta,
M., Preston, K., Buzan, T., Gremillion, M., Shaw, P., Stocker, K., Higuchi,
S., Sarma, A., Kosmal, A., Lawless, S., Marqusee, J., Lipschultz, F.,
O'Connell, R., Olsen, R., Walker, D., Weaver, C., Westley, M., and Wright,
R.: Nonstationary Weather Patterns and Extreme Events: Informing Design and
Planning for Long-Lived Infrastructure, Tech. rep., ESTCP, ESTCP Project
RC-201591, 2017.
</mixed-citation></ref-html>
<ref-html id="bib1.bib46"><label>Nair and Hinton(2010)</label><mixed-citation>
Nair, V. and Hinton, G. E.: Rectified linear units improve restricted boltzmann
machines, in: Proceedings of the 27th international conference on machine
learning (ICML-10), Association for Computing Machinery,  807–814, 2010.
</mixed-citation></ref-html>
<ref-html id="bib1.bib47"><label>Ouyang and Lu(2018)</label><mixed-citation>
Ouyang, Q. and Lu, W.: Monthly rainfall forecasting using echo state networks coupled with data preprocessing methods, Water Resour. Mange., 32, 659–674,
<a href="https://doi.org/10.1007/s11269-017-1832-1" target="_blank">https://doi.org/10.1007/s11269-017-1832-1</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib48"><label>Pradhan et al.(2018)Pradhan, Aygun, Maskey, Ramachandran, and
Cecil</label><mixed-citation>
Pradhan, R., Aygun, R. S., Maskey, M., Ramachandran, R., and Cecil, D. J.:
Tropical Cyclone Intensity Estimation Using a Deep Convolutional Neural
Network, IEEE Transactions on Image Processing, 27, 692–702,
<a href="https://doi.org/10.1109/TIP.2017.2766358" target="_blank">https://doi.org/10.1109/TIP.2017.2766358</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib49"><label>Rasp et al.(2018)Rasp, Pritchard, and Gentine</label><mixed-citation>
Rasp, S., Pritchard, M. S., and Gentine, P.: Deep learning to represent subgrid
processes in climate models, P. Natl. Acad. Sci., 115, 9684–9689,
<a href="https://doi.org/10.1073/pnas.1810286115" target="_blank">https://doi.org/10.1073/pnas.1810286115</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib50"><label>Robertson et al.(2015)Robertson, Kumar, na, and
Vitart</label><mixed-citation>
Robertson, A. W., Kumar, A., Peña, M., and Vitart, F.: Improving and
Promoting Subseasonal to Seasonal Prediction, B. Am. Meteor. Soc., 96,
ES49–ES53, <a href="https://doi.org/10.1175/BAMS-D-14-00139.1" target="_blank">https://doi.org/10.1175/BAMS-D-14-00139.1</a>, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib51"><label>Ronneberger et al.(2015)Ronneberger, P.Fischer, and
Brox</label><mixed-citation>
Ronneberger, O., Fischer, P., and Brox, T.: U-Net: Convolutional Networks for
Biomedical Image Segmentation, in: Medical Image Computing and
Computer-Assisted Intervention (MICCAI), vol. 9351 of <i>LNCS</i>,
234–241, Springer, available at:
<a href="http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a" target="_blank"/> (last access: February 2020),
(available on arXiv:1505.04597 [cs.CV]), 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib52"><label>Santer et al.(1990)Santer, Wigley, Schlesinger, and
Mitchell</label><mixed-citation>
Santer, B., Wigley, T., Schlesinger, M., and Mitchell, J.: Developing Climate
Scenarios from Equilibrium GCM Results, Tech. rep., Hamburg, Germany,
1990.
</mixed-citation></ref-html>
<ref-html id="bib1.bib53"><label>Shi et al.(2015)Shi, Chen, Wang, Yeung, Wong, and
WOO</label><mixed-citation>
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-k., and WOO, W.-c.:
Convolutional LSTM Network: A Machine Learning Approach for Precipitation
Nowcasting, in: Advances in Neural Information Processing Systems 28, edited
by: Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R.,
802–810, Curran Associates, Inc., available at:
<a href="http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf" target="_blank">http://papers.nips.cc/paper/5955-convolutional-lstm-network-a-machine-learning-approach-for-precipitation-nowcasting.pdf</a> (last access: February 2020),
2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib54"><label>Simonyan and Zisserman(2014)</label><mixed-citation>
Simonyan, K. and Zisserman, A.: Very deep convolutional networks for
large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib55"><label>Srivastava et al.(2014)Srivastava, Hinton, Krizhevkskey, Sutskever,
and Salakhutdinov</label><mixed-citation>
Srivastava, N., Hinton, G., Krizhevkskey, A., Sutskever, I., and Salakhutdinov,
R.: Dropout: A simple way to prevent neural networks for overfitting, J. Mach. Learn. Res., 15, 1929–1958, 2014.
</mixed-citation></ref-html>
<ref-html id="bib1.bib56"><label>Stocker et al.(2013)</label><mixed-citation>
Stocker, T. F., Qin, D., Plattner, G.-K., Alexander, L. V., Allen, S. K., Bindoff, N. L., Bréon, F.-M., Church, J. A., Cubasch, U., Emori, S., Forster, P., Friedlingstein, P., Gillett, N., Gregory, J. M., Hartmann, D. L., Jansen, E., Kirtman, B., Knutti, R., Krishna Kumar, K., Lemke, P., Marotzke, J., Masson-Delmotte, V., Meehl, G. A., Mokhov, I. I., Piao, S., Ramaswamy, V., Randall, D., Rhein, M., Rojas, M., Sabine, C., Shindell, D., Talley, L. D., Vaughan, D. G., and Xie, S.-P.: Technical Summary, in: Climate Change 2013: The Physical
Science Basis. Contribution of Working Group I to the Fifth Assessment Report
of the Intergovernmental Panel on Climate Change, edited by: Stocker, T. F., Qin, D., Plattner, G.-K., Tignor, M., Allen, S. K., Boschung, J., Nauels, A., Xia, Y., Bex, V., and Midgley, P. M., Cambridge University Press, Cambridge, United Kingdom and New York,
NY, USA, 2013.
</mixed-citation></ref-html>
<ref-html id="bib1.bib57"><label>Szegedy et al.(2015)Szegedy, Liu, Jia, Sermanet, Reed, Anguelov,
Erhan, Vanhoucke, and Rabinovich</label><mixed-citation>
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D.,
Vanhoucke, V., and Rabinovich, A.: Going deeper with convolutions, in:
Proceedings of the IEEE conference on computer vision and pattern
recognition, 8–10 June 2015, Boston, Massachusetts, 1–9, 2015.
</mixed-citation></ref-html>
<ref-html id="bib1.bib58"><label>van Vuuren et al.(2011)van Vuuren, Edmonds, Kainuma, Riahi, Thomson,
Hibbard, Hurtt, Kram, Krey, Lamarque, Masui, Meinshausen, Nakicenovic, Smith,
and Rose</label><mixed-citation>
van Vuuren, D. P., Edmonds, J., Kainuma, M., Riahi, K., Thomson, A., Hibbard,
K., Hurtt, G. C., Kram, T., Krey, V., Lamarque, J.-F., Masui, T.,
Meinshausen, M., Nakicenovic, N., Smith, S. J., and Rose, S. K.: The
representative concentration pathways: An overview, Clim. Change, 109,
5–31, <a href="https://doi.org/10.1007/s10584-011-0148-z" target="_blank">https://doi.org/10.1007/s10584-011-0148-z</a>, 2011.
</mixed-citation></ref-html>
<ref-html id="bib1.bib59"><label>Weber et al.(2020)</label><mixed-citation>
Weber, T., Corotan, A., Hutchinson, B., Kravitz, B., and Link, R. P.: A Deep Neural Network approach for estimating precipitation fields in Earth System Models, available at: <a href="https://github.com/hutchresearch/deep_climate_emulator" target="_blank"/>, last access: 24 February 2020.
</mixed-citation></ref-html>
<ref-html id="bib1.bib60"><label>Yao et al.(2007)Yao, Rosasco, and Caponnetto</label><mixed-citation>
Yao, Y., Rosasco, L., and Caponnetto, A.: On early stopping in gradient descent
learning, Constructive Approximation, 26, 289–315, 2007.
</mixed-citation></ref-html>
<ref-html id="bib1.bib61"><label>Yeager et al.(2018)Yeager, Danabasoglu, Rosenbloom, Strand, Bates,
Meehl, Karspeck, Lindsay, Long, Teng, and Lovenduski</label><mixed-citation>
Yeager, S., Danabasoglu, G., Rosenbloom, N., Strand, W., Bates, S., Meehl, G.,
Karspeck, A., Lindsay, K., Long, M., Teng, H., and Lovenduski, N.: Predicting
Near-Term Changes in the Earth System: A Large Ensemble of Initialized
Decadal Prediction Simulations Using the Community Earth System Model,
B. Am. Meteor. Soc., 99, 1867–1886, <a href="https://doi.org/10.1175/BAMS-D-17-0098.1" target="_blank">https://doi.org/10.1175/BAMS-D-17-0098.1</a>,
2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib62"><label>Yeh et al.(2018)Yeh, Cai, Min, McPhaden, Dommenget, Dewitte, Collins,
Ashok, An, Yim, and Kug</label><mixed-citation>
Yeh, S.-W., Cai, W., Min, S.-K., McPhaden, M. J., Dommenget, D., Dewitte, B.,
Collins, M., Ashok, K., An, S.-I., Yim, B.-Y., and Kug, J.-S.: ENSO
Atmospheric Teleconnections and Their Response to Greenhouse Gas Forcing,
Rev. Geophys., 56, 185–206, <a href="https://doi.org/10.1002/2017RG000568" target="_blank">https://doi.org/10.1002/2017RG000568</a>, 2018.
</mixed-citation></ref-html>
<ref-html id="bib1.bib63"><label>Yuan et al.(2019)Yuan, Huang, Duan, Zhu, Xoplaki, and
Luterbacher</label><mixed-citation>
Yuan, N., Huang, Y., Duan, J., Zhu, C., Xoplaki, E., and Luterbacher, J.: On
climate prediction: How much can we expect from climate memory?, Clim.
Dynam., 52, 855–864, <a href="https://doi.org/10.1007/s00382-018-4168-5" target="_blank">https://doi.org/10.1007/s00382-018-4168-5</a>, 2019.
</mixed-citation></ref-html>
<ref-html id="bib1.bib64"><label>Zhang and Sutton(2017)</label><mixed-citation>
Zhang, S. and Sutton, R. S.: A Deeper Look at Experience Replay, CoRR,
abs/1712.01275, 2017.
</mixed-citation></ref-html>--></article>
