Post-processing ECMWF precipitation and temperature
ensemble reforecasts for operational hydrologic
forecasting
Jan Verkade
1 2 3
, James Brown 4 , Albrecht Weerts
1 5
and Paolo Reggiani
6
November 2014 HEPEX meeting @ ECMWF
1
2
3
4
5
6
Deltares, Delft, The Netherlands
Delft University of Technology, The Netherlands
Rijkswaterstaat River Forecasting Service, Lelystad, The Netherlands
Hydrologic Solutions Ltd, Southampton, United Kingdom
Wageningen University and Research Centre, The Netherlands
University of Siegen, Germany
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
1 / 33
These slides are available from twitter.com/janverkade
(see www.hepex.org)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
2 / 33
Summary conclusions
Postprocessing of temperature, precipitation forecasts improves skill
Quality improvement does not proportionally propagate to streamflow
forecasts
We believe this is due to:
I
I
I
non-linearities in rainfall to runoff processes
presence of storages
inadequate space-time modelling using Schaake shuffle
These results are largely in line with those obtained in similar studies
(Zalachori et al., 2012; Kang et al., 2010)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
3 / 33
Introduction I
Setting the scene
hydrologic forecasting
ensemble prediction
reduction of predictive uncertainties
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
4 / 33
Introduction II
Problem statement
Numerical Weather Prediction products (NWP)
propagation of biases
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
5 / 33
Introduction III
Statistical post-processing
often applied to streamflow forecasts directly
can be applied to NWP also (‘pre-processing’)
Research question
To which extent can biases (mean, spread) in streamflow forecasts be
addressed through post-processing of the forcing ensembles?
1
To what extent are the ‘raw’ forcing ensembles biased?
2
How do these biases propagate to streamflow ensembles?
3
Can quality of ‘raw’ forcing ensembles be improved by
post-processing?
4
Does this quality improvement proportionally translate to streamflow
ensembles?
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
6 / 33
Research design
Predicted
forcings (e.g.
P,T,E-ensembles)
Predicted
forcings (e.g.
P,T,E-ensembles)
Verkade-Brown-Weerts-Reggiani
Predicted
streamflow
ensembles
Bias-correct
forcings
(“pre-process”)
Preprocessing for hydrologic forecasting
Predicted
streamflow
ensembles
November 7, 2014
7 / 33
Rhine basin
Rhine basin
Rhine tributary basins
HBV-subbasins
Hydrological stations
Outlet (Lobith)
Main tributaries
Headwater basins
Rhine stream network
State borders
0
100
200
300
400 [km]
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
9 / 33
Observations from the E-OBS dataset (www.ecad.eu/E-OBS/)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
10 / 33
Bias-correction principles and techniques I
Temperature
quantile-to-quantile transform
linear Gaussian regression
Precipitation
quantile–to–quantile transform
logistic regression (Hamill et al., 2008)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
11 / 33
Bias-correction principles and techniques II
Principles of conditional techniques
Predictand Y = observed temperature, precipitation or streamflow.
Assumed unbiased!
Potential predictors X = {X1 , . . . , X5 , . . . , Xm }; biased.
The bias—corrected forecast:
F (y |x1 , . . . , xm ) = P [Y ≤ y | X1 = x1 , . . . , Xm = xm ] ∀y
for each lead—time and each location separately
After bias-correction: “Schaake Shuffle” (Clark et al., 2004) to
maintain spatial and temporal patterns (“traces”)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
12 / 33
Bias-correction principles and techniques III
Combinations of techniques used
Uncorrected temperature, precipitation ensemble forecasts (raw–raw)
Quantile-to-quantile transformed temperature, precipitation forecasts
(qqt–qqt)
linear Gaussian regression (temperature) and logistic regression
(precipitation) (lin–log)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
13 / 33
Ensemble verification
Verification against simulations!
Skills shown are relative to sample climatology
Metrics expressed as function of P
Metrics shown here:
I
I
I
I
Relative Mean Error
Brier’s probability skill score
Mean Continuous Ranked Probability skill Score
Relative Operating Characteristic skill score
metrics computed using Ensemble Verification System (Brown et al.,
2010)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
14 / 33
Verification graphs
best
Metric
Baseline9where9skill=0
Sub-sample9ke.g.9pairs9with9
top91F9of9observations9at9
P=0.99)
Full,9unconditional9
sample9at9P=0
worst
0
Value9of9the9paired9observation
klabelled9with9climatological
probability9of9non-exceedence9P)
0.99
Figure: Sample verification plot
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
15 / 33
Temperature
24 hrs
0.1
120 hrs
240 hrs
RME [-]
0.0
-0.1
raw (median)
raw (q10-q90)
qqt (median)
qqt (q10-q90)
lin (median)
lin (q10-q90)
-0.2
-0.3
1.0
CRPSS [-]
0.8
0.6
0.4
0.2
0.0
1.0
BSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
ROCS [-]
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
Observed value (Logite axis labelled with climatological nonexceedence probability)
Temperature, 134 basins
0.99
Precipitation I
QQT
RAW
LOG
error = forecast - observation [mm/d]
40
20
0
-20
-40
ensemble mean and range
0
10
20
30
40
50
0
10
20
30
40
50
0
10
20
30
40
50
observation [mm/d]
Precipitation, I-RN-0001, 72-hour lead time (in Neckar basin)
Precipitation II
24 hrs
120 hrs
240 hrs
0.2
RME [-]
0.0
-0.2
-0.4
-0.6
-0.8
-1.0
raw (median)
raw (q10-q90)
qqt (median)
qqt (q10-q90)
log (median)
log (q10-q90)
CRPSS [-]
0.6
0.4
0.2
0.0
-0.2
-0.4
0.6
BSS [-]
0.4
0.2
0.0
-0.2
1.0
ROCS [-]
0.8
0.6
0.4
0.2
0.0
0.5
0.9
0.99
0.5
0.9
0.99
0.5
0.9
Observed value (Logite axis labelled with climatological nonexceedence probability)
Precipitation, 134 basins
0.99
Precipitation III
24 hrs
120 hrs
240 hrs
CRPSS [-]
0.6
0.4
0.2
0.0
-0.2
-0.4
raw (median)
raw (q10-q90)
qqt (median)
qqt (q10-q90)
log (median)
log (q10-q90)
CRPSS-rel [-]
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
CRPSS-res [-]
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
0.10
CRPSS-unc [-]
0.15
0.20
0.25
0.30
0.35
0.40
0.5
0.9
0.99
0.5
0.9
0.99
0.5
0.9
Observed value (Logite axis labelled with climatological nonexceedence probability)
Precipitation, 134 basins, CRPSS
0.99
Precipitation IV
24 hrs
0.6
120 hrs
240 hrs
BSS [-]
0.4
0.2
0.0
-0.2
raw (median)
raw (q10-q90)
qqt (median)
qqt (q10-q90)
log (median)
log (q10-q90)
0.0
BSS-rel [-]
0.1
0.2
0.3
0.4
0.6
BSS-res [-]
0.5
0.4
0.3
0.2
0.1
0.0
0.5
0.9
0.99
0.5
0.9
0.99
0.5
0.9
0.99
Observed value (Logite axis labelled with climatological nonexceedence probability)
Precipitation, 134 basins, BSS Type I
Precipitation V
24 hrs
0.6
120 hrs
240 hrs
BSS [-]
0.4
0.2
0.0
-0.2
raw (median)
raw (q10-q90)
qqt (median)
qqt (q10-q90)
log (median)
log (q10-q90)
BSS-tp2 [-]
0.0
0.2
0.4
0.6
0.8
BSS-dis [-]
1.0
1.0
0.8
0.6
0.4
0.2
BSS-sha [-]
0.0
1.0
0.8
0.6
0.4
0.2
0.0
0.5
0.9
0.99
0.5
0.9
0.99
0.5
0.9
0.99
Observed value (Logite axis labelled with climatological nonexceedence probability)
Precipitation, 134 basins, BSS Type II
Streamflow I
Three spatial scales:
1
Basin outlet at Lobith
2
Four main tributaries: Main, Moselle, Neckar, Swiss Rhein
3
∼40 headwater basins
Streamflow II
24 hrs
120 hrs
240 hrs
RME [-]
0.0
-0.2
-0.4
raw-raw (median)
raw-raw (q10-q90)
qqt-qqt (median)
qqt-qqt (q10-q90)
lin-log (median)
lin-log (q10-q90)
-0.6
1.0
CRPSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
BSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
ROCS [-]
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, 43 headwater basins
0.99
Streamflow III
24 hrs
1.0
120 hrs
240 hrs
CRPSS [-]
0.8
0.6
0.4
raw-raw (median)
raw-raw (q10-q90)
qqt-qqt (median)
qqt-qqt (q10-q90)
lin-log (median)
lin-log (q10-q90)
0.2
0.0
CRPSS-rel [-]
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
CRPSS-res [-]
0.4
0.2
0.0
-0.2
-0.4
CRPSS-unc [-]
0.0
0.1
0.2
0.3
0.4
0.5
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, 43 headwater basins, CRPSS
Streamflow IV
24 hrs
1.0
120 hrs
240 hrs
0.8
BSS [-]
0.6
0.4
0.2
raw-raw (median)
raw-raw (q10-q90)
qqt-qqt (median)
qqt-qqt (q10-q90)
lin-log (median)
lin-log (q10-q90)
0.0
-0.2
0.00
0.05
BSS-rel [-]
0.10
0.15
0.20
0.25
0.30
1.0
BSS-res [-]
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, 43 headwater basins, BSS Type I
Streamflow V
24 hrs
1.0
120 hrs
240 hrs
BSS [-]
0.8
0.6
0.4
raw-raw (median)
raw-raw (q10-q90)
qqt-qqt (median)
qqt-qqt (q10-q90)
lin-log (median)
lin-log (q10-q90)
0.2
0.0
BSS-tp2 [-]
-0.2
0.0
0.2
0.4
0.6
0.8
BSS-dis [-]
1.0
1.0
0.8
0.6
0.4
0.2
0.0
BSS-sha [-]
1.0
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, 43 headwater basins, BSS Type II
Streamflow VI
24 hrs
120 hrs
240 hrs
RME [-]
0.0
-0.2
-0.4
raw-raw (median)
raw-raw (q10-q90)
qqt-qqt (median)
qqt-qqt (q10-q90)
lin-log (median)
lin-log (q10-q90)
-0.6
1.0
CRPSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
BSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
ROCS [-]
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, 4 tributaries, BSS Type II
Streamflow VII
24 hrs
120 hrs
240 hrs
RME [-]
0.0
-0.2
-0.4
raw-raw
qqt-qqt
lin-log
-0.6
1.0
CRPSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
BSS [-]
0.8
0.6
0.4
0.2
0.0
-0.2
1.0
ROCS [-]
0.8
0.6
0.4
0.2
0.0
0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
0.99 0
0.1
0.5
0.9
Simulated value (Logite axis labelled with climatological nonexceedence probability)
Streamflow, Rhine outlet at Lobith
0.99
Summary conclusions
Postprocessing of temperature, precipitation forecasts improves skill
Quality improvement does not proportionally propagate to streamflow
forecasts
We believe this is due to:
I
I
I
non-linearities in rainfall to runoff processes
presence of storages
inadequate space-time modelling using Schaake shuffle
These results are largely in line with those obtained in similar studies
(Zalachori et al., 2012; Kang et al., 2010)
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
29 / 33
Journal of Hydrology 501 (2013) 73–91
Contents lists available at ScienceDirect
Journal of Hydrology
journal homepage: www.elsevier.com/locate/jhydrol
Post-processing ECMWF precipitation and temperature ensemble
reforecasts for operational hydrologic forecasting at various spatial
scales q
J.S. Verkade a,b,c,⇑, J.D. Brown d, P. Reggiani a,e, A.H. Weerts a,f
a
Deltares, PO Box 177, 2600 MH Delft, The Netherlands
Ministry of Infrastructure and the Environment, Water Management Centre of The Netherlands, River Forecasting Service, Lelystad, The Netherlands
Delft University of Technology, Delft, The Netherlands
Hydrologic Solutions Limited, Southampton, United Kingdom
e
RWTH Aachen University, Aachen, Germany
f
Wageningen University and Research Centre, Hydrology and Quantitative Water Management Group, Wageningen, The Netherlands
b
c
d
a r t i c l e
i n f o
Article history:
Received 20 January 2013
Received in revised form 21 July 2013
Accepted 28 July 2013
Available online 7 August 2013
This manuscript was handled by
Konstantine P. Georgakakos, Editor-in-Chief,
with the assistance of Ashish Sharma,
Associate Editor
Keywords:
Bias-correction
Post-processing
Ensemble forecasting
Verkade-Brown-Weerts-Reggiani
s u m m a r y
The ECMWF temperature and precipitation ensemble reforecasts are evaluated for biases in the mean,
spread and forecast probabilities, and how these biases propagate to streamﬂow ensemble forecasts.
The forcing ensembles are subsequently post-processed to reduce bias and increase skill, and to investigate whether this leads to improved streamﬂow ensemble forecasts. Multiple post-processing techniques
are used: quantile-to-quantile transform, linear regression with an assumption of bivariate normality and
logistic regression. Both the raw and post-processed ensembles are run through a hydrologic model of the
river Rhine to create streamﬂow ensembles. The results are compared using multiple veriﬁcation metrics
and skill scores: relative mean error, Brier skill score and its decompositions, mean continuous ranked
probability skill score and its decomposition, and the ROC score. Veriﬁcation of the streamﬂow ensembles
is performed at multiple spatial scales: relatively small headwater basins, large tributaries and the Rhine
outlet at Lobith. The streamﬂow ensembles are veriﬁed against simulated streamﬂow, in order to isolate
the effects of biases in the forcing ensembles and any improvements therein. The results indicate that the
Preprocessing
for hydrologic
forecasting
7, 2014Some
forcing ensembles contain
signiﬁcant biases,
and that these cascade to theNovember
streamﬂow ensembles.
30 / 33
Bibliography I
Brown, J. D., Demargne, J., Seo, D.-J., and Liu, Y.: The Ensemble
Verification System (EVS): A software tool for verifying ensemble
forecasts of hydrometeorological and hydrologic variables at discrete
locations, Environmental Modelling & Software, 25, 854 – 872, 2010.
Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., and Wilby, R.:
The Schaake Shuffle: A method for reconstructing space-time variability
in forecasted precipitation and temperature fields, Journal of
Hydrometeorology, 5, 243–262, 2004.
Hamill, T., Hagedorn, R., and Whitaker, J.: Probabilistic forecast
calibration using ECMWF and GFS ensemble reforecasts. Part II:
Precipitation, Monthly Weather Review, 136, 2620–2632, 2008.
Kang, T.-H., Kim, Y.-O., and Hong, I.-P.: Comparison of pre- and
post-processors for ensemble streamflow prediction, Atmospheric
Science Letters, 11, 153–159, doi:10.1002/asl.276, URL
http://doi.wiley.com/10.1002/asl.276, 2010.
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
31 / 33
Bibliography II
Zalachori, I., Ramos, M.-H., Gar¸con, R., Mathevet, T., and Gailhard, J.:
Statistical processing of forecasts for hydrological ensemble prediction: a
comparative study of different bias correction strategies, Advances in
Science and Research, 8, 135–141, doi:10.5194/asr-8-135-2012, URL
http://www.adv-sci-res.net/8/135/2012/, 2012.
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
32 / 33
Contact details
Jan Verkade
[email protected]
James Brown
[email protected]
Slides are available via twitter.com/janverkade
Verkade-Brown-Weerts-Reggiani
Preprocessing for hydrologic forecasting
November 7, 2014
33 / 33