Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting Jan Verkade 1 2 3 , James Brown 4 , Albrecht Weerts 1 5 and Paolo Reggiani 6 November 2014 HEPEX meeting @ ECMWF 1 2 3 4 5 6 Deltares, Delft, The Netherlands Delft University of Technology, The Netherlands Rijkswaterstaat River Forecasting Service, Lelystad, The Netherlands Hydrologic Solutions Ltd, Southampton, United Kingdom Wageningen University and Research Centre, The Netherlands University of Siegen, Germany Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 1 / 33 These slides are available from twitter.com/janverkade (see www.hepex.org) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 2 / 33 Summary conclusions Postprocessing of temperature, precipitation forecasts improves skill Quality improvement does not proportionally propagate to streamflow forecasts We believe this is due to: I I I non-linearities in rainfall to runoff processes presence of storages inadequate space-time modelling using Schaake shuffle These results are largely in line with those obtained in similar studies (Zalachori et al., 2012; Kang et al., 2010) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 3 / 33 Introduction I Setting the scene hydrologic forecasting ensemble prediction reduction of predictive uncertainties Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 4 / 33 Introduction II Problem statement Numerical Weather Prediction products (NWP) propagation of biases Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 5 / 33 Introduction III Statistical post-processing often applied to streamflow forecasts directly can be applied to NWP also (‘pre-processing’) Research question To which extent can biases (mean, spread) in streamflow forecasts be addressed through post-processing of the forcing ensembles? 1 To what extent are the ‘raw’ forcing ensembles biased? 2 How do these biases propagate to streamflow ensembles? 3 Can quality of ‘raw’ forcing ensembles be improved by post-processing? 4 Does this quality improvement proportionally translate to streamflow ensembles? Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 6 / 33 Research design Predicted forcings (e.g. P,T,E-ensembles) Predicted forcings (e.g. P,T,E-ensembles) Verkade-Brown-Weerts-Reggiani Predicted streamflow ensembles Bias-correct forcings (“pre-process”) Preprocessing for hydrologic forecasting Predicted streamflow ensembles November 7, 2014 7 / 33 Rhine basin Rhine basin Rhine tributary basins HBV-subbasins Hydrological stations Outlet (Lobith) Main tributaries Headwater basins Rhine stream network State borders 0 100 200 300 400 [km] Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 9 / 33 Observations from the E-OBS dataset (www.ecad.eu/E-OBS/) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 10 / 33 Bias-correction principles and techniques I Temperature quantile-to-quantile transform linear Gaussian regression Precipitation quantile–to–quantile transform logistic regression (Hamill et al., 2008) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 11 / 33 Bias-correction principles and techniques II Principles of conditional techniques Predictand Y = observed temperature, precipitation or streamflow. Assumed unbiased! Potential predictors X = {X1 , . . . , X5 , . . . , Xm }; biased. The bias—corrected forecast: F (y |x1 , . . . , xm ) = P [Y ≤ y | X1 = x1 , . . . , Xm = xm ] ∀y for each lead—time and each location separately After bias-correction: “Schaake Shuffle” (Clark et al., 2004) to maintain spatial and temporal patterns (“traces”) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 12 / 33 Bias-correction principles and techniques III Combinations of techniques used Uncorrected temperature, precipitation ensemble forecasts (raw–raw) Quantile-to-quantile transformed temperature, precipitation forecasts (qqt–qqt) linear Gaussian regression (temperature) and logistic regression (precipitation) (lin–log) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 13 / 33 Ensemble verification Verification against simulations! Skills shown are relative to sample climatology Metrics expressed as function of P Metrics shown here: I I I I Relative Mean Error Brier’s probability skill score Mean Continuous Ranked Probability skill Score Relative Operating Characteristic skill score metrics computed using Ensemble Verification System (Brown et al., 2010) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 14 / 33 Verification graphs best Metric Baseline9where9skill=0 Sub-sample9ke.g.9pairs9with9 top91F9of9observations9at9 P=0.99) Full,9unconditional9 sample9at9P=0 worst 0 Value9of9the9paired9observation klabelled9with9climatological probability9of9non-exceedence9P) 0.99 Figure: Sample verification plot Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 15 / 33 Temperature 24 hrs 0.1 120 hrs 240 hrs RME [-] 0.0 -0.1 raw (median) raw (q10-q90) qqt (median) qqt (q10-q90) lin (median) lin (q10-q90) -0.2 -0.3 1.0 CRPSS [-] 0.8 0.6 0.4 0.2 0.0 1.0 BSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 ROCS [-] 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 Observed value (Logite axis labelled with climatological nonexceedence probability) Temperature, 134 basins 0.99 Precipitation I QQT RAW LOG error = forecast - observation [mm/d] 40 20 0 -20 -40 ensemble mean and range 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 observation [mm/d] Precipitation, I-RN-0001, 72-hour lead time (in Neckar basin) Precipitation II 24 hrs 120 hrs 240 hrs 0.2 RME [-] 0.0 -0.2 -0.4 -0.6 -0.8 -1.0 raw (median) raw (q10-q90) qqt (median) qqt (q10-q90) log (median) log (q10-q90) CRPSS [-] 0.6 0.4 0.2 0.0 -0.2 -0.4 0.6 BSS [-] 0.4 0.2 0.0 -0.2 1.0 ROCS [-] 0.8 0.6 0.4 0.2 0.0 0.5 0.9 0.99 0.5 0.9 0.99 0.5 0.9 Observed value (Logite axis labelled with climatological nonexceedence probability) Precipitation, 134 basins 0.99 Precipitation III 24 hrs 120 hrs 240 hrs CRPSS [-] 0.6 0.4 0.2 0.0 -0.2 -0.4 raw (median) raw (q10-q90) qqt (median) qqt (q10-q90) log (median) log (q10-q90) CRPSS-rel [-] 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 CRPSS-res [-] 0.2 0.1 0.0 -0.1 -0.2 -0.3 -0.4 0.10 CRPSS-unc [-] 0.15 0.20 0.25 0.30 0.35 0.40 0.5 0.9 0.99 0.5 0.9 0.99 0.5 0.9 Observed value (Logite axis labelled with climatological nonexceedence probability) Precipitation, 134 basins, CRPSS 0.99 Precipitation IV 24 hrs 0.6 120 hrs 240 hrs BSS [-] 0.4 0.2 0.0 -0.2 raw (median) raw (q10-q90) qqt (median) qqt (q10-q90) log (median) log (q10-q90) 0.0 BSS-rel [-] 0.1 0.2 0.3 0.4 0.6 BSS-res [-] 0.5 0.4 0.3 0.2 0.1 0.0 0.5 0.9 0.99 0.5 0.9 0.99 0.5 0.9 0.99 Observed value (Logite axis labelled with climatological nonexceedence probability) Precipitation, 134 basins, BSS Type I Precipitation V 24 hrs 0.6 120 hrs 240 hrs BSS [-] 0.4 0.2 0.0 -0.2 raw (median) raw (q10-q90) qqt (median) qqt (q10-q90) log (median) log (q10-q90) BSS-tp2 [-] 0.0 0.2 0.4 0.6 0.8 BSS-dis [-] 1.0 1.0 0.8 0.6 0.4 0.2 BSS-sha [-] 0.0 1.0 0.8 0.6 0.4 0.2 0.0 0.5 0.9 0.99 0.5 0.9 0.99 0.5 0.9 0.99 Observed value (Logite axis labelled with climatological nonexceedence probability) Precipitation, 134 basins, BSS Type II Streamflow I Three spatial scales: 1 Basin outlet at Lobith 2 Four main tributaries: Main, Moselle, Neckar, Swiss Rhein 3 ∼40 headwater basins Streamflow II 24 hrs 120 hrs 240 hrs RME [-] 0.0 -0.2 -0.4 raw-raw (median) raw-raw (q10-q90) qqt-qqt (median) qqt-qqt (q10-q90) lin-log (median) lin-log (q10-q90) -0.6 1.0 CRPSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 BSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 ROCS [-] 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, 43 headwater basins 0.99 Streamflow III 24 hrs 1.0 120 hrs 240 hrs CRPSS [-] 0.8 0.6 0.4 raw-raw (median) raw-raw (q10-q90) qqt-qqt (median) qqt-qqt (q10-q90) lin-log (median) lin-log (q10-q90) 0.2 0.0 CRPSS-rel [-] -0.2 0.0 0.2 0.4 0.6 0.8 1.0 CRPSS-res [-] 0.4 0.2 0.0 -0.2 -0.4 CRPSS-unc [-] 0.0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, 43 headwater basins, CRPSS Streamflow IV 24 hrs 1.0 120 hrs 240 hrs 0.8 BSS [-] 0.6 0.4 0.2 raw-raw (median) raw-raw (q10-q90) qqt-qqt (median) qqt-qqt (q10-q90) lin-log (median) lin-log (q10-q90) 0.0 -0.2 0.00 0.05 BSS-rel [-] 0.10 0.15 0.20 0.25 0.30 1.0 BSS-res [-] 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, 43 headwater basins, BSS Type I Streamflow V 24 hrs 1.0 120 hrs 240 hrs BSS [-] 0.8 0.6 0.4 raw-raw (median) raw-raw (q10-q90) qqt-qqt (median) qqt-qqt (q10-q90) lin-log (median) lin-log (q10-q90) 0.2 0.0 BSS-tp2 [-] -0.2 0.0 0.2 0.4 0.6 0.8 BSS-dis [-] 1.0 1.0 0.8 0.6 0.4 0.2 0.0 BSS-sha [-] 1.0 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, 43 headwater basins, BSS Type II Streamflow VI 24 hrs 120 hrs 240 hrs RME [-] 0.0 -0.2 -0.4 raw-raw (median) raw-raw (q10-q90) qqt-qqt (median) qqt-qqt (q10-q90) lin-log (median) lin-log (q10-q90) -0.6 1.0 CRPSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 BSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 ROCS [-] 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, 4 tributaries, BSS Type II Streamflow VII 24 hrs 120 hrs 240 hrs RME [-] 0.0 -0.2 -0.4 raw-raw qqt-qqt lin-log -0.6 1.0 CRPSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 BSS [-] 0.8 0.6 0.4 0.2 0.0 -0.2 1.0 ROCS [-] 0.8 0.6 0.4 0.2 0.0 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 0.99 0 0.1 0.5 0.9 Simulated value (Logite axis labelled with climatological nonexceedence probability) Streamflow, Rhine outlet at Lobith 0.99 Summary conclusions Postprocessing of temperature, precipitation forecasts improves skill Quality improvement does not proportionally propagate to streamflow forecasts We believe this is due to: I I I non-linearities in rainfall to runoff processes presence of storages inadequate space-time modelling using Schaake shuffle These results are largely in line with those obtained in similar studies (Zalachori et al., 2012; Kang et al., 2010) Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 29 / 33 Journal of Hydrology 501 (2013) 73–91 Contents lists available at ScienceDirect Journal of Hydrology journal homepage: www.elsevier.com/locate/jhydrol Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting at various spatial scales q J.S. Verkade a,b,c,⇑, J.D. Brown d, P. Reggiani a,e, A.H. Weerts a,f a Deltares, PO Box 177, 2600 MH Delft, The Netherlands Ministry of Infrastructure and the Environment, Water Management Centre of The Netherlands, River Forecasting Service, Lelystad, The Netherlands Delft University of Technology, Delft, The Netherlands Hydrologic Solutions Limited, Southampton, United Kingdom e RWTH Aachen University, Aachen, Germany f Wageningen University and Research Centre, Hydrology and Quantitative Water Management Group, Wageningen, The Netherlands b c d a r t i c l e i n f o Article history: Received 20 January 2013 Received in revised form 21 July 2013 Accepted 28 July 2013 Available online 7 August 2013 This manuscript was handled by Konstantine P. Georgakakos, Editor-in-Chief, with the assistance of Ashish Sharma, Associate Editor Keywords: Bias-correction Post-processing Ensemble forecasting Verkade-Brown-Weerts-Reggiani s u m m a r y The ECMWF temperature and precipitation ensemble reforecasts are evaluated for biases in the mean, spread and forecast probabilities, and how these biases propagate to streamflow ensemble forecasts. The forcing ensembles are subsequently post-processed to reduce bias and increase skill, and to investigate whether this leads to improved streamflow ensemble forecasts. Multiple post-processing techniques are used: quantile-to-quantile transform, linear regression with an assumption of bivariate normality and logistic regression. Both the raw and post-processed ensembles are run through a hydrologic model of the river Rhine to create streamflow ensembles. The results are compared using multiple verification metrics and skill scores: relative mean error, Brier skill score and its decompositions, mean continuous ranked probability skill score and its decomposition, and the ROC score. Verification of the streamflow ensembles is performed at multiple spatial scales: relatively small headwater basins, large tributaries and the Rhine outlet at Lobith. The streamflow ensembles are verified against simulated streamflow, in order to isolate the effects of biases in the forcing ensembles and any improvements therein. The results indicate that the Preprocessing for hydrologic forecasting 7, 2014Some forcing ensembles contain significant biases, and that these cascade to theNovember streamflow ensembles. 30 / 33 Bibliography I Brown, J. D., Demargne, J., Seo, D.-J., and Liu, Y.: The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations, Environmental Modelling & Software, 25, 854 – 872, 2010. Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., and Wilby, R.: The Schaake Shuffle: A method for reconstructing space-time variability in forecasted precipitation and temperature fields, Journal of Hydrometeorology, 5, 243–262, 2004. Hamill, T., Hagedorn, R., and Whitaker, J.: Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: Precipitation, Monthly Weather Review, 136, 2620–2632, 2008. Kang, T.-H., Kim, Y.-O., and Hong, I.-P.: Comparison of pre- and post-processors for ensemble streamflow prediction, Atmospheric Science Letters, 11, 153–159, doi:10.1002/asl.276, URL http://doi.wiley.com/10.1002/asl.276, 2010. Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 31 / 33 Bibliography II Zalachori, I., Ramos, M.-H., Gar¸con, R., Mathevet, T., and Gailhard, J.: Statistical processing of forecasts for hydrological ensemble prediction: a comparative study of different bias correction strategies, Advances in Science and Research, 8, 135–141, doi:10.5194/asr-8-135-2012, URL http://www.adv-sci-res.net/8/135/2012/, 2012. Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 32 / 33 Contact details Jan Verkade [email protected] James Brown [email protected] Slides are available via twitter.com/janverkade Verkade-Brown-Weerts-Reggiani Preprocessing for hydrologic forecasting November 7, 2014 33 / 33
© Copyright 2024