Efficiency in Railway Operations and Infrastructure Management
Efficiency in Railway Operations and Infrastructure Management 12 Discussion Paper 2015 • 12 Dejan Makovsek, Vincent Benezech and Stephen Perkins International Transport Forum, Paris, France Efficiency in Railway Operations and Infrastructure Management Discussion Paper No. 2015-12 Summary and Conclusions of the Roundtable on Efficiency in Railway Operations and Infrastructure Management (18-19 November 2014, Paris) Dejan MAKOVSEK, Vincent BENEZECH, and Stephen PERKINS International Transport Forum at the OECD, Paris May 2015 THE INTERNATIONAL TRANSPORT FORUM The International Transport Forum at the OECD is an intergovernmental organisation with 54 member countries. It acts as a strategic think-tank, with the objective of helping shape the transport policy agenda on a global level and ensuring that it contributes to economic growth, environmental protection, social inclusion and the preservation of human life and well-being. The International Transport Forum organises an Annual Summit of ministers along with leading representatives from industry, civil society and academia. The International Transport Forum was created under a Declaration issued by the Council of Ministers of the ECMT (European Conference of Ministers of Transport) at its Ministerial Session in May 2006 under the legal authority of the Protocol of the ECMT, signed in Brussels on 17 October 1953, and legal instruments of the OECD. The Members of the Forum are: Albania, Armenia, Australia, Austria, Azerbaijan, Belarus, Belgium, Bosnia and Herzegovina, Bulgaria, Canada, Chile, China (People’s Republic of), Croatia, Czech Republic, Denmark, Estonia, Finland, France, Former Yugoslav Republic of Macedonia, Georgia, Germany, Greece, Hungary, Iceland, India, Ireland, Italy, Japan, Korea, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Mexico, Republic of Moldova, Montenegro, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Russian Federation, Serbia, Slovak Republic, Slovenia, Spain, Sweden, Switzerland, Turkey, Ukraine, United Kingdom and United States. The International Transport Forum’s Research Centre gathers statistics and conducts co-operative research programmes addressing all modes of transport. Its findings are widely disseminated and support policymaking in member countries as well as contributing to the Annual Summit. Discussion Papers The International Transport Forum’s Discussion Paper Series makes economic research, commissioned or carried out at its Research Centre, available to researchers and practitioners. The aim is to contribute to the understanding of the transport sector and to provide inputs to transport policy design. ITF Discussion Papers should not be reported as representing the official views of the ITF or of its member countries. The opinions expressed and arguments employed are those of the authors. Discussion Papers describe preliminary results or research in progress by the author(s) and are published to stimulate discussion on a broad range of issues on which the ITF works. Comments on Discussion Papers are welcomed, and may be sent to: International Transport Forum/OECD, 2 rue AndréPascal, 75775 Paris Cedex 16, France. For further information on the Discussion Papers and other JTRC activities, please email: [email protected] The Discussion Papers can be downloaded from www.internationaltransportforum.org/jtrc/DiscussionPapers/jtrcpapers.html The International Transport Forum’s website is at: www.internationaltransportforum.org This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Table of contents 1. Introduction .............................................................................................................................. 4 2. Is there a simple approach to assessing railway efficiency? .................................................... 5 A basic but balanced scorecard.................................................................................................. 5 Caveats to the simple approach to railway efficiency assessment ............................................. 7 The importance of the narrative ................................................................................................. 9 3. Taking account of multiple railway efficiency dimensions .................................................. 14 4. The policy maker and exogenous efficiency determinants ..................................................... 16 5. Conclusion............................................................................................................................... 18 Bibliography ................................................................................................................................ 20 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 3 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT 1. Introduction The ITF has produced a series of reports and discussion papers addressing the interrelated issues of railway structure and performance; see for example Beck et al. (2013), Thompson (2013), ECMT (2007) and, Thompson (2007). The academic literature on this subject is also significant, with good examples in Mizutani et al (2014), Nash et al. (2013), Van de Velde et al. (2012) and Kirchner (2002, 2004, 2007 and 2011). All of these studies have confronted the question of how to measure the performance, or efficiency, of railways both in the sense of how one railway compares with others (cross-section) and how railways have changed as a result of policy interventions (time-series). The purpose of the roundtable discussions was to revisit the issue of how to define and measure efficiency at the proper level of detail and with reasonably available data so that policy makers can benchmark the performance of their railways, evaluate the impact of past changes in railway structure, ownership or regulation and assess the likely outcome of future initiatives. The challenge is inherent in the phrases “proper level of detail” and “reasonably available data”. Efficiency entails maximizing the outputs from a set of inputs (technical efficiency) or creating an optimal mix of inputs to maximize output (allocative efficiency). When we consider efficiency, we are inclined to think in terms of a single dimension, a single number or percentage. The railway business is not that simple. Railways come in all shapes and sizes: vertically integrated, vertically separated, public and private, passenger or freight dominated or mixed, supported by subsidies or fully selfreliant. The provision of railway services is multidimensional. In economic terms the railway company is a multi-product firm. It is a very capital intensive business, economies of scale and density can be relevant and some natural monopoly characteristics are present. In most contexts and on most continents, a competitive railway market is not a straightforward concept. This complexity makes it difficult to produce a description of efficiency that is equally relevant from different viewpoints or to all stakeholders. The answers to questions on railway efficiency will depend on who is asking. Primarily, apart from the owners of rail companies, the questions may come from three main stakeholders: the government; the users of the network (in a vertically separated railway) or the users of railway services; and the economic regulator, if there is one. In most countries, governments pay large subsidies for railway infrastructure and passenger transport. In these cases, the focus of the government will be in asking if those subsidies are spent efficiently or how, through higher efficiency, they could be reduced. Train operators will be interested in efficiency in terms of availability and access costs of railway infrastructure, while the users of railway services will place their stress on punctuality/reliability or speed of transport. 4 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT The regulator will pursue both technical and allocative efficiency. It will require different information to examine each aspect. Its focus will include rate setting, user charge price caps, safety and performance in terms of operating and financial indicators. Against this background the current paper explores how railway efficiency can be conceptualized and measured at different levels of inquiry and depth, developing the work begun in the paper by Beck et al (2013). 2. Is There a Simple Approach to Assessing Railway Efficiency? A basic but balanced scorecard The simplest approach would be to derive key performance indicators (KPIs) from published data. Thompson and Bente (2014) show an example of what sources and types of publicly available data exist (supplemented by data they have collected themselves). Their sample includes some railways in the EU, plus the railways of Switzerland, Norway, China, US, Canada, Japan and Indian Railways. They propose an approach that can be used to produce some information on railway efficiency in a limited time-frame with limited data. The indicators proposed refer to basic indices of size and scale and from these parameters basic ratios of efficiency and productivity can be developed (e.g. traffic density, wagon/coach productivity…). Adding a few more data points, one might create a basic but balanced score card of railway efficiency that would consist of six types of indicators (Bente & Thompson 2014): system scope; asset utilisation; human resource utilisation; operational performance; financial performance; customer-centric service quality. An example of more detailed KPIs that could be included in each of these fields is presented in Box 1. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 5 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Box 1. A balanced score card for railway efficiency System scope Asset utilization Passenger service scope • Fleet • Transport units (pass-km) • Train-kilometres • Staff Freight service scope • Fleet • Transport units (ton-km) • Train-kilometres • Staff Infrastructure scope • Lines • Track • Stations • … Infrastructure assets • Train-km/line-km per year • Train-km/track-km per year • Transport units/line-km per year • Transport units/track-km per year Passenger fleet • Transport units/unit of fleet per year • Kilometres in service/unit of fleet per year Freight fleet • Transport units/unit of fleet per year • Kilometres in service/unit of fleet per year Human resources Operational performance Unit cost, infrastructure • Asset maintenance (per line/track-km per year) • Operational (per line/track-km per year) Unit cost, train operations • Passenger service (per train-km) • Freight service (per train-km) Infrastructure, number of staff in • Operations/traffic management • Asset maintenance • Network development • Total cost of labour by division Passenger service, number of staff in • Sales and marketing • Operations • Asset maintenance • Total cost of labour by division Freight service, number of staff • Sales and marketing • Operations • Asset maintenance • Total cost of labour by division Financial performance Customer-centric indicators Revenue (core, ancillary) Modal competitiveness • Relative speed between modes • Relative performance between modes • Relative cost/price between modes • Modal split (the points above) subdivided by • Freight into business lines (combined/block/…; container/bulk/…) Passenger into: • Between agglomerations • Cross-country • HSR Cost (core, ancillary) Asset values (book value, MEV, shadow price) • Infrastructure • Passenger service • Freight service Subsidy (OPEX, CAPEX) (NEW) Investment/capital employed (Re-) Investment/capital employed Indebtedness/EBITDA-ratio Cash-flow from financing activity (depreciation – CAPEX + subsidies-net borrowing) Source: Bente & Thompson (2014). 6 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Both cross-sectional (comparing systems) and time series (change over time) indices will be required to monitor performance. Time-series analysis makes it possible to compare the performance of a railway system compared to previous years but no matter how well the railway may appear to be performing against itself, it may still be relatively inefficient when compared to other railway companies. Using cross-section (with time-series) one could try to assess whether some railway companies consistently come out on top of the distribution in terms of efficiency over a number of these fields and over time. Both types of data have to be treated with caution. Time series indicators suffer from the risk that the categorisation of inputs or outputs change from one year to the next and they can be heavily influenced by major organisational or economic changes, for instance since 2008. Outsourcing services can, for example, have a very large impact on indicators of the efficiency of using human resources. Cross-sectional data similarly risk comparing systems where outputs or inputs have been delineated differently. The limits are addressed more thoroughly in the next section. It is clear that some subjective judgements in the creation and use of these KPIs will have to be made and that an integral part of using such an approach is significant input from railway experts in interpreting the results. A key weakness of the use of KPIs is that different measures will give different results (for example cost per train-km and cost per track-km) and it is not clear which measure should be used. The outcomes of such an exercise would be subject to multiple caveats, which we examine in the next section, and would give an observer a first indication of the efficiency of a railway system without explaining why observed efficiency differences arise, a point addressed in section 2.3. Caveats to the simple approach to railway efficiency assessment The simplest benchmarking approach is to compare a railway to its own past performance. Even in this case, however, “ceteris paribus” may not hold. Past data on financial performance may need to be adjusted as a result of organizational changes in the company, changes in accounting standards or policy, inadequate application of accounting standards1 or external policy decisions,2 where the State is the owner. There may also be times during which railway investment or maintenance is inadequate, thus storing up problems for the future. Cross-sectional benchmarking is equally challenging and requires two preparatory processes. The data used from different systems must be standardised and normalised. Standardisation refers to making sure that the activities or materials compared are alike. Normalisation requires taking account of differences in economies of scale and density. In railways, company size or scale of activity has a substantial impact on the perceived efficiency (most notably asset utilisation). A rich data set is required to enable standardization for cross-sectional analysis. Comparison is increasingly compromised by data availability as a result of organizational changes, changes in ownership and integration of railways into larger logistics chains. The UIC Lasting Infrastructure Costs Benchmarking initiative3 was an excellent example of how to undertake cross-sectional analysis and illustrates how many different factors need to be taken into account before the cost of maintenance and renewals can be examined on comparable grounds. Reported costs across different infrastructure 1 Nash and Smith (2014) cite examples where no depreciation or interest is entered into the accounts for assets that have been purchased with grants. 2 E.g. prior to privatisation, the state might decide to treat railway infrastructure as a strategic asset and remove it from the balance sheets of the railway infra manager. This would, at least in part, change its borrowing capacity against its assets and influence all future business activity. 3 Information on the initiative is available here: http://www.uic.org/spip.php?article582 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 7 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT managers were harmonized under this initiative to account for complexity of the network, for example in terms of the number of switches per track km. Another challenge is the use of demand to supply ratios to compare the output of railways (passengers carried) with inputs (e.g. train kilometres). In principle in a fully competitive market there should be convergence in these ratios between railways but in practice the demand or output of railway services is dependent on a number of exogenous factors and public policies. Having low traffic levels does not necessarily imply that train operations are inefficient, and vice versa. To provide two illustrations: heavily subsidized passenger traffic provided in a thin market under public service obligations might yield low passenger numbers per train-kilometre, even though it is managed highly efficiently, simply because the market is so small; a financially self-sustained, profitable freight railway business can still be managed below bestpractice efficiency standards. Bonnafous and Crozet (2014) illustrate how demand/supply ratios can be misinterpreted and how the use of a limited number of indicators can provide a misleading picture without thorough knowledge of the root causes of efficiency. The French rail system, when compared to its peers seems to be fairly successful in passenger transport. In fact, it ranks first in Europe with 406 million trainkilometres and an average of 227 of passenger per train. Other national incumbents like DB or SBB lie far behind with countries in Europe like Germany or Switzerland with 100 and 130 passengers per train, respectively. But when one probes deeper, it is revealed that this performance is the result of structural differences – in France 2/3 of passenger traffic is carried by the TGV. The high speed trains have a higher capacity, higher occupancy (due to use of yield management in setting fares) and they cover long distances. The illustration suggests, how basic structural differences, which are ultimately the result of geography public (transport) policy decisions can determine the outcomes of simple efficiency comparisons. The observed “efficiency” of a railway system through demand/supply ratios is influenced by factors exogenous to the system (topography, historic evolution, etc.), government policy and endogenous factors, which are dependent on the efforts of the management and policy maker. Without adjustment for exogenous factors, KPI’s such as “cost per transport unit” may be useless. Finally, a problem that requires special attention is data availability. Research efforts, especially in the EU have been hampered by lack of good quality data. As Thompson and Bente (2014) note, most railways do not see the need for detailed information for internal management purposes or do not think it is in their interest to release such information to permit public comparisons to be made. Given that most railways in the EU receive substantial financial support from government sources, it is more than questionable to what extent such a practice is acceptable, although it is a well-documented phenomenon in the broader context of state-owned companies, particularly where strong unions have developed (e.g. Salinger 1984, Rose 1987, Hendricks 1977, Savedoff and Spiller 1999). These authors identify concealing data as a strategy for protection of the company’s cash flow. Better information on efficiency might lead to the shedding of excess workforce or substitution of labour with capital or other changes, which the unions would perceive as a threat to their bargaining power. 8 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT The US example illustrates that information can be systematically collected and publicly reported, including from fully private businesses.4 Moreover, a lot of the collected information is publicly available, while a small part of commercially sensitive information is reserved for regulatory purposes only. Given the caveats, a general recommendation in developing “high-level” aggregate analysis, especially when time and resources are limited, is to build on simple indicators, preferably those that the railway organisation already routinely collects. These may simply be unit costs, so long as they are not dependant on external factors5 and are controllable by management. This should reduce issues with data availability and the interpretation of data6. Even in this case, it should be recognized that exogenous factors and the level of demand7 will heavily influence the outcomes of any comparison, especially the cross-section type. The example in box 1 tries to follow this approach, as much of the information mentioned is in many cases already publicly available. Developing indicators that require large amounts of data that are not readily available can undermine transparency and may be counterproductive for monitoring performance over time, making updating of indicators difficult. The importance of the narrative To illustrate the importance of the context in which a railway operates, and especially structural reforms and changes in policy that have affected the railway over recent decades, this section looks more closely at the cost efficiency of the railway systems in two countries, the Netherlands and Great Britain. The evolution of infrastructure costs in the Netherlands The transformation of the railway sector in the Netherlands started in the 1990s. Full vertical separation occurred in 1995 and by 1998 all network maintenance was outsourced. In 2005, infrastructure maintenance, capacity management, and traffic control activities merged to form ProRail BV (Public Limited Liability Company), which became the infrastructure manager in a clear, vertically separated framework. At first sight, from Figure 1, it would appear that vertical separation led to a decrease in the efficiency of the railway system. Maintenance costs soared from 1997 and the overall cost of running trains in the country steadily increased from the time of separation in 1993 (Swier, 2014). However, the main reasons behind the surge in costs are not directly linked to technical inefficiency and outsourcing; changing circumstances like a fast increase of rail contact fatigue, more severe safety rules, night work and increase in track utilization explain a lot of the cost increase. 4 The railway companies face charges in case of reporting inaccurate data. The incentive for accurate reporting is enforced through the fact that such data are often used in court proceedings with shippers or the regulator and are subject to intense scrutiny. 5 For example if the infrastructure manager is state owned and subsidized, it is the decision of the state how much money it wants to make available for maintenance or renewals so these costs are outside of the direct control of the infrastructure manager. 6 However, care should be taken when comparing different railway companies or countries because unit costs depend on economy of scale and density, which need to be controlled for. 7 Deciding whether demand should be treated or not as an exogenous factor is a complex question. On the one hand it is easier to compare situations for a given level of demand. On the other hand, growing demand is often an objective of railways companies, as exemplified by net cost contracts in franchising. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 9 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT In the first ten years after outsourcing, ProRail had input contracts for maintenance. Even though failure rates became lower than before outsourcing, costs did not go down. The performance/price ratio really started to improve after the introduction of powerful performance contracts defining precisely the expected outputs without setting out explicitly the technical ways of achieving these outputs8. Separation was accompanied by the establishment of more transparent financial reporting rules, with assets depreciated according to replacement value rather than historical cost accounting value and the inclusion of traffic control and capacity management in the Prorail balance sheet. Arguably, there is a noticeable increase in stewardship costs (measurement and accountance, taxes…) which is partly attributable to separation but these remain small compared to the increase in asset depreciation costs (see Figure 2). Figure 1. Evolution of maintenance cost of the Dutch railway network Fixed Maintenance slot schedule Rail Contact Fatigue Source: Adapted from Swier 2012. 8 10 One of the biggest challenges when writing these performance contracts, especially those related to daily maintenance, is to set out outputs that guarantee track quality in the long-term and prevent subcontractors from opportunistically reducing costs by only carrying out short-term maintenance. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Figure 2. Evolution of cost items in infrastructure spending in the Netherlands Source: Swier 2014. International comparison of ProRail gives a mixed picture. In terms of quality, ProRail fares significantly better than other European railway infrastructure managers, with less rail breaks and track or signal failures, but their maintenance unit costs are higher than the European average (Hansen et al, 2013), stressing the need for quality-adjusted metrics. The cost of the railway system in Great Britain The British Railway system underwent structural changes, including full vertical separation, around the same time as the Netherlands. The Railways Act of 1993 established the privatised framework for British railways, breaking up the historical BR into more than 100 separate companies, the relationships between which were to be set in contracts or through regulatory mechanisms. Infrastructure was taken over by Railtrack, a newly established public sector company whose shares were subsequently sold to the private sector and train operations were divided among 25 geographical franchises. Maintenance and renewal were subcontracted by Railtrack to several private engineering companies formed by privatising the relevant parts of British Rail. Railtrack went into administration as a result of the derailment of a train near Hatfield in 2001. The accident was caused by a faulty rail. Even though casualties were limited (four deaths), the derailment had major repercussions as it laid bare the company’s ignorance of the condition of its assets. In response, Railtrack imposed speed restrictions over large sections of the network. This resulted in the government compensating the train operating concessions, providing grants for track renewals and allowing the infrastructure company to raise additional debt. Railtrack was replaced by and their activities were taken over by Network Rail, a not for profit company eventually defined as State-owned for national accounting purposes. The dynamics of infrastructure spending in Great Britain can be divided in three periods. After privatisation and before the Hatfield incident, maintenance costs went down and total expenditure was stable. This was not only the result of efficiency gains but to a large extent due to less maintenance being carried out by subcontractors, which worked on fixed-cost contracts. Moreover, even if investment rose during this period, experts agree that it was inadequate considering the increase in passenger demand (Nash, 2002). This eventually led to a spiralling of costs after 2001, when Network Rail started making up for the inadequate levels of maintenance and investment under Railtrack and Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 11 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT adapted to tougher safety requirements9. Substantial inefficiency crept into Network Rail’s cost base as the regulatory mechanism was temporarily suspended (see Office of the Rail Regulator, 2003). Since 2004-2005, Network Rail, under regulatory pressure, has improved efficiency substantially. Unit costs are still higher than 1996-1997 levels and international comparison suggests that costs are still about 30% higher than what could be expected of them (McNulty, 2011). Two main reasons are given in the McNulty report. The first one concerns a lack of outsourcing, Network Rail having taken over all private contractors in 2003 for safety reasons, and because this would result in better coordination and economies of scale, or at least this is what Network Rail argued. The second reason is the misalignment of incentives as a result of vertical separation. The huge increase in indebtedness to support the investment in the years 2001-2004 also contributed to sustained higher costs up to 2011. The overall picture for train operating companies (TOCs) is similar. Right after privatisation, costs went down as the result of TOCs making low bids to win franchises; franchising was in this regard a success, with passenger numbers increasing and government subsidies decreasing significantly. From 2001 on, TOC costs increased sharply, with 35% unit cost growth from 2000 to 2006 (Smith and Wheat, 2012). This was the result of several factors, including fuel prices, emphasis of franchises on quality (cleanliness, information…) and an above-inflation increase in staff costs. Interestingly, despite this increase, train operating costs in Great Britain are comparable and even slightly lower than in other European countries (Civity, 2012). Once again, this figure needs to be interpreted looking at the railway context. For instance, the incumbent Dutch operator also has among the lowest unit costs but with much higher traffic density (passenger-kilometre per track–kilometre), thus enabling large economies of density. 9 12 The strengthening of safety requirements resulted from the public outrage over the Hatfield incident. However, it is now argued (Evans, 2007) that safety statistics had not deteriorated. The very high level of maintenance required after the incident rather stemmed from the fact that Railtrack did not know the state of their network and could thus not direct maintenance to the portions of the network that required it most. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Figure 3. Expenditure of the British railway system by cost item between 1996 and 201010 Two main conclusions can be drawn from these two case studies. First, unit costs need to be interpreted cautiously. In the case of infrastructure maintenance, unit costs are as much a reflexion of the network complexity, purpose and usage as one of the efficiency of its maintenance manager. An in-depth knowledge of the “engineering narrative” is required to carry out international comparisons. The same applies to TOCs and overall industry costs, where costs per passenger-kilometre are likely to be more favourable on systems with high track usage rates. Second, when looking at the evolution of costs for a single country, the broader railway context needs to guide interpretation. For instance, even though the evolution of infrastructure costs is similar between ProRail and Network Rail, the underlying mechanisms are very different. The role of outsourcing is opposite in the two countries. In the Dutch case, outsourcing proved beneficial to efficiency after a running-in period of ten years or so while in the UK it provided short term benefits but ended up being very costly. The difference lies in the way contracting is managed. When applying econometric methods, these elements need to be precisely taken into account.11 Moreover, not only the costs but also the quality of the railway system needs to be investigated: differences in performance (capacity, functionality, reliability etc.) often explain cost differences and if delaying maintenance is cheap in the short-run, it is often not economically justified. 10 ROSCO - Rolling Stock Leasing Cgenerally s of change in organizother conclusions then as well.about the effects of change in organization or it is a problem ofompany. 11 As noted by Nash and Smith (2014), vertical separation or outsourcing “is not a dummy variable in the regression” i.e. a variable, which can take the value 1 or 0 (e.g. is there vertical separation or not). Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 13 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT 3. Taking Account of Multiple Railway Efficiency Dimensions If the narrative determines the measured efficiency as well as its interpretation to the extent illustrated in the previous section, then a simple approach to railway efficiency will not yield a very accurate image. One could try to expand the set of the observed KPI’s, but given the dimensions which affect railway efficiency, this would not be easily manageable and there is also the question of which KPI to give greater weight. Two possible approaches that could translate this complexity into an understandable form, as outlined by Smith and Nash (2014), use econometric functions and Data Envelopment Analysis (DEA) to describe efficiency. The advantage of the DEA method over econometric approaches in terms of handling multiple inputs and outputs and not requiring the specification of a particular functional form have disappeared over time as econometric approaches have evolved. The major weakness of DEA is its inability to take account of random noise, which may lead to inefficiency being over stated. Econometric analysis is therefore preferred. The econometric efficiency measurement techniques rely on three approaches: cost functions, production functions, distance functions. The cost function assumes cost minimisation in the production process and relates cost to the level of outputs and input prices. It can deal with multiple outputs, economies of scale and density. It also allows for cost changes over time, for instance resulting from technological progress. It is the only one of the three approaches that accounts for both technical and allocative efficiency and railway efficiency measurement most commonly relies on cost functions. The relative cost efficiency of transport operators is also the key focus of policy makers and regulators, hence also their reliance on this approach. Another reason for preferring the cost function approach is that data on costs is more reliable12. Even though some of the caveats evoked for the simple scorecard approach to railway efficiency remain, one key advantage of the econometric method with costs is that it produces a single efficiency indicator, controlling for all the factors that can be modelled in the regression (such as cost changes over time, or some of the differences between countries). The cost function approach can be upgraded to allow the relative efficiency of companies to vary through time and allows an estimation of this variation. The expansions are termed SFA (Stochastic 13 Frontier Analysis). As the name suggests, the method is used to construct an efficiency frontier and to estimate the distance of individual observations (railway companies in our case) from the frontier 12 Getting comparable data may be difficult, for instance for financial costs, but at least cost data does not suffer from the problem of outsourcing. 13 “Stochastic” refers to something than can be estimated statistically, but cannot be precisely predicted. 14 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT (graphically similar to Data Envelopment Analysis). The approach can be and is used with panel data (cross-section and time-series data). SFA seeks to go beyond the cost function approach by recognising that firms' costs will deviate from the frontier due to random noise (e.g. random events such as bad weather) as well as inefficiency. It thus seeks to decompose the error in the model into these two components and obtain a better measure of inefficiency. Econometric approaches require the choice of an appropriate functional form but this is hardly a problem, as the commonly translog function is very flexible and Box-Cox transforms allow the data to dictate the functional form. The choice of distribution for the inefficiency term in stochastic frontier analysis is arbitrary. In the case of SFA, there is a risk that unobserved heterogeneity will be captured as inefficiency. More recent literature, cited in and including Smith and Nash (2014), have proposed a number of useful methods to address this issue when panel data is available. In practice, cost functions and increasingly SFA are being used by the regulators. Regulators usually apply a number of different methods and approaches to derive an average indicator of efficiency across the methods in order to account for the inherent uncertainty in the data and the modelling process. Econometric models require a lot of data. Panel data is particularly useful in this regard for separating unobserved heterogeneity from inefficiency. In addition to panel data, regional data within countries is useful to examine within system performance, enacting better like-for-like comparisons across countries, better capturing scale and density effects and increasing sample sizes. The econometric modelling approach produces a top-down view of efficiency that will benefit greatly from a bottom-up engineering narrative to provide insight into what determines performance. For instance, in its last review of pricing, the Office of Rail Regulation (ORR) in the UK commissioned a number of engineering studies to help understand differences in indicators of efficiency between Britain and other countries. In an international comparison exercise with a US railway company, Jan Swier (2014) shows an example of the importance of such an engineering analysis. He found that unit rail infrastructure costs (measured on a life–cycle basis) were as much as five times higher in the Netherlands. This is where another aspect of the narrative comes in: what is the purpose and level of use of the tracks? A precise statistical analysis of these questions for the US railway company and ProRail allowed Swier to establish the cascade diagram in Figure 3. Usage and complexity explain 60% of the cost difference. The biggest cost elements in the Netherlands are the large number of switches and signals, as well as the extension of electrified tracks. In contrast, the US company benefits from having dedicated freight traffic and high traffic density in ton-kilometres with few trains. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 15 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Figure 4. Life-cycle cost comparison between the Netherlands and a US railway company LCC comparison rail infrastructure; the Netherlands - US (INDICATIVE) 120 4 100 0 9 LCC index 11 80 7 60 100 40 20 4 4 5 5 17 19 9 10 11 12 19 M or L e CC (c o r M mp a il or le i nf e r d i x) m a st es a ru el rs ct re ha u re fu l lin N el i n g ya L s 60 N tall rd s % o ati Le Le ss l e ca on ss co 80 ss te n s (c m % sw ary o m p l l e i tc M p l ex ss h e or ex tra si s e d a Lo ) le f fic gn a M yw w ve c ls or o er l c o n e e f rk ( ma ro s trol fe 9 0 te si n c r Pu Hig tive % v i al gs rc he w ers cos r Sc ha s u ti o rk us ts C ale in g l iza in g 65 % a O rg o ad Po tio n h rs ) pe (7 v w ra rel an t e r in t /5) tio ate ag (e he na d e s stim U l e sp (e a S LC xce ecs sti m te d C l le (e a ) ra n c sti ted il i e m ) nf (e a t ra st ed st im ) ru a ct te d ur ) e U S 0 Differences Source: Swier 2014. 4. The policy maker and exogenous efficiency determinants The importance of exogenous determinants of costs, as argued in a previous section, can be a far greater determinant of differences in performance than railway company management. Of the three main stakeholders mentioned in the introduction, it is the policy maker that has the greatest power in determining the exogenous circumstances. The policy maker may not be able to change the topography or the density of settlement of a country, but he can define national transport policy, he can choose the structure of the railway company, its ownership and the form of economic regulation. Although it may be debated whether the structure, the ownership and the regulation of the railway company are exogenous reasons, this paper assumes them as such, as they cannot be decided by the management of the railway company. The private railways of the US, Mexico and Canada are vertically integrated companies, competing with other private companies across large areas of land and large markets. These railways are considered to be performing efficiently, but their exogenous (and historical) circumstances are different to other countries, those of Europe for example, where most traffic is carried by railway companies that are still state owned. 16 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT In the EU the dominant mode of private participation, where it has developed, is through exclusive concessions for passenger transport (with competition for the market). Open access freight and more recently passenger train operations (competition in the market / on the track) have gradually expanded since the mid-1990s. The effects of competition on efficiency are not obvious. The EU 15 railways do not show a particularly positive picture in terms of traffic growth or modal share. As Thompson and Bente (2014) note, it is possible that the restructuring reforms in the EU have not been implemented to a sufficient extent to have a significant impact. It is also possible to argue that the outcomes would have been worse without the reforms. The organizational structure in the case of most railways in the EU has not substantially changed. In some cases, a restructuring was made from a single company to a holding structure. There are now 14 cases of full vertical and horizontal separation in Europe. Recently, France reintegrated the infrastructure manager, Reseau Ferré de France (RFF), with the national train operator, SNCF, recreating a fully vertically integrated company. Bonnafous and Crozet (2014) note this will lead to further growth of unit costs due to the balance of power being shifted in favour of the company and the unions rather than the public regulatory authority. It will also likely undo any advances in infrastructure management efficiency achieved through devolving infrastructure management role to RFF. The UK is the most analysed case of railway privatization and vertical separation in the EU. The process of the privatization of the original British Rail was marked by a steep learning curve on the part of the infrastructure manager and the regulator. Smith and Nash (2014), in their literature review, show that not everything in the UK went “according to plan”. The events that led to the bankruptcy of the infrastructure manager and its administration resulted in a major loss of efficiency, which has not yet been recovered. In freight transport, privatisation effects were overridden by exogenous developments in the economy and especially the structure and geography/distance of coal transport. 14 In passenger transport, the growth of traffic can be attributed to exogenous developments in the economy for the major part. Cost of operations in passengers franchising were found to have increased, partly due to inadequate franchise size and overlap. These developments contrast with other countries like Germany and Sweden, where savings of 10% to 30% are reported as a result of the introduction of competitive tendering for exclusive concessions for passenger services. Vertical separation adds transaction costs for coordination and internal accounting and negotiation, although the exact magnitude of these costs is debated. The McNulty study (2011) noted in the UK that the major issue of vertical separation is not the transaction costs but rather a misalignment of incentives between the operators and infrastructure manager, which could increase the cost of the later from 2 to 20%. The most comprehensive study to date, EVES-rail (van de Velde 2012), concluded that the vertically separated model works less well on intensely used networks and provides the perhaps counter-intuitive result that vertical separation reduces cost on less intensely used networks. But the study also provides no evidence that the introduction of competition had any effect on cost. The analysis is subject to data issues and did not take account of some factors that may be important (e.g. differences in declared and effective economic regulation). Finally, the EVES study gathers data from a large sample of Asian and European railways so its results only reflect average behaviour. It is not able to explain some of the finer changes that have been observed, for instance in the Netherlands where vertical separation first introduced a disturbance in the system with worsening of performance before a net improvement was evident (Figure 5). 14 With the closing of the coal mines in the UK, coal power plants had to shift to imported coal. That meant the more than half of the increase in ton kilometres in rail freight could be explained by the increase length of coal hauls from the ports to the coal power plants. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 17 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT 2013 ProRail BV 2005 1998 Outsourcing Maintenance 1995 Separatio Figure 5. Example of technical failure performance of ProRail before and after vertical separation Source: Swier 2014. Given this discussion, a further element that could be added to exogenous performance drivers might be cultural differences between countries. This leads to further pertinent questions on the studies of organizational structure and ownership we have available today. To what extent do the changes in organisation and ownership drive the outcomes and to what extent are there deeper root causes influencing the results - do we have a confounding problem? If this is the case, what can we really say about railway efficiency, structure and ownership of railways in the EU or elsewhere? Clearly, the meaning of such questions is not that no progress has been made in our understanding of the problem. We need to emphasize, though, that an immense obstacle to informing the policy with solid analysis is the lack of quality data. 5. Conclusion Railway efficiency is a challenging concept due to the complexity of the production process in the railway business. A basic view of railway efficiency can be formed with a small set of cross-section and time series data producing a limited set of KPI’s. Railways generally produce this data as it is needed for internal management but it is not publicly available in every jurisdiction. A basic responsibility of government is to require companies report key data, with publication by companies or the government. Railway experts will inevitably be required for interpretation of the results but large deviations from efficient operation should be evident. For some purposes the set of basic indicators identified in Box 1 will be sufficient. 18 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT For monitoring efficient use of public subsidies and regulating monopolistic infrastructure managers and exclusive train operating concessions, more sophisticated econometric techniques (such as SFA) will need to be employed. These have much greater data requirements. When benchmarking the performance of several countries, the use of KPI requires detailed harmonization of data, as exemplified by the UIC study. There is a link between this harmonisation process and statistical methods: both are trying to take account of factors before getting to an efficiency judgement. Further, econometric approaches could even inform the harmonisation process as it informs on the factors that need to be controlled for. With both basic indicators and econometric analysis, a narrative of exogenous and endogenous drivers of railway performance will need to be developed to understand differences in railway performance properly. The narrative includes the function of the railway network (whether it carries mixed traffic or is dominated by one traffic type), the topographical characteristics of the country, historical policy decisions and other elements, outside of the scope of influence of the company’s management. Interpretation of any kind of efficiency analysis results without taking account of the narrative will lead to misinformed decisions. Along with econometric analysis, bottom-up engineering analysis can be employed to develop the narrative in sufficient detail to shape the parameters used in the econometric functions modelled. Bottom-up analysis can also shed light on whether the parameter estimates in a model make sense. As the brief review of the current state of the art in railway efficiency assessment revealed, both the simple and econometric approaches are very data intensive and suffer from problems with data availability and quality. While there are positive exceptions in data collection and availability, inadequate data explains why questions with regard to railway efficiency are often still very much open. The problem is not that it is unclear what data should be collected. In the US, both accounting regulations for private companies and data reporting requirements imposed by the regulator, the Surface Transportation Board, make most of the data required for construction of a “balanced scorecard” publicly available. US and Canadian data reporting requirements provide a model for the data needed for regulation of vertically integrated freight-dominated railways elsewhere. In Europe, good quality studies have been undertaken, in terms of data quality and common terms of reference, in a number of EU research projects on railway infrastructure expenditure (e.g. the RailCalc and GRACE projects). However, data is not collected routinely or when it is, it is only made public after it is has been processed to mask the identity of individual railways, as the UIC is constrained to do by its member companies. Such practice is at odds with substantial shares of public money involved in the operating of national railways. Much work by regulators and maybe supranational organization will be required to ensure collection of consistent data across countries. The European data situation could be greatly improved if a regulatory body, such as the European Railway Agency, were mandated to specify data to be reported by every railway, requiring annual production and publication. A small number of national regulators already do this and the annual network statements of some railways provide a wealth of information. Further, when maintenance and construction is outsourced, the necessary information is readily available in the contracts and invoices. While respecting subsidiarity and confidentiality, the specification of a common set of information to be produced for all the railways would be extremely useful for comparing performance. Finally, most studies of railway efficiency are focused on technical cost efficiency. Performance on costs reflects the inside view but what is ultimately important is service to the customer. More Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 19 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT effort needs to be invested in providing data and KPIs on the service quality related to how users choose between transport modes. Bibliography Beck, A.; Bente, H.; Schilling, M. (2013), “Railway Efficiency.” ITF at the OECD. Bente, H.; Thompson, L. (2014). A balanced Scorecard Model for System Efficiency. OECD/ITF – Railway Efficiency Roundtable. Paris. Bonnafous, A.; Crozet, Y. (2014). Efficiency indicators of Railways in France. Discussion Paper No 201424. International Transport Forum at the OECD. Paris. Civity Management Consultants (2012) European Benchmarking of the costs, performance and revenues of GB TOCs, report prepared for the Office of Rail Regulation. ECMT. (2007), “Competitive Tendering of Rail Services,” ECMT. Paris. Evans, A. W. (2007) Rail Safety and Rail Privatisation in Britain. Accident Analysis and Prevention Journal, vol. 39(3), pp 510–23. Hansen, I. A.; Wiggenraad, P. B. L.; Wolff, J. W. (2013) Benchmark analysis of railway networks and undertakings. Presented at the 5th International Conference on Railway Operations Modelling and Analysis, Copenhagen, Denmark, 13 -15 May. Hendricks, W. (1977). Regulation and Labor Earnings, Bell Journal of Economics, 8:183–96. Kirchner, C. (2011), “Rail Liberalization Index 2011,” IBM Global Business Services. Earlier issues are 2002, 2004 and 2007. McNulty, Sir R (2011) Realising the potential of GB Rail: final independent report of the Rail Value for Money study. Department for Transport and Office of Rail Regulation, London. Mizutani, F., Smith, A.S.J., Nash, C.A. and Uranishi, S. (2014) Comparing the costs of vertical separation, integration, and intermediate organisational structures in European and East Asian railways. Journal of Transport Economics and Policy (Fast Track Articles) Nash, C. (2002) Regulatory reform in rail transport – the UK experience. Swedish economic policy review, vol. 9, pp 257-286. Nash, C., Nilsson, J.E., & Link, H. (2013). Comparing three models for introduction of competition into railways. Journal of Transport Economics and Policy (JTEP), 47(2), 191-206. Office of the Rail Regulator (2003) Access Charges Review 2003: Final Conclusions, London. 20 Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 EFFICIENCY IN RAILWAY OPERATIONS AND INFRASTRUCTURE MANAGEMENT Rose, N. L. (1987). Labor Rent Sharing and Regulation: Evidence from the Trucking Industry. Journal of Political Economy, 95: 1146–1178. Salinger, M. E. 1984. Tobin's q, Unionization, and the Concentration-Profit Relationship. Rand Journal of Economics, 15:159–170. Savedoff, W.; Spiller, P. (1999). Spilled Water: Institutional Commitment in the Provision of Water Services in Latin America, Washington DC: Interamerican Development Bank. Smith, A. S. J., Nash, C. (2014). Rail Efficiency: Cost Research and Its Implications for Policy. Discussion Paper No 2014-22. International Transport Forum at the OECD. Paris. Smith, A. S. J., Wheat, P. (2012) Evaluating alternative policy responses to franchise responses to franchise failure: Evidence from the passenger rail sector in Britain, Journal of Transport Economics and Policy, vol 46, pp 25-49. Swier, J. (2014). Case study ProRail: Understanding the drivers of Railway (in) efficiency. MS PowerPoint Presentation at the ITF at the OECD roundtable: Roundtable: Efficiency in Railway Operations and Infrastructure Management Paris, 18-19 November. Swier, J. (2012). How ProRail successfully outsourced maintenance – Lessons learned. Presentation made at the Czech infrastructure conference, Prague, 27-29 March. Thompson, L. S. (2007). “Railway Accounts for Effective Regulation.” ECMT Thompson, L. S. (2013). “Recent Developments in Rail Transportation Services.” OECD. Thompson, L.; Bente. H. (2014). What Is Rail Efficiency and How Can It Be Changed? Discussion Paper No 2014-23. International Transport Forum at the OECD. Paris. Van de Velde, D.; C. Nash, Smith, A.; Mizutani, F.; Uranishi, S.; Lijesen, M.; Zschoche, F. (2012). EVES-Rail - Economic effects of Vertical Separation in the railway sector Amsterdam/Brussels. Publisher: inno-V (Amsterdam) in cooperation with University of Leeds – ITS, Kobe University, VU Amsterdam University and Civity management consultants. Dejan Makovsek et al. — Discussion Paper 2015-12 — © OECD/ITF 2015 21 International Transport Forum 2 rue André Pascal 75775 Paris Cedex 16 [email protected] www.internationaltransportforum.org
© Copyright 2024