eResearch Challenges in New Zealand Discussion Document eResearch 2020 Conversations for change. March 2015 eResearch 2020 TO VIEW ONLINE GO TO: eRESEARCH2020.ORG.NZ Overview March 2015—eResearch2020 High-quality science and innovation can have a transformational effect on a nation. Research and development investment offers the power to increase economic productivity and competitiveness and improve health, social and environmental outcomes in unprecedented ways. Indeed, New Zealand’s economic and social wellbeing depends on the productivity and competitiveness of our economy and the knowledge we have C to help make informed decisions as a society. Science provides that knowledge and informs those decisions. DR A FT NATIONA L STATEM ENT OF SCIENCE IN V EST M ENT: 2014–2024: M BIE, M AY 2014 Contents Executive Summary 1 Introduction: The future of research is digital 3 Research methods and expectations are changing 3 We have been falling behind 4 Some fundamental messages include 6 About eResearch 2020 7 Four key observations of eResearch 2020 10 #1 Skills Lag: 11 The chasm in skills is widening 13 The challenge of change 15 #2 Research Communities: 16 #3 Align Incentives: 23 #4 Future Infrastructure: 27 Where should we be aiming? 31 A few expectations for research in 2020 34 Rethinking incentives for quality 36 Summary 39 Recommendations 39 Introduction Executive Summary March 2015—eResearch2020 Executive Summary eResearch 2020 is a 12 month qualitative study launched by NeSI, REANNZ and NZGL in 2014. We spoke with researchers across a wide range of disciplines, as well as leaders of ICT and management at our research institutions. In each case we asked them to discuss the potential for change in their work over the coming decade, and to highlight the opportunities and challenges they see before us as a national research system. The full interviews are published at www.eresearch2020.org.nz. What we found: Our contributors made four major observations which we think need greater focus in the context of national strategy: 11 Skills Lag: as a research system, we are under-investing in the broad-based research skills and methodological training that underpins the international shift in research methods to digital evidence and data intensive discovery. Consequently we are putting at risk the quality of our research outputs and our opportunity to grow high tech capabilities in the country, today and into the future. Research Communities: greater explicit investment support for research-discipline led collaboration is likely to lead to greater cohesion, knowledge exchange, and collaborative research outcomes across the system. Align Incentives: governance and planning of contestable funds is poorly aligned with the governance and planning of institutional funds, leading to sub-optimal application of the balance of government investment in research overall. Future Infrastructure: provision should be made early for investment in data, visualisation, and digital research expertise. Executive Summary March 2015—eResearch2020 Where to from here: As a part of our conversation with the sector, we also sought recommendations for change. These recommendations fall into three broad areas: 1 Improve our professional research skills and standards: ·· Provide training and adjust incentives to address the growing skills gap in our research system. ·· Adopt comparative standards within and across disciplines for standardised data, common meta-data standards and ontologies for sharing information or making decisions. 2 Adjust our approach to strategy and planning: ·· Promote sharing of risk and investment into the infrastructure and capability layer in our research system. ·· Better align governance, planning and funding cycles for institutional and contestable funding in research. ·· Define our infrastructure requirements to address specific research community needs. 3 Focus our investments on future opportunities: ·· Emphasise strategies that leverage off-shore services and infrastructure to preserve the on-shore human capability and skills we require to be self-sufficient. 2 ·· Research data infrastructure has been the missing piece in our research system over the last decade. “Visualisation” will be the key enabling technology in innovation and discovery in the next decade. Introduction March 2015—eResearch2020 Introduction: the future of research is digital . . . The distinction between “Research” and “eResearch” is disappearing. The standards, skills and expectations that are currently the domain of a few, select, data, ICT, or eResearch groups are diffusing into the wider research sector at a greater pace than ever. The speed of technological change means rapidly adapting to new expectations must be a pervasive and habitual component of a high performing research system. Over the last few years, international research norms have begun a shift to digitally driven research methods, and to new standards in terms of evidence and publishing. The future impact of New Zealand research and researchers will depend on the ability of our research sector to adopt these new tools and to excel against new expectations in a data informed and digitally enabled research world. Research methods and expectations are changing The fourth paradigm of research — “data intensive discovery” — is expanding the tools and resources available to help researchers understand the world at an accelerating pace. We can now collect, share, and analyse data on a scale and speed unprecedented in research history — and this holds true across all fields of research and academia. In health, the natural environment, ecology, agriculture, urban planning, or in responding to security needs or natural hazards, we 3 see major growth in the use of telemetric sensors, genomic sequencing, radio telescopes, social media, geospatial and sources of real time information. To be meaningful, all of this new information must be linked with meta-data, making it searchable, storable, and reusable. In many cases, our power to collect, compute, and analyse this flood of information outstrips our ability to visualise, interpret and communicate the results. Introduction March 2015—eResearch2020 New technology is producing data that allows researchers to move beyond theoretical models of the world around us, and towards understanding and optimising systems —pastures, forests, factories, or energy grids—in real time. All the while, international standards for the quantity and quality of research evidence — data —are becoming more difficult for New Zealand researchers to achieve. M A R K GA H EGA N eR2020 We have been falling behind Our abilities as individual researchers, and as a national research system to collect, validate, analyse, visualise, store, and curate research data are not keeping up with international expectations. With regard to training and skills, research resources, and infrastructures needed, we are considerably behind. New Zealand research institutions have historically been late investors in new technologies as this minimises risk “If we take a backseat in science then we are really taking a backseat in economic development and global competitiveness.” 4 and reduces capital requirements; however the accelerating rate of change still presents the challenge of increasing the frequency and scope of our investments. In investment, as well as in developing research skills and shared research resources, our institutions appear to have struggled to coordinate their strategies at a regional or national level. The culture in our institutions overly emphasises the historical “information services” focus on their corporate needs at the expense of prioritising needs that support their research mission. Our major scientific endeavours appear to Introduction lack the effective, cooperative tools and support to be able to live up to modern expectations for research methods. Many of our New Zealand researchers are not sufficiently exposed to new information technology in research, nor are they familiar or confident with 21st century methodologies or requirements for reproducibility in science. Incentives for researchers and institutions are focused firmly on producing “more” research, rather than on “better” research — or on improving quality in research methods. Instead, we observe a planning and skills gap in the framework of our research sector that leaves our universities, our Crown Research Institutes, and our research community fragmented and often isolated in the research data space. Evidence of this gap includes the often weak links between the governance of research goals and long-term planning in our research institutions. Additionally, we have launched new initiatives, such as the National Science Challenges, without a strong understanding of the digital infrastructure, skills and governance implications of these initiatives. We are only beginning to build collaboration in data and research infrastructure into the foundations of our planning (and not yet into our funding). Only 5 of our universities make the list of the world’s top 750 research institutions in terms of research impact — the highest placed, Otago, sits at 249 on the current rankings1 . One scenario suggests that a significant number of New Zealand researchers may soon struggle to publish 5 their work in respected international journals. To ad“ Digital research infrastructure allows dress this, we need a coordiresearchers in NZ to compete internated approach to upskilling nationally at scale, to work on larger projects, and to make impactful our researchers, resourcing digital research from infrainsights without the complexities of set up, operation or overheadsof structure and tools, through to lifecycle management and major equipment.” 1 http://www.leidenranking.com/ranking/2014 Global rankings for Impact of papers (% in the top 10% of peer reviewed papers).3 unis didn’t rank in top 750 (#249 Otago 10.9%; #272 Auckland 10.5%; #420 Massey 8.7%; #514 VUW 7.5%; #522 Canterbury 7.4%) TON Y LOUGH NZGL March 2015—eResearch2020 Introduction March 2015—eResearch2020 expertise. Most importantly, we need to alter the culture in our research institutions towards quality engagement with digital research methods and 21st century standards for impactful research. Some fundamental messages include . . . We can expect a splitting of corporate and research data needs and consequently diverging investment strategies for each. As service providers and “all of government agreements” mature, more and more of corporate data needs and IT services can be provided effectively and at scale by third party providers through cloud services and consumer technologies. Less institutional level investment will be required. On the other hand, research IT, while technically advanced, is likely always to be behind the corporate curve in terms of mature commodity services — and therefore effective scale — as the performance requirements and demand curves are different. Therefore research IT investment strategies are likely to focus on institutions cooperating for procurement and delivery of research IT needs. 6 An increasing number of tools, both commercial and open source, are coming online focused specifically on digital research and the data lifecycle, but these are of quite a different nature to corporate and consumer technologies as they require considerable training and a change in researchers’ workflow. Furthermore, and right from the first stage of data generation, research equipment and instruments (such as gene sequencers) are increasingly complex and capital intensive acquisitions that often are not shared nor well-connected to enable data processing and in some cases remote operation. Large scientific instruments are becoming more complex and capital intensive, meaning that countries can afford fewer than in the past. The expertise required to de- Introduction March 2015—eResearch2020 sign experiments and analyse results with these instruments is increasingly hard to find, therefore the value of high speed, high quality connections to overseas facilities is increasing rapidly. STEV E COTTER R E A N NZ No single New Zealand research institution is likely to effectively meet the specialised and various needs of New Zealand research data at scale. Joint efforts, perhaps facilitated with leadership from government or other neutral infrastructure or policy agents, is likely to be required to ensure coordinated development across the research system. About eResearch 2020 The eResearch 2020 outreach programme is a future oriented, national conversation with thought leaders within the research sector that aims to assemble a comprehensive vision of researcher needs and essential skills over the coming decade. eResearch 2020 is led by NeSI, REANNZ and NZGL as co-patrons together taking a combined approach to facilitating national dis“ We’re only doing things in half meacussions. eResearch 2020 sures, such as funding National Scibrings researchers across ence Challenges without thinking disciplines together to focus about what support structures are on particular themes, be it on necessary. NZ needs a coordinated research sector cloud strateeResearch ecosystem—it needs a gies, skills gaps, institutional comprehensive science data policy, governance of research capaappropriate governance, incentives 7 bilities, or the infrastructure for institutions to invest in data inneeds of the National Science frastructure, a willingness to develop Challenges and the Centres of data-intensive research skills over a Research Excellence. long period of time, and a relentless focus on quality.” 8 Introduction 9 March 2015—eResearch2020 Four key observations March 2015—eResearch2020 Four key observations of eResearch 2020 1 2 10 3 4 As a science system, we are under-investing in the broad-based research skills and methodological training that underpins the international shift in research methods to digital evidence and data intensive discovery. In addition, our major capital and programme investments are disconnected, either geographically, or organisationally. Consequently, we are putting at risk the quality of our research outputs — our stock of knowledge and our opportunity to grow high tech capabilities in the country, today and into the future. RESEARCH COMMUNITIES: Impactful research collaboration usually occurs at the inter-personal, research-discipline level, not at project or institutional level. Greater explicit investment support for research-discipline led collaboration (rather than institutional collaboration), such as CoREs and NSCs, and support for discipline-based national research societies is likely to led to greater cohesion, knowledge exchange, and collaborative research outcomes across the system. INCENTIVES DRIFT: Institutional funds, policies and practices for ICT are applied to research methods with potentially ruinous consequences, distorting research design, subverting resources, blocking collaboration, and leading to sub-optimal application of the balance of government investment in science overall. FUTURE INFRASTRUCTURE: Major innovation or discovery in the coming decade is likely to stem from teams of researchers working across disciplines, institutions, and national borders, based on data from a proliferation of sources — including collaboration with industry — and with significant reliance on compute, integrated systems, sensor networks, and complex data analytics. Rather than lag behind comparator countries in science, provision should be made early for investment in capacity and capability in data, visualisation, SKILLS LAG: #1: Skills Lag March 2015—eResearch2020 and digital research expertise. New Zealand’s science system might then continue to contribute to economic productivity and competitiveness, and improve population health, societal and environmental outcomes at first world levels. SKILLS LAG: Ensure researchers keep up with the pace of change 11 A critical problem within our science system and New Zealand as a whole, is a lag in the adoption of digitally driven methods and eResearch skills, both in our research sector and in industry. As more and more research activity moves into the digital domain across all disciplines, we observe that many New Zealand researchers are struggling to keep up with changing standards within their international research community and with the associated quality expectations for evidence, data, and research methodology. Our abilities both as individual researchers and as a national research system to operate in a digital research world don’t appear to be keeping up with international expectations. This should not be unexpected. Many organisations struggle to keep up with the pace of technological change, and many research disciplines are particularly susceptible to this struggle. Additionally, the competitive and commercial imperatives we instil in our research system encourage a focus on shorter term outcomes than may be required for long term skills acquisition. An informal survey quickly reveals that over 75% of researchers do not have high speed internet at their desks, usually in spite of their institution’s membership of REANNZ, which suggests a lack of focus on researcher needs within institutions. Yet researcher needs are changing: until very recently, almost all field research data collection in New Zealand was #1: Skills Lag March 2015—eResearch2020 done with forms and pencils. Not so long ago, researchers sketched images of fauna or flora samples, recorded birdsong with sheet music, and noted broad based observations of habitat rather than precise location data. Similarly, soil samples were collected in jars, and geographical information was limited to 10km2 plots. Digitisation of information, use of connected mobile devices, high definition geo-spatial data — these technologies and many more are changing the level of scrutiny, the detail and complexity of information available to our researchers. NICK JON ES NeSI “Our current approach (to investment) means our research communities are excluded from applying the most recent advances in research methods aided by the latest digital and information technologies, therefore excluding a growing and highly innovative raft of methods and approaches to research.” 12 New tools in computation and modelling permit not only a deeper understanding of the world around us, but also lift expectations for scientific discovery and research quality. Many of our nationally significant databases are no longer considered particularly large or complex data sources (though the information they contain still maintains its relevance). What we can observe is a gap in our ability to optimally leverage the information we are collecting in our national databases, and a growing gap in the ongoing funding mechanisms for supporting data sources in the long term. #1: Skills Lag March 2015—eResearch2020 The chasm in skills is widening Key areas where New Zealand researchers and research institutions need help to lift their game include: Reproducibility: much of our research output is not sufficiently reproducible science, due to poor methods in evidence, a lack of published method, or poorly interpreted results. Greater rigour is required in designing and publishing methods and data. In order to reproduce research findings in digital research, we need to record and reproduce the provenance of research results, including the data semantics, workflows, code etc. that produced the research output. No matter the research field, features that drive reproducibility need to be monitored and incentivised as keenly as we currently monitor and encourage the rate of academic publication. In some international research systems, data management planning and resourcing has become a first step in any research proposal. Long term sustainability of data/evidence: almost universally in New Zealand research, no support is available for storing and managing data beyond the end of the research project that generated it — this is especially the case for public-funded research data. In several cases, New Zealand research institutions have, or intend to, delete publicly funded research data that is no longer supported by research project funding after the project has ended. Standardised toolsets for research disciplines: we need our graduates to emerge from university already equipped 13 with digital skills, tools and standards accepted and applied in the fields they are entering. We also need to adopt comparative standards within and across disciplines for standardised data, such as geo-spatial, genomic, or health informatics data. Research methodology and standards that flow into economy and society: in leading digital research sectors, such as genetics and human health, new technologies and CRISTIN PRINT eR2020 #1: Skills Lag March 2015—eResearch2020 digital data-driven methods are rapidly moving from research to reality. In these fields in particular our researchers and research institutions need to be able to lead the way in terms of methods and practice. Unfortunately, even our leading research institutions often still lack common meta-data standards and ontologies for “ At clinical scale the quantity of sharing information or making (genetic) data, and the processdecisions. ing required to make sense of it, Whether we expect the will quickly eclipse the current pace of technological change to infrastructure capabilities in New remain the same, or to accelerZealand.” ate, the future implications of this lag in digital research methods and evidence are concerning. Purely from a research sector perspective, we can expect our lack of engagement with new methods to begin to limit our ability to collaborate internationally, to publish impactful research in leading journals, to access international research funding, and to generally lower the quality of New Zealand research compared with first world nations. There are without doubt exceptions to these consequences — we can quickly identify stars in our research communities who are eResearch leaders on the world stage; however the gap in skills between our research stars and our general research population is worrisome. If we consider our socio-economic wellbeing to be linked in part to our research sector and our ability to understand the world around us, then arguably this lag in our eResearch adoption may create limits 14 to our productivity, our social cohesiveness, or to our capacity to monitor our environment, our borders, or our economy. #1: Skills Lag March 2015—eResearch2020 The challenge of change As with much of society, change can be a challenge for researchers, sometimes individually, often as a community. To a certain extent, negative engagement with new, technology driven research methods is a feature of researcher culture: in the pursuit of productivity researchers tend to stick with the methods and tools they know. Researchers can be early JA N E A LLISON eR2020 “A lot of the computer people in research value their independence, so are reluctant to cede control at root level to ICT staff. We spend a lot of time finding ways around corporate limits to share data and back up.” 15 adopters — consumer tools (i.e. Dropbox, Twitter) are widely used across all our research organisations. Researchers are also highly independent, often preferring the self-sufficiency and control that consumer applications allow them to the institutional systems and attempts to homogenise that are common in corporate ITS. This independence is often seen to be at the expense of the scale, cost efficiency, and collaborative opportunity institutional IT services are expected to achieve (though this needn’t always be the case). In general, researchers are excited about the potential new technology brings to their work; however they often believe themselves too busy with research and teaching to have time to learn new tools or methods. In most cases, researchers will do whatever is required to get funding, suggesting a strong motivation to upgrade capability if funding is attached to a higher level of digital research skills. #2: Research Communities March 2015—eResearch2020 Many researchers will encourage their post-doctoral and junior fellows to do the leg-work with digital methods and analysis rather than learn these methods themselves. They do this recognising an underlying change in the standards of evidence and the expectations for impactful research. There’s considerable scope for learning opportunities at the institutional and community level to address this skills gap if the mandate to provide the training were taken up. Furthermore, offering our researchers a better understanding of the competitive research environment — both in New Zealand and internationally — and of the accelerating pace of change, would contribute to expanding our researchers’ engagement with modern expectations for quality in research. RESEARCH COMMUNITIES: Design research infrastructure around users 16 Until now, our approach to infrastructure usually has been centred on investment with research institutions, however we observe that researchers and research are often better connected within discipline-centric research communities rather than at an institutional level. We also note that research communities are at very different levels of maturity when it comes to digital research methodologies and data. A clear concern we have seen in the research sector is that the wide range of funding mechanisms are not explicitly linked to infrastructure or long-term capability. In the case of the National Science Challenges for example, the questions of collaborative infrastructure and data management arguably needed to be dealt with in the upfront design of the Challenges. Yet as infrastructure was specifically excluded from being funded, it does not appear to have been treated as a priority during the development of proposals. A A RON MCGLINCH Y eR2020 #2: Research Communities March 2015—eResearch2020 There appears to be an opportunity to gain greater cohesion across the sector through enabling and promoting sharing of risk and investment into the infrastructure and capability layer that underpins the National Science Challenges. We have found considerable demand for standards and tools in the research sector that might help individual researchers, research communities, and institutions align on meta-data, quality and “ Small groups of staff are forming method. To a certain extent, expert groups within Landcare, as international standards such pioneers to create unofficial groups as ISO standards for research, that are the first port of call for scior those standards imposed entists with questions (rather than by international research inigoing to NeSI).” tiatives are beginning to bring order; however the concern is that as tools and data proliferate, it will become more difficult to share data for collaboration rather than easier. In the drive to produce science of the highest quality that is also linked to economic and societal outcomes, we’ve already observed that our research sector has matured unevenly in terms of skills, capacity, and impact. In general, those elements or disciplines most closely linked to large primary industry sectors are the most mature in terms of the degree to which they collaborate, the high-tech skills retained, access to equipment, and the ability to leverage advanced methods. Conversely, disciplines linked to less 17 well-off sectors of the economy, or those research domains that contribute more to social cohesiveness rather than economic output, are in general less mature in their use of advanced data tools or sophisticated digital and eResearch methodologies. If a knowledge-based economy is a policy goal, then we need to develop tactics that produce excellent science of the highest quality across all of our relevant research JOCH EN SCH MIDT eR2020 #2: Research Communities March 2015—eResearch2020 disciplines. This is not just for those with the highest potential for short-term impact, but also those that focus on sectors of future need or growth. A key criterion in an increasingly data informed and digitally driven research sector is to ensure researchers maintain the capability and skills to be able to innovate and to make serendipitous discovery into the future. This requirement has implications not only for investment strategies in infrastructure and skills, but also in the potential for incentives to adopt technology and digital methodology to be harmonised and embedded in government contestable funding. There is arguably a gap in explicit funding for collaborative, inter-discipline endeavour. Possibly the most impactful links for collaboration and impact in science occur at the “research-discipline community” level, within and across groups of researchers who share a research domain or discipline, rather than contained to an institution or a locale. This suggests that mechanisms which promote connec“ We need off-shore input to advance tivity, knowledge sharing and science, and we make contributions cohesiveness within research back to off-shore science. Concerns disciplines, yet on a nationof sovereignty are likely to have lital scale, will impact quality, tle impact on this fundamental scicollaboration, and the pace of ence paradigm.” progress. A fundamental tenet of eResearch, both in New Zealand and overseas, has been shared risk and investment 18 into capability and capital equipment. Shared investment is most effective when needs and risk are aligned among the participants, which is more often the case across research disciplines that between research institutions. The National Science Challenges are clear examples of a mechanism which promote connectivity in our research system, however the collaborative underpinnings of shared risk and investment in capability and equipment for these initiatives SY DN EY SH EP eR2020 #2: Research Communities 19 March 2015—eResearch2020 is missing: instead the fundamentals of evidence and methodology appear to have been left to chance. While international collaboration for education often occurs at institutional level, international collaboration for research is almost exclusively based around research communities or communities of practice. At an international level, research data and infrastructure is consistently designed around the needs of a particular research community; however this does not mean that it is limited in scale — CERN data infrastructure is quite different to Elixir data infrastructure yet both are large scale, high performant, and significant investments. While the scale may differ, the observation is that research communities have specialised data needs, and that we should avoid making “one size fits all” investments across the entire research system as these too frequently suffer from sub-optimal implementation and use. The argument for designing research infrastructure, and in particular research data infrastructure, around particular research com“ Humanities in particular are still stuck munities, follows from the on the monograph as the output of observation that evidence, research endeavour. Even electronpractice, and collaboration ic journal publishers are resistant to in research differs from disDigital Humanities methods.” cipline to discipline. Applied engineers operate and think differently to pure scientists; agricultural genomics has different needs and standards to clinical genetics; humanities and demographics deal with different kinds of meta-data. To be adopted and become embedded, research and data infrastructure in New Zealand needs to fit both the digital technology maturity of the research community concerned, and the particular data and meta-data standards of that research discipline. The New Zealand Data Futures Forum notes that research data, whether publically funded or not, is likely to be Introduction 20 March 2015—eResearch2020 Introduction Small countries still have big problems; we can’t rely on small science to solve them. For example, to be at the leading edge in dairy genomics is an economic necessity for New Zealand. A small percentage difference in growing grass, or dairy nutrition, or milk production all have major impact on our comparative advantage and economic fortunes in the global economy. For NZ, genomics is an essential eResearch, compute driven discipline. If we take a backseat in the science, then we are really taking a backseat 21 in economic development and global competitiveness. M A R K GA H EGA N eR2020 March 2015—eResearch2020 A RIA N DE WIT eR2020 #2: Research Communities March 2015—eResearch2020 treated differently in terms of security, confidentiality, and meta-data integrity, depending on the type of data and the research discipline involved. To take the National Science Challenges as an example, we’ve learned that not all of the NSC teams have the same demands of infrastructure, and that most NSC needs can be addressed with consumer technologies. What’s missing for the NSCs is the organising principle and leadership to help them make coordinated, well-informed choices. Without well thought out policy driving key behaviours and incentivising decision-making at both a researcher and an institutional level, we will struggle to achieve the desired outcomes from our national investment. Our overarching comment is that, in making investment decisions in infrastructure, particularly research data infrastructure, the needs and maturity of the research community in question should be considered early in the process. “ With both collaboration and data, This does not mean that our personal relationships and networks research institutions do not are still very important, especially at play an important role — the start of a project, as these build only the institutions have the trust and uncover data and expertise organising structure and lonthe project needs to succeed.” gevity to invest in and manage major investment programmes and infrastructure. In the coming decade, New Zealand is likely to have one shot at developing digital infrastructure at a national scale in our research sector. We need to begin that development process 22 with cross-sector research communities, as well as with individual institutions, if we aim to design the best outcomes. To accommodate such a wide range of needs, we can identify some central principles: Stop defining infrastructure needs at an institutional level, instead digital infrastructure should be designed for particular research communities #3: Align Incentives March 2015—eResearch2020 Encourage institutions to make connected investments right from RfP stage —build sector infrastructure planning into institutional strategy making processes where possible Define 8–10 research communities and then have them specify their digital infrastructure needs to be met — and don’t invest in digital infrastructure until we have these research community needs defined Recognise digital research communities have wide variety of maturity, so embed tactics that lift skills within immature communities, and across institutions, without slowing down advanced communities. ALIGN INCENTIVES: Connect planning and research governance 23 We observe that “institutional governance” and “research governance” work to different funding timeframes, operate independently when it comes to planning, and are not jointly engaged in the rapidly changing research environment. Our research institutions are medium to large scale New Zealand enterprises that plan strategic activity against corporate timeframes over 5 to 15 years; however our research committees and research funders operate research strategy against 2 or 3-year funding horizons. The outcome we observe is research projects that are designed with limited goals to avoid needing major infrastructure resources, but that have budgets larger than necessary in order to purchase small scale capital items that do not lend themselves to shared use — or long term management. This misalignment between research governance and institutional planning arguably limits funding for major capital items, as well A LLISON JA N E eR2020 #3: Align Incentives March 2015—eResearch2020 as limiting the resources our researchers can access and share. To the extent that the government employs both institutional and contestable funding across the science system, we also can see misaligned incentives and disconnected governance of these different types of funding at an institutional level. Indeed, many researchers complain that senior institutional management do not understand or wish to engage with the changing paradigm of research methods and data. Investments in line of business applications necessarily dominate institutional budgets, yet there “ I think the barrier is existing funding may be room for incentivising models and individual institutions begreater cross institution proing limited by both their ownership and curement of shared corporate internal funding models. For example, systems such as HR, finance, putting funding for computing hardlibrary, learning management, ware and/or time on a grant often isn’t grant management, and other possible. We can now put NeSI as a corporate services platforms. consumable on a grant, but if we were While a level of shared instisticking to the rules, we’d still have tutional information services a hard time accessing the compute investments might be worth equipment we need for our students. investigation, devising new We can’t guarantee we’ll always have incentives in the institutiona grant, therefore we like to keep a lital management system that tle hardware around and available just release investment to drive in case.” shared capability in research and education, especially for 24 our best researchers and educators, are even more desirable. It would be fair to observe that our institutions are littered with minor capital equipment acquired through project funding, often poorly maintained or supported, and that few others in the research sector or even the same institution are aware of —or use. The culture is to keep such assets in the “department” and out of sight of wider institution. If these assets do become shared (i.e. commoditised use) the #3: Align Incentives March 2015—eResearch2020 fully-costed use of the asset is typically revealed to be much higher than expected; thus sharing is discouraged. This represents a misalignment between research governance and institutional planning, and arguably limits funding for major capital items as well as limiting the resources our researchers can access and share. The National Science Challenges and the National Statement of Science InvestNIC M AIR eR2020 “A challenge for us is that we don’t have the organisational structure in place to harness the infrastructure and services that are available to us.” 25 ment both suggest an attempt to improve whole of system alignment, however it may be worthwhile for government to investigate aligning the funding periods of the institutional and contestable funding mechanisms — at least for minor capital expenditures. Our Crown Research Institutes, created to operate broadly around commercial research models, are extremely well connected to each other at the administrator and executive level. The CRIs are relatively small enterprises which have recognised that at an infrastructure level they cannot afford to act in isolation. Coordinated tactics and responses at the CRI level are becoming more common. Unfortunately this is not the norm in New Zealnd research —the chief architects of long-term planning and strategy appear to be the leading researchers in our institutions, not the administrators or executives. Researchers, ingenious and experimental by nature, devote much of their ingenuity in New Zealand to acquiring project funding. Once again, this leads to the situation where our research institutions feel continually #3: Align Incentives 26 March 2015—eResearch2020 “cash-strapped” and our research infrastructure is unable to scale or is poorly suited to shared use. Some of our institutions already recognise this issue, yet still struggle to coordinate their activities. We observe that no two universities in New Zealand are taking the same approach to crafting a long-term investment strategy that can also support mandated short to medium term research goals. NeSI, NZGL and REANNZ represent the few coordinated, significant long-term investments our research institutions have made in a joint fashion, each one with government leadership, and of these only REANNZ has comprehensive representation in its membership / customer base. Half of our universities have formally or semi-formally instituted committees dedicated to progressing digital and eResearch methods; half do not. At a sector level, our system is highly fragmented — in many cases, our institutions appear to struggle with collaboration, despite deliberate mechanisms for coordination within their respective industry bodies. In general, it is not clear where institutions should draw “pre-competitive” lines so they can collaborate on national infrastructure (for example) while creating healthy competition in research and academic. What we can observe is that collaboration occurs more easily in the realm of contestable funding than in institutional funding. In general, this is arguably due to the close, collegial nature of individual research discipline communities. For this reason, government initiatives that fund collaborative research based around research disciplines, appear more likely to produce impact than those focused on institutional collaboration. Furthermore, improving the cohesiveness, knowledge exchange, and skills within research discipline based communities appears to produce greater quality scientific outputs than cross-discipline investments at an institutional level. #4: Future Infrastructure March 2015—eResearch2020 FUTURE INFRASTRUCTURE: Visualise data for innovation and discovery 27 Our capacity to innovate and make serendipitous discovery is linked to our ability to understand and leverage information. A major part of human interaction with information is visual — the ability to see what is occurring is a crucial aspect of understanding and of cooperative endeavour. Good visualisation is a powerful force in innovation e.g. the (ever larger screen) smartphone is now a key tool for productivity and society, and is fundamentally altering almost every facet of daily life. Visualisation of data, the ability to bring the power of vision to computational modelling, data analysis and collaboration, is a future critical capability across all research fields (and ultimately in major public services areas such as health, law enforcement, and managing natural hazards). In many instances current New Zealand research practice is linear; data-sets inform batch compute processes which produce models or answers that can then be assessed as an end-product. As more and more research activity becomes digitally driven, researchers (most likely working in teams in multiple locations and even time zones) will need to visualise this linear process as it occurs, so that they might intervene to adjust, to experiment, and to essentially make discovery. Critical gaps in our research capability for the coming decade are in the tools, standards, and infrastructure to support advanced data, analytics and visualisation. In the very near future, our ability to interact with and visualise data will determine our capacity to innovate, to make serendipitous discovery, to manage our economy and institutions, and to respond to crises. In these terms, visualisation in research will become less concerned with merely displaying the outputs of science, and instead will be the interface through #4: Future Infrastructure March 2015—eResearch2020 RICH A R D TEM PLER eR2020 which researcher understanding and discovery occur. This will be even more important in research collaboration, where researchers must work from the same observed points of reference, or with datasets of enormous size and complexity. “There’s an assumption of benevolence on behalf of large corporations that is not necessarily justified, so a practice of not putting too many eggs in one basket could be worthwhile.” 28 The NZ Data Futures Forum foresees a future where “data will be abundant and ubiquitous; it will be connected and linked to people, things and places; it will be used and reused, processed and reconfigured”. In this future, our capacity to innovate and make serendipitous discovery is closely linked to our ability to understand and leverage information not only in the research sector but across the wider economy. Sensor proliferation is increasing the volume of data that we have access to, not just research sensors but also the “internet of things” and the ability to monitor our health, economy, and environment. In the short term this leads to spectrum crunch — the limited ability of our communication technologies to convey all of this data without interference or delay — which is a concern of its own. In the long term we expect massive growth in data generation and transmission — continuously generated, real time data, flowing towards analytical systems that enable decisions to be made, and data to be stored and later retrieved for action. In such an environment, our researchers move beyond constructing theoretical models of systems and can begin to analyse and STUA RT CH A RTERS_eR2020 #4: Future Infrastructure March 2015—eResearch2020 optimise systems in real time. This capability, to understand a system in near real time, will present an innovation in research methods that may offer considerable advantage to those researchers who can access it. Data intensive discovery is the analysis of data to discover trends or new knowledge; however it presupposes that the data is discoverable. The library system and now online published journals have been the key research tools in finding secondary information, yet significant data exists in New Zealand institutions that is opaque to our researchers. Even something as ubiquitous as water monitoring becomes complex in New Zealand as the water management data produced and held by our Regional Councils is heterogeneous, and not particularly integrated — we’ve little standardisation or transparen“ Ultimately, the visualisation cy in this area. As a result, our becomes the key interaction ability to understand, respond and the tool for operation, to and manage flooding hazards debugging and understand- and weather events is unnecessarily constrained. Unless they ing the model.” have interpersonal relationships across their community, researchers in New Zealand have very little opportunity to discover the data that resides in institutions other than their own. Without a doubt, a degree of reliance on off-shore infrastructure providers is inevitable and welcome — Dropbox, for example, is the number one shared tool in our research 29 community today. Indeed, there are certain things we really want the market to run, those services where price and reliability matter should be provided by a market — provided the market is willing and able to provide them. Research infrastructure, including networking, data, and compute, are clear outliers here — the market is seldom able to provide the reliability, flexibility and peak performance required for impactful research at an acceptable price. There’s a culture #4: Future Infrastructure 30 March 2015—eResearch2020 change required to come up to speed with the technological change that’s occurring, as previously niche or specialised services become mature and commoditised. New services, such as cloud computing permit easy access to hardware as a service, while potentially releasing resources to focus on ever higher performant computing, more advanced networking, or greater specialisation in data technologies. In our small, first world economy, the researcher consensus is that we should not aim to be isolationist, that the speed of technology advance is such that we can only keep up through extensive use of off-shore providers and services. We should also endeavour to be self-sufficient in those strategic circumstances that require it, and we should actively invest to ensure the underlying human capability to understand new technologies is never lost to us. Researchers consistently identify the key capability for New Zealand as the ability to “tightly couple” data and computational analysis to the analytical tools and visualisation that enable decision support. This “tight coupling” ability implicitly includes elements of technical capacity, model manipulation, and human skill — i.e. it is represented in both the skills of the people we have access to, and the infrastructure and technology we are able to leverage. A challenge in New Zealand institutions is that the skills required to provide and operate this level of technical capacity are not research skills, nor are they general ICT skills, but require specialists in high performance computing and networking, genomic sequencing and informatics. One factor in the ability to attract and retain people with such skills is the access to the infrastructure and equipment they are trained to work with. If the ability to visualise data, compute and analyse in timeframes sufficient to allow decision support and crisis response needs to become the governing principle of infrastructure and technology investment, then consequently the development and retention of talent pools Aiming March 2015—eResearch2020 that support this capability are equally vital. Our observation is that placing requirements such as “real time analysis”, “computational steering” and “collaborative visualisation” at the forefront of on-shore investment decisions will ultimately contribute to New Zealand’s research and support key national capability in crisis response, decision support, and goals in health, energy, the environment and the economy. JOH N R AIN E eR2020 Where should we be aiming? The government’s draft National Statement of Science Investment is clear about the links between science investment and the economic and social development of New Zealand. We need our research sector to have the skills that will eventually translate into “ It’s vital we understand that eRescience making a greater search isn’t something separate; contribution to our economic rather it’s how all research (including competitiveness and social primary sector research) will be done well-being. A contribution in the future—so eResearch needs to also to improving our prebe built into all our plans, our projects paredness to respond to criand our institutions into the future.” ses, be resilient to hazards, influence our economy, and make informed decisions as a society. A clear focus of current policy has been increasing R&D as a percentage of GDP, largely from business investment. We observe that New Zea31 land has a gap to close to ensure our researchers become sufficiently familiar with digital and eResearch methodologies, and with the skills and capabilities expected of world leading researchers, to be able to contribute to the policy goals for the science system. To achieve these national goals, our research sector needs to function as a best-in-class small country “research system”, leveraging larger resources off-shore, but Aiming March 2015—eResearch2020 maintaining key skills and core capabilities in New Zealand. To increase cohesiveness at a national level while maintaining competitive tensions in the system we need to adjust to an eco-system view of digital research skills and infrastructure. This means taking an end-to-end approach to digital research resources — not only in technology investments (such as data or networking) but also across incentives and skills training, policy, funding and governance. The more consistent and aligned our policies, strategies and incentives are, the more efficiently we can implement digital methodologies and enhance skill pools with our limited resources to achieve strategic outcomes for New Zealand. Some of the core capabilities and skills we could consider strategic outcomes include but are not limited to: The ability to understand and optimise at a system level (e.g. a pasture; a water catchment; a hospital) in real time, as data streams in. The ability to plan for and collaboratively respond in real time to an emerging situation (e.g. a volcanic eruption, an earthquake, a disease outbreak) across multiple research institutions and government departments. The ability to visualise tightly coupled, complex data and compute models, to allow interactions with data that enable innovation and serendipitous discovery. 32 The ability to collaboratively deliver on research goals across institutions, inter-disciplinary communities, with industry, and with local, regional, and central government. The ability to track, store, secure, manage, and share private, public, and proprietary data across social networks, public services, and research institutions. The changing pace of tools and technologies, and the explosion in the use of models and data for decision making in government, industry and society make our ability to Aiming March 2015—eResearch2020 work with advanced computational and data analysis techniques in the research field even more vitally important. New Zealand research is strong on foundational capabilities such as open source software platforms and data lifecycle leadership in specific domains (e.g. climate, bioengineering, geonet), however these eResearch capabilities are yet to be well-engaged by governance, strategy and policy in the re- CRISTIN PRINT eR2020 “At clinical scale the quantity of (genetic) data, and the processing required to make sense of it, will quickly eclipse the current infrastructure capabilities in New Zealand.” 33 search system, nor are they linked within and across our research institutions. There is need for clear policy positions to be developed for all publicly funded research which requires and resources long term access to data and analysis models (the evidence) as well as the research output. This may be most easily achieved by aligning data requirements across all the different research funding mechanisms, and ensuring funding is available for ongoing data management (i.e. after the original funded project has completed). Technology is changing the future of research just as rapidly as other areas of our society, yet by taking a silo-ed approach we actively run the risk of not keeping up, or not being able to participate. The infrastructure, skills and resources needed for research and data intensive discovery differ from corporate and consumer needs, and access to those skills and resources are a necessity in contemporary global research. It is these research methods and data tools Expectations March 2015—eResearch2020 devised in our universities and research institutes that ultimately diffuse out to our society to then drive innovation, improve health care, increase our social and economic development, and ultimately ensure New Zealand’s status as a first world country. A few expectations for research in 2020 Research Quality By 2020, evidence and methodology in research will be completely digital. Research quality is assessed through the visibility and repeatability of research methods and evidence in the public domain — in a digital world this places new requirements on researchers. Open access to publications and data will put researchers’ methods on display and play an increasing role in research impact assessment, leading to enhanced researcher skills and infrastructure services. Digital Research Communities By 2020, research, compute and data infrastructure will be designed and deployed for particular research communities, rather than for individual research institutions. Data communities will have a wide variety of maturity — some communities will need significant development while other research communities will demand advanced analytics and data management infrastructure. Very advanced research 34 communities may move beyond national infrastructure and become self-supporting in a global context. Integrated Research Sector Planning By 2020, sustained engagement and linkages will be in place between the governance of research activity and the governance of research institutions. Integrated planning for research strategy, institutional development and national Expectations March 2015—eResearch2020 infrastructure strategy will operate against aligned funding horizons. STUA RT CH A RTERS eR2020 National Capability Beyond 2020, the ability to innovate and make serendipitous discovery will depend on data, compute and networking that is tightly coupled with visualisation. Visualisation is one of the key ways in which you turn data into usable information. It aids the discovery process and enhances capabilities — computation is key to creating visualisation, and networking 35 “Ultimately, the visualisation becomes the key interaction and the tool for operation, debugging and understanding the model.” is key to interacting collaboratively with visualisation. Technology investment decisions for on-shore infrastructure will need to lend themselves to this tightly coupled functionality in order to support key national capability needs (including crisis response and decision support), economic development, and goals in health, energy, and the environment. More and more, data and compute that is tightly coupled to and interacted with through data visualisation will play a role in how we understand real-time systems such as urban systems, pastures and farming systems, demographic systems and other myriad macro and micro socio-economic system-level drivers. Rethinking incentives March 2015—eResearch2020 Rethinking incentives for quality 36 Our project funding and resourcing models for research need to be applied to a different scale and timeframes if we are to understand, manage, and leverage data for scientific insight, social development, or economic growth. For many of our leading thinkers and major research institutions, the quantity, frequency, and detail of data we can now gather simply overwhelms the scope of funding and resource available to work with it. A number of factors are contributing to this situation: We typically invest late in infrastructure and big ticket items, and many of our investments are designed to catch up to international standards rather than to advance with them. This strategy reduces risk in the investment and ensures we take advantage of opportunities to collaborate; however it limits our researchers’ access to new technologies, capabilities and techniques, and to the ability to make serendipitous discovery. This lag in access appears to follow through to a corresponding lag in adoption and ultimately in research impact. Our research sector is both small and highly fragmented — we haven’t a strategy for reaching a critical mass of researchers and infrastructure. In many cases, any particular New Zealand research institution may only have one or two researchers with skills in data intensive research or computational analysis. Little cross-fertilisation of knowledge and skills occurs at a research institution level except when researchers change institution (this is not to say that researchers don’t share information at a community level, below the institutional engagement layer). As different research disciplines engage with digital methods at different speeds, expectations of research output and evidence have become uneven across disciplines. GAVIN M A RTIN eR2020 Introduction Rethinking incentives 37 March March 2015—eResearch2020 2015—eResearch2020 “ NZ likes to do per capita calculations to show how good we are or how we manage our investments, but a comparative per capita measure is the wrong approach in this area for such a small country — it is leading us to under-invest. Small nations, particularly those in Scandinavia (and elsewhere too) are investing substantially more — and more diversely, that is in basic sciences as opposed to a focus on applied science. It’s not because they want to, or they love scientists and engineers, but they see they have to in order to build the science based research ecosystems necessary to underpin any successful economy in the future. To make the point, a comparative per capita approach to making infrastructure investments simply would not be affordable for China — and would inevitably lead to over-investment. China has to use a different measure to be able invest at appropriate levels for their economy — and so must we.” Rethinking incentives 38 March 2015—eResearch2020 To a certain extent, our funding system for research rewards quantity of published journal articles but does not incentivise continuous improvement in research quality or adoption of new methods. Our fully-funded science system means funding for generating data is captured within projects, which arguably limits cross-project visibility of data and promotes duplication in data collection. A clear fault occurs when project funding ends and project generated data is subsequently lost. We need to incentivise our researchers to upskill / improve their research methods and expectations by ensuring funding and resources are tied to quality as well as quantity — where “quality” is measured beyond the simple check box of “published”. Different publications are of differing standards, and have differing expectations for evidence, reproducibility, and impact. Government and institutions have already taken significant but partial steps, including brokering or providing affordable access to infrastructure such as high performance computing. A further step will be the implementation of training and guidance resources, with accompanying expectations of performance that allow researchers to upskill themselves and gauge their improved ability. Finally, to overcome inertia and incentivise research engagement, we might consider reserving premium research funding for researchers who’ve demonstrated engagement and adoption of world-class methods and tools that have been made available to them, and can therefore certify the quality of their research. In the same vein, we would need to ensure funding also reached promising young researchers who are native to digital research and whose skills will spill-over to lift the abilities of those they work with. Summary March 2015—eResearch2020 Summary The aspiration is for the New Zealand research system to be functioning as a best-in-class small country sector in 2020. One facet of this will be a determined effort to augment the capability of New Zealand researchers with stronger skills in reproducibility, and the evidential integrity of our digital research output. Only a small fraction of New Zealand researchers are really engaging with the methodologies, skills and practices that likely will be required of all researchers within the coming decade. Another aspect of best-in-class sector design for small country systems will be comprehensive integration with major off-shore research resources, as well as smart, coordinated use of market and commercial providers for efficiency and scale. Finally, we will need to maintain within New Zealand the core digital skills and capabilities that are integral to functioning as a country, and that will be increasingly central in supporting our well-being as a society. These include the ability for essential government services to function, our ability to respond to hazards, and the capacity to innovate and design new products and services. Recommendations 1 Provide training and adjust incentives to address the growing skills gap in our research system when it comes to digital methodologies and research quality. As funding ap39 pears to be the prime motivator of researcher behaviour in our system, we should consider reserving a portion of current funding as a “premium” or top-up for researchers — both experienced leaders and promising younger generation researchers. In parallel, we need our research system to better support those seeking to improve their skills in new research areas and meet new expectations in terms of quality. In particular, we need our graduates to emerge from university al- Summary March 2015—eResearch2020 ready equipped with digital skills, accepted standards and applied tools in the fields they are entering. 2 Too much of our thinking and strategy is focused around the silos of our institutions, rather than across a broad community of researchers. We should take the opportunity offered by the National Science Challenges to enable and promote sharing of risk and investment into the infrastructure and capability layer in our research system. This includes promoting cohesiveness and knowledge exchange within national research communities. As well it recognises our digital research communities have wide variety of maturity, thus implementing tactics that lift skills within immature communities without slowing down advanced communities (and their associated industry R&D capabilities). To achieve this we should try to stop defining infrastructure at an institutional level, and instead design digital infrastructure for particular research communities. We should resist investing in digital infrastructure until we have the target research community’s needs properly defined. We need research communities that guide and coordinate infrastructure investments across our research institutions, just as we need our research institutions to be central to the operation and provision of infrastructure services if our infrastructure investments are to be effective. 40 3 The rapid speed of technological change may actually offer us the opportunity to move away from a fragmented approach and take a national eco-system view of digital research capability and how we meet changing expectations in research quality. Eco-systems don’t develop overnight, however taking a national view of digital research capability as an end-to-end system begins to suggest some early policy and strategy opportunities. In practical terms we need to adopt comparative standards within and across disciplines for standardised data, common meta-data standards and ontologies for sharing information or making decisions. We Summary March 2015—eResearch2020 JA M ES SMITHIES_eR2020 also need to front-end data management planning and resourcing as a first step in research and public project planning so that ongoing support is available for storing and managing digital resources beyond the life of the projects that generate it. At a more generic level, we can improve the efficacy of allocating our limited resources by better aligning governance, planning and fund“ New Zealand needs to remember, ing cycles for institutional though, that we’re living in an increasand contestable funding in ingly globalized world. That offers exresearch. We want to incencellent opportunities —and I for one tivise both of these towards am optimistic about the opportunities quality as well as quantity digital technologies offer to engage of outcomes, and avoid the with the wider world —but we can’t sub-optimal allocation of rely on the likes of Google, or even the capital towards project reInternet Archive or the United Nations, sources that don’t support to preserve and develop our culture.” shared use. Particularly desirable would be any newly devised incentives in the institutional management system that drive capability in digital research capabilities and infrastructure for our best up-and-coming researchers and educators. 41 4 As a small, first world economy, the researcher consensus is that we should not aim to be isolationist, that the speed of technology advance is such that we can only keep up through extensive use of off-shore providers and services. At the same time we should endeavour to be self-sufficient in those strategic circumstances that require it, and we should actively invest to ensure the underlying human capability to understand new technologies is never lost to us. Researchers consistently identify the key capability for New Zealand as the ability to “tightly couple” incoming data and computation analysis to the analytical tools and visualisation that enable collaboration and decision support – in Summary March 2015—eResearch2020 PETER H U NTER eR2020 fact, “visualisation” is likely to be the key enabling technology in innovation and discovery in the coming decade. As such, it could be the best foundation skill and infrastructure driver for strategic investment decisions. While it will take significant expertise in the field to define investments at a technical level, our observation is “Inevitably, our society’s problems in the future are going to have a data and computation aspect to them.” that placing requirements such as “real time analysis”, “computational steering” and “collaborative visualisation” at the forefront of New Zealand’s on-shore investment decisions and research capabilities will ultimately contribute best to New Zealand’s research goals. It will also best support key national capability in crisis response, decision support, and goals in health, energy, the environment and the economy. 42 Acknowledgements March 2015—eResearch2020 Acknowledgements eResearch 2020 would like to acknowledge the input of our expert advisory group and thank them for their generous support and efforts in guiding this project. • Professor Penny Carnaby Lincoln University • Professor John Hine Victoria University of Wellington • Professor Keith Gordon University of Otago • Associate Professor Blair Blakie University of Otago • Professor Shaun Hendy University of Auckland • Associate Professor Cristin Print University of Auckland • Tim Chaffe Enterprise Architect University of Auckland • John McMaster CIO Plant & Food Research • Lynley Smith IS Manager (Acting) GNS Science • Steve Cotter CEO Research and Education Advanced Network New Zealand (REANNZ) 43 • Tony Lough CEO New Zealand Genomics Limited (NZGL) • Nick Jones Director New Zealand eScience Infrastructure (NeSI) To view the document online go to: eresearch2020.org.nz 2020 eResearch If we consider our socio-economic wellbeing to be linked in part to our research sector and our ability to understand the world around us, then arguably a lag in eResearch adoption may create limits to our productivity, our social cohesiveness, or to our capacity to monitor our environment, our borders, or our economy.
© Copyright 2024