New Approaches for Evaluating and Improving the Efficacy of Digital Courseware: Request for Proposal Application Deadline: March 20, 2015 – 5:00 pm (PST) Overview Introduction In 2013, The Bill & Melinda Gates Foundation invested $2 million in new approaches to facilitating short-cycle feedback so that teachers, school decision-makers, and parents have better information about the effectiveness of digital courseware, and developers of such products could receive rapid input to improve their offerings. That investment supported the launch two types of new initiatives: Test bed networks, organizations that support schools in piloting innovative courseware products and help schools and entrepreneurs to collaboratively define a structured process to evaluate and improve the efficacy of digital courseware; and Efficacy platforms, technology platforms that use validated diagnostic and formative assessments to facilitate rapid iteration and improvement of digital content and tools. To date, these initiatives have supported more than 50,000 students and 2,000 teachers in piloting new courseware. Based on that success, we are requesting proposals for a second cohort of test bed networks and efficacy platforms that can help improve digital courseware’s efficacy in: 1. Student Learning: effective digital courseware should not only enable students to achieve explicit learning objectives but also accelerate the pace at which they do so. 2. Teacher Support: effective digital courseware should be easy to implement and integrate into instructional plans, saving teachers time and helping them support their students’ learning needs. 3. Engagement: effective digital courseware should provide for a rich, compelling, and interactive experience that will encourage repeated, prolonged, and self-motivated use. 4. Satisfaction: effective digital courseware should sufficiently meet student and teacher needs and provide for a consistent, inviting, and intuitive experience. This document is an invitation for proposals for rapid digital courseware evaluation and improvement approaches that: Focus the development and adoption of personalized learning products on helping students achieve desired learning outcomes. Put teachers and school decision-makers at the center of the shift toward personalized learning. Lower risks and barriers to all parties of adopting new, potentially transformative products. Encourage the rapid development of a healthy, transparent market for highly effective learning technologies. Page 1 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Vision for Personalized Learning Our vision for education is that all students have learning experiences that are tailored to their individual needs, skill levels, and interests. For some, this will mean accelerating instruction beyond the fixed pace of many of today’s classrooms. For others it will mean being given opportunities for extended practice of skills or content they have yet to master. For all students, it will mean learning experiences that are tailored specifically to their progress against a clear set of academic standards; receiving constant, actionable information on their progress; experiencing more one-on-one engagement with their teacher and developing deep connections with other students. Implementing this vision, which we and many others call personalized learning, at scale will require new tools to help teachers, parents, and students diagnose gaps in knowledge and skills and adapt learning experiences based on student progress. Technology can help support effective teachers, by giving them new ways to design and provide personalized instruction to their students. Comprehensive courseware products such as Carnegie Learning’s Cognitive Tutor, Reasoning Mind, ST Math, and Dreambox are facilitating significant learning gains for students in even the highest-need schools, while apps like Refractions, DragonBox, and Motion Math are demonstrating that even rigorous academic content can be highly engaging. Our goal is for all students to have ready access to these types of tools, enabling them to learn what they need, when they need it by personalizing content, learning experiences, and support in real time. Market Context Fueled by the growth of personalized learning schools, technological advances, and an influx of new talent, “start-up companies and individuals are developing digital learning resources at a rapid pace.” (US Department of Education, 2013). At the same time, tech-savvy teachers are increasingly seeking products to support their efforts to personalize learning and improve student outcomes. More than 90% of all teachers in a recent Harris Interactive survey indicated that “they would like to use more education technology in the classroom than they do now” (Harris Interactive, 2013); however, the market is not yet rich enough with choices for pioneers implementing personalized learning, nor is information on the impact of digital courseware on student learning readily available enough to meaningfully drive purchasing decisions or to support the transition to a performance-driven market. As a result, many incumbent providers are able to compete based on the strength of their sales and marketing efforts alone, while innovative players with promising products struggle to achieve sustainability and scale. We envision a healthy, rational US K-12 digital courseware market characterized by effectiveness, affordability, and transparency. In such a market, vendors would have ready access to short-cycle efficacy data and information that would be used to drive continuous product improvement. In turn, teachers and school decision-makers would use standardized, easily comparable efficacy data in deciding whether to continue using existing products or to adopt new ones. Making this data and information readily available would help evolve the market to one in which school decisionmakers are well supported in identifying the most effective tools for student learning, vendors develop and compete primarily based on performance, and all students have ready access to highly effective personalized learning tools. One way in which we support these types of market shifts is by providing resources that cast a light on gaps and opportunities that are critical to realizing the vision of personalized learning for all students. Over the past year, our analysis of the market – informed by input from leading teachers and school decision-makers, innovative vendors, and a range of experts – identified three major shifts required to move towards a transparent, performance-based, and responsive K-12 digital courseware market: 1. A dramatic increase in the number of effective personalized learning products, particularly in high-need areas such as literacy. While investment in education has increased dramatically over the past several years, critical areas of the market remain unserved. For example, of the $427 million in venture funding provided for K-12 education entrepreneurs in 2012, only 6% of it was invested in digital instructional content (NewSchools Venture Fund, 2012). 2. Providing readily available, independent information on learning technologies that allow teachers and school decision-makers to discover new products, quickly identify product alignment to standards, and share feedback on learning technologies based on personal experience. Nearly 80% of teachers agree that “it’s at least somewhat difficult to find out about high quality educational technologies,” and nearly a third of all teachers spend an hour or more each week researching which tools to use (Harris Interactive, 2013). 3. The emergence of lightweight, rapid feedback loops between teachers, school decision-makers and entrepreneurs that use student learning and engagement data to drive decision-making and continuous improvement. The federal Page 2 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal What Works Clearinghouse was designed to show what programs and products are most effective. However, due to its methodological requirements, it does not sufficiently address the needs of schools or vendors. Increasing the speed and transparency with which valid data is produced and made available should increase incentives for vendors to improve the performance of their products. Our Literacy Courseware Challenge invested $6 million to help facilitate the first shift. Our support of organizations that provide ready information on digital learning products, such as Graphite, EdReports, and EdSurge, seeks to facilitate the second shift. This proposal will seek to facilitate the third shift. Thus, we are requesting proposals for new approaches to facilitating short-cycle feedback on product efficacy to both developers of innovative, digital courseware and the students, teachers, and school decision-makers they serve. Detailed information on the problem we seek to solve, our definition of product efficacy, grant awards, and eligibility is contained within this request for proposal. Scope and Approach This request for proposal will support a three-year grant program targeting product entrepreneurs, market catalysts, and school networks, or any partnerships or combinations of these parties. The Bill & Melinda Gates Foundation does not envision being a sole funder of these initiatives during the two-phase process or thereafter. Applicants should have the ability and inclination to generate additional funds and to support on-going operations. Solution Requirements This RFP was developed by the Bill & Melinda Gates Foundation with the input and guidance of an advisory group composed of eight innovative school and non-profit leaders, and entrepreneurs. While the content and viewpoints expressed herein are those of the foundation, the vision and solution requirements for the RFP were shaped through extensive research on existing solutions and in-depth interviews with the advisory group. Throughout 12 interviews, advisory group members consistently described product development and piloting as an ambiguous, time-consuming, and often isolated process that poses large risks to schools and creates a very high barrier to entry for entrepreneurs with potential solutions. Faced with an increasing number of product options, school decisionmakers reported both a lack of data on what products work for which students under which circumstances and a lack of means to quickly evaluate whether a given tool is the right fit for them – a particular concern at a time when lean school budgets have only heightened the need to justify and assess the return on any investment in technology. Entrepreneurs, in turn, cited barriers including limited feedback channels and an insufficient number of teachers and students to building effective products that satisfy school needs. Both groups agreed that currently available data sources were insufficient for evaluating product efficacy, in particular a product’s impact on student learning. Among the major limitations they cited are: The rigorous effectiveness studies that have traditionally produced this data are time-consuming, expensive, and unable to keep pace with the abundance of emerging tools. For example, the federal What Works Clearinghouse lists fewer than 50 K-12 literacy interventions that show positive or potentially positive impact on student learning. Data produced by existing methods is often based on infrequent interim or end-of-year tests that assess student performance at a single point in time, forcing school decision-makers to rely on data that is inconsistently available and lags far behind ongoing product improvements. Product efficacy data from credible sources typically provides only a single dimension of product quality and statements about large, general populations. This type of data is useful in assessing a product independent of context, but is not helpful to school decision-makers who need to understand how a particular product is affecting an individual student’s learning and take immediate action based on this information. Self-reported vendor claims of efficacy cannot be easily evaluated and corroborated. This leads to even validly produced studies being viewed with distrust, and schools in some cases unnecessarily duplicating efforts by contracting with independent evaluation firms to conduct their own efficacy studies. Page 3 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Vendor-produced efficacy data is often presented in unwieldy, hard-to-interpret, and inflexible formats, eroding its usefulness for already-oversubscribed school staffs. School Needs In a recent survey of more than 1,000 teachers, the most cited reason for frustration with technology in the classroom was that “they don’t do exactly what I want” (EdShelf, 2013). Still, nearly all teachers “agree they would like to use more education technology than they currently do,” citing its potential positive impact on student outcomes and personalizing learning (Common Sense Media, 2013). In order to achieve this, schools need technologies that are both efficient and effective at delivering personalized learning experiences, and that make the best use of school and teacher time, effort, and expertise. More than ever, teachers and school decision-makers need practical, credible evidence on the effectiveness of new tools – and that evidence’s applicability to their particular needs before adopting the tools. As a part of that evidence, they need to understand how to implement new tools based on prior success, their unique student needs, and fit between test conditions and real classroom environments. Post-implementation, teachers need regular data showing whether the tools they are using are having a positive impact on student learning and engagement. If something is not working optimally, they need to have at their disposal built-in feedback channels with entrepreneurs to help diagnose and address those deficiencies through product refinements in a timely manner. Vendor Needs Entrepreneurs tell us that these shortcomings pose a Catch-22: their products are often not trusted by schools due to a lack of independent, long-term evidence of efficacy – but they also cannot get their products used by a critical mass of people to generate the evidence that they need. Furthermore, while entrepreneurs are increasingly adopting an agile product development process that creates opportunities to refine and strengthen products in response to feedback in a matter of weeks, the lack of real-time, product efficacy data makes this process at best one guided primarily on engagement alone rather than actual student learning. As a result, some entrepreneurs build products that are engaging enough for individual teachers to employ in classrooms as an instructional change of pace, but that are not optimized for student learning. When successful, this approach generates an initial surge in use, but often fails to build an evidence base that justifies wider deployment. Other entrepreneurs focus immediately on demonstrating impact on student learning, but struggle either to test their products with enough students to generate statistically significant data or to show clear, positive impact without incurring major expenses from hiring independent evaluators. In both cases, innovative entrepreneurs struggle to establish themselves in the market and to compete against incumbents. Entrepreneurs, therefore, need a critical mass of pilot teachers and students to help inform and validate their product efforts. They also need to be able to lower the risk of trying new, untested products by quickly proving to schools that they have the processes in place to gather and provide credible, actionable efficacy data at intervals short enough to support short-cycle adjustments to student learning experiences without compromising the entrepreneur’s own ability to test and refine products. As adoption increases, entrepreneurs need robust mechanisms within their product development processes to continuously measure efficacy, to learn about the effects of different implementations on student learning and engagement, and to rapidly iterate and optimize their products in response to both qualitative and quantitative feedback. Rapid cycles of modification, analysis of results, and redesign are “key to producing dramatic change while reducing risk” (Tony Bryk, 2011). Solutions Requested To address these needs, we are requesting proposals for test bed networks, platforms that utilize A/B testing with validated assessments, or solutions combining the two. Successful proposals will address at least one of the solution descriptions below and will rate highly against the evaluation criteria below. Test Bed Networks Forums that bring teachers and entrepreneurs together in the hopes of facilitating collaborative product development are increasingly being convened across the country by organizations such as the Education Technology Start-up Collaborative. These events typically feature teachers “test-driving” personalized learning products across various stages of development for free or at a reduced price and shaping the product development process by offering valuable feedback to entrepreneurs. Events such as these are a step in the right direction, but they are limited in continuity and scope of impact. A sustained test bed network with consistent and frequent interactions between teachers, their Page 4 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal students, and entrepreneurs on a larger scale – in either an online or offline medium – can facilitate a meaningful feedback loop about the qualitative and quantitative dimensions of product efficacy. For entrepreneurs, these networks would provide a range of benefits that would help them compete more effectively in the market and increase their chances of success, including: access to a critical mass of pilot teachers and students; the feedback necessary to build more effective products; the ability to generate early evidence of their products’ impact on student learning and engagement, and greater teacher satisfaction. For schools, these networks would create a way to quickly determine which products merit further exploration, greater influence on product design, and better, more regular data about the effectiveness of the learning technologies used in their classrooms. As a part of these test bed networks, schools and entrepreneurs would collaboratively define a structured process to evaluate and improve the efficacy of digital courseware, including: (1) the implementation model for each product tested, (2) the format and frequency of any requisite professional development, (3) the specific measures of product efficacy to be evaluated and by whom, (4) the diagnostic and assessment to be administered, (5) the nature of data reporting and data-sharing agreements between school and vendor, and (6) the format and structure of the feedback loop. Platforms (Utilizing A/B Testing with Validated Assessments) While software programs such as Google Analytics and Mixpanel have provided an effective means by which to digitally measure engagement and satisfaction in web-based products, the largest gap when it comes to evaluating product efficacy remains in measuring a product’s impact on student learning. The randomized controlled trial (RCT) has long been recognized in many fields, including education, as the gold standard for evaluating the efficacy of an intervention. While this experimental model is viewed by researchers as the most conclusive means by which to establish a causal relationship between a given practice and a learning outcome, in its traditional form it poses a number of often-insurmountable challenges in schools, including in a digital learning context. Namely, RCTs can be cost-prohibitive for resource-starved schools and entrepreneurs, costing hundreds of thousands of dollars (Coalition for Evidence-Based Policy, 2012), and can often take years to fully execute, a timeline which is incompatible with the velocity of change and decision-making in technology and education. Random assignment of students to treatment and control groups is also very difficult to accomplish, given the way schools typically assign students to teachers and programs. For these reasons, fully leveraging the transformative potential of innovative, personalized learning products must first involve “accepting that the strongest level of causal evidence is not necessary for every learning resource decision" (US Department of Education, 2013). A/B testing, a widely-practiced product development heuristic employed by many education entrepreneurs, captures the essence of random assignment in RCTs and allows for systematic comparison of different product features – without the prohibitive cost or timeline required. Under this method, two randomly-assigned groups of students are given versions of the product that vary in a single, deliberate manner – thus, allowing comparisons to be drawn between the experiences and outcomes of the two groups, and influential product changes identified. For early-stage entrepreneurs with new products and teacher and student bases insufficient for large-scale, multi-site RCTs and schools with short decision-making timelines and low tolerance for risk, a collaborative form of A/B testing utilizing validated diagnostic and formative assessments represents the ideal vehicle with which to carry out small-scale efficacy studies. The information gleaned from these studies can serve as crucial inputs into the product development and purchasing processes in the near-term by proving that impacts on student learning are maintained even outside of the specific product environment. In the long-term, it will also form the foundation upon which larger-scale experiments can be performed to demonstrate fidelity of results across a broader diversity of teachers, students, and contexts. Page 5 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Exhibit 1: Evolution of Efficacy Studies Use Cases The scenarios depicted below are representative of our overall vision (Phase I + Phase II). They are presented here as potential examples of proposals to help guide potential respondents only and should not be perceived as prescriptive or limiting in any fashion. Test Bed Networks Better Schools for All (BSA), a non-profit organization that supports the expansion of high-performing charter schools, submitted an application for Phase I of the New Approaches to Evaluating and Improving the Efficacy of Digital Courseware RFP. BSA, whose network of twenty schools nationally serves a population of over 8,000 students, had been thinking about a systematic means of evaluating the efficacy of a seemingly ever-increasing pipeline of new digital learning products up for consideration for use in classrooms. Intrigued by the vision of a test bed network that would facilitate the structured piloting of digital courseware in classrooms, connect teachers and students with entrepreneurs in a mutually-beneficial way, and ultimately enable rapid decision-making based on product efficacy data, BSA formed a working team composed of a select group of entrepreneurs, school decision-makers, and teachers, to begin exploring a potential solution. Over a series of planning meetings, the team agreed upon the following parameters of the test bed network: Implementation format for each product (station rotation blended model, 30 minutes a session, two times per week) Professional development format and frequency (half-day, in-person, vendor-facilitated overview combined with justin-time online PD as needed) Efficacy metrics to be tracked and evaluated (impact on student learning, teacher support, student engagement, and student/teacher satisfaction) The external measure of impact on student learning (a standards-aligned, initial diagnostic and formative assessment already in use presently) The content, level of detail, and visual format of data outputs for all stakeholders The structure of the feedback loop (weekly reports on student engagement, monthly reports on student/teacher satisfaction as measured by surveys, monthly reports on impact to student learning, real-time IT/trouble-shooting support) The team applied for and won a Phase 1 grant for a six-month pilot of the solution for three new literacy products, across ten sixth-grade classrooms within the BSA network. Page 6 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal For entrepreneurs, the pilot generated a continuous stream of valuable feedback from real teachers and students that could be used to improve the product experience. It also exposed inconsistencies between how the products were designed to be implemented and how they were actually being utilized in the classroom. For teachers, the pilot provided a way to quickly assess how effective new products actually are in the classroom, and an opportunity to help shape the product development process as well. For school decision-makers, the pilot revealed significant and persistent relative differences in student learning gains and engagement across the three products, allowing them to make faster decisions about which products to explore further or discontinue. Based on the success of this initial pilot, BSA is currently seeking Phase II funding to deploy the solution across its entire portfolio of schools. Platforms In hopes of building a decision-making tool for teachers and school decision-makers to quickly evaluate the efficacy of digital courseware products in a systematic fashion, DecisionEngine, an enterprise software company specializing in data analytics, submitted an application for Phase 1 of the New Approaches to Evaluating and Improving the Efficacy of Digital Courseware RFP. The proposal was for ProductAssess, a web-based evaluation system that would: Integrate with web-based products so as to be able to measure student learning and engagement resulting from use of those products Create a student portal within which students would be able to securely log-in, access, and use the digital products under evaluation, and subsequently complete standards-aligned diagnostics and formative assessments Create teacher/administrator/parent portals within which stakeholders would be able to access actionable data on student engagement and learning gains in short cycles, and in both granular and aggregate levels of detail as needed Provide continuous feedback on product efficacy and fidelity of implementation to personalized learning vendors, who can then quickly translate the feedback into product refinements/enhancements Have the capacity to perform automated randomized assignment tests on a large scale After winning the Phase I grant and building out a beta version of ProductAssess, DecisionEngine was able to secure a six-month pilot with a large, urban public school district and a subset of its portfolio of product vendors. Though this initial pilot was insufficient in scope to generate any statistically-definitive conclusions, the data suggested that there were indeed significant relative differences in student engagement and learning gains across the various digital learning products currently employed by the district. It also established for school decision-makers a standardized framework with which to measure product efficacy, provided useful feedback for ProductAssess improvements back to DecisionEngine, and provided information regarding fidelity of implementation back to product vendors. DecisionEngine is currently seeking Phase II funding for a national launch. With the efficacy data gleaned from a broader cross-section of schools and vendors, DecisionEngine hopes to utilize the full capability of ProductAssess to conduct automated randomized assignment tests in order to: Quickly assess the efficacy of newly-introduced personalized learning tools A/B test new product features/enhancements Recommend different implementations of products based on data from other schools Page 7 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Rules & Guidelines Eligibility Funding Criteria For this RFP (Phase I), we are targeting product entrepreneurs, market catalysts, and school networks/districts - or partnerships or combinations of these parties – with proposals for the following: Test Bed Networks: collaborative partnerships between schools and entrepreneurs that facilitate broad usage of digital courseware by teachers and their students in order to build a continuous feedback loop on product efficacy for school decision-makers and entrepreneurs. Platforms: technology products that will support data collection on digital courseware in order to assess product efficacy and that can support large-scale, short-cycle A/B testing. Combination: solutions that combine Test Bed Networks with Platforms. While we anticipate that most respondents will seek to address either a platform or test bed network in their proposals, special consideration will be given to organizations that – whether on their own or through proposed or already established partnerships – can deliver on a comprehensive solution that combines both. We will consider only those proposals that adequately address all four components of product efficacy as defined in this document, with the heaviest emphasis on impact on student learning. We encourage for- and non-profit organizations with demonstrated capacity for or promise of undertaking the work described herein, to respond. Teams proposing partnerships are required to submit a single application and identify the financial recipient of the grant funds. Given the nascent role of product efficacy efforts in the K-12 education ecosystem currently, we will evaluate and grant awards to ideas both in the Concept Development and Testing/Deployment stages. Type of Award Stage of Development Concept Development New concepts pre-beta testing Definition In use by <10 teachers and 300 students Size of Award Number of Awards Grant Period Up to $100K Up to 6 12 months Purpose of Grant To facilitate concept development and testing in preparation for pilot Testing/Deployment School-ready products In use by >10 teachers and 300 students and poised to grow Up to $350K Up to 4 12 months To facilitate solution refinement and pilot in preparation for broad implementation and independent, third-party evaluation Do I retain intellectual property ownership? Yes, all entrants, regardless of winner status, will retain intellectual property ownership of their software applications. However, winners must agree to support global access objectives by conducting and managing the project and information in a manner that enables (a) the knowledge gained during the project to be promptly and broadly disseminated and (b) the intended products to be made available and accessible for free or at reasonable cost to the people most in need. For more details, please see the Representative Terms and Conditions that will be included in the grant agreement. Page 8 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Exclusion Criteria We will NOT consider funding for: Respondents that do not propose either a platform and/or test bed network solution Proposals that fail to adequately address all four dimensions of product efficacy (as defined in this document) Single LLCs, or individuals not affiliated with a for- or non-profit company o If you are an LLC, please provide evidence that you're taxed as a partnership, or that you've made an election to be taxed as a corporation Respondents without an existing, minimally-viable solution Individual schools Isolated, one-off events bringing together entrepreneurs and school leaders (e.g. “School Start-Up Days”) Proposals with budgets >$350,000 or >12 months in duration Evaluation Criteria Submissions received by the deadline will be evaluated on the following criteria and rubric: Scoring Criteria Alignment with Objectives Potential for Impact Capacity to Deliver TOTAL Points 25 50 25 100 Alignment with Objectives (25%) o Fully addresses the problem statement and vision raised herein o Proposes to adequately and comprehensively measure all dimensions of product efficacy (as per the definition described herein) o Will serve a diverse cross-section of schools, teachers, and students o Is affordable and accessible to all schools Potential for Impact (50%) o Establishes robust, continuous, and scalable feedback channels and processes between stakeholders o Integrates diagnostics and formative assessments that are broadly-applicable and have been validated by instructional leaders and the research community o Includes short-cycle reporting interfaces and outputs that are user-friendly, flexible, intuitive, and actionable o Proposes a realistic implementation plan that requires minimal training and IT support by vendors and/or pilot schools o Utilizes technology that is interoperable, secure, scalable, and has the analytics capability to perform largescale A/B testing (if applicable) Capacity to Deliver (25%) o Led by a team with the skills and experience necessary for the proposed work o Proposes a work plan that can be realistically executed on time and within budget o Attests to the long-term financial health, sustainability, and self-sufficiency of the organization o Engenders confidence that the team has the human and financial capital to viably scale the solution if it succeeds in pilot testing o Demonstrates buy-in from proposed partners (if applicable) Page 9 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Activities & Timeline February 18, 2015: Request for Proposal (RFP) Open. February 24, 2015 – 1:00 PM (PST): Q&A call/webinar. March 20, 2015 – 5:00 PM (PST): Completed applications due. April 2015: Finalists notified. June 2015: Countersigned Grant Agreements received. Note: Grant awards will be contingent on the successful completion of due diligence processes. How to Apply Response Requirements To apply, please submit a proposal using the ‘Proposal Narrative’ template that includes the following: Summary (1/4 page or less): o Award category: Concept Development or Testing/Deployment o Amount requested ($) o Brief description: 1-2 sentence summary of your proposal Organization (3 pages or less): Please tell us about the people and organization(s) that will develop the solution, and provide: o Background about how long the organization has been in operation, its mission, and any products that are already available o A list and description of key existing or pipeline users of the solution o Historical information on number of existing users currently supported and anticipated growth trajectory o A list of all relevant team members complete with biographical information. Describe how your team members know one another and specify the activities each person will be involved in o An explanation for why this team has the unique capability to develop and implement this solution Solution Description (5 pages or less): Please provide a detailed description of the platform and/or process you aim to develop, and address the following: o What is a use case that accurately depicts how you envision your solution to work? o What short- and long-term goals do you propose for measuring and evaluating the success of your solution? o What specific metrics of product efficacy do you seek to measure, facilitate a feedback loop around, and optimize for? o What technologies will the solution utilize, and how will such technologies support facilitating a short-cycle feedback loop around product efficacy? o What assessments and/or feedback channels will be employed in the solution? To what extent have the assessments been validated by instructional leaders and the research community? What quantitative and qualitative feedback will teachers, school decision-makers, and entrepreneurs receive based on these assessments and channels, how, in what form, and how often? o What stakeholders and/or partners need to be involved and what is the desired nature, number, frequency, and structure of intercepts linking them together? o How do you specifically plan to deliver feedback on product efficacy to drive continuous improvement efforts for vendors? o If a platform, service or product, how will you price it? If a process, what rubric will you employ to determine who gets to participate and when? o What training, professional development, and ongoing support will be required to enable a pilot? o What is your cost to develop the platform? o How does your pricing compare to similar existing products? Additionally, respondents may provide up to five additional pages of appendices that specifically relate to solution description, such as screenshots, sample assessments, etc. Page 10 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Work Plan (1 page or less): Please outline the major activities associated with the development of the proposed solution, when they will be completed, and by what resources. Recruitment Plan (1 page or less): Please tell us how you plan to attract customers of the solution to demonstrate proof of concept. Include specific information on partnerships, distribution channels and/or marketing strategies. Submission Instructions Start by clicking the application link below. The first time you visit the portal you will need to create an account. After you enter your information, a temporary password will be emailed to you. Return to the portal to personalize your password and complete your account profile. 1. Click on the Application Portal link. 2. Create an Account 3. Once you are logged in, select “If you have received an invitation… you may respond here.” on the home page and enter the following information: Request ID: SOL1128135 Access Code: educ@tion2015 Please Note: You can save the application and return to it later by logging in to the Application Portal at any time before the submission deadline with your email address and password. The Request ID and Access Code are only necessary the first time you access the application. You can only hit “submit” once using the application tool, so we suggest iterating on documents offline and submitting your final versions. You will receive a confirmation email once you have successfully submitted your full application materials. If you do not receive the email (check your spam folder), please return to the portal and try again. To view frequently asked questions about submitting your application, click Help on the Application Portal. We use the application portal mainly to create an entry in our internal database. After you use it once to submit your documents, you’ll send all subsequent documents as attachments via email. The Application Portal will be updated periodically with information regarding the Q&A Call/ Webinar currently scheduled for February 24, 2015 – 1:00 PM (PST) Filling out the Online Application Follow the directions on the application portal. 1. Getting Started tab - Please review the all Guideline and Reference documents noted below. Guidelines and Reference Document New Approaches for Evaluating and A Improving the Efficacy of Digital Courseware - RFP B BMGF Representative Terms and Conditions C BMGF College Ready Strategy D BMGF Indirect Cost Policy E ER Guidelines F Data Stewardship Principles Page 11 of 15 Purpose Invitation to participate and guidelines for the RFP Provides the Representative Terms and Conditions for the grant. If you’re proposal is selected, you should be prepared to agree to the terms as stated in this document. Please review it carefully. Provides information about the Gates foundation USP College Ready Strategy Provides guidance about the foundation’s Indirect Cost policy Provides guidance on private foundation expenditure responsibility requirements Provides guidance on private foundation stewardship principles to protect student data and privacy © 2013 Bill & Melinda Gates Foundation Request for Proposal 2. Describe Project tab - Please fill this out to the best of your ability, using the following guidelines: Project Title – a short title, including the name of the solution. Project Description – a short description about the solution and any unique features. Project Duration – 12 months. Project Amount – enter the amount you are applying for. Please consult the RFP Information above for guidance on the maximum amount for each award. Referred By – please note where you heard about this RFP. 3. Upload Files tab - Please complete and upload the documents noted below using the templates provided. Templates and Organizational Documents Document Purpose Organization’s Current Year Budget Narrative Guidelines and template for narrative responses Captures broad budget and spending categories for the work Identify how the proposed work under the Project will fulfill the foundation’s Global Access objectives A collection of information about your organization (Note: Please gather the documents below into PDF form, and then submit as one document) A copy of your IRS tax determination letter or certification of formation (depending on what type of entity you are), as different grant terms and conditions may apply Applicable only to LLCs. If you are an LLC, please provide evidence that you are taxed as a partnership, or that you have made an election to be taxed as a corporation Audited financial statements for the past two years, or a proxy if you have not had an audit Your organization’s current-year budget List of Board of Directors List of your board of directors, if applicable 1 Proposal Narrative 2 Budget 3 IP Due Diligence Module 4 Organizational Documents IRS Tax Determination Letter or Certificate of Formation Evidence of Partnership Last 2 Years of Audited Financial Statements (or proxy) 4. Add Contacts tab - Please make sure you add a contact for the person who will sign the grant agreement if you are awarded a grant in order to facilitate the award process. Help Contact(s) Please contact [email protected] for questions. Selection and Award Process Initial Application Review After all applications are submitted, Foundation staff will screen submissions to address whether the proposal addresses the key needs described in the topic and adheres to the application requirements. Applications excluded during the screening process will be notified that their proposals were declined. Page 12 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Application Review, Scoring, and Finalist Notification Next, a panel made up of Foundation and external reviewers will review submissions and issue a preliminary scoring based on the scoring rubric. At this point, a select number of organizations will be notified that they have been designated as finalists. Final Application Review and Winner Selection Organizations selected for investment will be notified in November. We will ask organizations selected for investment to answer a final set of questions that enable us to perform a more detailed due diligence review. For-profit grantees or other entities subject to expenditure responsibility will have to answer a specific set of questions to enable the Foundation to comply with its IRS obligations as a private foundation. For more on expenditure responsibility, please review the ER Guidelines in the application portal. Upon successful completion of this review, finalists will be asked to sign grant agreements with the Foundation. Execution of Grant Terms and Conditions All grant awards will be contingent on execution of a definitive grant agreement. The Representative Terms and Conditions can be found on the Application Portal. These Terms and Conditions have been developed specifically for this RFP and are not negotiable. You are advised to ensure that your institution can accept these Terms and Conditions at the time of proposal submission. Grantees subject to expenditure responsibility may be subject to additional terms and conditions If your proposal is selected, you will have a very limited amount of time after the notification of award to accept the grant and return the award letter with an appropriate institutional signature. You must return a fully executed Grant Agreement to the foundation post-marked no later than the date indicated by the Grants Management team to receive a grant award. Intent and Disclaimer This RFP is made with the intent to identify organizations to build solutions as described in this RFP. The Foundation will rely on an organization’s representations and consider them to be truthful as described. The Foundation assumes it can be confident in an applicant’s ability to deliver the activities described in this RFP. The responses will be incorporated into a future grant agreement should the Foundation wish to support the proposal submitted by the applicant. This RFP is not an offer to enter into a funding agreement. The Foundation assumes no responsibility for your cost to respond to this RFP. Until a written funding agreement is fully executed, the Foundation will have no obligations to any applicant. The Foundation has put in place policies and procedures to restrict public dissemination of grant application materials including, when possible, having external reviewers sign confidentiality agreements and requiring that reviewers destroy or return to the foundation all copies of information acquired or created during the course of performing a review. In some instances, we are unable to put in place confidentiality agreements or to police the use of grant application materials. As a general policy, the Foundation does not publicly disseminate or "publish" proposals or supporting information related to grant applications. For IRS compliance reasons, we are required to publish a list of grants that we have made. We also provide a general description of the grant on our web sites including www.gatesfoundation.org. These brief descriptions are also made available in press releases and other marketing materials. To identify and avert conflicts of interest among reviewers, reviewers will not be permitted to submit proposals, or to review proposals from organizations in which they have self-identified conflicts of interest. Page 13 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal More Information Reference Blank, Steve. The Startup Owner’s Manual: The Step-by-Step Guide for Building a Great Company. Pescadero: K&S Ranch, 2012. Blank, Steve. “Why the Lean Startup Changes Everything.” Harvard Business Review. May 2013. Chatterji, Aaron and Benjamin Jones. 2012. “Harnessing Technology to Improve K-12 Education.” Coalition for Evidence-Based Policy. 2012. “Rigorous Program Evaluations on a Budget.” Costa, Kristina. “Finding ‘What Works’ in Education.” Center for American Progress. 9 Feb. 2012. 15 July 2013. http://www.americanprogress.org/issues/open-government/news/2012/02/09/11053/finding-what-works-ineducation/. Education Market Research. 2012. The Complete K-12 Report®: Market Facts & Segment Analyses . Fullan, Michael and Katelyn Donnelly. 2013. “Alive in the Swamp: Assessing Digital Innovations in Education.” Gage, Deborah. “The Venture Capital Secret: 3 out of 4 Startups Fail.” The Wall Street Journal Online. 19 Sep. 2012. 16 July 2013. http://online.wsj.com/article/SB10000872396390443720204578004980476429190.html Global Silicon Valley Advisors. 2012. “Fall of the Wall: Capital Flows to Education Innovation.” Gray, L., Thomas, N., & Lewis, L. 2009. Teachers' Use of Educational Technology in U.S. Public Schools. Washington, D.C.: U.S. Department of Education. Havens, David. 2012. “A Closer Look at K12 EdTech Venture Funding.” Innovation and Entrepreneurship in Education – NewSchools Venture Fund Blog. 20 Dec. 2012. 15 July 2013. http://www.newschools.org/blog/closer-look IDEO & The Bill and Melinda Gates Foundation. 2013. “Catalyzing Ed Tech: User Journeys.” Lee, Mike. “The Top EdTech Frustrations of Educators.” EdShelf Blog. 9 Apr. 2013. 15 July 2013. http://edshelf.tumblr.com/post/47541656401/the-top-edtech-frustrations-of-educators Maxwell, Lesli A. “Study: Hybrid Algebra Program ‘Nearly Doubled’ Math Learning. Education Week. 20 June 2013. 15, July 2013. http://blogs.edweek.org/edweek/inside-schoolresearch/2013/06/government_study_finds_gains_f.html?cmp=ENL-EU-MOSTPOP mind the gap & The Bill and Melinda Gates Foundation. 2013. “Teachers and Technology.” Purcell, Kristen and Alan Heaps. 2013. “How Teachers are Using Technology at Home and in their Classrooms.” Ries, Eric. “Minimum Viable Product: A Guide.” Startup Lessons Learned. 3 Aug. 2009. 15 July 2013. http://www.startuplessonslearned.com/2009/08/minimum-viable-product-guide.html. Rogers, Everett M. Diffusion of Innovations. 5th ed. New York: Free Press, 2003. The Bill and Melinda Gates Foundation. 2012. “Innovation in Education: Technology and Effective Teaching.” US Department of Education Office of Educational Technology. 2013. “Expanding Evidence Approaches for Learning in a Digital World.” About the Bill & Melinda Gates Foundation Page 14 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal Guided by the belief that every life has equal value, the Bill & Melinda Gates Foundation works to help all people lead healthy, productive lives. We work with partner organizations worldwide to tackle critical problems in four program areas. Our Global Development Division works to help the world’s poorest people lift themselves out of hunger and poverty. Our Global Health Division aims to harness advances in science and technology to save lives in developing countries. Our United States Division works to improve U.S. high school and postsecondary education and support vulnerable children and families in Washington State. And our Global Policy & Advocacy Division seeks to build strategic relationships and promote policies that will help advance our work. Our approach to grant making emphasizes collaboration, innovation, risk-taking, and, most importantly, results. To learn more about the foundation's work, visit www.gatesfoundation.org. Page 15 of 15 © 2013 Bill & Melinda Gates Foundation Request for Proposal
© Copyright 2024