Evaluation Capacity Development (ECD) project – Uganda “How To”: Develop a Theory of Change (TOC) By Olga Nakato Mugerwa Uganda Management Institute (UMI) 1. Introduction This guidance note expounds on the Theory of Change (TOC) as used in development and social programming for evaluation practitioners. It is a methodological approach to understanding the theory of Page | 1 any initiative, and is pivotal in informing the evaluation’s purpose and focus, arriving at the key questions that it will address as well as in helping to derive the selection methods. 1.1. What is a “Theory of Change”? A THEORY OF CHANGE (TOC) is a description of how an intervention is believed to deliver the desired results. It is a representation of how and why a complex change process will succeed under specific circumstances. TOCs embody beliefs about what is needed by the target population and what strategies will enable them to meet those needs. They establish a context for considering the connection (causation) between a system’s mission, strategies, and actual outcomes, while creating links between who is being served, the strategies or activities that are being implemented, the assumptions, and the desired outcomes. The TOC provides the following: i. ii. iii. A big picture analysis of how change happens in relation to a specific thematic area; An articulation of an organisation or program pathway in relation to how change will occur; An impact assessment framework which is designed to test both the pathway and the assumptions made about how change happens. Because of these components, TOCs are usually presented in the form of a diagram and these can take any format as long as they aid in the process of uncovering and developing the assumptions within the TOC. Understanding the TOC of any program or project is essential for carrying out any evaluation, given the cause and effect focus of an evaluation research. 1.2. Why develop a Theory of Change? Similar to the definition of TOC, there is a variety of purposes for using a TOC. Nonetheless, it is a given reality that in our everyday lives we hold beliefs (theories) that guide our analysis of problems and the way in which we respond to them. Having these theories of change therefore helps to clarify problems and provides a framework on how to address them. TOCs help us to plan development interventions, as well as aiding us when it comes to evaluating these interventions. Specifically: i. ii. iii. iv. TOCs help move stakeholders from being passive collectors and reporters of information to active users of information for system planning and service delivery; TOCs help system and program staff better understand the kind of evaluation information they need in order to make day-to-day decisions; TOCs help the evaluator develop research questions that focus measurement on changes that can occur given the particular strategies that are operative at the system, program, and client level; Because they facilitate understanding the link between strategies and the achievement of outcomes, TOCs facilitate the integration of data from broader evaluation and accreditation requirements into local evaluation efforts. TOCs are best used when one desires to: i. ii. iii. Design a complex initiative and have a rigorous plan for success; Evaluate appropriate outcomes at the right time and in the right sequence: this may involve in some instances reconstructing a TOC for an intervention to better understand/assess outcomes and impact; Explain why an initiative worked or did not work, and what exactly went wrong. Evaluation Capacity Development (ECD) project – Uganda 2. Theory of Change and Logic Models Logic models focus on aligning the component parts of a program into a hierarchy of clearly-specified goals, outcomes/objectives, outputs/results, inputs/activities, (usually) together with a set of measurable indicators to demonstrate progress. For instance, a results chain is a logical linkage of a transformative process that describes how inputs are used to carry out activities, these activities will produce outputs which in turn will produce outcomes Page | 2 and finally how these outcomes will produce impact. Inputs Activities • Financial, Human & other Resources Mobilised to support activities Budgets, Staffing + other available Resources Outputs • Actions taken or work performed to convert inputs into specific outputs Outcomes • Products resulting from converting inputs into tangible outputs Series of activities undertaken to produce goods and services Implementation (Supply Side) Goods and services produced and delivered, under the control of the implementing agency Final Outcomes • use of outputs by target population • The final objective of the program. Long term goals Not fully under the control of the implementing agency Changes in Outcomes with multiple drivers Results (Demand + Supply) A TOC, by contrast, is broader. It seeks to establish the “miracle” that happens between the components of the results chain and logical framework. It lays out an ultimate goal (sometimes called a ‘long-term outcome’, ‘vision’, or ‘mission’) which is broken down into a causal pathway with preconditions, intermediate outcomes, and assumptions for each step along the way. It is usually presented as a flow chart. Secondly, Logic Models require that program components are identified so that it is possible to observe at a glance if outcomes are out of sync with inputs and activities. It does not explain why activities are expected to produce outcomes. The TOC on the other hand requires justification at each step, i.e. asking why and how at each step. A hypothesis about why something will cause something else needs to be articulated. This underlies why it is often termed as a causal model. Thirdly, Logic Models do not always identify indicators, whereas TOCs require identification of indicators. This is so because it is necessary to understand within the TOC causal pathway, how well a precondition needs to be met in order to get to the next goal. In summary: Logic Models  Representation  List of Components  Descriptive Theories of Change  Critical Thinking  Pathway of Change  Explanatory Nonetheless, TOC approaches can and are often used together with Logic Models. 3. Steps in developing a Theory of Change It is crucial to take into consideration that there is no set or standardized format for deriving or arriving at a TOC. However there are common elements. 1. Identifying the Goal: at times referred to as the Long-term Outcome or Vision. This should be clearly articulated in a specific statement on what goal the organisation/group is working on and as to how success will be identified. Evaluation Capacity Development (ECD) project – Uganda 2. Develop a Pathway of Change1: TOC uses backwards mapping requiring planners to think in backwards steps from the long-term goal to the intermediate outcomes and then early-term changes that would be required to cause the desired change. This creates a set of connected outcomes known as a “pathway of change”. A “pathway of change” graphically represents the change process as it is understood by the initiative planners and is the skeleton around which the other elements of the theory are developed. Note that during the backwards mapping process the intermediate outcomes that link the early and long-term steps should be as strong and robust as possible, and ideally grounded in a body of best practice. 3. Define interventions: Identify the activities and outputs that will lead to each identified outcome, it is prudent to select which outcomes are plausible and within the means of the project to deliver upon. 4. Articulate assumptions: In creating the pathway of change one is required to articulate as many assumptions2 about the change process as plausible so that they can be examined. There are typically three important types of assumptions to consider: a) Assertions about the connections between long term, intermediate and early outcomes on the causal map; b) Substantiation for the claim that all of the important preconditions for success have been identified, and; c) Justifications supporting the links between program activities and the outcomes they are expected to produce. A fourth type of assumption which outlines the contextual or environmental factors that will support or hinder progress toward the realization of outcomes in the pathway of change is often an additional important factor in illustrating the complete TOC. The principle for a TOC is to include an explanation of how the intervention’s activities contribute to the results – not simply a list of activities followed by the results, with no explanation of how these are linked apart from the arrow. The TOC should articulate the theories and assumptions which underpin the anticipated change process. The TOC should be presented both in a diagram to give a visual summary and also a narrative text which gives more detail on each of the results, causal links and assumptions. 3.1 Pitfalls to be cautious about while developing a TOC 1. Confusing accountability with hope. A TOC must clarify what results must be delivered to be successful. Defining results in this way will force your organization to get real about the impact you are signing up to create, not just what you hope will happen. While dreaming big and setting lofty goals, such as ending world hunger, can inspire your stakeholders, these are better left for your mission statement rather than your TOC. 2. Creating a mirror instead of a target. A good TOC does not simply reflect what an organization is already doing; rather, it articulates what the organization wants to be held accountable for, and works backward to identify necessary activities, strategies, resources, capabilities, culture, and so on. 3. Failing to take the external context into account. The best TOCs explicitly integrate the anticipated actions of regulators, the work of peer organizations, expected changes in the economic climate, and other factors. A deep understanding of the external context will help you create a more realistic TOC. 1 In some literature referred to as the “Causal Pathway”: which is a step by step, backwards mapping process through which a group determines all the preconditions necessary to reach an ultimate vision or goal. 2 Assumptions within the TOC formation process refer the necessary conditions for change, or the underlying conditions or resources that need to exist for the planned change to occur. The identifying and questioning of assumptions is generally deemed central to the process of articulating a TOC. Page | 3 Evaluation Capacity Development (ECD) project – Uganda 4. Not confirming the plausibility of your theory. While internal dialogue is a common starting point for TOC development, the process should not conclude without a concerted effort to verify whether your “theory” is plausible. 5. Creating a theory that is not measurable. To be able to test, refine, and improve your theory of change over time, you need to be able to measure its key elements. A common way to operationalize your TOC is to get specific — articulate the input, output, and outcome indicators the Page | 4 TOC suggests you should track. If you cannot actually gather these indicators at the right frequencies to learn what is working, you may not be specifying your TOC at a deep enough level. 6. Assuming you have figured it all out. To get the most out of your theory of change, you need to recognize and explicitly account for the uncertainties that underlie your plan. Learning organizations carefully specify their assumptions, regularly reflect on whether those assumptions are bearing out, and consider what new assumptions they might test to further improve impact. An example of a TOC diagram Goal •Assumption Outcome Outcome Outcome Intermediate outcomes Pre conditions Assumption Strategies Pre conditions Assumption External contextual assumptions 4. Conclusion TOCs are now being used more and more widely, especially for evaluation research, both as a substitute and as a complement to more familiar logic models. This guidance note has provided a workable definition of what a TOC is, provided a sketch on how to arrive at a TOC, expounded on when to use a TOC as well as explained the difference between Logic Models and TOC. References and Recommended Readings − Center for Theory of Change. (n.d.) Toco software. Center for Theory of Change. Available from: www.theoryofchange.org/toco-software/. − Anderson, A.A. (2005). The community builder’s approach to Theory of Change. The Aspen Institute. Available from: www.seachangecop.org/node/215. − O’Flynn, M. (2012). Theory of change: What’s it all about? Ontrac: The newsletter of intrac. International NGO Training and Research Centre (INTRAC). Available from: www.seachangecop. org/node/566 − DFID Working Paper series 38- Broadening The Range Of Designs And Methods For Impact Evaluations (2012): Elliot Stern, Nicoletta Stame, John Mayne, Kim Forss, Rick Davies, Barbara Befani. − Understanding ‘Theory Of Change’ In International Development (2012); A Justice and Security Research Program (JSRP) Paper Series: By Danielle Stein And Craig Valters.