3/30/2015 EUROPEAN STUDIES CENTRE ST. ANTONY’S COLLEGE, OXFORD The Ethics of Accountability in Education Assessment Professor Paola Mattei Associate Professor in Comparative Social Policy University of Oxford [email protected] Symposium: Teacher Ethics in Assessment Oxford University Centre for Educational Assessment (OUCEA) & Ofqual The Wire—Louie and Jane “ I don’t want to go to school…It’s not good…the teachers don’t know anything, they’re mean and tired…like why is there even an 26 March, 2015 St Anne’s College, University of Oxford Public Accountability: why bother? Why is accountability important? Public Accountability and democratic theory (Waldron, 2014) Accountability essential for effective public-private partnership and for market-based collaboration and new tools of government (John, 2011; Ranson, 2003; Christensen and Laegreid, 2007) America?” (Jane) Accountability as institutional mechanism Five formally structured relationships that influence public organization functions and performance Political Central elements of accountability (Mulgan, 2014; Finer, 1941; Mattei, 2012) Delegation of authority from accountor to accountee (agency) Public process (transparency) Consequences (responsiveness) Ministerial accountability (pre-NPM); financial auditing; top political executive controls; accountability for performance and results (“managerial accountability”) Professional Legal Social Principal-agent delegation to elected officials Administrative/managerial Codes, standards, norms Courts Clients, customers, interest groups through media, public panels 1 3/30/2015 Multiple and competing accountabilities Measured Performance Indicators: New Accountability in Government Types of accountability relationships Political Managerial Professional Clear democratic accountability lines from electorate to elected politicians Accountability to owners/shareholders (private) or autonomous agencies if public. Accountability primarily to professional forums and peers Logic Emphasis on broader public good/interest Emphasis on “value for money” Emphasis on medical/ educational evidence Focus Process dimensions (openness, involvement, due process etc.) and politically determined goals Output dimensions: bottom line, business strategy; market based coordination Clinical output/outcome Direction M A new approach to public services governance in the 2000s: targets and measured performance indicators linked to negative feedback/rewards (Osborne and Gaebler, 1992; Chubb and Moe, 1990) Was it a decisive breakthrough in governance – or a partial repeat of the some of the history of the Soviet Union (Bevan and Hood, 2006)? Key question: how far is the world of cheating and output distortions “unethical”? Source: Mattei et al, 2013; Mattei, 2012 Governance by league tables and targets: a necessary evil? Targets threshold standards that a person, organization, country is expected to reach at a specific time They have powerful incentive effects in organizations They help organizations to focus on performance deficits League Tables Use of indicators to compare the performance of different organizations They attract media attention Can encourage good performers to continue Should they be scrapped all together (as in Scotland and Wales in 2001-2002)? Performance management systems: does the removal of league tables matter? 1979-1997 England Quasimarkets V Inspection League Tables Targets V V X Wales V V V X Scotland X X X X 1997-2009 Quasimarkets Inspection League Tables Targets England V V V V Wales V V X( abolished in 2001) V Scotland X V X (in 2002) V 2 3/30/2015 England, Wales and Scotland: Pupils with 5+ A* to C Assessment as an accountability tool GCSEs and SCQF at Level 5 (Mattei, 2012) 65% 1960s: “secret garden of the curriculum” (David Eccles, Conservative Secretary for Education, cited in Timmins, 1995) Late 1980s onwards: accountability through measured assessment and performance indicators (Baird, 2014) 2000s: output targets (based on exam results) and new performance management systems (Lawn, 2014; Mattei, 2012) to address underperforming schools 60% 55% 50% 45% 40% 35% 30% 25% 20% 1993 1995 1997 1999 Wales 2001 2003 England 2005 2007 Scotland Performativity accountability regimes and controversies Pro-reform claims Anti-reform claims External public scrutiny of the teaching Teaching to the tests and negative Information available to parents to bring curriculum reallocation the sanction of ‘exit’ (market Gaming and cheating by teachers accountability) Cream skimming (entry selection) Learning and professional self corrective Schools give up on low performing students and focus on those on the Minimize teacher assessments and stereotyping against ethnic minorities (Burgess and Greaves, 2009) margins Threshold effect incentive to concentrate on meeting the minimum target Narrowing of the curriculum profession through measurable outcomes measures Some problems with targets and PIs e.g. teachers concentrate on narrow band of students on the margins to achieve targets Output distortion incentive to those subject by targets to concentrate on achieving success at the expense of other factors which are not measured by the target Need for broader indicators (child wellbeing) e.g. teaching to the test (at the expense of sports, arts) 3 3/30/2015 Grid-Group cultural theory and four control systems Output distortions and cultural conditions Role of culture (Mary Douglas, 1966; Christopher Hood, 2000; Dan Kahan, 2006) in policy making and administrative process. Design performance indicators sensitive to organisational culture. Why? GRID How individuals and groups respond to measures depends on four types of control systems/cultures Strength of distortions Ofqual survey questionnaire on acceptability of cheating behaviour is a measure of “culture” Fatalism Hierarchy Low group cohesion, apathy, sense of disorder and distrust Oversight through within a hierarchy characterised by strong regulation and rulebound institutions Individualism Egalitarianism Privileges markets, unbridled entrepreneurialism competition, and deregulation Mutuality, solidarity, communal governance, participative decisionmaking GROUP Illustration from another field: gaming the transplant system in the United States Types of actors/motivation (Hood 2007) 1)‘Saints’: who may not share mainstream goals, but whose public service ethos is so high that they voluntarily disclose shortcomings to central authorities 2) ‘Honest triers’: who broadly share mainstream goals and do not voluntarily draw attention to their failures, but do not attempt to spin or fiddle data in their favour 3)‘Reactive gamers’: who broadly share mainstream goals, but aim to spin or fiddle data if they have a motive or opportunity to do so. 4)‘Rational maniacs’: who do not share mainstream goals and aim to manipulate data to conceal their operations (gross misconduct) For a patient in need of an organ transplant, life is a waiting game! Low Group Low Grid—Individualist control system and cultural conditions Majority of hospitals: private non-profits 78% of all medical procedures performed on a fee-for-service basis (incentives volume targets and rewards) Most remunerated procedure (transplant), highest DRGs for hospitals High transplant prices – average liver transplant now more than $577,000 and bonuses for surgeons Policy challenge: do market pressures combined with high prices transplant surgeons command, and financial incentives and bonus rewards encourage unethical cheating behaviour? 4 3/30/2015 Cases of cheating by surgeons: control systems or motivation? Public trust (O’Neill, 2011) and morality in public policy 2002: University of Illinois teaching hospital Federal lawsuits against three centres in Chicago ($2mil to settle the lawsuit) Misleading medical information and inappropriate hospitalisation Admission to Intensive Care Unit to jump the transplant waiting list in the region (Department of Justice, 2003) Public policy discourse of efficiency/accountability is now increasingly tied to morality and trust (Simpson and Baird, 2013) Malpractices in public services may diminish public trust In live organ transplant (scarce supply), this is detrimental-- drop in organ donation has severe implications for patients Strengthening “defensive medicine” and “defensive teaching” Erosion of professional ethos and demoralised professions MELD introduced in 2002 (new medical standards) sharp decrease in admissions to ICUs (Snyder, 2010) Largest decrease in ICU admissions in markets with highest provider competition (controlling for surgeons’ rewards system) Findings: cheating associated with type of target (volume expansion, 12 procedures per year) and type of reward (bonus) Future research and policy challenges: “Ethics and Gaming in Education” Is cheating in the education system caused by regulatory failure or individual human action? What is the relationship between cultural conditions (GridGroup theory) and educational assessment? Do some control systems deliver a better game-proof design? What are the potential effects of changing administrative values and cultures (e.g. increase relational distance between teachers and regulators) on the strength of output distortions? Why ethics and accountability? Move beyond institutional mechanisms and formal arrangements in the field of accountability Understanding accountability through human relationships and social interactions (Bovens, 2002; Dubnick, 2006) Multiple diverse conflicting expectations (MDCE) from policy makers who face dilemmas and make choices among opposing values (Dubnick and Romzek, 1993; Mattei, 2015) Accountability is intrinsically an ethical question. Prestige management increasingly significant with PIs governance Achieve moral status in the eyes of others (not only about bonuses) Avoid moral blame that might result from wrongdoing 5 3/30/2015 Nozick’s ethical theory and public policy: moral pushes and pulls Action as outcome of tensions between moral push and moral pull Moral push—my own values Conclusion If production and economic values internal motivation, individual, that determines own moral conduct based on self-worth Internalisation of values Moral pull—the others’ values external, institutional values and demands, conduct based on the others’ values (structures and procedures) Moral pull of ‘A’ puts a moral constraint on ‘B’ and determines the behaviour of ‘B’ in accordance to ‘A’ values When does ethical action occur? Moral push is equal or greater than the moral pull Then, conflict between compliance and fidelity 6
© Copyright 2024