Assessment I: Issues Jim Ridgway Why Study Assessment? I. What you test is what you get... Externalise your values Set new educational agendas Defend yourself! "Most of the teachers in this study were caught in conflicts among belief systems, and institutional structures, agendas, and values. The point of friction among these conflicts was assessment, which was associated with very powerful feelings of being overwhelmed, and of insecurity, guilt, frustration, and anger. This study suggests that assessment, as it occurs in schools, is far from a merely technical problem. Rather, it is deeply social and personal." (US secondary language teachers-Johnston et al., 1995) Black Box p13 Why Study Assessment? II. There are BIG educational gains associated with better assessment Black and Wiliam (1998) – reviewed 580 publications ‘All… studies show that… strengthening… formative assessment produce significant, and often substantial, learning gains. These studies range over ages (from 5-year olds to university undergraduates), across several school subjects, and over several countries… typical effect sizes…between 0.4 and 0.7’. P3 of ‘Black Box’ Effects are often most noticeable for low attaining pupils BUT…There are no ‘magic bullets’ in education Effect Size? • An effect size of 0.4 would mean that the average pupil involved in an innovation would record the same achievement as a pupil just in the top 35% of those not so involved. • • A gain of effect size 0.4 would improve performances of pupils in GCSE by between one and two grades. • • A gain of effect size 0.7, if realised in the recent international comparative studies in mathematics (TIMSS), would raise England from the middle of the 41 countries involved to being one of the top 5. What? How? Why? Who tests whom? For whom? Driving Test Neonatal Assessment Job Interview Choosing a University Course Assessment in Education Students Teachers Parents Schools Curriculum Design Government Big Ideas in Assessment Reliability and Validity Summative and Formative Assessment Diagnosing and addressing pupil understanding Diagnosis Beware of interview demand characteristics! Look for: Use in everyday language Phenomenological evidence Existence in the history of science Use multi-methods: Interview Objects Drawing Examples in Mathematics Examples in Science Examples in Mathematics Multiplication makes bigger 6 * 0.9 = ?? Understanding place value 1.3 To multiply by 10, just add a zero Fractions … 1.6 1.9 ?? Student Conceptions and Misconceptions? "Blood flows down one leg and up the other." "Dew is formed on leaves when the sun shines down on them and makes them perspire." "A super-saturated solution is one that holds more than it can hold." "Mushrooms always grow in damp places and so they look like umbrellas." "The skeleton is what is left after the insides have been taken out and the outsides have been taken off. The purpose of the skeleton is something to hitch meat to." "The tides are a fight between the Earth and moon. All water tends towards the moon, because there is no water in the moon, and nature abhors a vacuum. I forget where the sun joins in this fight." "Water is composed of two gins, Oxygin and Hydrogin. Oxygin is pure gin. Hydrogin is gin and water." "Respiration is composed of two acts, first inspiration, and then expectoration." Inside the Black Box A poverty of practice "Marking is usually conscientious but often fails to offer guidance on how work can be improved. In a significant minority of cases, marking reinforces under-achievement and under-expectation by being too generous or unfocused. Information about pupil performance received by the teacher is insufficiently used to inform subsequent work." (OFSTED general report on secondary schools 1996, p.40. ) Government Initiatives in Testing National curriculum testing Development of the GCSE League tables of school performance more frequent assessment, and thorough inspection UK Government has REDUCED teacher assessment Does more pressure on the classroom lead to better education? Might make it HARDER Unfair? Classroom as Black Box Literature review of problems with assessment Teachers' tests encourage rote and superficial learning The questions and methods are not discussed or shared between teachers • For primary teachers particularly, there is a tendency to emphasise quantity and presentation of work and to neglect its quality in relation to learning. • Giving marks is over-emphasised, while the giving of useful advice is underemphasised. Pupils are compared with one another - the prime purpose appears to them to be competition rather than personal improvement. Feedback teaches pupils with low attainments that they lack 'ability' Pupils look for ways to get better grades, not necessarily how to learn the materials From the Research Literature - Effective Formative Assessment Involves Changes in classroom practices more feedback between teacher and student real use of the feedback to change things Active involvement of students in their learning Attention to student motivation and self-esteem Focus on the quality of teacher-pupil interactions Understanding pupil understanding Guiding and remediating Encouraging generalisable ‘learning habits’ – taking responsibility for learning This is a BIG DEAL – from Transmission model to Constructivist - and will be very hard to achieve Culture change will be slow Inside Paul Black’s Black Boxes Twenty four science and mathematics teachers, drawn from six comprehensive schools in two local education authorities (LEAs), Medway and Oxfordshire Introduce the research findings suggest ideas for innovations in their classroom practices lesson observations and discussions help teachers reflect on their innovations twelve one-day meetings of the whole group, teachers with researchers and with LEA officers, to reflect on progress, share ideas teachers wrote reflections on the work Changes in teachers’ practices: Questioning Research shows that the average ‘wait time’ is less than one second. I’d become dissatisfied with the closed Q&A style that my unthinking teaching had fallen into, and I would frequently be lazy in my acceptance of right answers and sometimes even tacit complicity with a class to make sure none of us had to work too hard.....They and I knew that if the Q&A wasn’t going smoothly, I’d change the question, answer it myself or only seek answers from the ‘brighter students’. There must have been times (still are?) where an outside observer would see my lessons as a small discussion group surrounded by many sleepy onlookers. (James, Two Bishops School) Most teachers found it hard to break long-standing habits in the classroom: Increasing waiting time after asking questions proved difficult to start with - due to my habitual desire to 'add' something almost immediately after asking the original question. The pause after asking the question was sometimes 'painful'. It felt unnatural to have such a seemingly 'dead' period but I persevered. Given more thinking time students seemed to realise that a more thoughtful answer was required. Now, after many months of changing my style of questioning I have noticed that most students will give an answer and an explanation (where necessary) without additional prompting (Derek, Century Island School) Questioning • My whole teaching style has become more interactive. Instead of showing how to find solutions, a question is asked and pupils given time to explore answers together. My Year 8 target class is now well-used to this way of working. I find myself using this method more and more with other groups. No hands • Unless specifically asked pupils know not to put their hands up if they know the answer to a question. All pupils are expected to be able to answer at any time even if it is an ‘I don’t know’.. Supportive climate • Pupils are comfortable with giving a wrong answer. They know that these can be as useful as correct ones. They are happy for other pupils to help explore their wrong answers further. (Nancy, Riverside School) Not until you analyse your own questioning do you realise how poor it can be. I found myself using questions to fill time and asking questions which required little thought from the students. When talking to students, particularly those who are experiencing difficulties, it is important to ask questions which get them thinking about the topic and will allow them to make the next step in the learning process. Simply directing them to the 'correct answer' is not useful. (Derek, Century Island School) Summary • teachers shifted role from presentation, to exploration of student ideas, involving everyone • students become more active as participants, and express and discuss their own understanding • teachers spend more effort on framing questions to explore issues critical to the students’ development of understanding Changes in teachers’ practices: Marking and Comments Students ignore comments when marks are also given (Butler, 1988). Many teachers were led to abandon the giving of marks or grades. My marking has developed from comments with targets and grades, which is the school policy, to comments and targets only. Pupils do work on targets and corrections more productively if no grades are given. CL (a researcher) observed on several occasions how little time pupils spent reading my comments if grades are given as well. My routine is now, in my target class, to: i) not give grades, only comments ii) comments highlight what has been done well and what needs further work iii)the minimum follow up work expected to be completed next time I mark the books (Nancy, Riverside School). Crazy?? At no time during the first fifteen months of comment only marking did any of the students ask me why they no longer received grades. It was as if they were not bothered by this omission. I found this amazing, particularly considering just how much emphasis students place on the grades and how little heed is taken of the comments generally. Only once, when the class was being observed by a member of the King's College team did a student actually comment on the lack of grades. When asked by our visitor, how she knew how well she was doing in Science, the student clearly stated that the comments in her exercise book and those given verbally, provide her with the information she needs. (Derek, Century Island School) More Work? After the first INSET I was keen to try out a different way of marking books to give pupils a more constructive feedback. I was keen to try and have a more easy method of monitoring pupils’ response to my comments without having to trawl through their books each time to find out if they’d addressed my comments. I implemented a comment sheet at the back of my Yr 8 class’s books. It was A4 in size and the left hand side is for my comments and the right hand side is for the pupils to demonstrate by a reference to the page in their books where I can find the evidence to say whether they have done the work. ............................................................... The comments have become more meaningful as the time has gone on and the books still only take me one hour to mark. (Sian, Cornbury Estate School) Significant features of the changes in practice: • teachers changed their view of the role of written work in promoting students’ learning; • teachers felt challenged to compose comments about the subject matter and about the learning needs of each individual student; • teachers have had to give more attention to differentiation in the feedback given to students; • students have changed in their perception of the role of their written work as part of their learning. Self-assessment and peer-assessment We regularly do peer marking - I find this very helpful indeed. A lot of misconceptions come to the fore and we then discuss these as we are going over the homework. I then go over the peer marking and talk to pupils individually as I go round the room. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .. . . . . . . . . . . . . . . Pupils regularly read their own work or another pupil’s as a matter of course. This has made them realise how important it is to write clearly. Previously I would have said that I could not read their work - their peers saying they cannot read the writing has more of an impact. (Rose, Brownfields School) Revision They did not mention any of the reviewing strategies we had discussed in class. When questioned more closely it was clear that many spent their time using very passive revision techniques. They would read over their work doing very little in the way of active revision or reviewing of their work. They were not transferring the active learning strategies we were using in class to work they did at home. (Tom, Riverside School) One teacher describes how he started to change this situation : One technique has been to put the students into small groups and give each student a small part of the unit to explain to their colleagues. They are given a few minutes preparation time, a few hints, and use of their exercise books. Then each student explains their chosen subject to the rest of their group. Following the explanation, each of the other students then gives the speaker a ‘traffic light’ mark – Green means it was explained better than they could have done so themselves. Amber means it was explained at about the level they would have been able to do, and red means that they feel it was not explained as well as they feel they could have explained it. Students are very honest in doing these, and in doing the follow up, where all those who gave a green or a red mark justify their ‘score’. Here students are quick to point out things such as, ‘I thought that the examples you chose to explain were very good as they were not ones in our books, so I don’t think I would have thought of those.’ Or, ‘I expected you to mention particles more when you were explaining the difference between liquids and gases.’ These sessions have proven invaluable – not only to me, in again being able to discover the level of understanding of some students, but to the students too. (Philip, Century Island School) Sharing The idea of using the ‘traffic lights’ icons spread rapidly across the other teachers and schools in the project. A teacher can identify at a glance the main learning difficulties that have arisen without lengthy interrogation of each student individually. It also facilitates communication between students Ask Students to set test questions Pupils have had to think about what makes a good question for a test and in doing so need to have a clear understanding of the subject material. As a development of this, the best questions have been used for class tests. In this way the pupils can see that their work is valued and I can make an assessment of the progress made in these areas. When going over the test good use can be made of groupwork and discussions between pupils concentrating on specific areas of concern (Angela, Cornbury Estate School) The changes initiated by the focus on self- and peer-assessment may be summarised as follows : Students learn to • • • • • assess their own and one another’s work develop criteria for judging its quality develop the habits and skills of collaboration in learning come to see that tests can play a positive part in reviewing learning become participants, rather than victims, in the testing process Lessons for us as learners The central role of assessment in education Collaboration Mutual monitoring and feedback Awareness of sources Lessons for us as teachers The central role of assessment in education Collaboration Mutual monitoring and feedback Awareness of sources Some ideas to try in class Awareness of the ‘situated’ nature of educational practices Search for good questions – and share them (especially with me!) Read Hargreaves, and Black and Wiliam– teachers should be major players in advancing knowledge about education