1. education

 What do institution leaders do that is effective in facilitating quality improvements in teaching and learning? A literature review conducted by the Centre for the Use of Research and Evidence in Education on behalf of the Learning and Skills Improvement Service 1
Contents Introduction........................................................................................................................ 3 Calibrating the impact of leadership on teaching and learning ....................................... 5 Leadership practices that have an impact on teaching and learning............................... 6 Developing people ......................................................................................................... 6 Managing the teaching programme .............................................................................. 7 Setting directions ........................................................................................................... 9 Relationships ................................................................................................................ 10 Implications of this review for approaches to leadership within the further education system............................................................................................................................... 12 Methods for conducting the literature review ............................................................... 14 References ........................................................................................................................ 15 Appendix I ‐ Leadership literature review search strategy ............................................ 16 2
Introduction There is a large body of research – both in the UK and internationally into the characteristics of organisational leadership in general, with a substantial subsection focusing on school leadership in particular. A specific focus on leadership in the further education system (FES) is relatively rare. However recent research in England by PriceWaterhouseCoopers (PWC) on behalf of the DCSF (2007) has established that organisational leaders, whether of schools, private sector organisations or higher and further education all face a constantly changing environment in which accountability and value for money have become increasingly important. Research by the Centre for Excellence in Leadership (CEL) makes similar links. However, this review, in exploring the leadership of teaching and learning, chose to focus in particular on studies that set out to make connections between leadership contributions and student success – however complex and great the intervening variables. This is an ambitious endeavour but one we thought was crucial in this particular context. An exploration of current research evidence into effective leadership behaviours and characteristics is particularly relevant to the current emphasis on a whole organisation approach to quality improvement in FE. In addition, leaders are being asked to take on new responsibilities as the FES moves towards greater self‐regulation. This means developing greater understanding within organisations of the kinds of leadership characteristics and activities that lead to positive outcomes for learners. This review set out to explore the strategies and practices that the evidence suggests are effective in doing this. After consultation with leaders and practitioners from all sectors of the FES, and with LSIS, we developed the following review question: What do institution leaders do that is effective in facilitating quality improvements in teaching and learning? For the purposes of this review ‘institution leaders’ included principals, senior managers and departmental leaders within FE, universities and schools. Evidence for the review was drawn from six studies based on the level of detail they provided on what constitutes effective leadership, and the robustness of the research. Two of the studies focus specifically on post‐16 education (Estyn, 2007; Muijs et al, 2006), one on effective leadership in higher education institutions (HEIs) (Gibbs et al, 2007), and three draw on studies with a school focus (Robinson, 2007; Leithwood et al, 2006; Waters et al, 2003). While every attempt was made to identify substantial studies on leadership in post‐16 education, we have included the three school‐centred studies because of the quality of the evidence they present and the efforts taken by their authors to identify the links between the practices of leaders and student outcomes. 3
The depth and scope of the exploration of leadership and student outcomes in the school‐centred studies was not present to the same degree in any of the post‐16 studies identified in our searches. Two of the reports included in this review (Robinson, 2007; Waters et al, 2003) are meta‐analyses. The researchers based their findings on studies which measured student outcomes and described leadership actions and characteristics. They established an effect size for each leadership element based on student data, and then an average effect size for each element across the studies. Elements of leadership were grouped in ‘dimensions’ such as ‘communication’ or ‘strategic resourcing’, so the researchers could report on their impact on student learning. We refer to the effect sizes of leadership dimensions from these meta‐analyses throughout the review. In order to gauge the importance of a dimension in terms of effect size Robinson suggests the following general guide: effect size interpretation < 0.2 no effect or a weak effect 0.2 – 0.4 small but possibly educationally significant effect 0.4 – 0.6 a moderate educationally significant impact > 0.6 a large and educationally significant impact. (2007: 9) In our report we outline the evidence of effectiveness in terms of teaching and learning in the organisations involved and highlight the findings about institutional leadership in terms of the facilitation of quality improvements. We first present an overview of what the studies had to say about the impact of leadership on teaching and learning in general. We then describe the activities and characteristics of institutional leadership under the following headings taken from our data: • developing people • managing the teaching programme • setting directions • relationships The sections are organised in the order of the degree of impact the studies suggest each leadership category has on student outcomes. So those with evidence of the strongest impact appear first. Where there is a range of effect size we have used the Robinson effect size to determine the order a characteristic appears in the review – because of the methodological rigour in the approach to recalculating effect sizes. The report concludes by explaining a small number of the implications of the evidence for leadership development in the FES. 4
Calibrating the impact of leadership on teaching and learning The studies in this review took differing approaches to establishing the impact of leaders’ practices on teaching and learning. Muijs et al selected sites for their study based on performance data which showed high levels of improvement over a three year period, and on successful leadership within institutions evidenced by high inspection grades. Similarly, Estyn used inspection judgements to identify effective leadership practice. Their analysis led them to the conclusion that “ineffective leadership and strategic management often leads to weaknesses in many areas of an organisation’s work. This has a significant impact on the quality of teaching and learning, the standards that learners achieve and the value for money that the organization provides.” (2007: 11) Gibbs et al identified for the focus of their study university departments deemed to be “outstanding at teaching” based on a range of internal and external performance indicators. These included student grades, progression and retention, and the adoption by other universities of the department’s teaching methods. They then visited the sites and interviewed staff and students to identify what role, if any, leaders had played in creating and supporting excellent teaching. The link between leadership, teaching and student outcomes was therefore established through a mix of outcomes data and reports from a range of perspectives. In the event, the researchers rejected two cases where staff and student report provided little evidence of leadership playing a major role in creating teaching excellence. According to Leithwood et al the research about leadership identifies different scales of impact depending on how and where it is carried out. Whereas small scale case studies have tended to focus on exceptional school settings and usually report large leadership effects, large‐scale quantitative studies have identified small but educationally significant impact on student outcomes. Leithwood et al estimate that in terms of what schools do to affect student learning, leadership factors account for about one quarter of variation in student achievement. By way of comparison, what happens in the classroom accounts for about a third of the variation. Robinson included in her meta‐analysis 22 studies which drew on students’ academic achievement and four which reported on social and personal well‐being outcomes. The latter included students’ attitude to learning, teachers and school; and how the students viewed themselves as learners. Waters et al, on the other hand, restricted their analysis only to studies which measured student achievement using objective measures, such as standardised tests. Waters et al found the average effect size between leadership and student achievement to be 0.25. 5
Leadership practices that have an impact on teaching and learning Developing people Leaders’ involvement in professional development and learning Robinson’s vigorous recalculation of the effect of a large number of leadership studies showed that the most important thing that leaders do to support learning in their institutions is to involve themselves in the professional learning and development of their staff. She found the average effect size to be 0.84 in institutions where the leadership “not only promotes, but directly participates with teachers in, formal or informal professional learning”. Direct involvement in CPD enabled leaders to keep on developing their own interests and skills, and also share ideas with colleagues and stimulate thinking. In the words of Gibbs et al “Developing excellent teaching and maintaining that excellence usually involved a great deal of talking about teaching” (2007: 2). This aligns with Leithwood’s et al’s “intellectual stimulation” dimension, in which leaders encourage colleagues: “to take intellectual risks, re‐examine assumptions, look at their work from different perspectives, rethink how it can be performed.” (2006: 37) Waters et al evaluated the impact of “intellectual stimulation” on student outcomes, and came to an effect size of 0.32. Intellectual stimulation meant leaders: • keeping abreast of current education research and theory • exposing staff to those ideas and systematically engaging them in discussion, and • encouraging staff to read articles and books about effective practice. Gibbs et al also found leaders in university departments placed much importance on teachers reflecting on and evaluating their practice. In this context there was an explicit connection between reflection on practice and performance review procedures. In some institutions teachers’ review of practice occurred more formally in the light of student performance and feedback ratings, and reflective accounts could also form the basis of salary reviews. Devolving responsibility and appointing staff Leaders in organisations that were effective in securing student success made specific efforts to support staff to take on the responsibilities their roles demanded. This 6
entailed not only helping staff to develop the knowledge and skill needed to accomplish organisational goals, but also the “commitment…resilience [and] the dispositions to persist in applying that knowledge and skill” (Leithwood et al, 2006: 36). Involving staff in decision‐making was one way of doing this, and was linked to improving student outcomes. Waters et al found the effect size for “involving teachers in the design and implementation of important decisions and policies” to be 0.30. Estyn also identified as a representative trait of effective FE institutions the motivation that follows involvement in decision‐making: “Staff accept that they are accountable for the quality of their work and they try actively to improve their performance as teachers and leaders so that learners can make the best possible progress.” (2007: 8) The creation of new coordinating posts and the establishment of teaching development groups was also a way of helping staff to move into more demanding and rewarding spaces (Gibbs et al, 2007). Leaders’ close interest in their staff as professionals began right at the recruitment stage. Robinson cites a US study which found a correlation between school principals’ direct involvement in the selection process and student achievement: “student achievement in schools where principals appointed a higher percentage of their teaching staff was higher than in otherwise similar schools where principals had appointed a smaller percentage of their staff.” (2007: 12) However, Robinson does add the caveat that this was only the case in institutions where principals ranked academic goals highly, “for principals who ranked them lower, the reverse was apparent.” Managing the teaching programme Direct involvement in teaching and learning by institution leaders was described across the studies, irrespective of the sector or phase, as a central activity to ensuring positive outcomes for students. Leaders’ concern for quality in teaching and learning meant they led on changes in teaching if they themselves were proficient practitioners, or they deployed colleagues who were, and who had the requisite knowledge and skills to lead changes in teaching (Gibbs et al, 2007). For the most part Gibbs et al found that most of the leaders in their study were recognised as effective teachers: “Several [leaders] had developed sufficient expertise in leadership of teaching that they were much in demand to give lectures and run seminars for other heads of department, sometimes on a national or international scale.” (2007: 4) 7
In the further education sector in Wales, Estyn also identified the importance of leaders involving themselves in the core business of teaching and learning: “The best leaders and managers make sure there is teaching and learning of good quality so that learners get the qualifications they set out to achieve. Where leadership and strategic management are weak, it often makes learners and institutions less successful than they could be.” (2007: 7) Both Robinson and Waters et al found statistical evidence to support the importance of leaders’ involvement in teaching and learning matters. Robinson calculated the effect size for leaders “planning, coordinating and evaluating teaching and the curriculum” at 0.42. This dimension included: • support and evaluation of teaching through regular classroom visits • provision of formative and summative feedback to teachers, and • direct oversight of curriculum through school‐wide coordination across classes and year levels and alignment to school goals. For Waters et al the effect size for this dimension was considerably lower at 0.16. The difference between the effect sizes in the two meta‐analyses may be attributable to Waters et al’s narrower conceptualisation for this dimension. They define direct involvement as “the extent to which the principal is directly involved in the design and implementation of curriculum, instruction, and assessment practices”. This includes such activities as leaders working with practitioners to address teaching and assessment issues, and ensuring teachers are resourced with adequate equipment and materials. Leaders’ intelligence about how teachers were performing came via ongoing monitoring and evaluation (Robinson, 2007; Waters et al, 2003), and also through student feedback (Gibbs et al, 2007). The latter was a particularly strong feature in HEIs: “Departments did not simply collect student feedback. They involved students in course planning committees and review processes and used them as collaborative evaluators of innovation and even as tutors in implementing new pedagogies. One head interviewed every departing student personally.” (2007: 2) For leaders to manage teaching and learning effectively assumes familiarity with pedagogic theory and processes. Waters et al found the correlation between leaders’ knowledge of curriculum, instruction and assessment and learner outcomes to equate to an effect size of 0.24. This dimension included leaders providing “conceptual guidance for teachers regarding effective classroom practice”. The importance of being knowledgeable about teaching and learning manifested itself in other ways. Robinson, 8
for example, cites on study that found that school principals “were significantly more likely to be nominated as sources of advice in higher achieving schools” (2007: 16). Being knowledgeable professionals meant that leaders also gained the respect of their colleagues (Gibbs et al, 2007). Leaders’ effective co‐ordination of teaching and learning entailed understanding of both current practice of individual staff and departments, and their planned trajectory for the future. This was important for several reasons, as it: • enabled leaders to bring into the open and address problems (Gibbs et al, 2007) • facilitated strategic selection of additional resources so that teaching staff were not overloaded with initiatives or detracted from planned improvements (Robinson, 2007; Waters et al, 2003), and • informed staff recruitment decisions (Waters et al, 2003). Leaders of effective departments were knowledgeable professionals who had gained the respect of their colleagues (Gibbs et al, 2007). Setting directions The formation of values and beliefs, and putting them into practice were described across the studies as an important foundation for effective leadership. The research suggests that it is not enough to simply claim a value, but in order to have an impact on outcomes, leaders need to: • behave in ways that are consistent with their beliefs (Waters et al, 2003; Leithwood et al, 2006) • translate their vision for their institution into concrete goals (Estyn, 2007; Robinson, 2007; Waters et al, 2003), and • communicate these clearly (Estyn, 2007; Robinson, 2007; Waters et al, 2003). Holding strong ideals and being clear about the direction an organisation is moving in were found to be linked to student outcomes by Waters et al, who calculated an effect size of 0.25. Communicating their vision meant leaders shared beliefs about education, teachers and learning with staff and parents. Translation of the vision into a strategy and specific goals was also important. The vision provided the common principles and key ideas which in turn informed institutional priorities (Robinson, 2007). On a practical level, being clear about the principles they were working to enabled leaders to be strategic about the selection and allocation of resources, and to do this effectively (Waters et al, 2003). Effective leaders made clear their expectations to staff. The two meta‐analyses found the effect size for establishing, monitoring and keeping a focus on goals was 0.24 9
(Waters et al, 2003) and 0.35 (Robinson, 2007). In successful further education organisations leaders operationalised such expectations by: • setting out action plans, and • using targets to make sure planned changes occurred swiftly to minimise disruption for learners (Estyn, 2007). Relationships The extent to which establishing good working relationships both with and among their staff appeared to vary depending on whether professional or personal relationships were the focus of analysis. Waters et al calculated an effect size of 0.33 for the dimension of ‘situational awareness’ – that is the extent to which leaders are “aware of informal groups and relationships among staff”. On the other hand, they found that “the extent to which the principal demonstrates an awareness of the personal aspects of teachers and staff” had an effect size of 0.19. This ties in with the finding that “the extent teachers identified principals as close personal friends or as participants in discussions was not significantly related to student outcomes” (Robinson, 2007: 16). Other studies describe the importance of building relationships in broader terms. Leithwood et al, for example, cite a large body of research indicating the central role collaboration plays in school improvement, and the need for leaders to create collaborative cultures. Central to relationship building was establishing and keeping open effective lines of communication. This meant leaders: • were easily accessible to teachers • developed effective means for teachers to communicate with one another, and • maintained open and effective lines of communication with staff (Waters et al, 2003). Where well functioning professional relationships existed there was a higher likelihood of trust between colleagues (Leithwood et al, 2006), which in turn has an impact on organisational improvement. In discussing the leadership dimension of “ensuring an orderly and supportive environment” Robinson cites large scale research in Chicago schools which found a strong link between improvements in relational trust and academic outcomes. The researchers argued that ‘relational trust’ is a key element in the school improvements they observed, and that this requires “a willingness to be vulnerable under conditions of risk and interdependence” (2007: 18). The studies provided some detail on what leaders do to create productive relationships. These can be summarised in two main categories: • distribution of leadership 10
•
establishing an orderly environment. Distribution of leadership Muijs et al (2006) defined three broad types of leadership: • transactional leadership – concerned mainly with exchange, eg of financial rewards for extra employee effort • transformational leadership – which brings about change in individuals and organisations through an appeal to values and long‐term goals, and • distributed leadership – in which all staff take on some form of leadership in their organisation. Muijs et al went on to explore in depth the issue of how different approaches to leadership have an impact in the FES. Their literature review identified evidence from a range of studies that distributed forms of leadership are most likely to bring about enduring change – a finding reinforced by Leithwood et al. In contrast, the views of Muijs et al’s own sample of managers and staff were equivocal with regard to the desirability, effectiveness and implementation of distributed leadership: “While transformational leadership seemed well supported in these organisations, distributed leadership received far less support from respondents, both with regard to its perceived effectiveness and to its application.” (2006: 103) Indeed this research found that transactional styles of leadership were not uncommon for certain activities. This suggests that a mix of leadership approaches may be required for an organisation to be successful. Leaders could not be dogmatic about decision‐
making. They needed to be flexible and adapt their approach according to current circumstances, and to be “comfortable with dissent” (Waters et al, 2003). Gibbs’s et al found distributed leadership was the norm in successful university departments, and that this occurred via the organising hand of departmental heads rather than arising organically: “Despite the common collegial culture, ‘cultural’ dispersal of leadership, in which individuals take responsibility informally without being asked, was rare while formal allocation of teaching development roles or duties to individuals was very common.” (2007: 2) Establishing an orderly environment 11
The studies suggest a pre‐condition for creating an orderly environment and pre‐
empting the need for extensive problem‐solving is for leaders to be well‐informed via strong lines of communication. According to Waters et al, it is important for leaders to be aware of issues arising in their institutions, of “details and undercurrents”, and to use this information to address existing and potential problems. They established the effect size of this dimension of leadership, described as ‘situational awareness’, as 0.33. Being well informed about patterns in achievement through analysis of ‘robust information’, and presenting this clearly to managers was key practice for effective leaders in further education (Estyn, 2007). Estyn concluded: “Weak leaders and managers nearly always fail to use management information robustly and do not focus enough on analysing trends in the attainments of learners.” (2007: 4) Having identified where problems lay, leaders targeted resources and energy on dealing with them. Gibbs et al found that effective leaders did not ignore problems they became aware of, even if they had been ‘festering’ for some time, but tackled them head on: “These heads openly acknowledged the problems, diagnosed them convincingly and mobilised others to work to address them in a positive way.” (2007: 2) The Waters et al and Robinson meta‐analyses describe the benefits of leaders creating and maintaining systems that ensure an orderly environment. According to their calculations, effect sizes were 0.26 for “order” and 0.24 for “discipline” (Waters et al, 2003) and 0.27 for “ensuring an orderly and supportive environment” (Robinson, 2007). Both studies include protecting instructional time in this category – that is holding off as much as possible anything that might distract teachers from the central task of teaching. Waters et al add that establishing an orderly environment also means: • providing and enforcing clear structures, rules and procedures for students and staff, and • establishing routines regarding the running of the organisation that staff understand and follow. Implications of this review for approaches to leadership within the further education system A strong message from the studies was the link between success and the participation of leaders in supporting the professional learning and development of their staff. • What opportunities does the 30 hour CPD entitlement offer your leadership team to be more strategically involved in CPD? How can you develop and/or build on 12
•
their skills in supporting CPD in order to extend opportunities and signal your interest in it? Are there ways you could role‐model professional learning, perhaps by participating in a teaching and learning group? There was also a correlation between leaders’ interest and involvement in teaching and learning matters and student outcomes. • To what extent do you feel able to be actively involved in discussions on teaching and learning? Is there scope for you to reallocate responsibilities to enable you to spend more time on the core business of teaching and learning? • Could college self evaluation discussions probe the teaching and learning processes more actively? • To what extent are members of your leadership team involved in discussions on teaching and learning? Are they in demand to lead sessions on teaching and learning, for example? • Do management systems track the learning needs of teachers alongside those of students? Being aware of what was happening in their institutions put leaders in a good position to apply resources strategically. • What are your current arrangements for understanding the quality of teaching across your institution and the practical concerns and opportunities facing your staff and students? • Is there scope for broadening your sources of intelligence? This might mean, for example, attending team meetings, observing lessons as the basis for professional discussions with staff, or texting suggestions for change to students to gauge their opinions and stimulate comment. The studies found that vision and its translation into specific goals, that align with underlying values, had a substantial impact on the quality of provision. • How explicit are the principles on which your organisation is based to your leadership colleagues, the wider staff, and the learners? Would understanding what they believe to be the values of the institution help you reflect on how well you have communicated your underlying beliefs? • To what extent does the leadership team walk the walk and not just talk the talk? 13
Methods for conducting the literature review In order to identify studies most likely to provide authoritative evidence in relation to the review question the search process comprised of: • keyword searches on ERIC, BEI, the UK Educational Evidence Portal, IngentaConnect, Education‐Line, CERUK, Regard (ESRC), ASSIA • searches of five specific journals most likely to contain relevant studies o Journal of Further and Higher Education o Research in Post‐Compulsory Education o Education Management, Administration and Leadership o Management in Education o International Journal of Leadership in Education • scanning government and other relevant national agency websites, including CEL, DIUS, LSN, sector skills councils, and Ofsted, and • following up recommendations from experts in the field. The searches were based on the keywords already established for systematic reviews (see for example Cordingley et al, 2007), with additional key terms that are relevant to the FES. In order to make the sifting process manageable, the search was limited to studies published from 2002 onwards. In total 205 abstracts and titles were retrieved and subjected to initial filtering which excluded those studies which did not: • have a focus on leadership strategies and practices • refer to methods of data collection and analysis, and • explore the impact of leadership on teaching and/or learning. The full text of the seventeen studies which were retrieved were subjected once more to the first stage of filtering, and to the second stage criteria which stipulated the studies should include: • evidence of attempts to establish validity of data and data analysis • a clear description of context, and • detailed information on leadership strategies and practices. The search strategy is available in Appendix I. 14
References Studies included in the review Estyn (2007) Leadership and strategic management in the further education, work‐based learning and adult community‐based learning sectors. Cardiff: Estyn. Gibbs, G., Knapper, C. & Piccinin, S. (2007) Engaging with leaders in higher education. In: In Practice. London: Leadership Foundation for Higher Education. Leithwood, K., Day, C., Sammons, P., Harris, A. & Hopkins, D. (2006) Successful school leadership: What it is and how it influences pupil learning Nottingham: DfES (RR800). Muijs, D., Harris, A., Lumby, J., Morrison, M. & Sood, K. (2006) Leadership and leadership development in highly effective further education providers. In: Journal of Further and Higher Education. 30 (1), pp. 87:106. Robinson, V. (2007) School leadership and student outcomes: Identifying what works and why. Melbourne: Australian Council for School Leaders. Waters, T., Marzano, R. & McNulty, B. (2003) Balanced Leadership™: What 30 years of research tells us about the effect of leadership on student achievement. Denver: McREL. Other reference Price Waterhouse Coopers (2007) Independent study into school leadership. Nottingham: DfES (RR818). 15
Appendix I ‐ Leadership literature review search strategy 1/ Journals for searching: • Journal of Further and Higher Education • Research in Post‐Compulsory Education • Education Management, Administration and Leadership • Management in Education • International Journal of Leadership in Education 2/ Search strategy – articles appearing from and including 2002 Leadership Sector Leader* Manage* Organisation* Institution* Principal* Headteacher* Adult education Continuing education Vocational Post secondary Post‐16 Work based learning College Community college Community learning Training Adult learning Community education Prison education Lifelong learning School Quality improvement Quality Improve* Teaching Learning Perform* Outcome* Higher education Student Universit* Develop* Success* 3/ Apply inclusion criteria to studies Stage 1 filtering criteria (titles and abstracts) • Focus on leadership in the FES and/or HE • Explores impact on teaching and/or learning • Includes information on leadership strategies and practices • Describes methods of data collection and analysis • Builds on previous research Stage 2 filtering criteria (full documents if we have more than 25 studies) • Evidence of attempts to establish validity of data and data analysis • Clear description of context • Includes detailed information on leadership strategies and practices 16