SEE PROGRAM MODERATION WORKSHOP 2014
SEE PROGRAM MODERATION WORKSHOP 2014 LWA is a contracted supplier of assessment validation services, moderation workshops and professional development services under the Skills for Education and Employment Program funded by the Department of Industry © LWA 2014 1 AGENDA © LWA 2014 9.00 – 9.30 9.30 – 9.40 9.40 – 10.10 Registration, on arrival tea and coffee Welcome, introductions, overview and purpose of the day SESSION 1: IV IN THE SEE 10.10 – 10.55 SESSION 2: MODERATION PTAs 10.55 – 11.15 11.15 – 11.35 Morning tea SESSION 2: CONTINUED Group feedback and discussion of PTA moderation 11.35 – 12.55 SESSION 3: MODERATION Learning tasks Group feedback and discussion of Learning moderation 12.55 – 1.30 1.30 – 3.00 Lunch SESSION 4: MODERATION Oral communication tasks 3.00 – 3.15 3.15 – 3.45 Afternoon tea SESSION 4: CONTINUED Group feedback and discussion of Oral Communication moderation 3.45 – 4.00 Wrap up and evaluation forms 2 SESSION 1: IV IN THE SEE Handout 1: What is needed for verification Do you have any questions about what you need to provide? © LWA 2014 3 IV REQUIREMENTS - PTA KPI 3 Accurate Assessment Were the assessment tasks appropriate for the ACSF level? PTA kit appropriately aligned to ACSF indicators PTA kit contains a sufficient range of tasks to adequately assess ACSF indicators Is there sufficient evidence that the provider has assessed the client accurately against the ACSF? Student response demonstrates exit competence Sufficient assessor analysis of student performance against the ACSF indicators claimed © LWA 2014 4 IV REQUIREMENTS – IPA/LOA KPI 3 Accurate Assessment Were the assessment tasks appropriate for the ACSF? Tasks appropriately aligned to the ACSF Task materials are appropriate for use as assessment tasks Is there sufficient evidence that the provider has assessed the client accurately against the ACSF? Student response demonstrates exit competence Sufficient assessor analysis of student performance against the ACSF Range of text types demonstrated KPI 4 Quality Training Are the tasks relevant to the client’s learning needs? Tasks are appropriate for the client cohort Tasks are appropriate for adult learners © LWA 2014 5 TODAY WE ARE MODERATING, NOT VERIFYING Tasks and client responses stand alone for moderation purposes Consistency in the interpretation and application of the ACSF Build on professional knowledge Support teaching and assessment practices © LWA 2014 6 SESSION 2 Moderation Activities: PTAs © LWA 2014 7 SESSION 2: PTA TASKS 1, 3, 4 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? © LWA 2014 8 PTA 1 :TASK, LEARNING & OC TASK © LWA 2014 TASK: Aligned Detailed instructions to the assessor Appropriate prompts to the assessor Both learning indicators comprehensively addressed Excellent prompts to the assessor to observe .02 evidence throughout the PTA More familiar questions could be used at the start of the interview, e.g. introductory comments relating to personal details Modified questions may be required for Language clients, e.g. personal details, background 9 PTA 1: CLIENT RESPONSE, LEARNING 2.01 Client Sample: Demonstrated • • 2.02 © LWA 2014 Client identifies some goals and areas of learning need e.g. numeracy, work related skills Evidence of a learning goal and the investigating of relevant learning pathway e.g. has investigated requirements for roadside slashing business • Evidence of barriers e.g. family, financial skills • Assessor has recorded detailed client quotes Client Sample: Not demonstrated Evidence more aligned to 1.02 Further questions could have been asked e.g about ‘White Card’ training, to prompt client to expand on strategies used to gain this qualification Source of information about web search unclear Evidence could have been strengthened if assessor had noted any strategies used by the client during the assessment 10 PTA 1: CLIENT RESPONSE, OC 3.07 Client Sample: Not demonstrated Evidence suggests potential level 3 but it is not clear for example what are the client’s actual words and what has been paraphrased • More in depth discussion of some topics, for example, the CFS may have allowed for gathering of stronger evidence at level 3 than a short response for every question e.g. What is involved with being a CFS volunteer? Tell me about the training required for the CFS. What type of person is suited to the CFS? • Quotes captured need to demonstrate appropriate levels of complexity • Assessor comments need to be supported by reference to specific quotes. The reporting sheet included assessor comments such as ‘used a range of tenses…vocabulary sufficiently broad’ but these are too general Client Sample: Demonstrated • 3.08 • • • © LWA 2014 Client was able to engage in conversation on familiar and some less familiar contexts Assessor comments need to be supported by reference to specific quotes of questions asked by the assessor and to the client’s ability to maintain a conversation e.g. turn taking Where assessor noted ‘asked questions to clarify discussed volunteer responsibilities’ a few examples of actual questions may have provided stronger evidence 11 PTA 3: TASK & CLIENT RESPONSE, LEARNING TASK TASK: Aligned Consider review of layout of Assessment Interview Form to provide more space for recording responses Suggest starting the Assessment Interview Form with questions relating to highly familiar, personal details to put low level clients at ease Prompts are too long if read as a block of text 1.01 1.02 © LWA 2014 Assessor could have extended the conversation to demonstrate full extent of competence Client Sample: Demonstrated Evidence supports reported indicator Further questioning, prompting may have allowed for higher competence to be demonstrated Client Sample: Demonstrated Evidence supports reported indicator Further questioning, prompting may have allowed for higher competence to be demonstrated Assessor could have made observations of strategies throughout the PTA 12 PTA 3: TASK & CLIENT RESPONSE, OC 1.07 1.08 © LWA 2014 Client Sample: Demonstrated Quotes on ‘Assessment Interview Form’ support reported indicator Extensive supporting quotes are recorded on the ‘scribed assessor’ sheet Ensure all assessor comments pertain to indicator performance features Client Sample: Demonstrated Assessor comments supported with referenced examples Evidence is strengthened with quoted examples of requests for clarification and repetition 13 PTA 3: TASK & CLIENT RESPONSE, READING TASK TASK: Aligned • Pre 1.03 Pre 1.04 © LWA 2014 It would be useful to include a range of texts in a PTA kit relevant to the client groups Client Sample: Demonstrated Assessor comments address Writing and Numeracy performance features Ensure comments address the performance features of the indicator being claimed Client Sample: Demonstrated Some assessor comments give evidence for learning 14 PTA 4: TASK & CLIENT RESPONSE, LEARNING TASK TASK OC & LEARNING Assessment Interview Form: Aligned • Suggest starting the Assessment Interview form with questions relating to highly familiar, personal details to put low level clients at ease • Prompts are too long if read as a block of text • Assessor could have extended the conversation to demonstrate full extent of competence A self-assessment form: Not aligned • Not an appropriate task for a low level client • Limited application against ACSF Performance features • Assessor noted ‘high level of support ‘was provided. What form did the support take e.g. was the client expected to read the task but found it too difficult? Core LLN Skill Assessment Summary Sheet: Aligned • Assessor has provided examples of client performance against the ACSF Focus Areas Pre 1.01 Client Sample: Demonstrated Pre 1.02 Client Sample: Demonstrated © LWA 2014 Assessor notes together with client performance support reported indicator Assessor notes together with client performance support reported indicator Assessor observed strategies throughout the PTA 15 PTA 4: CLIENT RESPONSE OC Pre 1.07 Pre 1.08 © LWA 2014 Client Sample: Demonstrated Based on evidence client sample aligned. Evidence on Assessor Notes sheet suggests more highly familiar questions may have produced evidence of 1.07 No clear delineation of comments addressing of individual indicators Client Sample: Not Demonstrated Client responses demonstrated understanding of several questions of varying complexity thereby indicating 1.08 was achieved Need to include a few examples of questions that need to be rephrased Assessor comments span both indicators. The .08 indicator is not clearly addressed 16 PTA 4: TASK & CLIENT RESPONSE READING TASK Both text types formatted. No prose text used Task: Emergency Procedures: Not aligned Task complexity spans several levels of reading In order to award Pre level 1, a task aligned to level 1 must be used Task: Calendar: Not aligned Task contains complex language Pre 1.03 Pre 1.04 © LWA 2014 Too many questions. Questions 5 – 11 do not address level 1 reading performance features Client Sample: Not Demonstrated • • Comment ‘have difficulties if the text has unfamiliar vocabulary and complex structure’ indicates Emergency Procedures task is too complex The client may well have demonstrated competency at level 1 if appropriate texts were used given that she has answered questions 1 - 4 correctly 17 PTA 4: CLIENT RESPONSE WRITING TASK Task: aligned Pre 1.05 Pre 1.06 © LWA 2014 Two appropriate text types were used Parts of the form exceed level 1 complexity. To ensure consistency between assessors maybe modify the form so there is a clear distinction between the sections to show level 1 and level 2 Client Samples: Demonstrated • Personal details on form copied as noted by assessor indicate the client is still prelevel 1.05 • Although the client demonstrated some 1.06 features (as acknowledged in the assessor notes e.g. assessor’s comments relating to punctuation) pre-level 1 is appropriate e.g. the information in the form was copied 18 PTA 4: TASK & CLIENT RESPONSE NUMERACY TASK Task: Not Aligned • Tasks are not real life. They consist of a series of decontextualized activities. • Portions of the task are maths questions (questions 3-7) • Some of the task activities are beyond level 1 (questions 15, 16, 17, 18, 200) 1.09 1.10 Pre 1.11 © LWA 2014 Client Sample: Demonstrated • Some comments span indicator levels • Assessor notes together with client performance supports reported indicators despite the unsuitable tasks 19 SESSION 3 Moderation Activities: LEARNING © LWA 2014 20 SESSION 3: LEARNING TASKS 1-5 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? (If supplied) Would you adapt this task to use in your own context? How? © LWA 2014 21 LEARNING TASK 1: TASK & CLIENT RESPONSE MY GOALS & HOW DO YOU LIKE TO LEARN 1.01 © LWA 2014 It is acknowledged that Distance learning presents particular challenges for assessment and documentation Tasks: Aligned • Could be broadened to cover past experiences of successful learning, any barriers and planning and organising • Task 2 provides evidence of preferred learning approaches but client may be confused by the graphic • Task 2 could be redesigned to allow for demonstration of more focus areas Client Sample: Demonstrated • Evidence would be strengthened if assessor noted any relevant details gained in conversation with the client • Assessor notes map evidence to focus areas although the notes are very brief • Assessor comment on strategies is a .02 performance feature • Assessor could note client’s ability to engage in learning activities with appropriate scaffolding as additional evidence for learner identity 22 LEARNING TASK 2: TASK & CLIENT RESPONSE BOOKLET & STRATEGIES FOR LEARNING 2.01 Tasks: Aligned Covers a broad range of performance features Client Sample: Demonstrated © LWA 2014 Prompting and advice box ticked but no information provided Variation in scaffolding provided Assessor comments are succinct yet thorough with mapping of evidence to relevant performance features Assessor prompting with Qs 1 and 7 could have strengthened the evidence Updated SEE program task Cover Sheet should have been used 23 LEARNING TASK 3: TASK ONLY LEARNING IN THE WORKPLACE 2.02 © LWA 2014 Task: Aligned • Depending on responses, the task allows for a range of levels to be demonstrated • Good task to facilitate evidence from a work placement • Task could be modified to suit a range of workplace contexts • Some of the statements could be reworded as questions e.g. statement 1 to allow for a broader range of responses 24 LEARNING TASK 4: TASK & CLIENT RESPONSE RANGE OF ACTIVITIES 2.02 Tasks: Aligned Task well constructed covering a broad range of performance features Client Sample: Demonstrated • Responses may have been more detailed if the assessor had scribed the student responses • Assessor Comments template is well structured • Not all client responses show strong exit level 2 competence e.g. task 1, task B question e. However, there is sufficient evidence overall to support the outcome • Some assessor prompting may have allowed for stronger responses e.g. completion of task 5 © LWA 2014 25 LEARNING TASK 5: TASK ONLY MY WORKPLACE – A REFLECTION 3.01 3.02 © LWA 2014 Task: Aligned • Good task for gathering evidence from a work placement and task could be modified according to workplace context • Could be used as part of a project about the work experience. This could broaden the possible demonstration a wider range of Learning focus Areas as well as demonstration of other Core Skills 26 SESSION 4 Moderation Activities: ORAL COMMUNICATION © LWA 2014 27 SESSION 4: ACTIVITY, TASK 2 Q & A BOARD GAME FILM CLIP REMOVED You will see a group of students playing the Questions and Answers game. Pre-teaching includes practising question forms. After watching, discuss in your group: What OC outcomes can be assessed with this game? Are there any other core skills that can be assessed? How? How can the game be adapted to assess different ACSF levels? © LWA 2014 28 SESSION 4: OC TASK 2 MODERATION: Q & A BOARD GAME TASK & CLIENT SAMPLE Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? Would you adapt this task to use in your own context? How? © LWA 2014 29 OC TASK 2: TASK & CLIENT RESPONSE Q & A BOARD GAME 1.07 Task: Aligned Creative task with potential to be used effectively across OC levels Instruction sheet provides guidance on participation (supported by examples on the white board. See video) Assessor sheet is very comprehensive and allows for reporting of strong evidence against the performance features Client Sample: Demonstrated © LWA 2014 Teacher notes provide strong demonstration of both the ability to ask questions and give short responses Assessor comments support reported indicator well 30 SESSION 4: ACTIVITY, TASK 8 ABC 7:30 REPORT FILM CLIP REMOVED This is a segment from the ABC 7.30 Report entitled Casualising the Workforce. Discuss: What OC listening level would you assess with this story? Could you also use it to assess speaking? How? Could it be used as a springboard to assess other core skills? How? Would you use this idea? How? © LWA 2014 31 SESSION 4: OC TASK 8 MODERATION: 7:30 REPORT, CASUALISING THE WORKFORCE TASK & CLIENT SAMPLE Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? Would you adapt this task to use in your own context? How? © LWA 2014 32 OC TASK 8: TASK & CLIENT RESPONSE CASUALISING THE WORKFORCE 4.08 Task: Aligned • • • • • • Excellent task relevant to the needs of the learner cohort Context of task in classroom learning well explained Text demonstrates appropriate complexity for the indicator Could be strengthened in Focus Area of Audience and purpose. Consider some questions to elicit client understanding of the purpose and motives of different speakers Could be strengthened in Focus Area of Comprehension. Consider some question reformulation to allow client to clearly demonstrate ability to relate pieces of information in an oral text rather than treating them as separate pieces of information Evidence strengthened through the provision of a part transcript to give an indication of the complexity of the listening text. It is not necessary to include the entire transcript Client Sample: Demonstrated • • • © LWA 2014 Demonstrates understanding of complex oral texts (multiple speakers) Understands abstract nouns, e.g. casualisation Understands vocabulary specific to the topic 33 SESSION 4: ORAL COMMUNICATION TASKS 1,3,4,5,6,7 Use the tasks to decide: Is the allocated ACSF core skill and level accurate? Is the task sufficient to support the indicator? Does the client response demonstrate the indicator? (If supplied) Would you adapt this task to use in your own context? How? © LWA 2014 34 OC TASK 1: TASK ONLY AT THE BUTCHER 1.07 Task: Not Aligned • • • © LWA 2014 Appropriate real life task Role play cards too heavily prompted to allow for demonstration of exit level competency Assessment Criteria sheet does not closely reflect indicator performance features 35 OC TASK 3: TASK & CLIENT RESPONSE G & S TRANSPORT TICKET 1.08 Task: Aligned • Authentic relevant task for learner cohort supporting community engagement • Well structured and scaffolded task • Student reflection on page 5 could potentially provide some learning evidence Client Sample: Demonstrated • Assessor annotations together with quotes of client’s speech strongly support indicator performance features • Transcript of transaction provided • Detail of classroom context provided on page 3 of task © LWA 2014 36 OC TASK 4: TASK & CLIENT RESPONSE DENTAL APPOINTMENT & CASUAL CONVERSATION 1.07 Tasks: Aligned • Clear instructions to learners • Prompt cards provide appropriate level of support Client Samples: Demonstrated • Strong evidence aligned to the relevant performance features both on the Task Cover Sheet and on the tasks • Clear transcripts of conversations provided © LWA 2014 37 OC TASK 5: TASK & CLIENT RESPONSE STUDENT RANG ME 1.07 Authentic sample: Aligned Client Sample: Demonstrated © LWA 2014 Good example of capturing authentic evidence from the student Evidence supports the indicator claimed 38 OC TASK 6: TASK ONLY PHONE MESSAGE 2.08 Task: Aligned • • • • © LWA 2014 Need to consider students with spiky profiles i.e. will writing ability impact on task outcomes? Task designed for use with multiple core skills (Listening, Writing, Reading) Listening text is appropriate context and complexity The message pad may be confusing as it does not readily relate to the information in the text e.g. ‘while you were out’ could be changed to ‘from’ 39 OC TASK 7: TASK ONLY PART TIME & CASUAL WORK 3.08 Task: Aligned • • • • © LWA 2014 The task is highly relevant for students planning to enter the workforce Explanation of context in classroom learning would need to be explained on the Task Cover Sheet Question 8 could be reformulated to give students the opportunity to express their opinion on the benefit of this information to listeners Some level 4 complexity in the text but scaffolding of questions supports the 3.08 outcome 40 PLEASE FILL IN YOUR EVALUATION FORM Contact us: T: 03 9429 7551 F: 03 9429 7221 www.lwa.net.au SEE admin: [email protected] Office admin: [email protected] Verifiers: [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] © LWA 2014 41
© Copyright 2024