[MUSIC] Welcome to this third session, in this third week on reporting. Today, we're going to focus on how we should report from a test. Our fundamental message is we should aggregate and group things according to their objectives, rather than trying give feedback on individual test questions or items. And in terms of our curriculum map, we're again looking at the whole process of how do we communicate validly the quality of our assessments? When we're thinking about an assessment test, there are multiple approaches to doing an assessment. And we're going to get into those details in the next two weeks. What do we want people to know? We want people to know their total score. On the whole, did they get an A, a B, or a 65? And we sometimes want to tell people their rank. But these things, as we've said, don't really improve the quality of teaching and learning. And don't really improve the quality of reporting. On the screen you can see a ten item quiz made up of ten questions covering three different contents A, B, and C. And the answers to each question by the student ticked to show that the student got it right and an x to show they got it wrong. And the score says five correct, or 50%, and rank in class 11th out of 21 in the class. For some students, this might be a high score, but for most parents, this is a barely, probably, not-good-enough report. But because we identified that the questions belong into three content areas, it makes much more sense to look at the pattern of how the students did on Group A compared to Group B compared to Group C. And so, the next slide shows how I've grouped the questions numbers one, two and six here are part of A, three, five, nine and 10 are B, and so on. And then what we want to look is what is the profile across these teachable content areas. And when we look at the analysis by content areas, we see that in content A, the student got three correct out of three questions. So, that's clearly a strength, And the take home message is whatever content A is, we need to move the difficulty up for the student. In B, the student got two out of four - needs some work, they know something, but they need some more practice and they probably need some more teaching. In contrast, in topic C, they got none of those three questions correct, so that's a definite weakness. And it's time to move the teaching back a little. You can't keep teaching at the level you were, for a student who gets none out of three correct. That student needs something more basic. So, the teacher needs to adjust the teaching to address the fact that this student has a definite weakness in this content area. The other thing we need to remember when we're making a report, I've done this - you walk into class, you give a ten minute, ten question quiz and you think this means something very large. Usually a ten minute quiz is only on the work we did this week or the work we did on this chapter or the work we did on one skill. But we're teaching a subject which is much larger than just this little component. So, when we get a score for this component, we have to take it into consideration that this does not represent the whole content that we're trying to teach. And we'll need multiple assessments before we can begin to generalise, how good is this student in this domain? How good is this student in mathematics or in English or in science? So, we need multiple tests structured well around multiple teachable areas to find out something useful. And then we need to consider, what does this score mean? Any score has to be interpreted to be meaningful. So, simply telling a parent you got 11 out of 20, or you got five out of ten, or six out of ten, is not very communicative. Unless we have some way of saying, and what is that worth. Now, many systems will have ways to interpret scores. For example, in Hong Kong, 40% is considered a pass. In New Zealand you have to get 50% to get a pass. In the United States, in many places, you have to get 60% to have passed. So, every society has a way of interpreting scores. And these rules of interpretation have to be included when we're communicating to parents and students. And when we say this is a good score or a poor score, we have to be able to explain on what basis we made that decision. And there are ways of doing that. We could consult. What does the curriculum expect students of this grade or age to be able to do? What is the average of other students who have taken this test? How hard were the questions? Getting 50% on a test of Einstein's theory of relativity when you're only ten years old might indicate an amazing future scientist Nobel prize winner. However, if it's after A level physics this student might be a disappointment. And of course, how hard were the questions and how good was the scoring? These things all need to go into the mix when we think about what does a score mean. But we must, as teachers, give instructions to parents on how to interpret scores. So, when we're reporting something we need to make sure that our reports are useful. We need to design our assessments around teachable objectives. We need to analyze student performance against those teachable objectives. We need to identify the strengths and weaknesses and needs, according to those teachable objectives. And report what's happening against those teachable objectives. Reporting test question after test question is not a sound way to make use of a test. In the next session, we're going to learn more from Doctor Keegan about analyzing and making use of assessment reporting with cultural minorities. [MUSIC]
