What does comparative judgement tell you that other forms of assessment don’t?

In my previous blog, I said that the results from comparative judgement assessments tended to agree with teacher judgements.

Almost invariably, the pupil you know to be the best writer in the class comes out near the top, and the pupils you know are struggling come out near the bottom. Broadly speaking, comparative judgement tends to reinforce your relative assessments of the pupils in your class.

This is reassuring as regards the validity of comparative judgement, but it might also make you think — well why do we need comparative judgement if our own assessments are already so accurate? Isn’t it just telling us something we already know?

There are two really important things comparative judgement can tell you which you can’t get just through traditional teacher assessment.

One: individual outliers

Whilst your judgement and comparative judgement will tend to match up, there will always be a couple of individual pupils whose outcomes diverge — pupils who do much better or worse on comparative judgement than you’d have predicted. These are the pupils you can investigate further. In some cases, the difference will be the result of a pupil having an unusually bad or good day, and you’ll conclude that the teacher assessment is the correct one. But in some cases, comparative judgement will be telling you something about a pupil that you had overlooked before. It’s a check on some of the inbuilt and unconscious biases that we all have — for example, bias that can lead to disadvantaged pupils and SEN pupils being marked down. This is a very well-researched phenomenon, and one that comparative judgement helps to counter.

Two: how your pupils are doing relative to other pupils in other schools

This is perhaps the biggest benefit of taking part in a national comparative judgement trial. Whilst teachers are very good at understanding how their pupils are doing relative to each other, they are much less good at transferring those judgements onto a shared national scale. The national moderation process is supposed to perform this function, by ensuring that pupils from different schools and local authorities are being placed on the same national scale. However, research from us and from Education Datalab has shown that this isn’t happening.

In practice, what this means is that a individual teachers and schools are good at working out how their pupils are performing relative to each other. But there is no effective process for putting all of those accurate school-level judgements onto one, shared, scale. The result is that what one school or moderator defines as greater depth or expected standard is different from another school and moderator.

We’ve seen this in particular at the greater depth standard. We’ve seen that schools with overall low attainment are very reluctant to award greater depth. The teachers in these schools probably have an accurate idea of how their pupils are doing relative to each other. But because they are in a low-attaining school overall, they may feel reluctant to award their top pupils greater depth, because they think that there are other ‘better’ schools out there where children’s writing is hitting a higher standard. But in actual fact, what we’ve found with both our primary and secondary research is something that lots of other studies have found too — which is that there is more variation within one school than there is between lots of different schools. Almost every school has low-attaining and high-attaining pupils. A school might have low attainment on average, but there can still be individual pupils who are performing very well. Similarly, a school can have high attainment on average, but there can still be individual pupils who are struggling. Comparative judgement helps you to place your assessments of your pupils onto a larger national picture.

And, it’s worth pointing out that both these advantages of comparative judgement will probably lead to a fairer and less biased assessment of disadvantaged pupils in particular.

Sharing Standards is our primary comparative judgement project which involves schools taking part in national judging sessions. Click here to learn more and sign up.