Capping Ambition – The Return to Norm ReferencingPosted: September 7, 2012
One of the side effects of the English grading fiasco has been to bring to wider attention Ofqual’s doctrine of comparable outcomes. In effect this is a slightly more sophisticated return to norm referenced exams – in other words we can expect to see roughly the same exam results every year.
In Ofqual’s own words, written before this year’s disaster:
“We want comparable outcomes when:
- the cohort for the subject is similar, in terms of ability, to previous years
- the syllabus and the exams and other assessments are fit for purpose
- the purpose, requirements and nature of the qualification is the same
- there has been no substantial improvement (or drop) in teaching and learning at a national level
- where previous grade standards were appropriate. “
The emphasis is very clearly on stability in results. All the ministerial rhetoric about maintaining standards makes it clear that this is the brief Ofqual have been given.
The implications of this change are far reaching and have had very little public debate as yet. Ofqual itself has set out some of the issues that arise in a letter to Michael Gove. Perhaps the most breathtaking statement in the letter is that
“For future years, we will explore whether there is scope to develop the approach so that genuine increases in performance can be more easily demonstrated, where there is evidence for that.”
So it would seem that Ofqual is admitting here that it has no adequate way of telling whether overall students’ performance has improved, stayed the same or (presumably) got worse. So it would seem that there is really nothing we can say about how standards have changed because regulators don’t know how to measure it. Can this really be true?
The implications for schools and for the whole school improvement industry are equally dramatic. Gove has decided to increase the floor target at GCSE this year to 40% and ultimately to 50%. But if results are going to stay the same overall, how can this possibly be achieved? If one school’s results go up, someone else’s will have to go down. If the average of 5 A to C with English and maths stays in the high 50’s, it is arithmetically obvious that a great many schools will be below a 50% floor target.
Or as Ofqual has told Michael Gove:
“Our approach means that whilst some schools will see improvement in their exam results, due to comparable outcomes the overall results will not show significant increases. So, it will be difficult to secure system-level improvements in exam results which you have said you want to see.”
We await a response.
So what is to be done? Obviously the first thing will be to make them all academies. But simple maths says that this won’t make any difference. The whole premise of the last 20 years school improvement activity has been that low performing schools can catch up with the rest. If the examiners are going to make that impossible, where do we go from there?
More importantly though, we – that is pupils, teachers, employers, colleges – will have real problems knowing how to interpret results. We will have a league table of pupils – we’ll know who did best and who didn’t. That of course is one purpose of exams and is not to be dismissed. But will we still know what any given grade really means? A college or an employer won’t really know what an applicant can knows and can do – just where they come in the pecking order.
We got rid of norm referencing because it was seen to be a cap on ambition. As Keith Joseph said when GCSE was introduced , it “would be fairer because pupils would be judged by what they could do and not how they compared to someone else.”
The objective set by Keith Joseph was to get the majority of pupils up to the standard then achieved by only a minority. That still needs to be our aim. But we have now fallen into a situation where we’ve effectively capped the number getting Grade C and have no way of knowing whether or not we are getting nearer to the objective we set all those years ago.
To get out of this muddle is going to require some hard work and a degree of intellectual honesty that has been sadly lacking over many years. We have to find ways of knowing how well pupils and schools are actually doing against real criteria as well as against each other. This will require a much more sophisticated approach to assessment than – reverting to simplistic end of course exams are unlikely to do the job.