Ofsted says “unprecedented improvement” – can we believe it?

“The Chief Inspector of Ofsted today hailed the ‘unprecedented rate of national improvement’ in school performance across England.” That’s how Michael Wilshaw introduced the latest inspection outcomes covering the period from September 2012 to June 2013. He was echoed by Michael Gove in the House of Commons.

In his normal modest way, Wilshaw is in no doubt that it’s all down to Ofsted and his new framework: “Sir Michael said he believed changes to Ofsted’s school inspection framework that came into force 12 months ago was clearly having a galvanising effect on England’s schools system.” In particular he claimed that the change from satisfactory to requires improvement had motivated heads to push through improvements – we are left to assume that they wouldn’t have bothered otherwise.

So what are we to make of this. Apparently now 78% of schools were good or outstanding at their last inspection compared with less than 70% a year ago. First let’s look at some figures. If we look at the actual inspections carried out in the last year, only 64% resulted in an outstanding or good outcome. 30% required improvement and 6% were inadequate.

How can this be? The answer of course is that Ofsted no longer inspects a random sample of schools. Inspections are heavily skewed to schools in grades 3 and 4. These are of course the schools that are most likely to improve – if you’re already outstanding you can’t improve. For many schools their last inspection was years ago and we have no way of knowing whether they are still worth their grade.

Even so, let’s assume that the figures do work out and there has really been an improvement in inspection findings. Most people when faced with a change in data that is entirely unprecedented would ask the question, has something gone wrong with the process because this kind of change doesn’t really happen very often.

Self-doubt is not a characteristic we associate with Ofsted. But a sensible reaction would be to ask whether there is any other data that might support or otherwise these findings. So perhaps we are seeing a dramatic improvement in exam results? Of course the answer to that is no, both GCSE and A level results went down. It seems strange that exam results are going one way and Ofsted is going the opposite.

We ought to ask then, is there any other hypothesis that could explain this sudden change. One plausible idea is that this is another example of perverse outcomes caused by a change in the system of incentives. Could it be that inspectors aren’t prepared to accept that what used to be satisfactory now needs improvement? Perhaps they are saying to themselves, this school is ok, it shouldn’t be in a category so we’ll give it good – whereas before they might have settled for satisfactory.

The other possibility is that heads have found ways of managing these new-style inspections to get the outcomes they want. It could be that the much vaunted new framework is not as rigorous as has been suggested. Certainly the approach to observing teaching with its emphasis on observations of 25 minutes or less and the lack of any statistical basis for summarising teaching quality would seem to be wide open to the making of inappropriate judgements. Then if the teaching judgement has priority over data on pupil achievement, the chances of error creeping in becomes even greater.

Whatever the reason, any sensible reaction to this inspection data would be to say “some mistake surely”. You just don’t get that kind of change in the whole system at that speed. Before blowing its own trumpet quite so loudly, Ofsted would do well to have a hard look at its own data and systems.

Advertisements

2 Comments on “Ofsted says “unprecedented improvement” – can we believe it?”

  1. David Pavett says:

    Excellent piece. Very helpful. Thanks John.