Ofsted – Part 2
The birth of the revised inspection framework
Many Ofsted inspectors must have experienced something of a Damascene moment in 2017 when confronted by the reality that learning is invisible and not something that they can ‘see’, even if the inspection framework and the training process told them otherwise. (See: http://educonned.co.uk/ofsted-1)
With the prestidigitation of a conjuror (‘watch the judgment, keep your eyes on the judgment at all times’), Ofsted then deftly whisked away the inspection judgment that referred to ‘the quality of learning’ and replaced it with something more general and open to interpretation (that is to say woollier and more contentious).
The evidence trail that tells us how this came about consists mainly of three documents:
‘Education Inspection Framework Overview of research’ [1] (January 2019)
‘An investigation into how to assess the quality of education through curriculum intent, implementation and impact’.[2] (December 2018, Phase 3 Research)
‘How valid and reliable is the use of lesson observation in supporting judgements on the quality of education?’ [3] (June 2019)
These are listed in this order because the ‘Overview of Research’ was commissioned first, in 2017. The Introduction explains that:
‘The purpose of this research was to ensure that Ofsted could assess the quality of education in a valid and reliable way.’
So the intentions were: to introduce a new key judgment – ‘the quality of education’ – to replace the old judgment of ‘the quality of teaching, learning and assessment’; to assess this new judgment only in terms of curriculum, but using a new definition of curriculum; and to continue to use lesson observation, which had been shown to be a process that was neither valid nor reliable, as supporting evidence about the validity and reliability of the judgments made by inspectors using this new model of inspection.
Everything flows from Ofsted’s new definition of curriculum:
‘Our working definition of curriculum is that it is a framework for setting out the aims of a programme of education, including the knowledge and skills to be gained at each stage (intent); for translating that framework over time into a structure and narrative, within an institutional context (implementation); and for evaluating what knowledge and understanding students have gained against expectations (impact).’ (p.4)
Where to start? It’s probably appropriate to revisit the parallel with the Care Quality Commission, the body that inspects and regulates health and care services. The parallel to Ofsted redefining what is meant by curriculum, when the term has always been understood as ‘the content of a course of study’, would be for the CQC to decide that, solely for the purposes of inspection, it has constructed a new definition of ‘medicine’.
‘Our working definition of medicine is that it is a framework for setting out the aims of a programme of health care, including the health gains to be made at each stage (medicinal intent); for translating that framework over time into a structure and narrative, within an institutional context (medicinal implementation); and for evaluating what health benefits patients have gained against expectations (medicinal impact)’.
Considered only as a passage of text, this may make grammatical and semantic sense but as a real framework for inspecting health care settings it seems to be completely artificial and somewhat nonsensical, probably because it is.
What Ofsted’s new definition of curriculum attempts to do is get us to accept that ‘education’ and ‘curriculum’ are synonymous – that when we talk about education, all we are really talking about is the curriculum.
Validity
Ofsted claims that the revised inspection framework is ‘the most researched, evidence-based and tested framework in Ofsted’s history’, so its validity should be unquestionable. But it isn’t.
Within the research study that underpins the framework, the Phase 3 research that investigated ‘how to assess the quality of education through curriculum intent, implementation and impact’, inspectors reached their judgments by using a checklist of 25 ‘curriculum indicators’. However, as the Chief Inspector’s commentary on the research [4] explains,
‘These indicators will not be directly translated into the new inspection framework. First, they were only tested in schools, not early years provision or further education and skills providers. Second, 25 indicators is too many for inspectors to use on an inspection, especially given the short timescales of modern inspection practice.’
So, notwithstanding questions as to whether it is possible to arrive at valid judgments on the quality of education using a checklist of 25 curriculum indicators, it was not intended that school inspections would be carried out using the same methodology as the research study which purports to establish the validity of this approach to inspection (because inspectors don’t really have the time).
So how instead will evidence to establish the quality of education be gathered? Mostly, by ‘considering’ and sometimes by ‘evaluating’.
194. Inspectors will consider the school’s curriculum, which is the substance of what is taught with a specific plan of what pupils need to know in total and in each subject.
195. Inspectors will consider the extent to which the school’s curriculum sets out the knowledge and skills that pupils will gain at each stage (we call this ‘intent’). [….] Finally, inspectors will consider the outcomes that pupils achieve as a result of the education they have received (we call this the ‘impact’).
196. In evaluating the school’s educational intent, inspectors will primarily consider the curriculum leadership provided by school, subject and curriculum leaders. (My underlining)
If you’ve read Part 1 of this, you’ll be aware that lesson observation of the type carried out by Ofsted for many years hasn’t stood up to scrutiny in terms of its validity. But undeterred, Ofsted uses lesson observation evidence to establish the validity and reliability of its judgments on the quality of education. Not the validity of the new judgment itself, but the validity and reliability of inspectors’ judgments.
Within the research study to establish the validity and reliability of lesson observation in providing supporting evidence for inspectors’ judgments, Ofsted again devised a checklist, this time featuring 18 ‘indicators’, focusing on two of the four inspection judgments: the quality of education and behaviour and attitudes.
“We developed 18 indicators across three domains of interest – curriculum, teaching and behaviour – along with detailed guidance to help inspectors with assessing the indicators. The indicators were scored on a high-inference, five-point scale.”
However,
“These indicators were strictly for the purpose of the research study, as they allow us to carry out quantitative analysis. They were not intended for use on inspection”
If the methodology used within the two key research studies was not intended to be used within inspections, what and where is the research and evidence base that Ofsted claims underpins the revised inspection framework? Where is the research that shows how the quality of education can be judged by inspectors doing a great deal of ‘considering’?
There is an extraordinary internal interdependence within the whole structure, because its central components – the new key judgment itself, the new definition of curriculum, the new assessment device that is based on the new definition of curriculum and the (new) use of lesson observation to provide corroborating evidence for the validity of judgments – all rely on one of the other new components for definition, or validation. The new judgment is defined by the new assessment device, which in turn relies on the new definition of curriculum and the validity of it all is nominally established by an approach to lesson observation which has an entirely new focus and purpose.
It would be a very significant stretch to call this an evidence-based inspection framework.
