John Hattie observed that we often think our profession is defined by autonomy or the ‘let me do it my way approach’. The problem with autonomous practice is that it isn’t informed by data and evidence but by assumptions and ideology. Without looking at student data, we aren’t looking at the reality.
I came across an article a few months back in The Atlantic on Andreas Schleicher, a German scientist working for the OECD. According to the Atlantic, he is the ‘most influential education expert you’ve never heard of’.
Schleicher is a number-cruncher and one of the instigators behind the Programme for International Student Assessment (PISA). His slogan is ‘without data, you are just another person with an opinion.’
We don’t just want teachers with opinions, we want teachers who use data to frame the right questions. Student data is not a judgment tool, it is a diagnostic tool in the same way doctors use blood tests. Doctors can make assumptions about a patient’s health but unless the assumptions are tested, you cannot diagnose and treat. It’s the same rigour that must be applied to the practice of teaching. The relational aspect of teaching will not be subverted by the use of data but enhanced by it.
As a profession, we need to move away from I think to we know. We know that using data to improve practice is fundamental to being a good school. We know that one of the most powerful ways of bring about improvements is teachers learning about and reflecting on their practice. Practice that is evidence based.
That’s the work of Helen Timperley and the inquiry model. Teachers looking at the learning needs of students using the data/feedback and then identifying what they need to know in order to improve the learning outcomes. It underscores the links between teacher learning and student learning. This is what gives us the sharper focus.
A recent report by the OECD highlighted a lack of knowledge among teachers ‘about how to interpret information on student performance, saying they are not adequately prepared in analysing student test results and using the data to improve their teaching.’ (The Australian, 18 August). Unfortunately, we get too many examples particularly in the media of the poor use of the ‘rank and order’ data that sheds little light on student learning or teacher practice.
Part of the reason why there is fear of evidence-based practice is that becomes an accountability measure for what teachers are or aren’t doing in classrooms. When teachers set ambiguous targets, you can’t measure progress so you can’t be held responsible for the learning outcomes. When teachers set ambitious and measurable targets for students, they have something tangible to work towards.
If we are to get serious about student data then we need to make this the focus of teacher training and teacher learning. When Schleicher analysed the PISA data he found a common thread across the best school systems: teacher training schools were made rigorous and selective, systems emphasised building the capacity of leaders and teachers above ‘reducing class sizes or equipping sports teams’ and once they had quality teachers and leaders in schools, they found ways of holding teachers accountable for the results while allowing creativity to flourish.
Four years ago, McKinsey released How the World’s Most Improved School Systems Keep Getting Better, observing that the quality of an education system cannot exceed the quality of its teachers (and leaders). I would also argue that no teacher can exceed the quality of the data.
bluyonder.wordpress.com 
Eric Mazur also has a good quote for evidence based research; ‘the plural of anecdote is not data’.
Good quote – thanks for sharing.
While data is a critical educational tool, there is also the danger of become paralysed by data. One of the key tools of a teacher is instinct. Instinct is borne from knowledge and wisdom and often lives outside the realm of statistics. While it is both nebulous and qualitative, let’s not lose the ability to trust and listen to the ‘gut feeling’ of teachers when they give their opinion – even when there may not be data they can quote. There’s a balance to be struck.
I agree jennifer,either side taken to extreme is as bad as the other. Professional know their craft and “sense” and “feel” things as they go. This is not a random thing but borne out of thousands of hours experience. I should never be taken lightly or ignored.But data used wisely and appropriately can add to this capacity
Data is ideology too. How data is gathered (and used) is inherently ideological. Ironies abound, especially in Australia, the UK and the USA when we look at our leaders use data and then plunge on, regardless, with more of the same. Their ideologies shine bright.
John Ralston Saul’s, The Unconscious Civilization is fast approaching its 20th anniversary. It still resounds.
I don’t agree, Data is neutral how it is used or misused is where ideology comes in. Too often we blame the data and miss the central issues.
This said I agree that there are some appalling misuses based on narrow and utilitarian purposes. On th eother hand it has been ignored and trivialized for too often.
Professional teachers know that they need evidence of student performance as well as their own if they are to improve this performance. We need wise heads!
Greg,
I promise not to play the contrarian too often at your blog but there are some very important issues to be discussed. Of course, how the data is used is the major issue but my opening sentence, written very consciously, is, to my way of thinking, essential for practitioners to understand. ‘Data’ is collected, based on some system or framework, with humans framing their approach for a variety of (mostly) intellectually honest reasons. However, by definition it is not natural or neutral as it has to be collected from some designed process. Yes, intellectually, many can see that, make the necessary allowances but as you point out, there are ‘some appalling misuses’. Data is not neutral. IQ testing showed us that, as did the poor uses systems put that data too.
We would not, I suspect, disagree about too much when it comes to what a good teacher should do. Yes, they should use data from a variety of sources to reflect on what a student can do and where they need to develop. I agree with the main thrust of your post. I do believe that what led me to actually comment, was you sentence:
‘The problem with autonomous practice is that it isn’t informed by data and evidence but by assumptions and ideology.”
How do you know this ‘autonomous practice’ is not informed? Is this not an assumption made from your ideological POV? I understand the point of your post, having worked in schools for a while too, recognising we need to improve how educators instruct students. I also agree that many educators trivialise data, often not making the professional judgements required to assist students to genuinely improve.
I suspect that ‘education systems’ looking for economies of scale in the way improvements are implemented, need to be very careful that they have ‘wise heads’ or more of our best young teachers, autonomous by nature but excellent practitioners who stay professionally up-to-date and connected, may continue to leave our systems in droves (as the data currently shows us). They may not enjoy data being ‘the focus of teacher training and teacher learning’ as one can imagine, the utilitarian purposes that would abound.
Finally, Greg, I know of no other Australian blog, from a person working at a similar level to you in any of our schooling systems, or higher. Can you post some links?
Darcy, always happy for you to play contrarian. In relation to your point about data, the concern is that it becomes the conclusion rather than the beginning of insightful and reflective discussions. What is critical is not the data per se but the quality of questions that arise from examining the data and feedback without prejudice or judgment. In this context, data is neutral.
As for autonomous practice – it isn’t necessarily informed. Informed practice or as Darling Hammond et suggests ‘adaptive expertise’ is the ability to learn from others. I’m not sure autonomous practice promotes the skills or willingness to collaboratively reflect, evaluate and adapt one’s practice in order to make the best decisions for students and their learning. What is informing practice if a teacher is not working in collaborative contexts? It is unrealistic for teachers to think they have all the answers. That is why we need to move from I think to we know. We know supports life-long learning.
I don’t know of any directors’ blogs but hopefully others may be able to share some useful links.
Thanks for the fascinating discussion. My very simplified understanding of data is that it provides me with a “picture” of today. What I do with the data,how I do it and with who provides me with the “picture” of the future.
Where I want to move to is framed by the data I collect.
Thanks,great conversation.
Peter
Just picked up your conversation about how data can inform teaching. I must say I’m now more than ever a convert to the idea that we must support teachers to collect and analyse student data so that they might pose questions, discuss with colleagues and form inquiries about shared problems of practice that once solved would lead to better outcomes for students.
I’ve just arrived back from completing a data wise course at Harvard and looking to embed this data wise inquiry process across the school. I’m starting to write about how we do this on my blog so I’d be interested in your comments.
Mark, I look forward to reading your posts on how we can use data more effectively to improve teacher practice.
I love this quote from Cliff Stoll & Gary Schubert:
“Data is not information,
Information is not knowledge
Knowledge is not understanding
Understanding is not wisdom”
For me, decisions we make in teaching or school improvement need to be based on evidence. And it is the conversations we have around data that is the key to making improvements.
Having said that I think that too many of our teachers and leaders have been led by some system leaders to believe that data is mystical and inviolate. I agree that one thing we need to improve is data literacy in our profession. For instance, the OECD report makes the point well that some teachers are using NAPLAN data in a way that can lead to unreliable conclusions. And (agreeing with Darcy) that included in that is an examination of the philosophical underpinnings of the or data collection tool. Involving teachers in discussions about how to collect data that focuses on the instructional core and deep and meaning learning for students is a place to start.
You raise good points here. The problem I think is we blame the data for the problems. In reality, it is the way we chose to use the data. It’s a challenge for teachers, leaders and administrators to ensure that the data is being used and interpreted in ways that inform practice and improve student learning. Ultimately it is about the intelligent questions that come from the data – something that we all need to get better at.
Barbara and Greg,
You both raise some points here:
Teacher conversations are important in looking at student learning data however I am learning that these conversations need to be structured.
When I say structured I mean both in technicial sense of a structured meeting with clearly defined protocols and data presented in ways that make sense and is easy to interpret (often this is new learning for teacher leaders), that we use multiple sets of data that is e.g. triangulated looking for patterns and that data is used to inform an inquiry that is linked to teacher practice (the need to use instructional observational data collected by teachers for teachers – this is certainly new learning for most of us).
The second point is that this is a continuous weekly process not a once off only looking at say NAPLAN. We should be looking at data on a very regular basis collecting not only test scores but other data [e.g. writing samples) that can inform our instruction.
Again I’m still at the early stages of trying to upskill my teachers in some of theis work so that we can embed this process into our regular work.
Mark