So the PISA results were released today and with them a flurry of online articles providing various analysis and conclusions from the data. It is my intention to post a couple of times over the coming weeks in relation to PISA and standardized testing. As my first post my inspiration comes from a twitter discussion from the weekend, as part of the weekly #sltchat, where recruitment was being discussed. The below tweet highlights the particular strand of the chat which I would like to highlight:
This strand of the discussion revolved around this constant need to bang on about how UK education is failing, is poor, isn’t working and a variety of other less than positive descriptions.
So what does this have to do with PISA. Well, during my usual browse through social media and the news I came across an article in the Guardian looking at the PISA results. You can read the article in full here. The title:
“UK schools fail to climb international league table “
The use of “fail” in the article title; Not exactly a positive start. It become more interesting when you dig around in some of the figures. Lets just take the Science results; The results show a fall from 514 in 2012 down to 509 this year which seems to align with the less than positive reporting however this doesn’t tell the full story. It should also be noted that a drop of 5 represents a less than 1% variation. Could this variation be explained through uncontrollable random variation within the sample group? Is this drop statistically significant? I doubt it.
Ignoring the issue of statistical significance for a moment, the UK science position in the rankings rose 6 places between 2012 and 2015 which seems to be a slightly more positive picture. Also looking at the average of all countries we find that this fell from 501 to 493, representing a drop of 8 yet the UK only dropped by 5. This UK difference, the drop of 5, could be considered as an improvement against the average across the period. Also we should note that UK score of 509 is above the average, which again sounds reasonably positive. A US article on the TIMSS data from last week had proclaim merrily that US students were “above average” however in this negative article we in the UK make no such claim despite the fact it would be valid. The article was quick to point out falling UK results but didn’t report changes in the OECD averages across the period for comparison.
The article also didn’t share any information with regards the numbers of students involved in testing within each country and how this sample compares to the overall population of study within each country. From a statistical analysis point of view this information would help in establishing the reliability, or lack there of, for the data.
So all in all I feel the negativity of the article doesn’t truly tell the story plus there is a lot of information missing which may cause us to question or at least assign less weight to the findings.
And all of the above is before I start discussing the issue of using standardized testing as a way to direct how individual students are taught in individual schools within individual geographical areas each with their own individual needs and context (did I use “individual enough to get my point across?). Not to mention possible discussions in relation to the statistical value of the findings and also the impact of natural random variation on the results.
Do I like or value the PISA finding? No not really, but that isn’t the point here. My point, and I may have gone the long way about it, is why are we allowing such a negative view to be projected onto our education system when even the data seems to have some possible positive indicators. Lets have some celebration of successes for once, of first steps in a positive direction. Lets have anything except finger pointing!
Technology and Learning