Tag: exams

Coursework moderation, exam bodies and technology

One of the big advantages of productivity suites like Google Workspaces for Education or Office 365 is the ability to easily share and collaborate.   This is great within schools, allowing students and staff to share and work together on documents and projects, however I also believe it starts to provide some other benefits for education in general, such as in relation to coursework moderation and exam bodies.

I have long believed the exam boards have lagged a bit behind in terms of technology use.   I remember being a visiting moderator for a vocational IT qualification some 15yrs ago and being presented by mountains of print outs.   I was visiting schools across England to carry out moderation activities relating to the evidence students had created in working with technology tools such as website development tools, spreadsheets, email clients, etc, yet it was all being printed out for me to look at.   I dread to think, on reflection, how many trees were cut down in the process.     It was around this time that I decided as a teacher of the same vocational qualification that I wouldn’t repeat this mistake, so I worked with our network manager to come up with a way to structure student evidence such that it was easy to extract and burn onto a CD (remember this was 15yrs ago!) which could be handed to the visiting moderator, rather than trying to bury them under mountains of paper.  Through the CD the moderator would have access to all student evidence in a structured and easily navigable form as opposed to a pile of A4 folders of printed evidence.

Office 365 and Google both allow for the easy sharing of digital evidence, which solves the above issue which I had previously solved with a CD, some network setup and a few batch files.    Recently working with one department who were using OneNote to store student evidence, we made use of the Parental Link functionality (See instructions here) to share the content with a moderator.    In some other areas we are using SharePoint for example to share video evidence of student work.   Now some planning does need to go into this, as some schools will have external sharing turned off in relation to data protection, however with a bit of thought and configuration, a solution can be found. 

This all highlights for me the need for exam boards to catch up.   Why arent exam boards providing more guidance to schools in terms of easily sharing digital student evidence with moderators?  Why is it being left to schools and their moderators?   Given most schools will now have either Microsoft’s or Googles suite in place, now is the time to drive things forward.   And it is about time, as looking back, I was trying to go paperless 15yrs ago, prior to the bandwidth and sharing tools which now greatly enable this to easily occur.

Additionally, and looking a little broader, why are we still making so many students sit in large exam halls to complete paper-based examinations following 2 years where the pandemic has meant that students and teachers all over the world have been reliant on technology to collaborate, communicate and engage in learning.   Why arent we looking at how technology can facilitate exams?  Now I note some initial pilots are being trialled but to me it all feels a little late in the day.   Again, there is a need for things to be driven forward here, and I don’t sense the drive and urgency I would expect.  

I feel schools have driven forward their use of technology over the last few years, urged on by needs resulting from the pandemic.   This has been great to see and has left schools in a stronger positive in my eyes.   But why are some of the services which underpin our current education system, such as the exam bodies, not working harder to do the same.   My main concern is that these services may serve to drag schools back, losing some of the technology-based advances we have so recently made.

Delaying exams; why?

So, a research study has arrived at the conclusion that due to Covid19 students may be 3 months behind in their studies.     The delaying of exams to allow students more time to catch up has also been discussed.   This all seems like rather simplistic thinking.

There are for me a number of issues with delaying the exams.

The first is that we already accept that exams differ each year and therefore there is already tinkering in place to adjust the grade boundaries to keep some consistency across academic years when looking at the statistical outcomes of students in general.   This is why the result show small but steady changes year on year rather than being more volatile. It seems to me to be fairly easy to just adjust this process to normalise the exam results next year should they be, as would be expected, lower than previous years and should it be important to maintain parity in results across different calendar years. And this statistical fiddle would be more acceptable than the algorithm proposed for 2020 results as it doesnt differ from the statistical adjustments of GCSE and A-Level results in 2019, 2018, etc.

Another issue, if we were to delay the exams, is that it simply knocks on to following years.   So, delay the GCSE exams would mean teachers would lose some teaching time they would likely use to start A-Level studies or to start Year 13 teaching of A-Level subjects following Year 12 exams.  As such it doesnt solve the issue, but rather displaces it. Is the focus not on learning rather than measuring learning? As such how can any solution with a knock on to teaching and learning be acceptable.

Also, the point students should be at the end of each academic year has been arbitrarily determined.   At some point the curriculum for each subject was developed and the content decided for each year or stage however it could have easily been decided that more or less content be added.   Why, therefore, is the point students should be at perceived to be so immovable?Why not simply reduce content for the year based on the reduced time available to students? Surely this is an alternative option.

There is also the point that next years results will be compared with this years results, where it has already been reported this years results were significantly up.   This obviously resulted from the use of centre assessed grades, provided by teachers, without any of the normal annual statistical manipulation in relation to grade boundaries.    This comparison is unavoidable.So, despite any delay, etc, there is still a high likelihood of negative reporting in the press with regards the 2021 results, with knock-ons in terms of students/parents being disappointed.

This bring us nicely to the big question I have seen a number of people ask, which is 3 months behind who or what?     Is it 3 months behind where teachers think they would be had Covid19 not arisen?   A prediction based on a predication doesn’t provide me with much confidence as to its statistical reliability.   Is it three months behind in terms of curriculum content covered at the predicted rate that content is covered?   Again this suffers given it relies on predicated rate of coverage of materials plus could the content be covered at a faster rate but in less depth possibly?

Maybe this issue is an opportunity to reassess our assumptions and to question our current approach regarding education and how it is assessed or are we simply going to accept that this is the way things are done around here and that any changes should be limited and only in maintaining the status quo? I believe we have reached a fork in the road, however I worry that we may look to take the route which looks easier.

Time to stop adjusting grades/grade boundaries?

If using an algorithm to adjust marks is unfair, as it has been deemed to be this year, then surely this practice must cease going forward.

The last few weeks have been filled with issues surrounding exam results.   One of these was being how the A-Level results were adjusted from centre assessed grades based on a statistical algorithm.   This was deemed to be unfair as it penalised some students or groups of students more than others.    The lack of equity was clearly evident due to the ability for schools to compare their centre assessed grades with the finally awarded grades.   It was therefore evident how the statistical adjustment, carried out in the interests of keeping results generally in line with previous year’s results, impacted on individual students.  The faces and lives of individual students could be attached to the grade adjustments.  This was deemed unacceptable.

My worry here is that this statistical adjustment has always gone on.   Normally students would sit exams with their resulting score undergoing adjustment in the form of changes in the grade boundaries.   Again, this was done in the interests of keeping results generally in line with previous years results and again some groups of students would likely be penalised more than others.    The grade boundaries changed due to the exam being deemed generally easier/harder.   The focus on the difficulty of the exam meant that seldom did we associate resulting grade changes with individual students; we don’t generally attach faces to this change, yet some students would have received lesser grades than had the adjustment not been carried out, the same as happened this year.    This seemed acceptable, and has been the way things have been done for decades, but I don’t see how this is any fairer that what happened this year.  

Maybe following this years issues, we need to take another look at how we assess/measure students learning and achievement including the associated processes.

Exams: A fair system?

Covid19 has forced the cancellation of this years GCSE and A-Level exams.   As a result of this schools are being asked to submit grades for their students with a number of people expressing concern over the fairness of the grades which will result.   But were the exams ever fair?

Public exams were introduced in the mid eighteen hundreds, over 150 years ago and I would suggest that they haven’t changed that much.    They are now a foundational element of the world we live in.   Your qualifications, or the grades you get in these qualifications largely determined by exams, dictate your progress and the options available to you as you progress through the education system.  From secondary education to further education then onto university and eventually careers each influenced by the grades you achieve.     To pass each exam a key skill is memorisation.  To memorise key facts, to memorise approaches to using these facts and to memorise and practice techniques to manage exams.   The issue is that in a world of technology, a world where most of us carry Google in our pockets complete with the worlds biggest collection of knowledge plus videos and animations to demonstrate how to use this knowledge, is it really critical that we memorise things?

For some student memorisation is easy as is exam technique and therefore the exam system is perfect for them.   For others memorisation is more difficult and exams bring out exam stress.  In terms of the end outcome both types of students may be capable of the same things, albeit one may need to quickly review Google for assistance.    This seems to be biased in favour of those students who are better suited to exams which is unfair.

Lots has also been written about differing achievement rates in different types of schools, different social-economic backgrounds, etc.   I am not going to expand on this here, only to say that most discussions finish with a lack of equity for students in differing circumstances being identified; that students access to learning and preparation for the all important exams isn’t fair.

Taking “fairness” a stage further, we would expect all students who answer a particular question in a particular way to achieve the same number of marks however, outside of fact-based questions, answers need to be marked by human examiners which introduces variability.  Yes, there are standardisation/moderation processes to try reduce the probability of students receiving unfair outcomes but these only reduce the probability rather than eliminating it.

We also have the annual adjustment of marks boundaries to reduce unfairness where a particular years exam questions might be more or less difficult that the previous years.   On one hand this is good as it seeks to reduce the probability of unfair outcomes, however the existence of this process, and of the standardisation/moderation processes mentioned above both acknowledge the fundamentally unfair nature of exams.

For 2019/20 there will be no exams and instead teachers who have worked with students throughout the year will be making professional judgements as to what students would have achieved.   Schools will then be seeking to check that results across the school are fair, followed by the exam bodies to checking fairness across all schools.   I am not sure how this will be any more or less fair than what we had before.   It may be that the sudden nature of the introduction of this process may be unfair, but outside of this I am unsure that it will have introduced any greater variability than that which existed previously.

I think those asking about the fairness of the 2019/20 results are asking the wrong question.  I suspect some may be invested in the exams machine, while some maybe so used to the exam system that they are scared of potential change and of the unknown.   The big question I find myself with is, if we can issue final qualification grades in 2019/20 without the need for final exams, do we really need these standardised exams in 2020/21 and beyond?

Exams: Why should 1/3 of students fail?

Not so long ago I read of a discussion in relation to whether the GCSE English Language should be scrapped.   Part of the reasoning behind this is identified as being due to the subject identifying a third of students as having failed.    As a headline I think it is difficult to disagree with.  How can identifying a third of students as having failed be an acceptable thing to do.    On reflection my view is that this issue is less about English Language subject and more about the educational system as it is now and as it has been for over one hundred years.

I remember when I worked within an FE college and I was involved in enrolment following the release of the GCSE results.   A-Level and Level 3 BTec courses had clear admissions requirements in terms of the minimum number of B’s or C’s required to gain entry to each course.   This often included the need for a minimum of a C in Maths or English.    I also remember working with students on their university applications, post A-Levels, where once again universities have entry requirements which students must achieve to gain entry.    Once again there might be a need for three C’s to get on their preferred university course.

The issue with the above is that a certain set of grades will gain entry and other lower grades will not result in entry.    It is easy to therefore perceive some grades as being passes and as a result the other remaining grades must be fails.     The education system as we know it is built on the ability to group students in terms of their ability, as described by their grades, and through this identify the opportunities which will be available.     As a result of this, independent of the U, or ungraded option, there will always be a perception as to some grades, those that easily permit entrance to the next level of education, being perceived as being passes and the remainder as being fails.

An alternative is to have qualifications which allow all students to pass.   From the headline point of view, improving from only two thirds of students passing to one hundred percent of students passing sounds logical and a success worth celebrating.   The issue is that it is unlikely to result in any real change.     FE colleges will still need to set requirements, meaning some passing grades will permit entry while others will not.    Universities will also set their requirements and again some grades will allow students to pass onto the next level whereas others will see their application fail to get them in.

The above alternative continues to be based on an education system where students pass through the system based on their age.    Given this there is a need to differentiate the students hence assigning grades to students based on their exams and coursework.

If we are to consider a system where all students are to achieve, we need to acknowledge the students learn at different rates.   We therefore need to allow students to progress through education at different rates.    The different rates of progress can therefore be used to differentiate students and identify when they are ready to progress to the next educational level.   Again this seems like an enviable solution in that students either complete or can be considered as having not yet completed or achieved.   They haven’t failed as the opportunity to complete always exists, being available for them at a time that suits their learning and rate of progressions.     The issue here is once again perception in that quickly there will become a view as to what the expected rate of progression will be.   This might be that by the age of 18 students will progress to university.   Instantly with this perception the media will be able to quote the percentage of students who proceed on or ahead of this target and therefore the percentage which do not.    Again we have those that progress as normally expected, those which pass, and those who progress at a slower rate, and therefore have not passed;  those which are perceived to have failed.

I don’t like the idea of one third of students failing.  It simply doesn’t feel right.   That said it is difficult to find an alternative solution that wont simply see us back in the same position a couple of years in the future.