I recently spoke at an AI event for secondary schools in which one of the topics I spoke on related to AI and its impact on Assessment. As such I thought I would share some of my thoughts, with this being the first of two blogs on the first of the sessions I delivered..
Exams
Exams, in the form of terminal GCSE and A-Level exams still form a fairly large part of our focus in schools. We might talk about curriculum content and learning but at the end of the day, for students in Years 10,11, lower 6 and upper 6 the key thing is preparing them for their terminal exams as the results from these exams will determine the options available to students in the next stage of their educational journey. The issue though is that these terminal exams have changed little. I provided a photo of an exam being taken by students in 1940 and a similar exam in recent terms and there is little difference, other than one photo being black and white and the other being colour, between the photos. The intervening period has seen the invention of DNA sequencing, the mobile phone, the internet and social media, and more recently the public access to generative AI but in terms of education and terminal exams little has changed.
One of the big challenges in terms of exams is scalability. Any new solution needs to be scalable to exams taken in schools across the world. Paper and pencil exams, sat by students across the world at the same time accommodates for this. If we found life on Mars and wanted them to do a GCSE, we would simply need to translate the papers into Martian, stick the exams along with paper and pencils on a rocket and fire them to Mars. But just as it is the way we have done things and the most easily scalable solution doesn’t make paper and pencil exams the best solutions. But what is the alternative?
I think we need to acknowledge that a technology solution has to be introduced at some point and the key point is the scalability based on schools with differing resources. As such we need a solution which can be delivered in schools with only 1 or 2 IT labs, rather than enough PCs to accommodate 200 students being examined at once as is the case with paper based exams. So we need a solution which allows for students to sit the exams in groups, but without compromising the academic integrity of the exams where student share the questions they were presented with. The solution, in my view is that of adaptive testing as used for ALIS and MIDYIS testing by the CEM. Here students complete the test online but are presented different questions which adapt to students performance as they progress. This means the testing experience is adapted to the student, rather than being a one size fits all as with paper exams. This helps with keeping students motivated and within what CEM describe as the “learning zone”. It also means as students receive different questions they can sit the exam at different times which solves the logistical issue of access to school devices. Taken a step further it might allow for students to complete their exams when they are ready rather than on a date and time set for all students irrespective of their readiness.
AI also raises the question of our current limited pathways though education, with students doing GCSES and then A-Levels, BTecs or T-Levels and then onto university. I believe there are 60 GCSE options available however most schools will offer only a fraction of this. So what’s the alternative? Well CalTech may provide a possible solution; They require students to achieve calculus as an entry requirement yet lots of US schools don’t offer calculus possibly due to lack of staff or other reasons. CalTechs solution to this has been to allow students to evidence their mastery of calculus through completion of an online Khan Academy programme. What if we were more accepting of the online platforms as evidence of learning and subject mastery? There is also the question of the size of the courses; GCSEs and A-Levels and BTec quals are all 2 years long but why couldn’t we recognise smaller qualifications and thereby support more flexibility and personalisation in learning programmes? In working life we might complete a short online course to develop a skill or piece of knowledge on a “just-in-time” basis so why couldn’t this work for schools and formal education? The Open University already does this through micro credentials so there is evidence as to how it might work. I suspect the main challenges here are logistical in terms of managing a larger number of courses from an exam board level, plus agreeing the equality between courses; Is introductory calculus the same as digital number systems for example?
Coursework
Coursework is also a staple part of the current education system and summative assessment. Ever since Generative AI made its bit entrance in terms of public accessibility we have worried about the cheating of students in relation to homework and coursework. I suspect the challenge runs deeper as a key part of coursework is its originality or the fact that it is the students own work but what does that look like in a world of generative AI. If a student has special educational needs and struggles to get started so uses ChatGPT to help start, but then adjusts and modifies the work over a period of time based on their own learning and views, is this the students own work? And what about the student who does the work independently but then before submitting asks ChatGPT for feedback and advice, before adjusting the work and submitting; Again, is this the students own work?
There is a significant challenge in relation to originality of work and independent of AI this challenge has been growing. As the speed of new content generation, in the form of blogs, YouTube videos, TikTok, etc, has increased year on year, plus as world populations continue to increase it become all the more difficult to be individual. Consider being original in a room of 2 people compared with a room of 1000 people; The more people and the more content, the more difficult it is to create something original. So what does it really mean for a piece of work to be truly original or a students own work?
The challenge of originally and students own work relates to our choice of coursework as a proxy for learning; It isnt necessarily the best method of measuring learning but it is convenient and scalable allowing for easy standardisation and moderation to ensure equality across schools all over the world. It is easy to look at ten pieces of work and ensure they have been marked fairly and in a similar fashion; Having been a moderator myself this was part of my job visited schools and carrying out moderation of coursework in relation to IT qualifications. If however generative AI means that submitted content is no longer suitable to show student learning, maybe we need to look at the process students go through in creating their coursework. This however has its own challenges in terms of how we would record our assessment of process and also how we would standardise or moderate this across schools.
Questions
I don’t have a solutions to the concerns or challenges I have outlined, however the purpose of my session was to stimulate some though and to pose some questions to consider. The key questions I posed during the first part of my session were:
- Do we need an annual series of terminal exams?
- Does there need to be [such] a limited number of routes through formal education?
- Why are courses 2+ years long?
- Should we assess the process rather than product [in relation to coursework]?
- How can we assess the process in an internationally scalable form?
These are all pretty broad questions however as we start to explore the impact of AI in education I think we need to look broadly to the future. In terms of technology the future has a tendency to come upon us quickly due to quick technology advancement and change, while education tends to be slow to adapt and change. The sooner we therefore seek to answer the broad questions or at least think about them the better.
Technology and Learning
One thought on “AI and assessment (Part 1)”