Month: March 2023

Technology and Exam boards: Time to modernise?

I recently received a request from a teacher in relation to getting some software installed on their school device to support them in marking for an exam board.    Now I know this isn’t part of their school role however having been a standards moderator in the past, I understand the benefits to schools and colleges of having markers or moderators within teaching departments.   I am therefore eager to try and enable staff by supporting such requests however this request involved a piece of software which requires admin rights to the laptop, both for install and for the operation of the application according to the exam board.   When the concern re: cyber security was raised the exam boards final reply was that the staff member should install the software on a personal rather than school laptop.   This got me thinking about how technology has changed but how exam boards have been slow to change.   This is all the more evident currently.   Just look at the advances in Large Language Models (LLMs) with ChatGPT over the last six months.

Traditionally, examination boards have relied on paper-based tests and manual grading systems. However, these methods have several drawbacks, including the potential for errors and delays in results processing.    One way examination boards could modernize is by moving towards computer-based testing. Computer-based testing allows for faster and more accurate grading, as well as the ability to customize exams to the specific needs of each student.  I very much believe that adaptive testing is the way forward, with this also enabling students to take exams in their own time when they are ready as opposed to at a set time with all other students.   Adaptive testing also supports students taking their tests anywhere, including at home, rather than having to be crammed into a large exam hall where the conditions themselves are not exactly designed for optimum student performance.    Additionally the results would be available much quicker reducing the stress associated with a long waiting period between the exams and the results being released.   There is also the potential benefit in the reduction in the amount of paper used in exams, transporting of these papers, etc, which may help with making the exam process more environmentally friendly.

Another way examination boards can modernize is by utilizing artificial intelligence (AI) in the grading process. This appears all the more relevant at the moment with development in LLMs like Chat-GPT.   AI-powered grading systems can quickly and accurately grade exams, allowing for quicker results processing and reducing the potential for errors. AI can also analyse student performance data to provide insights into areas where students may need additional support and guidance.   Now I note here that the use of AI may introduce new errors to the marking process however I would suggest that the volume or magnitude of these errors when compared with human based marking is likely to be lower.  It isn’t “the solution” to errors but definitely a step in the correct direct.

Related to the above, exam boards need to acknowledge the existence of AI and LLMs and the fact that they will become an increasing part of life and therefore a tool which students will increasingly use in their studies be it for revision, to help in developing critical thought or for creating coursework or other learning content.   So far only IB (International Baccalaureate) have really acknowledged ChatGPT and how they see it impacting on their courses, providing at least some steer for schools on what appropriate or inappropriate usage might look like and proving at least some direction for schools and teachers for managing these new technologies.

Moreover, examination boards can use technology to improve exam security. Online proctoring tools can help ensure that students are taking exams in a secure and controlled environment, preventing cheating and other forms of academic dishonesty.    Related to this, I have seen exam boards continuing to send out resources on CDs or USB drives, or requesting student video or audio work using similar formats.   It is about time that they provided appropriate online portals to allow the quick, efficient and secure transfer of such exam and coursework data.  

Finally, examination boards can use technology to make their exams more accessible to students with disabilities or special needs. For example, screen readers, text-to-speech software, and other assistive technologies can help students with visual or hearing impairments to take exams on an equal footing with their peers.   This is already happening for a subset of students however I suspect eventually will need to acknowledge that all students are individual and having differing learning preferences including their device use and the online tools they use.  In classrooms teachers support students using a range of tools and techniques so it is only correct to seek to support the same in the final exams which are, at least for now, viewed as so critical in a students format education.   As such examination boards will need to adapt to this.

Conclusion

Technology has the potential to revolutionize the examination process, making it more efficient, accurate, and accessible. Examination boards must embrace these technological advancements to ensure that their exams are of the highest quality and that students receive accurate and timely results. By doing so, they can help prepare the next generation of students for success in a rapidly changing digital world.   

And at a time when the pace of technology, particularly in relation to Artificial Intelligence solutions, has never been faster, the exam boards will need to significantly increase their agility and their ability to adapt to and embrace change.

AI and Learning Platforms

Software learning platforms which come complete with learning content for students to work through are not new.   I remember an online Maths programme from my days as a university student as I was studying to become a teacher back in the late 90’s.   Basically, you worked through content and then were presented with different options as to how you progressed through the programme.    As a learner the individual modules of content were pretty much fixed, having been written into the software, but the path through the wider programme of learning was up to me.    I was provided options as to how I progressed from one module to the next.   Now, I was never a great fan of this as each module was presented in a given way and worked through examples in a given way, as it was programmed to do.  If you didn’t understand the way it was presented then there was no help or way to progress through this module although you could move to further modules in the hope they would provide you with insight which might eventually get you past this issue.    I liked the idea of online programmes and self paced learning however had concerns about user motivation, especially when you hit concepts which provide difficult for you to understand, and about the fixed nature of the content materials;   A great teacher adjusts and customises their learning materials and approach to their class and the individual students within it.   As such the self paced learning aspect was a step forward but this was about as far as it goes.

Fast forward to more recently and little progress had been made, at least as far as I saw it.   Newer learning platforms are capable of gather much more diagnostic data and analytics which allow the developers and content writers to adjust and improve their content.   So, the content is better than the content I experienced in the 90’s but generally it still provides largely linear and fixed content and if the content, its style, etc don’t match your needs then there is little that can be done.   As so, until very recently I have had a largely negative view of learning platforms which come complete with the vendors own content which teachers cannot adjust or customise to their content.   They have their place for example supplementary to classroom teaching or self paced learning when teachers are absent but that was it.

That was until recently when I saw a video of some new developments within the Khan Academy platform including its new use of the GPT4 Large Language Model (LLM).    Still the content in terms of problems set within the platform and the way they are worked through appears very linear and fixed.  So if it is maths problems it will work through the problem in a specific way;  no change there.   The difference, and the massive leap forward in terms of learning platforms is their new chat bot style assistant.   It prompts and supports the student using the platform.   It identifies common misconceptions and provides guidance.   It acts as a coach and facilitator but customising its responses to the efforts being made by the student using the platform and this includes providing motivational “well dones” and corresponding emojis.    Watching the demo it was almost as if there was a teacher sat behind the chatbot rather than an AI solution.    Now I note that this demo was short and was for the purposes of showing off what is possible in the Khan Academy platform so may not be fully representative of how it all looks and feels in real life, however if the final product is anything close to this then it is a major shift forward.

Flipped learning has been a concept long discussed looking at releasing teachers from supporting students practice of learning concepts however maybe AI solutions like GPT4 and its use in Khan Academy will allow us to release teachers from more of the basic learning.    Maybe the AI and learning platform can be used here, allowing teachers to act more as facilitators rather than delivering new learning, and allowing them to focus much more on the high order skills of creativity, critical thinking and the like.    

AI and large language models could potentially facilitate significant shifts in what learning in our schools and colleges looks like, not in the distant future, but in the very near future indeed.

Cyber culture

The enterprise org budgets being spent in relation to cyber security have, for a number of year, seen a steep increase however at the same time the volume of attacks and size of attacks have also seen a continuing and steep increase.  From a return in investment point of view this doesn’t look good.   In how many areas of a business or school would we be willing to accept increasing spends but worsening results?

Now this isnt such a big issue for schools and colleges as the available budget which might be applied to cyber security are very small indeed however viewed from a different perspective, this might mean it is all the more important to spend that which we have carefully and correctly.  

Or maybe we need to start looking at the problem differently?   If we accept that additional money and associated spends on technology tools and more staff won’t necessarily solve the problem then what can we or should we be doing?

Culture

I suspect this is key to how we need to approach cyber security.  It needs to be “how we do things around here” rather than something which is seen as an IT issue or, where things have progressed a little further, an IT and SLT issue.   Cyber security and appropriate cyber behaviours need to permeate a school, being the responsibility of everyone in the school community.    Everyone needs to understand why it matters and what part they play in keeping users, data and systems safe.     Now building such a culture isnt a quick process however I suspect it is something we need to start developing now, as part of a longer term journey to having more cyber resilient schools.

Measurement

Another area that is important is the need to have some form of measurement.  In order to make sure our cyber efforts are effective we need to be able to measure this effectiveness.   This might relate to awareness of phishing or a multitude of other measures we might create in trying to assess our cyber security.    The key however is the need for some sort of measurement so we actually have some data as how we are doing, to help identify areas we need to focus on and to assess whether our efforts bring about the positive change we are hoping for.    This measurement could be the data from a phishing awareness exercise, from help desk calls or even from a RAG (Red/Amber/Green) rating exercise.    It needn’t be overly complex but it needs to provide some meaningful data in terms of where we are at the point the measurement is taken.

Accountability

The third area which I think is key, and which was shared at a TEISS InfoSec event I attended, is the need for accountability.   We might have data as to where we are, or where a given department is or a school within a school group, but who is responsible and accountable for moving things forward?    We need to ensure this is clearly identified and again it isnt simply an IT issue and instead should belong to the business, the school.   It may therefore be that the HR manager is responsible for the HR dept, while the academic Head is responsible in terms of academic data, processes and staff.    Whatever the accountability lines are, they need to be clear and understood.

Conclusion

On reflection, the above isnt a quick fix;  culture takes a long time to develop and even establishing accountability and measures for assessing cyber readiness will take time.  We need to ensure we are measuring the right things and that accountability is set at the correct hierarchical level, with this taking some time to get right.   That said, the current approach, and complaint regarding lack of money/resources, doesn’t work as additional  money/resources havent solved issues for those which have more of both money and resources currently.    As such I think maybe we need focus on cyber culture in the same way we have previously focussed on safeguarding culture in schools.   Maybe we all need to be focusing on cyber culture?

TEISS European Information Security Summit

I try to step outside education at least once each year, looking at the bigger technology world by attending an industry event.  The most recent of these ways the TEISS European Information Security Summit on 23rd Feb in London.    I feel it is important to keep up to date with the wider technology world to sense check my thoughts and ideas and to benchmark technology in education against technology in other sectors.    During the course of the event it was interesting to have discussions from a diverse range of industries including highly regulated industries like banking.   Hearing that they suffer similar issues to education, such as shadow IT or issues identifying responsibility for data, but at a much larger scale was reassuring.

Given below are some of my takeaways and thoughts from the various sessions and discussions I had throughout the course of the conference.

Budgets and Cyber

One of the first takeaways from the event related to cyber security and budgets.    It was presented that cyber budgets and cyber spending has been on the increase for a number of years.   It was also however indicated that the volume of attacks and the size of attacks continue to increase.    For me this suggests that more budget, including more staffing associated with additional budget, does not necessarily solve or improve the situation in relation to cyber.   From the point of view of schools and colleges this is important given the limited budgets available.    I think this highlights the need to start approaching cyber and cyber risk a little differently including possibly being more accepting of the fact we will never reach 100% secure and therefore accepting cyber as a journey and simply trying to focus on our key “business” assets and on continual improvement in relation to cyber security in whatever form this may take, including where this may be simple and small improvements.

Gamification

User awareness and cyber security culture was one of the three main streams offered at the conference with one session looking specifically at the potential use of gamification in relation to cyber security awarenss training.   It is true that often cyber security and other online training can be a boring process of reading a screen of text and clicking next repeatedly before completing a test at the end.   Clearly not an engaging experience and therefore possibly an experience  where little long term or deep learning takes place;  We may remember for long enough to answer the test at the end, but ask the same questions a week later and I suspect the retention of the content will have dropped to very low indeed.   So this is where gamification comes in.    The presenters identified two types of gamification, being content or structure based.   In content based gamification the content is presented as a game.  In structured based gamification the content is the same but includes some sort of leader board, prize of other enticement to engage users.   As the session was presented I was thinking of the potential of doing a Kahoot quiz with heads of department where they need to identify whether emails are trustworthy or not for example.     I also thought about some sort of competition between departments so maybe a quiz or phishing test which results in a cyber score which can be reported and compared with other departments.   This is one area I will certainly be looking into in the short term to see how I can try to gamify user awareness materials and processes, and to see what impact that has.

Civic duty rather than organisational cyber security awareness

Another point that was made during the conference was to engage people on security awareness beyond simply keep the organisations data secure but to accept that we can also deliver a civic benefit in making users more secure, both personally and also professionally.   Where we seek to do this we are more likely to engage users and have them learn from awareness programmes plus additionally we address the risk of a personal cyber incident potentially impacting on the school or other organisation anyway.  Take for example the compromised personal mobile phone:  It may have organisational email on it or info about the individual which could be used in crafting attack against them in their professional context, among other data which could pose a risk to the organisation.

Regulation as a change agent

One of the panel sessions I attended involved discussion of change and of compliance with security standards, change processes, etc.     From a school and college point of view this can be difficult as although policies are in place sometimes these will be overlooked and busy staff, both teachers and support staff, as well as students, may fail to engage with requirements or training around cyber security.    One of the panellists in the session highlighted that this wasn’t an issue in financial technology (FinTech) due to the nature of the business being heavily regulated meaning the penalties for non-compliance, for both the individual and the organisation, can be quite extreme.   Taking this insight and applying it to education got me thinking of the potential for the DfE to set requirements and of ISI and Ofsted to then include this within the inspection requirements.   Now the release of the DfE standards is a small step towards this however I suspect that is about as far as things will progress, which without any monitoring or penalties for non-compliance, is very limited in terms of impact.

Cyber insurance

There was a good session discussing cyber insurance with a very clear take away.  The session talked about how the cyber insurance market has seen policy costs increase along with greater requirements to get insured.   The questionnaires which you need to complete were a particular focus of discussion in that some of the questions are not easy to answer or not appropriate in a given context.   I have never really thought about this however the panel highlighted that the purpose of these questionnaires is for the underwriters to get a view of the risk in order to provide their proposal.   As such if the questions don’t make sense, it is the underwriters which we need to discuss this with to find out what it was they were hoping to find out from a given question.   Apparently the underwriters often don’t have access to client information, with this handled by the broker, so it is for the client, the school or college, to request a discussion with the underwriter and to initiate dialogue.

Conclusion

Cyber security seems to me to very much be a business risk, including where that business is the education of students.    As such it impacts all organisations albeit the scope of impact and the scope of risk varies.    This means there is a lot to gain from sharing experiences and ideas across sectors rather than just within sectors.    Having attended this industry focused information security event, where I think I may have been one of very few from the education sector, I came away with a fairly long list of ideas and things to try.    

But if I am to leave this post with one thought it is that maybe we need to get past the doom and gloom of cyber and become more accepting of doing what we reasonably can and of seeking to constantly improve, even where these improvements might only be small and minor;   It is about risk management.Any progress in the right direction is progress after all.