The effect of AI on knowledge assessment 

In 2015, Mark Dawe, then Chief Executive of Oxford, Cambridge and RSA (OCR), told the Today programme that “allowing internet use in exam rooms reflected the way pupils learnt and how they would work in future". The idea behind this was that students should be assessed not on knowledge recall but on the understanding of their search output and the ability to apply the results in the context of the assessment.

There is some debate among employers about whether “typical” or “traditional” education is adequately preparing students for the workplace. Are companies placing enough focus on skills such as creativity, problem solving and effective communication?

During the COVID-19 pandemic, organisations needed to find a way to continue operations, including developing their learners. They achieved this mainly through video calls — this shift to technology to overcome current issues dates back to the 1800s when Bell Labs developed the concept of transmitting images and audio over wire, with the first instance recorded in 1927. This is not new technology. What’s new are further developments, such as screen sharing, virtual whiteboards and document sharing, that add value to the experience when working and learning remotely.

In online assessments, digital tests are also not new. Features such as remote proctoring, AI-assisted proctoring, and automated scheduling and certificates add value by enhancing the user experience and reducing time and resources spent developing and evaluating assessments. Moreover, an increasing variety of devices can be used to conduct an online assessment using minimal bandwidth.

As new technology enters the market and examples of AI-enhanced images and voice covers flood our newsfeeds, advocating an “approach with caution” style seems reasonable. But should institutions ban technology such as ChatGPT, or is it better to gain an understanding of the technology for future skills and employment?

Should educational institutions follow employers’ lead? Mercer’s recent Real-Time Insights Survey on Shaping the Future of Work shows that 58% of businesses plan to use generative AI in HR in the next 12 months, with 29% looking to use AI in recruitment-related tasks such as assessments and screening.

Most of us have limited understanding of the capabilities of AI technology. However, the output from generative AI is highly dependent on the information you put into it. “With ChatGPT, the magic happens when you chat to it and adjust the information you are putting into it.” (Dan Fitzpatrick, The AI Educator). Effective communication is needed here — a skill employers already look for. This reinforces Mark Dawe’s 2015 argument for using technology not just to find the answers but to apply them to the context of the question. Just 25 years ago, Google itself was the emerging technology.

An increased emphasis on listening to employees and understanding what they want from their employers has prompted companies to redesign work through reskilling and upskilling. Job roles are changing, and many jobs that future generations will seek don’t currently exist. Mercer’s Real-Time Insights Survey shows that HR leaders believe collaboration, technology and communication will be the most important workforce skills in the next 24 months. Other important skills highlighted included cognitive, management and entrepreneurial skills. Is “traditional” education doing enough to prepare future generations for these changes?

More and more employees are looking for companies that allow them to carry out projects that spark their interests alongside their day-to-day roles. As such opportunities become more common within organisations, job roles become less defined. So how does this affect the hiring process? Should employers put more emphasis on competencies?

Granted, many jobs require a certain level of knowledge. Therefore, we cannot simply swap out a specific skills assessment for a competency- or behaviour-based assessment. There is also the question of how to determine what makes a person proficient in a particular skill. Is it their knowledge in the area, their experience or their application of the knowledge?

Behavioural assessments examine the attitude needed to achieve something rather than assessing a person’s ability to pass a threshold. This goes hand in hand with assessing not only a person’s eligibility for a role (that is, the qualifications they hold) but also their suitability for the role — their personality and whether it will fit with the team’s culture and ethos.

There are many factors to consider when looking at how we assess knowledge and the role technology plays in the future of learning. As technology changes and job roles adapt, employers will seek more from potential candidates — not only in the knowledge and skills they hold but also in their competencies, behaviours and attitudes. 

How we assess these areas will be vital in the hiring process. However, preparing candidates for such assessments and the world of work requires input from their education. Educational institutions and their approaches to using technology play a large role in how we ready future generations for the world of work. This includes the development of vital skills, such as collaboration, technological proficiency, problem solving, adaptability and effective communication. Employers need a balanced view of students’ competencies both in and out of the classroom. 

Mercer Assessments helps organisations by providing a single cloud-based platform that supports the assessment of such competencies, from effective communication to technology. With 26 different question formats, Mercer Assessments can evaluate both skills and knowledge. 

Furthermore, Mercer’s talent assessments can assist in identifying skills gaps within organisations, while our skills-based talent solutions support strategic workforce planning, reskilling, succession planning and career development. Workforce transformation helps businesses understand their future people and skills needs and plan how to meet them. 

We don’t know where technology will take us in the next five or 10 years, but embracing change and innovation is a good place to start in implementing this technological revolution.