In a recent series of studies, it has been revealed that artificial intelligence (AI)-generated exam answers are not only going undetected but are also outperforming human students in several instances.
Researchers at the University of Reading in the U.K. have aired concerns about the integrity of tests after they surreptitiously submitted unedited artificial intelligence-generated exam answers, fooling exam markers and beating most human students.
In the study, the researchers created 33 fake student identities and used ChatGPT-4 to answer 63 questions in legitimate online assessments for undergraduate psychology courses containing short and essay-style answers.
Comment
byu/Maxie445 from discussion
insingularity
The professors on the course who marked the exams were not told about the experiment. The AI-generated students, on average, did better than the human students. The fact they were AI “verged on being undetectable,” said the researchers.
Only one of the papers handed in was flagged as possibly not being human. The authors of the study, who called it “the largest and most robust blind study of its kind” regarding trying to pull the wool over educators’ eyes, said this surely must mean the AI passed the “Turing Test” – when humans aren’t able to distinguish a computer from a human.
Comment
byu/waozen from discussion
intechnology
Dr. Peter Scarfe, one of the study’s authors and an associate professor at Reading’s School of Psychology and Clinical Language Sciences, called the study “particularly worrying,” adding that it should stand as a “wakeup call” to educators who, at some point are very likely going to get fooled by their students whose exam responses are the work of regular generative AI models.
“Many institutions have moved away from traditional exams to make assessment more inclusive,” explained Scarfe. “Our research shows it is of international importance to understand how AI will affect the integrity of educational assessments. We won’t necessarily go back fully to handwritten exams – but the global education sector will need to evolve in the face of AI.”
Concerns were somewhat ameliorated by the fact the AI was not as good at answering questions on tests given to third-year students, which required more abstract reasoning.
However, ChatGPT romped home for tests given to first- and second-year students. Since the study results were published, experts have said this could spell the end of take-home coursework.
Comment
byu/waozen from discussion
intechnology
Further studies at the University of Reading found that 94% of AI-generated submissions went undetected, with these submissions generally scoring higher than those from real students.
In a staggering 83.4% of cases, the AI submissions surpassed the grades of a randomly selected group of real student submissions.
According to the researchers, the findings suggest that students could successfully use AI to cheat and achieve superior grades compared to their peers.
This has raised questions about the effectiveness of current AI detection tools and the potential for widespread cheating using AI. While the AI itself did not have any time constraints typically associated with exams, the study design mirrored real-world scenarios where students could utilize AI within the given timeframes.
Comment
byu/Maxie445 from discussion
insingularity
The study focused on exams that required short answers or essay responses. The short answer exams had a 2.5-hour window, while essays had an 8-hour window.
Both were unsupervised take-home exams, so students could have easily used AI within those timeframes,” Associate Professor Scarfe highlighted.
They speculate that some real students might have also successfully submitted AI-generated work during the study.
However, the researcher suggests that question content, rather than type, can be a more significant factor.
This could correlate with question types e.g. essays generally allow more complex reasoning to be demonstrated than multiple-choice questions.
Comment
byu/Maxie445 from discussion
inChatGPT
Notably, the study employed two distinct approaches for comparing ChatGPT-generated answers with those of real students.
In comparisons in the paper, we either (1) compared all AI versus all student submissions (overall or specific modules), or (2) in the resampling analysis, randomly sampled from student submissions and compared it to the AI.
From an academic integrity standpoint, these findings are deeply troubling. Professor Elizabeth McCrum, Pro-Vice-Chancellor for Education and Student Experience at the University of Reading, also acknowledged AI’s transformative effect on education.
Comment
byu/Maxie445 from discussion
inOpenAI
They have undertaken a huge program of work to consider all aspects of teaching, including making greater use of technology to enhance the student experience and boost graduate employability skills.
These articles underscore the urgent need for the academic community to re-evaluate its approach to assessments and embrace the evolving landscape shaped by AI.
Educational institutions must adapt their assessment methods to maintain academic integrity, improve AI detection tools, and possibly return to supervised, in-person exams to mitigate the risk of undetected AI-generated submissions.
Comment
byu/waozen from discussion
intechnology
Educators and policymakers must develop new strategies that incorporate AI to enhance learning while preventing academic misconduct. Ongoing research and dialogue are essential to navigate the complexities introduced by AI in education.
For more news and trends, visit AI News on our website.
