As he sat in his office, working his way through a stack of essays for his undergraduate degree, Dr. Edward Skidelsky’s suspicions were aroused by a telling word: “dig deep.”
“The sentence was something like, ‘This essay dives into the rich tapestry of experience…’ and I thought, there’s that word ‘dive’ again,” says the University of Exeter philosophy lecturer. “The sentence is typical of the purple but empty prose that ChatGPT produces.”
ChatGPT, the AI software that creates free text on any topic in seconds, was launched by OpenAI in late 2022. Other models quickly followed – and their arrival has caused equal amounts of horror and excitement in the education world.
At its best, it’s a tool that streamlines research. A recent survey by the Higher Education Policy Institute (HEPI) found that more than half admitted to using generative AI to “prepare assessments.”
It’s fair to say that the uptake of generative AI software by university students is now endemic. But its ubiquity means that the routine task of grading essays has become an increasingly onerous task for thousands of tutors on campuses across the UK. For what began as a trickle of AI text appearing in student work has grown into a steady stream, resulting in “lots and lots of essays” written, at least in part, by generative AI.
In addition to ‘delve’, academic writing is now increasingly littered with AI favorite words such as ‘showcasing’, ‘underscores’, ‘potential’, ‘crucial’, ‘enhancing’ and ‘exhibited’.
There are other giveaways. Incorrect or incomplete references in students’ work to papers in academic journals may be because “ChatGPT has a hard time with page numbers.” Sudden changes in writing style within a single essay are another red flag, as is the lack of a sustained argument.
In this new world, bad grammar and spelling are comforting, but not annoying.
“Ironically, you know students aren’t using AI because they make mistakes in their grammar,” says Dr. Skidelsky. “ChatGPT content, while boringly empty, is error-free, and that immediately sets it apart from a student’s own work.”
Professor Steve Fuller, a professor of sociology at the University of Warwick, feels his antennae twitch when he encounters “certain words or phrases that are repeated in a mechanical way”, a sign that ChatGPT is mindlessly repeating phrases that often appear in the internet material it samples.
Fuller believes that most students do not cheat. That said, he does encounter what he believes to be AI-generated text on a regular basis. A telltale sign is when student responses contain little or no reference to course material.
“The required reading should appear in their answers,” says Prof Fuller. “But with ChatGPT there is no particular reason why it should. You end up with answers that may be correct but are very general and not really relevant to the course.”
Some professors have been blunt in their assessment of the impact. Des Fitzgerald, a professor at University College Cork, has said that the use of AI by students has “gone completely mainstream” and described it as “a machine for making crap”.
Meanwhile, academics are desperately trying to hold on to academic integrity. University policies around the use of AI can be vague and contradictory in practice.
Where the line of “appropriate use” is drawn is poorly defined. AI detection software is not very helpful. Its creators admit it is unreliable, and many universities don’t use it because of it.
The phenomenon of generative AI has so bewildered academics that they look back with nostalgia to the days of simple, old-fashioned plagiarism, which could be detected by software and checked against source material.
It is harder to prove that students cheated on ChatGPT. There is no source document to verify. As one academic puts it: the tutor can’t prove anything and the student can’t defend himself.
A HEPI study suggests that since the launch of generative AI, cases of academic misconduct have skyrocketed – doubling or tripling at some institutions. But academics say they are reluctant to report allegations without hard evidence.
“It’s impossible to prove and you’d waste a lot of time,” said Warwick’s Professor Fuller. “But if I have a hunch, you’ll see that in my figures and my observations.”
The professor recently gave an essay a score of 62 percent and wrote about it, “This looks like it was generated by ChatGPT.”
“I also gave feedback and explained that it was a very superficial treatment of the subject,” he says. “The student didn’t challenge me on it. I’m sure I didn’t get it all. [ChatGPT generated text] but I’m also pretty sure I’ve never given a scoop to anyone who’s used ChatGPT a lot.”
Dr Skidelsky in Exeter takes a similar approach: “You can condemn it because it is bad, but you can’t accuse it without evidence.”
Academics are caught between a rock and a hard place. Many believe that generative AI should used by students and integrated into courses because it is a given and employers expect them to use it effectively. However, overuse of ChatGPT risks students not putting in the effort needed to consolidate knowledge and develop critical skills and capabilities.
And it is naive of universities to treat generative AI as the equivalent of a calculator or an online thesaurus. As Dr. Skidelsky says, “It’s much more than just a tool; it actually replaces some pretty sophisticated cognitive processes, and if students are encouraged to use it, they end up not being able to think for themselves.”
One obvious way to ensure that students’ work is their own is through in-person exams – a method that has been replaced at many universities by coursework and unproctored online exams. So should they be reinstated?
Institutions argue that a diet of exams fails to assess the most important things or reflect the world outside the classroom. And, as one academic notes: “Students don’t like exams and they are the consumers of £9,250 a year and need to be kept happy.”
But as increasingly sophisticated versions of generative AI come to market, the clear signals that academics currently rely on to signal its usefulness are likely to disappear.
“The technology is moving very fast,” says Kieran Obermanan Oberman, an associate professor at the London School of Economics. “Eventually, essays generated by ChatGPT or something like that will not be bad and will not be obvious.”
He predicts a future with more “AI-resistant,” in-person assessments—tests, oral exams, and classroom presentations. In addition, policing could require students to save multiple versions of their essays to track edits, making “massive copy-and-paste jobs” a given.
“You think about it all the time,” Oberman says. “You look to the future and you know that the technology is going to get better and it’s going to be harder to detect and it’s going to be harder for students to avoid using it if everyone is using it. It’s like doping in sports, and academia, like sports, is extremely competitive.”