On November 30, 2022, OpenAI launched ChatGPT to little fanfare. Two months later, the platform reached 100 million monthly active users, making it the fastest-growing app in history. Any and every industry and field that dealt in this little thing called human language would soon face a sea change, and the realm of education was no exception. Universities and educational institutions at nearly every level entered a frenzied panic — how could teachers trust that anything their students were doing outside of the classroom had its origin in their own intelligence and not that of machines?
The answer lay in the same technology that created the problem to begin with. AI detectors, it was thought, could be a silver bullet for teachers looking to sift effort and reflection from laziness and guile. But even before programs like ChatGPT sprang onto the scene, plagiarism detectors were known for giving teachers false positives, leading to students getting unfairly reprimanded. AI has only exacerbated this problem. Students can now produce convincingly high-quality essays with just a few clicks of the keyboard, cognitively offloading the mental exercise meant to educate them in the process. Teachers, in an attempt to not fall behind in the technological arms race, are starting to rely heavily on AI plagiarism detectors that continue to mislabel original content as AI-generated — with disastrous consequences for the students accused of cheating.
Here, we’ll look at how AI has harmed schools and educational programs around the globe, from increasing the possibility of student cheating to tech dependency and more. Just as importantly, though, we’ll dive into the reason why the tech’s most flagrant drawbacks might also belie its greatest benefit: how institutions are being forced to consider how to make education increasingly human in the face of unprecedented technological change.
AI cheating
The rise of AI tools like ChatGPT has introduced a powerful new avenue for academic dishonesty, making it easier than ever for students to bypass traditional learning processes to get the grade and pass the class. Apart from the standard essay writing — the gold standard in assessing a student’s ability to think critically and synthesize information into a coherent and compelling argument — AI platforms can solve complex math problems (to varying degrees of success) and even generate computer code with relatively minimal input.
Instructors are not great at detecting when these are used. In one study published in the journal PLOS One, researchers posing as university students submitted fully AI-generated essays in examination systems at a psychology program in the United Kingdom. The result? 94% of the AI submissions went undetected. The potential for misuse extends beyond writing. In computer science classes, teachers are concerned that programs like GitHub Copilot, Microsoft’s code-suggestion tool, will mean they have to completely retool how they conduct their courses.
Students are jumping at the opportunity. Hundreds of cases of AI-related cheating in Scotland have been reported in the last two years. One student in Turkey was recently arrested for using AI on a university entrance exam. They had rather ingeniously utilized a camera disguised as a shirt button linked to AI software by way of a router hidden in their shoe. The software generated the correct answers, which were then relayed to the student through an earpiece.
Overreliance on tech
AI’s integration into classrooms has brought efficiency and innovation, but it has also raised concerns about overreliance. Research published in the journal Computers & Education indicates that student use of AI results in lower individual agency, meaning that rather than learning from the technology, they come to rely on it. Such studies indicate that developing new human-AI hybrid methodologies in the classroom could be of increasing importance.
Teachers, too, are increasingly dependent on AI tools like plagiarism detectors, which are known to generate false positives. According to one survey published by the Center for Democracy & Technology, at least two-thirds of a nationally representative sample of 460 instructors across the U.S. utilize such AI content detection tools, a figure that stands in stark contrast to evidence of those tools’ ineffectiveness. Interestingly, researchers working at OpenAI have developed a kind of AI watermarking technique that is consistent and effective in its detection methodology. However, market pressures keep that tool behind closed doors, as its release would likely result in ChatGPT users flocking to OpenAI’s competitors, doing little to abate the overall problem. However, new efforts from Google could make it easier to identify AI text as well.
Even grading processes have been adapted to incorporate AI tools. Instructors are turning to ChatGPT and its ilk to grade essays and other project work more efficiently, something that has caused a rift among educators on the ethics and effectiveness of doing so. Beyond that debate, even just uploading a student’s work to an AI platform without their consent could constitute a massive breach of privacy.
The spread of misinformation
Large language model-powered AI systems are known for their tendency to hallucinate; that is, present misinformation in a highly convincing and authoritative-sounding manner that can be surprisingly difficult to catch, even for experienced users. In the context of educational institutions, whose explicit goal is to relay facts and impart the ability to discern reality from falsehood to their student body, this is particularly problematic.
The issue is far from theoretical. In December 2023, a student at Hingham High School in Massachusetts received a failing grade on an AP U.S. History project after copying text from an AI program and submitting it as their own. The text the student submitted included several citations to nonexistent sources — pure hallucination on the AI’s part. Claiming the school had no such anti-AI policy on the books, the student’s parents sued the school to amend the grade and expunge the scandal from his record, an injunction that the U.S. District Court for the District of Massachusetts rejected.
Higher education is not exempt from the problem. One research study published in Scientific Reports found that 55% and 18% of literature reviews produced by ChatGPT-3.5 and ChatGPT-4, respectively, fabricated bibliographic citations for which no scholarly work actually exists. As always, instructors have their own pitfalls to watch out for as well. As more teachers begin to incorporate AI into their lesson planning procedures, the chances of an educational authority unwittingly presenting misinformation as factual to their students rise significantly.
Biases in the dataset
AI-powered tools have revolutionized personal learning, but their reliance on vast, unfiltered datasets has introduced a significant drawback: bias amplification. Algorithmic bias is when algorithms disadvantage certain groups more than others, whether the demographic characteristic is gender, race, nationality, or something else. When tech developers train models on data that replicates existing biases, knowingly or not, this can have serious downstream effects on the individuals using these systems.
Even OpenAI acknowledges this openly, stating on its website that educators should exercise caution when reviewing ChatGPT’s outputs, given that it leans toward Western views and favors English-language inputs. Bias against non-native English speakers in AI systems more broadly is also an issue, with some AI detectors labeling writing from such students as AI-generated at a higher rate than work from native speakers.
AI is also used in educational institutions as a predictive analytical tool. Often, these tools are employed to determine the likelihood of a student graduating from high school, for example. While they are meant to help educators better support students in their academic journeys, they often fall short of their aim, incorrectly labeling racial minorities as less likely to flourish. One such system in Wisconsin, introduced in 2012, consistently identified students of Black and Hispanic backgrounds as being at risk of not graduating high school. The problem is that the algorithm the state used was wrong nearly three-quarters of the time, creating an unfair perception of students from minority backgrounds in the minds of educators.
Harnessing AI for good in the classroom
For all the downsides of AI technologies, they have the potential to transform classrooms for the better when used thoughtfully to enhance the teaching and learning experience. If educators can effectively incorporate these tools into a more efficient, enriching, and human-centric experience for learners, the shock and challenges that ChatGPT and its competitors have delivered to institutions everywhere may counterintuitively end up being the source of the tech’s greatest benefit to the field.
Research is beginning to show that AI tools are great brainstorming partners for students, serving as the initial spark to streams of curiosity and inquiry. Crucially, however, students should be given chances to exercise and establish their creativity before using tools like ChatGPT, which can be brought in later to extend their learning. Essentially, AI may be great as a kind of tutor to provide explanations that enrich thoughtfulness, but using it as an answer generator is likely to impede learning more than anything else. For teachers, AI-driven feedback and even real-time activity generation are among the most inspiring of the tech’s applications, fostering a more responsive and guiding environment for learners.
As in all things, a considered balance with AI in education seems to be the key to unlocking its potential while minimizing its harm. If you’re interested in learning more about AI’s impact on the world, read our breakdown of the chilling concern Elon Musk had about Google’s DeepMind AI and the threat Google’s Gemini AI reportedly told a student.
