The other week, I was invited to contribute to a panel discussion for the Social Science Faculty of my university. The topic of the panel was "Impacts of AI on the Social Sciences". I have to admit that when I saw that the panel clashed with part of my teaching, I was relieved, but in the end I was able to present at the panel and head straight to my class after (which was, incidentally, also about generative AI and plagiarism). The reason I felt trepidatious is probably familiar to many people who a) work in natural languge processing, b) work in science and technology studies, c) work in linguistics or d) work in ethics and social impacts of language technologies (and probably many other areas): many of these discussions involve a utopian (or, I'd argue, actually oddly bleak) vision in which some sort of inevitable "AI revolution" will utterly change everything we do, from research to teaching, and we have no choice but to adapt and, as a result, benefit from the riches bestowed by the benevolent AI.
When I agreed to talk at the panel, I was planning to give a "critical perspective" and present some of my own research on algorithmic bias, and point to excellent and accessible work such as Emily M Bender's public scholarship. However, once I realised that some of the talks were, in fact, adopting this "positive view" of "AI" in the social sciences, and that the discussion as a whole really was framed as "benefits to social sciences", I decided to take a more confrontational stance and use my 15 minutes to present 10 reasons why I think we should (categorically) refuse generative AI in the university. This post is a summary of this talk. (Needless to say but opinions and errors are mine alone).
"Artifical intelligence" as a concept has always been ill-defined. As Yarden Katz points out in his brilliant history of AI, this vagueness has served researchers (and later industry developers) well while also inviting sustained critique (see e.g., Natale & Ballatore). Recently, I have found Ali Alkhatib's thinking around "defining" (and "destroying") AI very useful. Among other things, AI makes a particularly unequal and opaque power relation material -- and that political dimension alone could be enough for many analyses. For the purposes of this talk (and post), I am really only focusing on what has recently been termed "generative AI": transformer-based large language models which are used to generate text given a prompt (e.g., a question). Because of the nature of my own work, I'm focusing on text generation but many of the points here also apply to image generation. As a brief aside: "generative AI" is not a particularly useful term as it fails to distinguish between different types of text generation -- I think the crux of the matter is whether the LLM replaces a useful and meaningful process. I'm not sure how we best talk about this.
Yarden Katz. 2020. Artificial Whiteness: Politics and Ideology in Artificial Intelligence. Columbia University Press.
Ali Alkhatib, Defining AI: https://ali-alkhatib.com/blog/defining-ai
Natale, Simone, and Andrea Ballatore. 2020. ‘Imagining the Thinking Machine: Technological Myths and the Rise of Artificial Intelligence’. Convergence 26, no. 1 : 3–18.https://doi.org/10.1177/1354856517715164
As discussed in my intro post (and my professional website), I've got a background in sociolinguistics and speech technology, and I research and teach at a university. Which is to say, I approach this topic both as a "mechanic" and a "luddite" (to borrow from Jathan Sadowski, again). I know enough about the inner workings of these systems to know their limitations and potentials, and I also have a very good idea of how they affect not just my livelihood and way of life, but many others (that, really, is the core of my research). The arguments provided here are both scientific and political ones. I believe it's helpful to draw attention to the inherently political nature of not just science but, especially, technological artifacts. We can and should have opinions on technologies that supposedly deeply impact our ways of life and if we don't like it, we don't have to simply "adapt".
This is "scientific" argument that most readers are probably familiar with: large language models have been shown to:
Personally, I am not convinced that any of these issues can be resolved with the current model architectures. However, even if they do, it should be uncontroversial that, right now, they are not there yet.
Birhane, A., and M. McGann. (2024). ‘Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency’. Language Sciences. https://doi.org/10.1016/j.langsci.2024.101672
Shah, C., and E. M. Bender. (2022). Situating Search. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval (CHIIR '22). Association for Computing Machinery, New York, NY, USA, 221–232. https://doi.org/10.1145/3498366.3505816
This is closely related to reason 1. Some language generation tasks may be "low-risk" and genuinely save time. A routine email requesting specific information, for example. This is low-risk because the person generating the email can, in fact, through almost no effort, check whether the output is as specified. (Now, arguably, we might risk removing any and all human interaction from these exchanges, but that's a separate issue).
Writing as part of learning and researching at a university is fundamentally different. It is never "low-risk" in this sense. Not only because what we choose to communicate matters, but also because, by definition, we are all always working at the edge of our own knowledge. Students writing an essay are, in an ideal case, developing new (to them) ideas, testing out theories they're unsure about, and, sure, possibly getting things wrong. Researchers, too, are writing at the edge of their (and perhaps the collective) understanding of a particular thing. This is not a context where it makes any sense to "verify" the output of an LLM because we lack the understanding to actually do this. And this is before we get to the question of why do we write (essays and research papers) in the first place -- we will return to that one.
BBC report finds “51% of all AI answers to questions about the news were judged to have significant issues of some form”: https://www.bbc.co.uk/aboutthebbc/documents/bbc-research-into-ai-assistants.pdf
Haider, J., K. R. Söderström, B. Ekström, and M. Rödl. (2024). ‘GPT-Fabricated Scientific Papers on Google Scholar: Key Features, Spread, and Implications for Preempting Evidence Manipulation’. Harvard Kennedy School Misinformation Review, https://doi.org/10.37016/mr-2020-156.
Another well-known argument against generative AI: large portions of the enormous datasets used to train these systems were effectively taken without permission. We can squabble about copyright regimes, and we can disagree about what internet users can expect to happen with their data once it is online, but it is clear that the vast majority of people did not provide informed consent for their writing to be used to train LLMs. Right now, we are seeing interesting legal disputes about this, with the first major win for a publisher (Reuters) in a case where the LLM in question was seen to be intended to compete with the original training data. Personally, I would argue that even if LLMs are not illegal they are often unethical (especially in these cases where the generated text replaces or competes with the original). Do we want that kind of practice in our universities? Isn't it hypocritical if we also tell students to be careful never to plagiarise?
Kate Knibbs, WIRED, 11/02/2025: Thomson Reuters Wins First Major AI Copyright Case in the US. https://www.wired.com/story/thomson-reuters-ai-copyright-lawsuit/
LLMs require enormous amounts of human labour to work as desired. This includes a significant amount of fine-tuning and annotation (RLHF) of the model, as well as extensive data work to precede training. This work is both difficult (because language is hard!) and sometimes even traumatising (because some of these tasks are similar to content moderation: someone needs to make sure the model doesn't generate child sexual abuse material, for example). In any case, this work is at the moment wildly underpaid and precarious. Personally, what I find particularly egregious is the trade-off between harms and benefits here -- unlike, say, content moderation or physically dangerous labour such as mining, the "benefit" of having LLMs simply does not seem to warrant the human cost.
There are many great sources on working conditions of data workers, but I especially like the worker-led Data Work Inquiry: https://data-workers.org/
In last 6-9 months there has been a lot of media reporting about the environmental costs of LLMs (and "AI" more broadly). This complements a smaller (but well-cited) area of academic research on the topic. The short summary is: more LLMs translates to more data centres which in turn translates to higher energy consumption, more land use, more water use -- all disproportionately affecting areas with lax regulation which often have worse infrastructure and/or fewer resources. Personally, I don't know that citing water usage or energy is the argument against LLMs simply because it seems very difficult to quantify and supports a highly individualised rather than structural frame. I think it's more useful to think about trade-offs again: what additional value or convenience does the LLM provide for a particular use and is that worth a non-zero expense of limited resources? Even more useful: a structural approach. Rather than thinking: "Should this email be written by an LLM and how much does that cost?", we can think about resource allocation priorities on a societal or national level and go from there. If resoures are limited on a national level, I'd rather have high-speed rail than LLMs. If they are limited on a university-level, I'd rather employ a teaching assistant than pay for an LLM.
Paris Marx' brilliant Data Vampires Series discusses this in depth: First episode here
Without wading into a large debate on surveillance and privacy, it is clear that commercial LLMs are not secure. Anything you type into any cloud-based LLM is no longer under your strict control. What happens with that data precisely probably depends on the LLM and its developers but as a general rule, any data about students or research subjects you're not supposed to share with, say, your friends, you probably shouldn't share with an LLM.
Overall, LLMs have been shown to be vulnerable to fairly simply adversarial attacks to extract personal information from training data. I don't see a good reason why we shouldn't expect user interactions to feed into training data for future model iterations.
Accessible and comprehensive primer on security and privacy risks + technical architectures of LLMs: Badhan Chandra Das, M. Hadi Amini, and Yanzhao Wu. 2025. Security and Privacy Challenges of Large Language Models: A Survey. ACM Comput. Surv. 57, 6, Article 152 (June 2025), 39 pages. https://doi.org/10.1145/3712001
Now we're getting to the more explicitely political reasons. These, I think, also make better arguments because they are grounded in a normative stance, rather than limitations which may be resolved. This is the point where I would argue that the problem is not that AI is "not ready" or "biased" or "faulty" or "snakeoil", but instead that it's fundamentally the wrong approach to any tasks involved in education. As a result, it will never be appropriate here.
Scholars in education studies are probably better placed to talk about the theory here, but based on my experience as a former student and a current teacher at a university, I believe that using LLMs as part of teaching, for example to generate lesson plans or automate parts of marking, significantly devalues an already undervalued profession. I want to acknowledge the real economic pressures on educators here: many university departments are understaffed and incentives are such that working faster allows staff to do other parts of their jobs which are also required and often more highly rewarded (e.g., research, grant applications). Now, the obvious response we should have to understaffing is to demand more staffing. But perhaps because we know that this will not be granted in the current economic and political climate, "embracing" AI seems tempting. However, this appears a shortsighted and treacherous route. The last thing we should do, as workers, right now is to demonstrate to our bosses that a decent portion of our work can be automated. I think it is beyond naive to assume that we will be rewarded for this "efficiency" with, e.g., more research time. I think we would do well to look at other sectors and learn about the ways in which partial automation and devaluing of expertise has led to job losses. precarity and lower wages (e.g., long distance trucking in the US as discussed by Karen Levy, as well as (digital) journalism).
Beyond this self-preservation, we should also resist generative AI in teaching because teaching is a relationship. It is disrespectful to this relationship and our students outsource the (yes, hard, annoying, undervalued) tasks of reading, giving feedback, and preparing classes. As bell hooks taught us (or taught me at least), education is about empathy. The reason I love teaching is because I get to build real human connections with students, share my expertise and learn from their expertise. There are many days where I feel unsure about my research but I have yet to leave a classroom without having learned something (even if it's just a new slang word I'm too old for). It is one way how we get to, however briefly, contribute to the lives of others who are there to listen and engage. If that's too romantic an argument, then remember that universities only exist because we collectively believe that there is value in this type of education.
Many universities appear keen to "embrace" generative AI for students too. While there are usually rules against using them in assessment, they are often explicitely encouraged for other tasks such as summarisation of texts, creating outlines for assignments, and even checking understanding. As discussed above, these are all tasks that we, as teachers, should be able to help students with (e.g., skills for reading scientific papers, skills for planning written work, office hours and classroom discussions to check understanding). There are many reasons why students turn to LLMs instead, all of which I believe could be eliminated (e.g., many students are very scared of failure and/or being "wrong" -- I think it's on us to create an environment where they feel safe to make mistakes.) Even if the alternative to "brainstorming with LLM" is "brainstorming with another student" that's an improvement.
Fundamentally, even setting aside concerns of bias and accuracy, LLMs are not supporting learning as the important part in student writing (and reading) is usually the process not the outcome. Reading, synthesising and writing are core skills a university teaches. These skills are really difficult to acquire and need a lot of practice. They are often not fun. I don't think we should do hard things for the sake of doing hard things, but engaging with a difficult text or putting together an original argument are tasks that genuinely develop students' abilities to think critically. It doesn't matter if the outcome is imperfect (that's why we give feedback), what matters is that we (teachers, students, researchers) try.
Again, there is both a romantic and a pragmatic argument here. The romantic argument relates to the intrinsic value of higher education. But even setting that aside (which I wouldn't), if we encourage students to outsource all the most difficult parts of reading, writing, synthesising, arguing, and thinking -- what possible value are we providing and what skills are they gaining? The good thing is that students by and large do want to learn -- they are just very scared of making mistakes (for understandable, structural reasons). What we should do here, is talk to them and figure out ways to support them in their learning without incentivising them to take unhelpful shortcuts.
This relates to some of the previous reasons. The current paradigm of training and deploying LLMs is really only feasible for a small oligopoly of very large tech companies. I think this is unlikely to change given the sheer concentration of resources necessary for these systems. Tech companies are not uninterested parties in education (with many seeing profits), and as has become abundantly clear in recent months (but really has always been true), they have (geo)political interests. Against this background we should again be very careful in our engagement with them, lest we make ourselves redundant.
There is also a humanistic critique, which I want to end with:
LLMs do not replicate our processes, they can only mimick our outputs.