On Algorithmic Monkeys with Typewriters

2023-01-23

The academic essay, term paper, written assessment, whatever you want to call it, University, and education in general, requires a lot of writing. And for good reason. Writing forces you to organise your thoughts and understanding into something that doesn’t just clarify your own thinking, but makes it understandable enough for someone else (your assessor) to understand too. It’s a chance for the assessor to ensure that the concepts covered over the duration of a course have actually sunk in and are understood enough to write about in some capacity. It’s a chance for students to explore their understanding and expose areas of weakness that they will hopefully strengthen. It’s a chance for new ideas to emerge as part of the writing process.

Indeed this is something I’ve experienced time and time again. Taking some notes that eventually coalesce into a new project idea. Writing a research paper only to realise a tangent is interesting enough to merit writing a post about here. Putting down that sentence that finally makes a concept click. Writing plays a major role in education and it’s clear to see why, writing forces you to wrestle your understanding into something clear enough and consistent enough to put into words.

However, it’s important to acknowledge that writing doesn’t work for everyone. Whether because they have a different learning style or just don’t enjoy the act of writing. Figuring out how to join various ideas together, how to successfully argue a point, how to weave together a dozen or more different references; writing can be, and often is, hard. Thankfully however, at least in my experience, assessments that would usually be written are offered in different formats such as a spoken presentation or something more domain specific. And should students need it, there’s usually some degree of flexibility to make mandatory written assessments work for students regardless of ability or style.

Unfortunately these alternative measures sometimes don’t cover everyone’s situation and some feel the only way to pass is by cheating. Whether it be through liberal copy-pasting from online sources or pay-per-paper services. Cheating in written essays does unfortunately come up time and time again. However, so far attempts at cheating have been largely easy to spot: unusual similarities between two students submissions, stilted language that doesn’t fit the essay topic, a general vagueness that avoids tackling the issue in any depth.

But as Mike Sharples points out in a paper from earlier last year, we may be headed towards something much more transformative. In his paper for the International Journal of Artificial Intelligence in Education, Mike looks at what AI might mean for essay writing in education. Specifically, he focuses on the use of OpenAI’s GPT-3 model. A language model that, if you’ve paid any attention to tech news at all in the past few years, will certainly have been one of the topics discussed. Language models that would have produced bizarre non-sequitors of jumbled nouns, verbs, and prepositions just a few years ago have quickly become one of the most interesting and groundbreaking areas of AI. In fact a great comparison of how quickly things have changed is to watch this video of where we were at in 2016, and then this video of where we are now. These are just examples of presenters reading a script - an essay if you will.

While the usage of AI for academic writing doesn’t yet appear to be widespread, it is definitely something we are going to see become more commonplace. Perhaps most obviously because the barrier to entry for cheating is dramatically reduced. Why spend time copying from various sources when you can just enter a prompt detailing what you’ve been tasked to do and call it a day? Why pay potentially hundreds for an essay writing service when you can generate dozens of different essays for pennies at most. Why carefully fine tune your generated output to fix up any mistakes or errors when you can just roll the dice again to get another completely new essay? An infinite number of algorithmically powered monkeys with typewriters must produce something good eventually.

It’s worth noting as well that AI shouldn’t be considered a last ditch effort for cheating. Language models are now routinely producing work that is not only indistinguishable from human work, it appears to even best it in some cases. In fact, AI can now reasonably produce work that, although not a literary masterpiece, can reasonably achieve a B or a C grade.

Looking forward there seem to be differing paths available with regards to AI and writing. A return to in-person written exams which seems impractical if not unreasonable in an age of remote work. Attempts could be made to ban AI assistants from the writing process however that seems completely unenforceable - and impossible to detect too. Alternatively, we could look to try integrate AI in a more constructive way with the writing process. As Mike suggests, there may be room to pose students with a prompt and take turns writing one paragraph after another. Another approach which seeks to invert the process entirely is for students to be presented with AI generated essays on a topic. From here, students are then encouraged to critique those essays and eventually produce something better.

This latter approach of merging human and AI writing seems the most likely. Especially given that it doesn’t seem entirely out of the question that eventually every piece of text we write will be assisted by AI in one way or another. Indeed, even as I’m writing this piece Notion has just announced their AI assistant for generating blog posts, meeting agendas, and poems, among other features. Even Microsoft looks to be working to integrate such models into their products - potentially their Office suite.

Is this just a new direction for the kinds of AI we already use daily but rarely notice? The likes of your spell checker, predictive text, or intelligent replies? Or does it point towards a more foundational change in the way we approach writing? AI continues to work its way into new fields and to improve at a break-neck pace. But, as always, we need to be mindful of how we’re applying these technologies. AI can only ever be as good as the data we feed it and so we need to be mindful about what our language models do and don’t know about. By making AI much more deeply connected with the writing process we risk students shaping their understanding based upon the narrow window of the world that language models tend to have.

On Algorithmic Monkeys with Typewriters

Further Reading