Using ChatGPT to write for you is literally "1984"

A reminder that Orwell predicted way more stuff than just "Big Brother" and "Thought Police".

Last month, at the prompting of the book club that I am a part of, I finally overcame my aversion to reading “1984”. Having the impression that it was a depressing book, along with my dislike of reading or watching media that centres around the oppression and humiliation of some powerless protagonist that is so popular these days, I inevitably developed little interest in reading this trailblazer for that depressing and sado-masochistic genre.

Nonetheless, having downloaded an E-book edition of it from Global Grey eBooks, I was pleasantly surprised at how enjoyable it was to read and finished it in less than a fortnight. In hindsight I should not have been surprised at all, since I read one of Orwell’s earlier books, “Burmese Days”, a long time ago and liked it very much. But due to the “Folk knowledge” that I gathered of what “1984” was meant to be like, I was swayed from correctly guessing that if I enjoyed the latter book, then I would logically enjoy the former. It is amazing how much the popular “Folk knowledge” of a book can wildly diverge from the actual book itself. If you probe the common assumptions of what “1984” is about you will most likely receive the book’s very general plot, which is that it is about a man who lives in a grey and depressing totalitarian dystopia who upsets “Big Brother” and gets tortured with rats. You will also get a handful of the novel’s “Newspeak” words which have become everyday words, such as “Thoughtcrime” and “Thought Police”, along with a few tidbits about the setting such as the Telescreens and the Two minutes hate¹.

Utterly forgotten from the public’s consciousness is how much sexuality is present in the book, such as how it is repressed by Big Brother in order to induce neurosis in its citizenry, and the brief romance “The man” Winston has with Julia. Forgotten also is the fact that most of “Big Brother’s” attention is directed towards the Middle and Upper-class, while mostly ignoring the “Proles” as they are considered being unworthy of much notice, considered little better than animals². Gone also is the awareness of the subject of today’s essay, which are the Versificator machines.

In the world of “1984” the Department of Fiction, which is devoted to the production of entertainment media, uses a machine called a “Versificator” to generate text and lyrics for the trashy literature and music popular with the Proles. Below, you can read a few excerpts from the novel that introduce this interesting machine to the reader.

And the Ministry had not only to supply the multifarious needs of the party, but also to repeat the whole operation at a lower level for the benefit of the proletariat. There was a whole chain of separate departments dealing with proletarian literature, music, drama, and entertainment generally. Here were produced rubbishy newspapers containing almost nothing except sport, crime and astrology, sensational five-cent novelettes, films oozing with sex, and sentimental songs which were composed entirely by mechanical means on a special kind of kaleidoscope known as a versificator.

A solitary figure was coming towards him from the other end of the long, brightly-lit corridor. It was the girl with dark hair. Four days had gone past since the evening when he had run into her outside the junk-shop. As she came nearer he saw that her right arm was in a sling, not noticeable at a distance because it was of the same colour as her overalls. Probably she had crushed her hand while swinging round one of the big kaleidoscopes on which the plots of novels were ‘roughed in’. It was a common accident in the Fiction Department.

The tune had been haunting London for weeks past. It was one of countless similar songs published for the benefit of the proles by a sub-section of the Music Department. The words of these songs were composed without any human intervention whatever on an instrument known as a versificator. But the woman sang so tunefully as to turn the dreadful rubbish into an almost pleasant sound. He could hear the woman singing and the scrape of her shoes on the flagstones, and the cries of the children in the street, and somewhere in the far distance a faint roar of traffic, and yet the room seemed curiously silent, thanks to the absence of a telescreen.

It goes without saying that Orwell’s “Versificator” comes close to anticipating the use of “LLMs” such as ChatGPT that can create text that closely mimics human writing. Already people have used this technology to generate essays and short stories, along with sound and imagery. And just as with the Versificator, the kind of writing these LLMs produce may be decent enough that you can read, understand, and possibly enjoy reading or listening to it on some level, but nonetheless whatever stuff such a device creates is usually mediocre and stilted in style while occasionally containing unusual errors. But most importantly, by its very nature the LLM and Versificator undermines Art by removing the necessity of developing the relevant skills and imagination that being an artist or writer requires. This is one of the issues that the Versificator in “1984” and LLMs in the 21st century cause us to confront.

By creating and implementing what is essentially a prosthetic “Skill in Art and Writing” this allows whoever is in control of such technology to gain the material output of the artist or writer but without dealing with the inconveniences that artists or writers usually create, such as questioning the commands, ideology and working conditions of their employer. This means they can instead entrust the job of creating art and literature to people who ordinarily would have no business working in art or literature, such as obedient Mid-wits loyal to them, or at least controlled by them. It is likely that this is one of the reasons Orwell chose to include this technology in his Dystopian novel. By using some mechanical substitute to replace the people most dangerous and resistant to totalitarian ideologies, artists and writers with independent minds and imagination, “Big Brother” can ensure that its citizens can have some semblance of “Art and literature” in their lives, while still discouraging the formation and continuation of people that would create problems for its ideological dominance. Essentially, Big Brother can have its cake and eat it too.

However, despite the similarities, there are some differences between LLMs like ChatGPT and the Versificator which cause the former to have consequences the latter lacks.

Besides being mechanical, the Versificator seems to require a fair amount of labour and supervision from its operators, so much so that they can accidentally injure themselves when using it. LLMs don’t have this problem, as all labour necessary to keep its software, hardware and infrastructure running is out of sight and mind of its operators, being instead carried out by the thousands of coders, Data-centre workers and Energy infrastructure workers across the Earth. Apart from removing even more skill-requirements from the operator, it also broadens the range of people who can successfully operate it, thus giving whoever is in control of an LLM even more liberty to pick and choose who they employ rather than having to bargain with a limited amount of skilled and assertive workers.

Another interesting difference between the Versificator and LLMs is that while the former was only shown being used for mass-producing literature and Art, the latter is theoretically capable of handling various white-collar jobs such as accounting, financial analysis or legal advice. While this application is still experimental it would give LLMs a certain negative effect on society that not even the Versificator had, which is making much of the “White-collar” workers usually required for white collar jobs unnecessary. At least the white-collar workers of “1984”, if a Versificator-like device were used to do Big Brother’s accounts, would merely be made similar to the factory workers they customarily look down upon, manipulating machinery in easily taught ways. In the modern world they would instead be made equal to the unemployed.

All in all, from this subtle interpretation of a neglected aspect of Orwell’s great dystopian novel, and a comparison with its equivalent in the modern world, we can perceive one of the great motivations that corporations and governments have for the development and use of LLMs. By lowering the skill-barrier for being able to produce Art, Literature, and possibly provide legal and financial services, they can free themselves from the inconvenience of skilled, intelligent and assertive professionals who can bargain with or assert themselves against their employer, or leave if they think they are getting a bad deal from them. They can instead hire pretty much anyone competent enough to use a laptop and get work done on time. And with the professional and skilled workers marginalised and under- or unemployed in an environment no longer sympathetic to them, they are unlikely to have much influence on what corporations, or governments, decide to do to the world at large. They will instead be forced into positions of little power and influence, either that of a standard working class person, or worse a precariously-employed one living on the fringes of society, where their ability to threaten the goals and schemes of corporations or governments shall be minimal.

Fortunately, it is likely that even if corporations or governments were successful in implementing LLMs in this way, the duration of their rule will not last indefinitely.³ Besides the fact that all governments and corporations come to an end one way or another, the massive amount of energy and material required to run LLMs means that eventually they will become too costly to keep funding, either when some great societal crisis makes it too difficult to sustain such technology, or from some more significant physical phenomena that reduces the amount of energy and resources that is available for use by human beings. All in all, the chance that LLMs are going to be a problem that plagues humanity indefinitely is minimal.

1. “Memory hole” is another “1984” reference that has entered common usage, but without any “Folk knowledge” of its connection to the novel. In the novel, it referred to little slots within the wall of the ministry’s many offices that would suck in any unwanted waste-paper or written evidence of Big Brother’s manipulation of the truth into a central furnace to be burned.

2. Of all the elements of “1984’s” analysis of totalitarianism and ideological repression, this one seems to be both one of the most significant and the most overlooked. While a quick glance at previous totalitarian societies will probably prove that Orwell exaggerates the safety that “Proles” can expect in such societies, the kind of ideological conflicts and repression that plague middle or upper-class people, or anyone on middle and upper-class environments, in totalitarian societies is mostly absent as “Proles” have little interest in ideology and ideologues usually have little direct interest or interaction with “Proles”. One can see this in real-life, when you compare the violation of some politically correct shibboleth in some backwoods farm or ghetto, with doing so in an office or university.

3. Many readers of “1984” develop the false impression that “The Party/ Big Brother” will never be defeated, and will continue to oppress the world with no end in sight. But after the last chapter there is an essay on the Newspeak dialect used by that authoritarian regime, which is written in a way that strongly implies that the author is a scholar who lives sometime after its fall, distant enough in time that its existence is a matter of historical, rather than contemporary significance.