Why Chat-GPT isn't as clever as it seems.
Earlier this week, the UK government was found to be using AI chatbots in making decisions. Earlier this month, a man on trial for attempting to fire a crossbow at the Queen claimed he was supported by his AI girlfriend. Earlier this year, people exploited AI’s inherent biases to produce some of the most strangely offensive names possible. So what, exactly, is AI, and should we all start pledging our allegiance to it?
Now, my concern is not with AI proper, instead, it is those projects popularly called AI, but more accurately described as large language models or generative image models. Unless you’re a computer scientist, this difference may seem arcane but boils down to what the program does. True AI is something that is still, for the most part, a fantasy. A true AI would be able to have the kind of mental state that we as humans enjoy; consciousness, for example. What is often called AI are programmes that just appear as if they are experiencing cognitive processes, but no actual cognition is occurring. Plainly, a weak AI is one that can replicate what something with a mind would say, without actually having a mind.
If it looks like a mind, walks like a mind, and quacks like a mind – isn’t it a mind? This is where the difference becomes more apparent. Something like Chat-GPT is incapable of genuinely responding to any prompt, instead, it presents a string of words in their most likely order, based on a huge set of data. This method of production causes two issues. Firstly, it causes problems in the content it creates. Secondly, there are fundamental problems in the project and its apparent aims.
The first problem is probably the easiest to spot. Chat-GPT is trained on text written by humans, a species inherently good at lying. Whilst there are attempts to mitigate the harmful stereotypes Chat-GPT presents, there are still systemic issues. Chat-GPT consistently genders doctors, engineers, and mathematicians as male, whereas cooks, nurses, and other traditional feminine roles are all performed by women. This reveals a deeper issue. Chat-GPT serves to blindly repeat and reinforce our biases, which, when presented as unbiased, may further entrench stereotypes.
The second problem is the contradiction between the actual project and the language around it. It seems clear to most that Chat-GPT attempts to generate accurate responses to prompts offered. This is the basis of the current concern that students will replace getting someone else to do their homework, with getting Bing to generate it for you. This may be a concern when we can produce thinking machines, but Chat-GPT is not a logical, reasoning, or thinking machine.
There is no clearer example of this than chess — when challenged to a game, Chat-GPT will invent pieces, moves, and winning positions. You have to constantly re-educate it on basic chess rules.
The machine looks like it provides reasoned and well-structured responses, and those responses, alongside our calling it an AI, make Chat-GPT seem credible and capable of some kind of intelligence. This lends integrity to a machine that has large issues and huge limitations, and waters down AI as an exciting concept. As Chat-GPT becomes more lucrative, the incentive is not to investigate whether a true AI is possible, but instead, it seems, to produce the most convincing façade of intelligence.