Depiction of HAL 9000, the artificially intelligent computer in Stanley Kubrick’s 1968 film 2001: A Space Odyssey
What happens when an AI tries to do etymology?
21 December 2022
A new large-language-model, artificial-intelligence (AI) engine, ChatGPT, has debuted. When one types a query into the AI, it returns a response in grammatically correct and sensical English. It has caused a bit of a stir. For one thing, there are those who are foretelling the end of human writing, or at the least the end of the of the first-year university essay. (Mark Liberman over at Language Log has a non-sensational summary of the debate, including links to various articles opining on the AI and its impact, and an assessment of the ability to detect whether or not an essay was written by the AI.) But from my playing around with it, ChatGPT is indeed an impressive step-forward in machine-produced, natural language, but the fears, moral panic, and click-bait headlines it has inspired are overblown, at least for the immediate future.
The AI is really quite remarkable. It produces intelligible, grammatically and (mostly) idiomatically correct English sentences and paragraphs. For instance, I typed How are you doing? into the engine and got this response:
As an artificial intelligence, I don't experience emotions in the same way that humans do. I am a computer program designed to assist with tasks and answer questions to the best of my ability. Is there something specific you would like to know or discuss?
While this response would pass the Turing test, many other responses, while correct English, are problematic. I asked the AI a series of questions about the origins of various words and phrases. Most of the responses would earn the grade of D in my first-year composition course. A typical response to a question about the origin of a term is:
The phrase “an exception that proves the rule” is a common saying that is used to indicate that an exception to a rule or generalization actually serves to confirm the rule or generalization. The phrase is often used to emphasize that a particular exception does not invalidate a rule, but rather serves to demonstrate its validity.
The phrase “an exception that proves the rule” is believed to have originated in the late 16th century, and it is thought to have been derived from the Latin phrase “exceptio probat regulam,” which means “the exception confirms the rule.” The phrase was originally used in the context of legal reasoning, and it was used to indicate that an exception to a rule actually served to confirm the rule's validity.
Over time, the phrase “an exception that proves the rule” has come to be used more broadly in everyday language to refer to any exception that serves to confirm a rule or generalization.
The response is very general and does not contain citations for the sources of information. The AI’s responses are often structured in the three-paragraph “hamburger” essay format of introduction, body paragraph, and conclusion that is favored by high school and first-year university students. In this case, the introductory paragraph is merely a literal reading of the phrase. The body contains three facts, the date, the Latin origin, and the origin in legal contexts. The conclusion is simply a restatement of the introduction. This one is remarkable only in that the facts are basically correct, if superficial and cursory.
In the questions I asked, the AI occasionally got the origin correct. In addition to the exception that proves the rule, it also correctly sourced big bang to astronomer Fred Hoyle in the 1940s. And the response to my question about gaslight correctly flagged Patrick Hamilton’s play as the source, later popularized by the 1944 movie starring Ingrid Bergman. But correct facts and specificity were not usually the case.
In some cases, the AI plumped for a long-debunked false etymology. And it did so on my very first query when it gave the port out, starboard home origin for posh. When asked about the origin of the whole nine yards, the AI came up with the following:
The phrase “the whole nine yards” is thought to have originated in the United States in the mid-20th century. There are a few different theories about its origin, but the most widely accepted one is that it refers to the amount of fabric used in a military uniform. According to this theory, a standard U.S. Army uniform required about nine yards of fabric, so if someone had “the whole nine yards,” they would have a complete and fully outfitted uniform.
Another theory is that the phrase refers to the length of a football field, which is 100 yards. If someone goes “the whole nine yards,” they are going all out or giving their all.
It is also possible that the phrase “the whole nine yards” is simply a colorful and vivid way of saying “everything” or “all of it,” without any specific meaning or reference to a particular quantity.
First, the dating is wrong. I’ve seen many different fabric-related hypotheses, but the military uniform version is a new one on me. And as the AI phrases it, the football hypothesis makes no sense—how is going nine yards giving one’s all on a hundred-yard field? The hedging, as in thought to have and is also possible, is also typical of the answers generated, and also of first-year university essays. If it had emotions, I would say the AI lacks confidence in its answers.
In a few instances, the AI pointed to supposed Old English words as the origin. It came close with threshold, saying it comes from:
“þresceold,” which is a combination of the words “þresc,” meaning “threshold,” and “ceold,” meaning “cold.”
While the spelling is wrong, þerscold is indeed an Old English word and the source of the Present-Day English word. But the breakdown of the syllables is wrong.
In other instances, the AI gave fictitious, and sometimes laughable, Old English roots. For instance, of fuck it said:
The word “fuck” is believed to be derived from the Old English word “fōc,” which means “goat.”
Goat?! Where did that come from? The Old English word for goat is gat. Fōc is not a word found in the extant Old English corpus.
Sometimes it gets the origin mostly correct but makes other errors, as in the origin of the name of the element americium:
Americium is a synthetic chemical element with the atomic number 95 and the symbol Am. It was discovered in 1944 by a team of scientists at the University of California, Berkeley, led by Glenn T. Seaborg. The element was named “americium” in honor of the United States, where the element was discovered and first produced.
The name “americium” is derived from the Latin word “America,” which was the name of the region of the world now known as the Americas. The name “America” was chosen by the Italian explorer Amerigo Vespucci, who explored the region in the early 16th century and claimed it for Spain. The name “americium” was chosen for the new element because it was discovered and first produced in the United States, which is located in the Americas.
Vespucci did not name the continents; they were named for him. And there is also weird usage in the Latin word “America,” which was the name of the region of the world now known as the Americas; the Latin name did not exist until Europeans made their way there and named the region for Vespucci. Also, the naming of the element was more nuanced than simply naming it for the United States.
Or there is the case of political sense of lobby, which the AI correctly traces to the nineteenth century Britain and does not repeat the false story of the origin in Washington, DC hotel lobbies. But then it says lobby comes from the Old French word “lob,” which means “a place where one can lounge,” which is just wrong.
Or the linguistic term snowclone, which the AI correctly dates and credits to Geoffrey Pullum, but which also says:
The term is a play on the word “snowclone,” which refers to a type of fossilized snowflake that is preserved in ice or permafrost.
Now this sense of snowclone might very well be an obscure technical definition (although I’ve never heard of it), but it is definitely not the inspiration for the linguistic term.
But giving factually incorrect and long-disproven information is something that undoubtedly will be corrected as the AI improves. The AI does not search the internet; rather it relies on training data that has been fed to it by the developers, and garbage-in/garbage-out, as the saying goes. With better training data, the AI will make such mistakes less often. At least in writing aimed at a general audience. I’m more skeptical of its ability to produce acceptable writing aimed at experts in a field.
While I can easily see a future version of this AI producing solid, Wikipedia-style responses, its writing lacks style and affect. It’s bland and boring. I suspect this will be a harder problem to solve than just getting the facts right. Perhaps it will eventually be able to produce something that someone wants to read. At present, the AI seems to be programmed to avoid writing creative pieces. I asked it to write me a poem about Christmas and it spit out Clement Moore’s A Visit from St. Nicholas (without crediting Moore). Others, however, report that the AI does produce original, albeit bad, poetry.
Crediting sources will also be a tricky problem for the AI to solve. It’s one thing to give Hamilton the credit for Gaslight or Hoyle for big bang, but to actually follow a trail of scholarly works and properly credit the ideas within that discourse is quite another.
To be sure, the AI poses a problem for teachers of writing in that students will undoubtedly use it as a vehicle for plagiarism, but in its present incarnation it is not all that scary and does not presage the “end of the college essay” as some have predicted. The AI’s answers are very superficial, and a bit more care in phrasing essay prompts will go a long way toward defeating any attempts at plagiarism. The AI also gives the same response each time a question is posed, and the developers have produced a tool to help identify texts that have been produced by the AI (I have not tested this tool). And teachers can embrace it as an in-class tool to help students identify and edit sub-standard writing and superficial arguments. With a little bit of ingenuity on the part of composition instructors, the AI as it exists today and in the immediate future will be no more a threat to academic integrity than the myriad human-based essay mills already are.
All problems aside, it's an impressive achievement, and it will get better with time, but the doomsaying is overblown.
Image credit: Tom Cowap, 2021. Licensed under a Creative Commons Attribution-Share Alike 4.0 International license.
