In the previous post we lauded some
of the wonders of which the new generation of chatbots is capable. In this post
it we will look at some of the drawbacks.
To be fair, It must be said that
the home screen of ChatGPT displays a whole list of caveats: not up to date by
several years, may occasionally produce biased content, may give information
that is plain wrong, etc. The internet is buzzing with examples. Rather than
make an invidious selection, I decided to conduct my own experiment. Here’s a
foretaste.
There’s is a long-established
Irish-Spanish family in the city of Valencia, Spain, where I Iive, which played a
prominent role in its modernisation in the nineteenth and early twentieth centuries.
The family name is Trenor. I’d heard of them but wanted to know more. To my
consternation, the answer came back from ChatGPT that it had no information
about the family. The answer is all the harder to understand because there’s a
long article about the Trenors in Wikipedia! True ChatGPT adds, “I cannot access up-to-date information on
specific families or individuals without violating their privacy;” but I didn’t
specify “up to date.”
Undaunted, I decided to try a second experiment, this time
on a topic that should be more familiar to regular readers of this blog and so
make it easier for them to judge the result.. The question was, “Who first
proposed the natural translation hypothesis.” This time I did get a meaningful
response – only it was a wrong one. It was that the inventor was Eugene Nida.
Nida was an immensely influential translatologist; there was a time, when I was
beginning in translation studies, that no thesis was complete without a reference to him. And it’s true
that Nida recommended the use of natural-sounding target language. However, he
did not formulate a natural translation hypothesis and he used natural in quite
a different sense. Why then did ChatGPT make this mistake? We can only
speculate, because chatbots are black boxes whose workings are not revealed by
their designers if they even understand them themselves. It could not be for
lack of data, since a cursory search with Google finds lots of references to
the hypothesis. So a more likely explanation is failure to apply the longest match
principle. The principle states that when seeking to match strings of
characters or words, only the longest match is acceptable. Could it be that
ChatGBT mistakenly stopped seeking after translation and missed hypothesis?
The next thing that happened was my
mistake. I pressed a wrong keyboard key and as a result the previous question
was repeated. Again I soon got an answer – but it was not the same one! It was
this:
“The Natural Translation Hypothesis was proposed by Gideon Toury, an Israeli scholar and translation theorist, in his influential book "Descriptive Translation Studies and Beyond" published in 1995. Toury's Natural Translation Hypothesis suggests that translations are influenced by various factors such as the translator's language competence, the target culture, and the translation norms of the time, which shape the translation process and result in translations that are shaped by the target language and culture, rather than being an exact replica of the source text. Toury's work has had a significant impact on the field of translation studies, and the Natural Translation Hypothesis is considered a fundamental concept in translation theory.”
This is
much better. In fact Toury was very sympathetic to the hypothesis, as I know
from personal correspondence with him. However, he had his own model of how the
translation ability develops and he had his own term for it, which was native
translation. And he never called it a hypothesis.
Notwithstanding
the disappointments, I decided to have one last try. By this time there were already
imitators and competitors of ChatbotGPT on the market, so I turned to one of
them, BingAI. Here’s its equally fast response:
"Brian Harris is credited with proposing the Natural Translation Hypothesis. According to Harris, natural translation is a way of translating that occurs when a person translates without any formal training or knowledge of translation theory. The hypothesis suggests that natural translation is a more effective way of translating than traditional methods. Harris' work has been influential in the field of translation studies and has led to further research into natural translation."
This is a correct answer to the query, though the hypothesis doesn't suggest that natural translation is “more effective.” It seems that BingAI is capable of going beyond its data and making things up into what are now called hallucinations.
A single example isn’t enough to
declare one chatbot superior to another. However, if there’s a lesson we can learn
from it it's not to rely on only one chatbot. If there is the slightest
doubt, try another.
Sources
ChatGPT. https://chat.openai.com
BingAI.
https://bing.com/chat.
