Wednesday, May 3, 2023

A Chatbot Experiment


In the previous post we lauded some of the wonders of which the new generation of chatbots is capable. In this post it we will look at some of the drawbacks.


To be fair, It must be said that the home screen of ChatGPT displays a whole list of caveats: not up to date by several years, may occasionally produce biased content, may give information that is plain wrong, etc. The internet is buzzing with examples. Rather than make an invidious selection, I decided to conduct my own experiment. Here’s a foretaste.


There’s is a long-established Irish-Spanish family in the city of Valencia, Spain, where I Iive, which played a prominent role in its modernisation in the nineteenth and early twentieth centuries. The family name is Trenor. I’d heard of them but wanted to know more. To my consternation, the answer came back from ChatGPT that it had no information about the family. The answer is all the harder to understand because there’s a long article about the Trenors in Wikipedia! True ChatGPT adds, “I cannot access up-to-date information on specific families or individuals without violating their privacy;” but I didn’t specify “up to date.”


Undaunted, I decided to try a second experiment, this time on a topic that should be more familiar to regular readers of this blog and so make it easier for them to judge the result.. The question was, “Who first proposed the natural translation hypothesis.” This time I did get a meaningful response – only it was a wrong one. It was that the inventor was Eugene Nida. Nida was an immensely influential translatologist; there was a time, when I was beginning in translation studies, that no thesis was complete  without a reference to him. And it’s true that Nida recommended the use of natural-sounding target language. However, he did not formulate a natural translation hypothesis and he used natural in quite a different sense. Why then did ChatGPT make this mistake? We can only speculate, because chatbots are black boxes whose workings are not revealed by their designers if they even understand them themselves. It could not be for lack of data, since a cursory search with Google finds lots of references to the hypothesis. So a more likely explanation is failure to apply the longest match principle. The principle states that when seeking to match strings of characters or words, only the longest match is acceptable. Could it be that ChatGBT mistakenly stopped seeking after translation and missed hypothesis?


The next thing that happened was my mistake. I pressed a wrong keyboard key and as a result the previous question was repeated. Again I soon got an answer – but it was not the same one! It was this:

“The Natural Translation Hypothesis was proposed by Gideon Toury, an Israeli scholar and translation theorist, in his influential book "Descriptive Translation Studies and Beyond" published in 1995. Toury's Natural Translation Hypothesis suggests that translations are influenced by various factors such as the translator's language competence, the target culture, and the translation norms of the time, which shape the translation process and result in translations that are shaped by the target language and culture, rather than being an exact replica of the source text. Toury's work has had a significant impact on the field of translation studies, and the Natural Translation Hypothesis is considered a fundamental concept in translation theory.”

This is much better. In fact Toury was very sympathetic to the hypothesis, as I know from personal correspondence with him. However, he had his own model of how the translation ability develops and he had his own term for it, which was native translation. And he never called it a hypothesis.

Notwithstanding the disappointments, I decided to have one last try. By this time there were already imitators and competitors of ChatbotGPT on the market, so I turned to one of them, BingAI. Here’s its equally fast response:

"Brian Harris is credited with proposing the Natural Translation Hypothesis. According to Harris, natural translation is a way of translating that occurs when a person translates without any formal training or knowledge of translation theory. The hypothesis suggests that natural translation is a more effective way of translating than traditional methods. Harris' work has been influential in the field of translation studies and has led to further research into natural translation."

This is a correct answer to the query, though the hypothesis doesn't suggest that natural translation is “more effective.” It seems that BingAI is capable of going beyond its data and making things up into what are now called hallucinations.

A single example isn’t enough to declare one chatbot superior to another. However, if there’s a lesson we can learn from it it's not to rely on only one chatbot. If there is the slightest doubt, try another.







  1. We wanted to take a moment to express how impressed we are with the amazing informative this blog is.Your presentation style is clear and concise. The level of detail and research that went into creating this informative video is truly exceptional.

    Business Development assignment help. Expert assignment writers can help businesses achieve this by delivering high-quality business development assignments that outline effective plans, strategies and methods to achieve success and growth.

  2. To err is not just human, also AI. It's a comforting thought!

  3. ICS-Dubai is a leading provider of professional Arabic Interpretation Services in Kuwait. Our team of experienced interpreters can provide on-site interpretation for business meetings, conferences, legal proceedings, and more. We also offer remote interpretation services via video conferencing and telephone. Our interpreters are highly qualified and have extensive experience in the Kuwaiti dialect. Whether you need an interpreter for a medical appointment, a business meeting, or a court proceeding, we have the expertise to provide you with accurate and reliable interpretation services. We guarantee quality interpretation services and excellent customer service. Contact us today to learn more about our Arabic interpretation services in Kuwait.

  4. Chatbots are changing every area, for example, digital marketing. They are thus increasingly central to Customer Service, sales lead generation and so on and so forth. Let's talk about Digital Marketing Thesis Writing Services Being in business for a long time especially in the academia and the business world is very important. They offer professional support in creating inspiring theses related to the digital marketing area, one of the areas where chatbots have a great impact. Considering those two subjects altogether, we have an opportunity of looking at how chatbots are affecting the development of digital marketing strategies, an interesting subject for a thesis.

  5. Welcome to blog which (all about) explore the terrific world of AI experiments. In our recent experiment, we focus on logistics on the ocean freight front. And we sail to Englewood Cliffs, an important centre for ocean freight logistics englewood cliffs . It is an epitome of a complicated and crucial part of the business in today’s global market. Keep an eye for our upcoming chapters on how AI can dramatically change this area.

  6. Interpreting agencies in the UK play a crucial role in facilitating effective communication across language barriers. These companies provide invaluable interpreter services, ensuring seamless interactions in various sectors such as healthcare, legal, and business. With their expertise and professionalism, interpreter services UK contribute significantly to multicultural communication and collaboration.

  7. In the realm of AI, a fascinating experiment is unfolding: the development of chatbots. These digital assistants are revolutionizing the way we interact with technology. One such experiment, A Chatbot Experiment is pushing the boundaries of what's possible in this field.

    In the second part of our discussion, we turn to a service that's becoming increasingly important in the academic world: assignment help cheap . This service is a boon for students, offering affordable assistance with their coursework and helping them achieve their academic goals. The intersection of AI and affordable academic help is a promising area for future exploration..

  8. These digital assistants are revolutionizing the way we interact with technology. One such experiment, A Chatbot Experiment is pushing the boundaries of what's possible in this field.