Is the AI job replacement hype overstated? Artificial intelligence for translations NOT READY to replace humans, leading translation firm finds

Translations of Eurovision song lyrics expose flaws in replacing humans with AI-powered tools

Despite Google Translate recently axing its human feedback translation tool called Contribute, artificial intelligence-powered translations lack the accuracy and sophistication to replace humans, according to a study by a leading human-powered translation company.

Analysis conducted by The Spanish Group – a human-powered translation company counting U.S. government agencies among its client base, compared the official translations of some of this year’s Eurovision song entrant lyrics to translations done by ChatGPT and Google Translate.

The findings revealed that AI had misinterpreted the original lyrics into phrases that changed the tone and overall meaning of the songs.

The Spanish Group has also internally been able to study the difference between using AI-powered tools and humans for delivering translations to its clients.

Its overwhelming conclusion is that artificial intelligence cannot compete with humans to match its precision and its ability to pick up nuances and local language differences.

The importance of accurate translations is evident in Croatia’s entry Rim Tim Tagi Dim by Baby Lasagna, the bookies’ favourite to win this year’s Eurovision song contest.

A serious song about a young person ready to leave Croatia, but worried about fitting in and being accepted in a new country once translated takes an amusing turn.

The lyric “I won’t go far and I’ve sold my pride” turns into “I don’t go far and sold my cow” according to translations from Chat GPT and Google Translate.

Similarly, Google Translate somehow came up with “the jail is very nice and very sophisticated” instead of the original lyric “everything is very beautiful and very perfected”.

Spain’s Eurovision song Zorra described as an anti-female slur has not been without controversy. Its steamy lyrics include “I’m just a vixen to you”, transforming into “I’m just a fox” (Chat GPT) to the more provocative “I’m just a bitch” (Google Translate).

Estonia’s entry (nendest) narkootikumidest ei tea me (küll) midagi, or to the non-Estonian speakers We (really) don’t know anything about (these) drugs produced an eye-catching translation error.

Like the official English translation, Chat GPT used the wording ‘uniformed’ to describe police officers carrying out a raid in a village, generating the lyrics “uniformed men visiting” compared to the original version “a visit from men in uniform”. Google Translate went one step further with its fruity interpretation. Its reading of officers carrying out a raid in a village was “I’m fine with fit men in the village.”

Georgia’s entry Firefighter further underlined the inconsistencies that can arise from using AI tools to translate song lyrics. It showcased multiple nuanced differences, from “I’m like running like tigers”, which Chat GPT and Google Translate believe to be “I’m flying like a hawk.” This is alongside more obvious mistakes from the two translation tools thinking that ‘dove’ means the same thing as ‘firefighter’.

Language expert and founder of professional translation services provider The Spanish Group, Salvador Ordorica, believes the research shows the pitfalls of relying on free tools.

“AI-powered translation tools are really useful for informal, low-stakes tasks like understanding restaurant menus or learning simple phrases.

However, as this research shows, they aren’t suitable for complex tasks. Free translation software currently lacks the accuracy and sense of nuance required to do important work. I’ve heard terrible stories of visa applications being rejected based on mistranslations, for example.

On the translations highlighted, he said: “These discrepancies underscore the limitations of AI translation tools. While they’ve made strides, they often miss the subtle nuances and cultural context that human translators grasp effortlessly.

“This analysis highlights the unique value humans bring to the translation process, ensuring accuracy and preserving the intended meaning. In a globalised world, human translators remain essential for effective cross-cultural communication.”

As the anticipation mounts for the 2024 Eurovision Song Contest, there is a crucial yet often overlooked aspect of the event: translation. Behind every melody and verse lie linguistic intricacies, where a wrong translation can give a whole different meaning to a song.

The appetite among music lovers for translating song lyrics is shown in a recent study by WordFinderX. It found that Taylor Swift is the most translated artist in the world, with fans translating 365 of her songs into 57 different languages, equivalent to nearly 5,000 translations in total.

So, if you and your friends are having your Adele Carpool Karaoke moment; you probably want to make sure that you’re singing from the same translation sheet.

Research Methodology

For the study, a direct comparison was made between: the native lyrics as supplied on the official Eurovision website, Eurovision’s official English translation of native lyrics, and the native lyrics translated via Google Translate and ChatGPT respectively. English-speaking entrants, or countries with an entrant singing in English only and which did not have a translation, were not included in the research.