AI in localization: What could possibly go wrong? (podcast)

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Spotify | Amazon Music | Email | TuneIn | RSS

In this episode of the Content Operations podcast, Sarah O’Keefe and Bill Swallow unpack the promise, pitfalls, and disruptive impact of AI on multilingual content. From pivot languages to content hygiene, they explore what’s next for language service providers and global enterprises alike.

Bill Swallow: I think it goes without saying that there’s going to be disruption again. Every single change, whether it’s in the localization industry or not, has resulted in some type of disruption. Something has changed. I’ll be blunt about it. In some cases, jobs were lost, jobs were replaced, new jobs were created. For LSPs, I think AI is going to, again, be another shift, the same that happened when machine translation came out. LSPs had to shift and pivot how they approach their bottom line with people. GenAI is going to take a lot of the heavy lifting off of the translators, for better or for worse, and it’s going to force a copy edit workflow. I think it’s really going to be a model where people are going to be training and cleaning up after AI.

Related links:

• Going global: Getting started with content localization
• Lessons Japan taught me about content localization strategy
• Conquering content localization: strategies for success (podcast)
• The Scriptorium approach to localization strategy
• Get monthly insights on structured content, futureproof content operations, and more with our Illuminations newsletter

LinkedIn:

• Sarah O’Keefe
• Bill Swallow

Transcript:

Introduction with ambient background music

Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations.

Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it.

Sarah O’Keefe: Change is perceived as being risky, you have to convince me that making the change is less risky than not making the change.

Alan Pringle: And at some point, you are going to have tools, technology, and process that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off.

End of introduction

Sarah O’Keefe: Hey, everyone. I’m Sarah O’Keefe, and I’m here today with Bill Swallow.

Bill Swallow: Hey there.

SO: They have let us out of the basement. Mistakes were made. And we have been asked to talk to you on this podcast about AI in translation and localization. I have subtitled this podcast, What Could Possibly Go Wrong? As always, what could possibly go wrong, both in this topic and also with this particular group of people who have been given microphones. So Bill.

BS: They’ll take them away eventually.

SO: They will eventually. Bill, what’s your generalized take right now on AI in translation and localization? And I apologize in advance. We will almost certainly use those two terms interchangeably, even though we fully understand that they are not. What’s your thesis?

BS: Let’s see. It’s still early. It is promising. It will likely go wrong for a little while, at least. Any new model that translation has taken has first gone wrong before it corrected and went right, but it might be good enough. I think that pretty much sums up where I’m at.

SO: Okay. So when we look at this … Let’s start at the end. So generative AI, instead of machine translation. Let’s walk a little bit through the traditional translation process and compare that to what it looks like to employ GenAI or AI in translation.

BS: All right. So regardless of how you’re going about traditional translation, there is usually a source language that is authored. It gets passed over to someone who, if they’re doing their job correctly, has tools available to parse that information, essentially stick it in a database, perhaps do some matching against what’s been translated before, fill in the gaps with the translation, and then output the translated product. On the GenAI side, it really does look like you have a bit of information that you’ve written. And it just goes out, and GenAI does its little thing and bingo, you got a translation. And I guess the real key is what’s in that magic little thing that it does.

SO: Right. And so when we look at best practices for translation management up until this point, it’s been, as you said, accumulate assets, accumulate language segment pairs, right? This English has been previously translated into German, French, Italian, Spanish, Japanese, Korean, Chinese. I have those pairs, so I can match it up. And keeping track of those assets, which are your intellectual property, you as the company put all this time and money into getting those translations, where are those assets in your GenAI workflow?

BS: They’re not there, and that’s the odd part about it.

SO: Awesome. So we just throw them away? What?

BS: I mean, they might be used to seed the AI at first, just to get an idea of how you’ve talked about things in the past. But generally, AI is going to consume its knowledge, it’s going to store that knowledge, and then it’s going to adapt it over time. When it’s asked for something, it’s going to produce it with the best way it knows how, based on what it was given. And it’s going to learn things along the way that will help it improve or not improve over time. And that part right there, the improve or not improve, is the real catch in why I say it might be good enough but it might go wrong as well, because GenAI tends to … I don’t want to say hallucinate because it’s not really doing that at this stage. It’s taking all the information it has, it’s learning things about that information, and it’s applying it going forward. And if it makes an assumption based on new information that it’s fed, it could go in the wrong direction.

SO: Yeah. I think two things here. One is that what we’re describing applies whether you have an AI-driven workflow inside your organization where you’re only allowing the AI to access your, for example, prior translation. So a very limited corpus of knowledge, or if you’re sending it out like all of us are doing, where you’re just shoving it into a public-facing translation engine of some sort and just saying, “Hey, give me a translation.” In the second case, you have no control over the IP, no control over what’s put in there and how it’s used going forward, and no control over what anyone else has put in there, which could cause it to evolve in a direction that you do or do not want it to. So the public-facing engines are very, very powerful because they have so much volume, and at the same time, you’re giving up that control. Whereas if you have an internal system that you’ve set up … And when I say internal, I mean private. It doesn’t have to be internal to your organization, but it might be that your localization vendor has set up something for you. But anyway, gated from the generalized internet and all the other people out there.

BS: We hope.

SO: Or the other content. You hope. Right. Also, if you don’t know exactly how these large learning models are being employed by your vendors, you should ask some questions, some very pointed questions. Okay, we’ll come back to that, but first I want to talk a little bit about pivot languages. So again, looking at traditional localization, you run into this thing of … Basically many, many, many organizations have a single-language authoring workflow and a multi-language translation workflow. So you write everything in English and then you translate. So all of the translations are target languages, they are downstream, they are derived from the English, et cetera. Now let’s talk a little bit about… First of all, what is a multilingual workflow? Let’s start there. What is that?

BS: Okay. So yeah, the traditional model usually is author one language, which maybe 90% of the time is English, whether it’s being authored in an English-speaking country or not, and then it’s being pushed out to multiple different languages. In a multilingual environment, you have people authoring in their own native language, and it should be coming in and being translated out as it needs to be to all the other target languages. Traditionally, that has been done using pivot languages because infrastructures were built. It is just the way it is. It was built on English. English has been used as a pivot language more than any other language out there. There are some outliers that use a different pivot language for a very specific reason, but for the sake of this conversation, English is the predominant pivot language out there.

SO: So I have a team of engineers in South Korea. They are writing in Korean. And in order to get from Korean to, let’s say, Italian, we translate from Korean to English and then from English to Italian, and English becomes the pivot language. And the generalized rationale for this is that there are more people collectively that speak Korean and English and then English and Italian than there are people that speak Korean and Italian.

BS: With nothing in between, yeah.

SO: With nothing in between. Right. Directly. So bilingual in those two languages is a pretty small set of people. And so instead of hiring the four people in the world that know how to do that, you pivot through English. And in a human-driven workflow, that makes an awful lot of sense because you’re looking at the question of where do I find English … Sorry, not English, but rather Italian and Korean speakers that can do translation work for my biotech firm. So I need a PhD in biochemistry that speaks these two languages. I think I’ve just identified a specific human in the universe. So that’s the old way. What is a multilingual workflow then?

BS: So yeah, as we were discussing, the multilingual workflow is something where you have two, three, four different language sources that you’re authoring in. So you’re authoring in English, you have people authoring in German, you have people authoring in Korean and, let’s say, Italian. And they’re all working strictly in their native language, and those would go out for translation into any other target language. It’s tricky because the current model still uses a pivot language, but I think when we talk about generative AI, it’s going to avoid that completely. It’s going to skip that pivot and just say, “Okay, I know the subject matter that you’re talking about and I know the language that you’ve presented it in. Let’s take this subject and meaning and just represent it in a different language and not even worry about trying to figure out what does this mean in English. It doesn’t matter at this point.”

SO: Right. And so I think the one caveat here as we’re looking at this issue is to remember that GenAI in general is going to do better when it has larger volumes of content. And a lot of the generative AI tools are tuned for English. That’s kind of where they started. But it’s also useful to remember that GenAI is math. GenAI doesn’t really have a concept of knowledge or learning or any of these other things. It’s just math. So math is a language of its own, and we should be able to express mathematical concepts in a human language of choice. So there’s some really interesting stuff happening there. Okay. So stepping back a little bit from this, let’s talk about where this is coming from and the history of machine translation in translation localization. Where did we start? And isn’t it true that localization really was one of the leaders in adopting AI early on?

BS: It really was. So way, way, way back, you had essentially transcription in a different language. So people were given a block of text and asked to reproduce it in a different language, and they went line by line and just rewrote it in a different language. Then you start getting into the old-school machine translation or statistical machine translation. What this did was it kept, essentially, a corpus of the translations that you’ve done in the past, and it also broke down the information that you were feeding it into small segments. And it would do a statistical query, taking one segment from what your source said and throwing it out into its memory and say, “Okay, is there anything out here? Was this translated before? And give me a ranking of these results of what was done before.” And essentially, the highest result floated to the top, and it used that. Translators could modify those results over time based on actual accuracy versus systematic or statistical accuracy. But that is forever old. Over the past 10, 15 years, we’ve seen neural machine translation come out, which is getting a lot closer to AI-based translation. So it takes away the text matching and replaces it with more pattern matching. So it’s better at gisting. It will find, let’s say, a 95% match and can fill in those gaps for the most part, or at least say, “Hey, this gets us 95% of the way there. I’m going to put this out over here, and then the translator will essentially verify that translation going forward.” It’s a bit more accurate, but it still relies on this corpus of translation memory that you build over time. And now we’ve got generative AI machine translation, which completely takes everything that was done before, and it doesn’t necessarily throw it away, but it says, “Thank you for all the hard work you did. I will absorb that information and move forward.”

SO: Does it actually say thank you?

BS: It could. It depends on the prompt you use. But I mean, really, you’re looking at a situation where the generative AI model, it uses a transfer learning model to do the translation work. So it takes everything that it knows, applies it to what you feed it for translation, produces an output, learns a lot of things along the way in getting that translation to a point where you say, “Okay, great, thank you. This was good,” and then applies what it learned to the next time you ask. And it keeps doing that and doing that and doing that. On the plus side is that, yes, you can train your generative AI to get really, really, really good if you train it the right way. If someone … And I am not saying it’s malicious or anything, but if you train your GenAI translation model to start augmenting how it translates, then you’ll start getting these mixed results over time because it’s going to learn a different way to apply your request to provide an output.

SO: So the question that I actually have, which I’m not going to ask you to answer because that would be mean, is whether AI is actually storing content in a language, like in English, or is language, in the case of GenAI, just an expression of the math that underlies the engines? You don’t want to tackle that, no. Moving on.

BS: Well, it’s worth poking at, at least, because … Does GenAI actually do anything with the language that we give it now, just for answers? If we’re asking it to write a stupid limerick about a news event, or are we asking, “Summarize this document,” does it care that it’s written in any language? I honestly don’t know.

SO: As meta as it is to ask the question, what is the math that underlies it, the other thing that’s helpful to me, and again, we’re grossly oversimplifying what’s going on, but what is very helpful to me is to think of AI as autocorrect, or autocomplete, actually, on steroids. It’s more than that, but not a lot more. It has just learned that every time I type certain words in my text app, certain other words are likely to follow and it helpfully suggests them. And sometimes it’s right and sometimes it’s wrong, but it’s just doing math, right? Autocorrect learns that there are certain words that, when misspelled or that I do not wish to have corrected, or perhaps it introduces the concept that that word needs to be corrected to the word that I use more commonly, which can be extremely embarrassing. We had some questions about this. We’ve done some prior localization AI conversation, and I wanted to bring in a question that came from one of our audience members. Their question was, “Will we get to the point where we can effectively ask an AI help system a question in a foreign language, the AI system will parse the source language content, and then return the answer in the user’s language? Will translating documentation eventually be no longer necessary?” And what’s your take on that?

BS: Well, I think the answer is yes, and my take is that we are nearly there already. We already have… even apps that you can run on your phone. We have apps that can translate on the fly from verbal language. And I have used them when I travel abroad and I don’t know the language very well, to be able to speak it into my phone and it essentially translates the text for the person I’m trying to communicate with. There are other apps that take a step further and use a synthetic AI voice to read it so that they don’t have to look at my screen. They can just hear what the phone has to say because obviously I’m unable to say it myself.

SO: There’s also a version that does that through the camera. So you point the camera at a sign or a menu, more importantly, and it magically translates the menu into your language while you’re looking at it through your phone, or through your camera.

BS: That has been so helpful.

SO: Yes. Now that is actually a really good example, though, of a place where this kind of translation is hard because there’s very little context, and there’s a tendency in food culture to have very specific terms for things that maybe are not part of the AI’s daily routine. We were talking not too long ago about … What was it? We came up with half a dozen different words in German for dumpling. And we got into a big argument about which one was what and which one is correct for this type of dumpling and all the rest of it. So yeah. The thing I would point out here is that the question was, if someone comes in and asks the AI help system a question in, let’s say, French, but the underlying system is in, let’s say, English, but it would then return French. It’s a very English-centric perspective, to say, “Well, the French people … Our AI is going to be in English, essentially. Our AI database.” And that is a really interesting question to me. Is the AI database actually going to be in English? And maybe not.

BS: Probably not.

SO: I tried this about a year ago with ChatGPT. And you might experiment with this if you speak another language, or combine it with machine translation, which should work as well. I asked ChatGPT a specific question, and I got an answer. Cool. And then I asked the same question again and added, “Respond in, in this case, German.” The answer that I got in German was, obviously, it was in German, step one, which I wasn’t actually sure it could do. But step two, the reply that I got in German, the content was different. It wasn’t just a translated version of the English content. It was functionally a different answer. So it’s like in English, I said to ChatGPT, “What color is the sky?” And it said, “The sky is blue.” And then I said the same thing, “What color is the sky? Respond in German,” and it came back with, “The sky is green.” Now, it was actually did a DITA-related question, which kind of explains what happened here. But what happened was that ChatGPT, even though the prompt was in English, it pretty clearly used German language sources to assemble the answer. And those of you who know that DITA is more popular in the US than it is in Germany would not be too surprised that the answer I got regarding something DITA-specific in German was very much culturally bound to what German language content about DITA looks like. So it was processing the German content to give me my answer, not the English content. Now, if you ask an AI help system, the next question is what’s sitting in that corpus? Because if you ask it a question in French and it has no French in the corpus, then it’s probably going to generate an answer in English and machine translate. But if it has four topics in French and you ask it something in French, it is probably going to try and assemble an answer out of that French content, which could be…

BS: Before it falls back, yeah.

SO: Fascinating, which brings me to my next meta question that we’re not going to answer, which is can we capture meaning and separate it from language? And a knowledge graph is an attempt to capture relationships and meaning. And that can be rendered into a language, but it is not itself specifically English. It’s a database entry of person, which has a relationship with address, and you can say, “Person X lives at address Y,” but that sentence is just an expression of the mathematical or the database relationship that’s sitting inside the knowledge graph. I want to talk about the outlook for LSPs, for localization services providers. What does it look like to be an LSP, to be a translation service provider, in this AI world? What do you think is going to happen?

BS: I think it goes without saying that there’s going to be disruption again. Every single change, whether it’s in the localization industry or not, has resulted in some type of disruption. Something has changed. I’ll be blunt about it. In some cases, jobs were lost, jobs were replaced, new jobs were created. And I think that for LSPs, I think AI is going to, again, be another shift, the same that happened when machine translation came out, when neural machine translation came out, all of this. They’ve had to shift and pivot of how they approach their bottom line with people. GenAI is going to take a lot of the heavy lifting off of the translators, for better or for worse, and it’s going to force a more copy edit workflow. And perhaps, I guess, a corpus editing role or basically an information keeper who basically will go in and make sure that the information that the AI model is being trained on is correct and accurate for very specific purposes, and start teaching it that when you talk about this particular subject matter, this is the type of corpus we want you to consume and respond with, versus someone who actually does the translation work and pushes all the buttons and writes all of the translations. It’s really going to be a model where I think people are going to be training AI and cleaning up after it, essentially. And I don’t know any further than that. I mean, it’s still pretty young. I think also you will see LSPs turning more into consultative agencies with companies, rather than just a language service provider. So they will help companies establish that corpus and train their AI and work with their corporate staff to make sure that they are writing better queries, that they are providing better information out of the gate, and so forth. So I think it’s going to be a complete shift in how these companies function, at least between now and what’s to come.

SO: Yeah. The cost of a really bad translation getting into your database when it was human-driven… this AI thing is going to scale. There’s going to be more and more of it, everything’s going to go faster and faster. And we already have these conversations about AI slop and the internet degrading into just garbage because there’s all this AI-created stuff. And so if you apply that vision to a multilingual world, it’s quite troubling, right? So I think you’re right. I mean, this idea of content hygiene. How do we keep our content databases good, such that they can do all this interesting math processing instead of becoming more and more and more and more error-riddled is really interesting. We started by saying clearly this is a disruptive innovation. Disruptive innovations start out bad, clearly of lower quality than the thing they’re disrupting, but they’re cheaper and/or faster and/or have some aspect that they can do that the original thing cannot. So mobile phones are a great example. They were worse than landlines in every possible way, but they were mobile, right? They were not tethered to a cord in the wall. And then over time, a mobile phone turned into something that really is a computer that is context and location-aware and can do all sorts of nifty things. It doesn’t look at what resemblance it bears to POTS, to plain old telephone service. And we hear people. Oh, I don’t use my phone to make phone calls. Why would I do that? That’s terrible, because we have all these other options.

So from a localization point of view, any organization that is using person-driven, manually-driven, inefficient, fragmented processes is going to be in trouble. And that stuff’s all going to get squeezed out. And I think it’s actually helpful to look at the structured authoring concept and how it eliminated desktop publishing, right? It just got squeezed right out because it all got automated. We do the same thing with localization. I think AI is going to have a similar impact, whether it’s on content creation in any language, that it’s going to remove that manual labor over time. And I think that maybe we’re going to reach a point where content creation is just content creation. It’s not creating content in English so that I can translate it into the target languages. I think that that distinction between source and target is really going to evaporate. It’ll just be somebody created content, and then we have ways of making that available in other languages, and that’s where this is going to go. I’ve talked to a lot of localization service providers recently, and certainly this is one of the things that they are thinking about and looking at, is the question of what it means, to your point, to be a localization service provider in a universe where language translation specifically is automatable, maybe. Okay. Bill, any closing thoughts before we let you go here?

BS: I think this is a good place to end this one.

SO: We’ll wrap it up, and they will come and take away our microphones and put us back in the corner. Good to see you, as always.

BS: Good to see you.

CC: Thank you for listening to Content Operations by Scriptorium. For more information, visit scriptorium.com or check the show notes for relevant links.