How I Support More Than 100 Languages in My React app...and You Can Too!

I'm Richard Kerrigan and I'm thrilled to be here today to show you how you can add multilingual support into your React apps. Before we dive into today's topic, let me do some quick housekeeping. Item number 1. Although I do work at Microsoft, I'm in no way affiliated with the Azure AI team, nor am I looking to convince you to adopt Azure as your cloud service provider. As you'll see by the end of this talk, my goal is to demonstrate one possible solution for automatically translating website web app content into other languages and dialects. But you could absolutely build something similar using AWS, GCP, or another cloud service provider that provides a document translation service.

Item number 2. There are already libraries such as React, IATN that allow you to translate static content such as your homepage, header, footer, etc. into other languages using the translations you provide. The solutions I'll be presenting to you today are for automatically translating the dynamic content, such as blog posts created by your team's content creators or even by end users.

Alright, so with those housekeeping items out of the way, let's dive in. First, let's clearly state the problem we're trying to solve. The problem we're trying to solve is that most websites, web apps, only publish content in one language, which is not convenient for users and is not inclusive for people for whom that language is not their primary language. In other words, we need a way to support other languages and dialects on our websites and web apps, without adversely affecting our team's productivity. Now there are various ways we could approach solving the problem, depending on the resources we have available to us. But for the sake of this talk, let's assume that we don't have much budget to dedicate to this feature, nor do we have a team that can translate content for us. So our solution must be low cost and require human interaction only to validate the results. This is a fantastic use case for AI and automation.

When my team started tackling this problem, for obvious reasons, we looked at Azure first. Through that research, I found the Azure AI Translator service. This service not only allows for ad-hoc text translation, where you send text directly through the translator and it returns the translated text as JSON, but it also offers document translation, which takes as input one or more files from a blob storage container, translates them, and outputs the translated files into a different blob storage container.

Also, Azure AI Translator service has a few key features that ultimately convinced us that it was the right fit for our needs. The first key feature that was important to us was language support. With over 100 different languages and dialects supported, even including Klingon, Azure AI Translator was definitely the right choice for us. The next key feature for us was cost. Azure AI Translator has a generous free tier of up to 2 million characters translated per month. Another important feature is accuracy. Azure AI Translator uses Neural Machine Translation, or NMT, which takes into context the full sentence and produces more fluid and human-translated-looking translations. The final key feature for us is the ability to use custom glossaries to tweak the translation process.

It's not all sunshine and rainbows, though, so I wanted to also take a moment to call out some of the limitations of Azure AI Translator. Although the service can handle many different file types, it currently isn't able to handle MVX or JSX TSX, so your content will need to be sourced separately from these files in order to be translated. Another limitation is that all translation responses are returned as horizontal, left-to-right, or right-to-left text, so you may need to add rendering logic if you want to display content in a vertical format for applicable languages. Finally, as with any AI implementation, you'll still want to validate the results before deploying them abroad, and you may find that additional tweaking may be needed, which would also require a redeployment and revalidation.

Depending on how the content is stored, there are essentially three options for how to translate it. If your content is stored as JSON, you would most likely want to translate the content during runtime using the Azure Translator API. Here's an example of how you can achieve this. As you can see, we're taking just the pieces of data that need to be translated, namely the title, excerpt, and the content, and sending each piece through the translator. Then, once we have all of the translated content, we're sending it as a response.

Finally, as with any AI implementation, you'll still want to validate the results before deploying them abroad, and you may find that additional tweaking may be needed, which would also require a redeployment and revalidation.

Alright, enough talk about Azure AI Translator. Let's get into how we're going to design this workflow. Depending on how the content is stored, there are essentially three options for how to translate it. Scenario 1, your content is stored as JSON. As you can see from this JSON snippet, we have an array of posts, which are objects containing the various data our frontend would need in order to render each post. If your content is stored like this, you would most likely want to translate the content during runtime using the Azure Translator API, probably using a React Server component or a separate custom API that would cache the translated content in order to reduce how often the translation service is being run, which is ultimately going to keep your cost down. Here's an example of how you can achieve this. As you can see, we're taking just the pieces of data that need to be translated, namely the title, excerpt, and the content, and sending each piece through the translator. Then, once we have all of the translated content, we're sending it as a response. This example could be added into a React Server component or an existing API route that sends the content to the React app.

Scenarios 2 and 3, your content is stored as separate HTML or Markdown files. In either of these cases, rather than executing the translation at runtime, we can actually incorporate the translation process into our CICD workflow. Firstly, we'll need to upload the files to be translated into an Azure Blob Storage container. Next, we'll need to create and then trigger an Azure Function app, or similar, which will orchestrate the translation process, including specifying the input and output languages and corresponding Blob Storage containers. Once the Function app has completed, then we need to download the translated files from the output Blob Storage container, and add them to the app's source code files. Now, finally, we're ready to deploy our React app. You'll want to be sure to deploy the app to a test environment first, so you can validate the translation outcome before deploying to Prompt.

Alright, enough talking about concept. Let's look at some code. First, let's walk through what the solution looks like for JSON content translated at runtime. So, in here, if we look at the API route for my Next.js app, you'll see that there's this translate function, just like we looked at earlier in the slides. And if we dive into that translate function, you'll see it's essentially just a post API call going to Microsoft Translator. And what we're doing is we're taking the locale that was requested. So, you could think of this as the route being the domain of the website, and then slash en-us, or es-es for Spanish, or similar to that. And so, we're grabbing just the first portion of that, so the en or the es, in order to understand which language they're trying to translate to. And then, we're putting that together with the API endpoint, and we're specifying that as the to language. And then, in here, we're sending this post request to our translator service, and we're sending in the text in JSON form. And then, once the translation comes back, then we're digging into that JSON and grabbing just the text translation out of it, and sending that as a response. So that's the solution for translating JSON.

Next, let's look at how we would translate HTML files from within a CICD pipeline. So, I'll switch over to my HTML branch. Looking at this same API route, you'll notice that we're no longer doing any translation work inside the API. We're just parsing through the head and grabbing the data. The translation process has been moved to our GitHub Actions workflow. In the workflow, we provision all the necessary Azure resources, including the Next.js static web app, the function app for translation, the translation service, and the storage account with input and output containers.

After provisioning the resources, we build and deploy the function app. Then, we perform the translation process. In the function code, we set up the necessary configurations. It's important to note that the translation endpoint for document translation is different from the endpoint for text translation. We specify the correct endpoint and ensure that they are not used interchangeably.

And then we have our translation service. And then we have our storage account, which contains an input files container and an output functions container.

So, back over to our GitHub workflow. So, then, after we provision all these resources, then we build our function app inside of the workflow. And then, once that's built, then we deploy it out.

And if we look into our actual function, so, this is where essentially all the magic happens. So, here we're just setting everything up. This isn't really any different than what you would do any time you're interacting with blob storage. And down here is where it starts to get more interesting. So, we're specifying the translation endpoint that we're going to be going up against. And one thing to call out is the translation endpoint, when you're doing document translation, is very different than the translation endpoint that you're going to use for just text, like JSON. And so, you want to make sure to not use the same domain for each of those. And so, in here, we're specifying our glossary that allows us to tweak our translations. And then, we're going through each of the blob files that's in our input.

And here's where we configure how we do the translation, when we go to execute. So, we're saying we want to translate to Spanish, and that we're providing it a glossary that's in a CSV format, and that we're doing file translation. And then, we send our request. And then, we submit that data in there. And then, once it's done, then, we go ahead and we create the new blob inside of our output location.

And then, we're just checking to make sure that everything got done, until we can close this out.

And then, we're going through each of the blob files that's in our input. And here's where we configure how we do the translation, when we go to execute. So, we're saying we want to translate to Spanish, and that we're providing it a glossary that's in a CSV format, and that we're doing file translation. And then, we send our request. And then, we submit that data in there. And then, once it's done, then, we go ahead and we create the new blob inside of our output location.

Yep. Yep. Yep. And then, we're just checking to make sure that everything got done, until we can close this out. And this is necessary because we want our GitHub workflow to wait until after all the translations have been done, before it resumes with the next step. Because, in the next step... No. No. After we run the translation, then we're uploading that artifact, and then downloading it in the next stage, and we're taking that artifact, and we're uploading it as our next JS app. And so, it's really important that all the translations get done first, because otherwise, you could end up in a situation where, like, you have half of your content translated, and you go to deploy it. And so, then, when someone switches to that language, they see half of your content, and then, they get errors for the other half, because the file is not found.

No. So, honestly, when I first set this up, it felt kind of like magic, because you're essentially just sending a request over to the translator, and then you just magically get this translated file, and it's all good. And you just have to do a little bit of tweaking. Alright.

So, finally, let's look at how we would do this same translation pipeline, but for markdown content instead. No. Alright. So, the pipeline stays exactly the same. There's no difference here. We're still just uploading the files, triggering the translation, and then grabbing the output files, and deploying our app. So, we don't need to go into that again. And if we look at our translate content, this is where it's a little bit different. So... Hey. In here, if I go down... No. No. No. All of this is the same. Nothing's changed yet. No. No. No. Yeah. So, let me compare this real quick. Let me switch back over to my HTML. You can see what that looks like first. No. No. No.

No. No. Yeah. So, here's the big difference. With HTML, because all the metadata for each of these posts is stored in the head of the file. So, if we look over at one of these files, you see we have our title here, our name, we have our title here, our excerpt, and then down at the bottom we have all our content. And we want to translate this. Well, by default, when you send an HTML file through, it's not going to translate the head. So, instead, that has to be part of our process. So, inside of our function, you see here that we're parsing the document, and this is after the translation's been done. So, we're taking the translated file and we're parsing through it and then what we're doing is we're going through the head in order to get all the metatags and then we're specifically looking for the title and the excerpt. And once we find it, we're sending it through our translate function to get it translated into the destination language. And after that's all done, then we're reassembling our head, including that translated metadata, and then we're replacing the translated file with the new translated file that also has the translated head.

So, if we jump back over to our Markdown version, this is where it's pretty different, because in here, after we do the initial translation, you'll see that we don't have a second step of needing to translate the head. However, what we do have is we have to translate, we still have to translate the metadata separately. And this is actually an issue that I ran into with Azure AI Translator, where it has, it attempts to translate the front matter. So, if we look at one of these posts, all this section, if you're unfamiliar, all of this section is metadata in our Markdown file. And so, it attempts to translate it, but I've run into several issues. So, I've run into issues where this period gets moved outside of the quote, which will prevent this file from loading. And also, I've run into issues where it tries to translate some of the keys in the objects, and then also, it tries to translate some of the words in the URL, like path itself. And so, this obviously causes it all to break. And so, in our translation function, it's an interesting setup. So, we have to translate not the destination files, so like the translated files, front matter, because it's actually unparsable right now because of some of the issues. So, what we actually have to do is we have to parse and then translate the original files metadata and then put that translated metadata in with the translated files content in order to get it matched up. And so, that's what you're seeing right here, where we're replacing or using regex to find the front matter, and we're replacing it with the input files translated metadata. And then, after that, then we're uploading. And so, this is how it all works.

So, now, if we jump over to GitHub, we can actually look at the GitHub actions workflow running. And so, here's the HTML version. And so, you see, first, we provision the Azure resources.

Then we build our function app. Then we deploy it out. Then we run our translation process. And then, with those translated files, we then deploy out. And similarly, now, we can go in and we can run this over our markdown branch. I should kick off here in one second.

And so, now, it's going and it's making sure that there's a resource group in Azure for this.

No. No. No. No. No. No. No. No.

No. No. No. No. No. Okay. And so, now, it's going and it's making sure that there's a resource group in Azure for this.

No. No. No. No. No. No. No. No. No. There we go. No. No. Now, it's going to deploy out our ARM template that contains all the various resources we looked at earlier.

No. No. Just to recap what we've discussed, the problem we're trying to solve, which is the fact that many websites and web apps don't offer their content in additional languages. No. We discussed the key benefits and features of Azure AI translator. But we also acknowledged that similar solutions could be built with other cloud service providers translation services. Finally, we looked at the three different versions of the same app showcasing how to translate JSON, HTML, and markdown content without requiring any members of your team to learn a new language. No. With this being a prerecorded remote session, I can't tell, but I sincerely hope that you're as excited about this solution as I am. If you'd like to explore these solutions in more depth, here's the link and the QR code to the repo on GitHub, as well as links to a couple of resources to learn more about Azure AI Translator.

No. Now let's go back over to our workflow and see how it's doing.

Okay. Great. So it's deployed out our Azure resources. It's built our function out. And now it's in the process of deploying that out.

No. This is the part that we really want to look at, because this is where it's translating our markdown files.

No. No. No. Okay. So at this point, it's downloading the files from Blob Storage. And so now it's triggered the translation, and it's just waiting on the translator service to say that all the jobs have been finished.

No. And this is a really important part, because we don't want to end up with not all of our files translated when we go to deploy next.

No. No. No. No. Awesome. And so you can see in here, it even specifies how many files have been translated.

No. And so now it's taking those translated files, putting them back in with the rest of our repo code.

No. No. And now we're deploying out.

No. No.

And just a quick word about, like, infrastructure as code. So I have this set up, so that way it guarantees that all the resources will be out there. You don't necessarily have to have that part. If you want to have, like, a separate GitHub Actions workflow that deploys out your function app separate from the Next.js app, you could totally do that. I chose to do it this way because the function app is a dependency of this Next.js app, and I want to make sure that it exists and is configured right before I go trying to run translations against it. But that's just my own personal preference.

All right. So it's deployed it, and now it's just waiting on the static web app service to respond back saying that the static web app is actually ready to be viewed. And this takes about two minutes. And then once this is done, then we'll be able to look at the completed app.

No. No. Okay. So it took a bit longer than expected, but our web app did deploy out. And so let's go ahead and open that up and check it out. All right. Here we go. So we have our blog, and we have different posts in here. And for each post, this is where the title from the metadata comes in, and then this is the excerpt from the metadata. Now we have other posts down here, and if we open up one of these posts, we got our title again, and then we have our actual markdown being parsed into HTML to be rendered on the client. And you can see we have different formatting. So, like, we have headers. We have a link right here. We have some bolded text. We have an unordered list. And if we go back and we change languages.

And if we go back and we change languages.

No. No. No. Now we see the title has been translated. The excerpt has been translated. It's for all of these. No. And then if we open up one of these posts. No. No. You see that it's kept the original formatting. So, like, our headers are still there. Our link is still working as expected. No. The text is still the same. Unordered list. Everything is rendered exactly the same. No. No. No. So I'll leave this up for a second. No. No. No. No. No. No. No. No.

So also, I'd love to hear about any multi-lingual solutions that you build based on the concepts we covered today. So please feel free to connect with me on LinkedIn or X. Thank you all for attending. And I hope you enjoy the rest of your conference. Take care. ♪♪♪♪