Frontend Internationalization That Reads Like English (Thanks, gettext!)

Hello, everyone. Hope you're enjoying React Summit US. Today, I'm going to share about a frontend internationalization methodology that I've been using in my project that I like a lot and I would like to share more with everyone. It reads like English. You can tell from the title, it's probably about gettext, but let me go into more detail. First of all, a bit about me. My name is Henry Lee. I'm currently a technical lead at Henge. A little bit of fun fact about me, I like keyboards, ergonomic keyboard. I really like building my own keyboard. And also, I use NeoVim. And a little bit about my company, my company is Henge. We're building B2B cloud security solutions in Japan. And if you're interested in software engineering jobs in Japan, do check this link out.

With that out of the way, let's start with the state of frontend I18N. So if you're building a React project, you want to add internationalization, that's quite a word there. This is what you would normally do. You install a JSON-based library, React I18N, React Intel. You prepare translations as JSON dictionary file, and then you load those translations at runtime and you replace the key in your code with the value that is in the dictionary. This is what the dictionary file looks like for a project with English and Japanese. You create your own key name, like welcome, description. And you have the English version and the Japanese version here. So it looks nice. It looks simple. It works well. But is it actually that simple? This workflow is actually fundamentally broken in some ways. Let me go into more detail. So the first one. Naming keys is hard. As engineers, we know that writing code is our job and inventing variable names is not as easy as it sounds.

There are two hard problems in computer science, cache infiltration, naming things, and off-by-one errors. Naming things are one of the things we engineers always find challenging. The first one is the cognitive load. If we have a JSON dictionary structure, what kind of naming structure do we want? Should it be categorized by page, pages.buttons.save? Or should it be categorized by the feature inside each page, so page.createUserForm.save? Or should it even be extracted into its own group, like common.buttons.save? These are cognitive load that we have to deal with. This is also prone to being inconsistent. Like for example, we have a naming scheme, we agree to it, but then we kind of forget in a minute like, what was that again? Is it buttons.save or is it like undercommon.buttons.save? Those are the pain about naming things. The second one is the context is lost when you're using a JSON dictionary. With the JSON dictionary, the translators need to understand how you structure your app and then infer from there. As an example, if we have this translation file for a game, we have like two keys that are called save. One is inside user actions and the other is just on top. Translator not knowing the context, they would guess, okay, they both mean the same thing. They want to save the changes like in a form. So they choose in this case, the Japanese word, hōzōn. But for example, in this case, the app could mean a different thing. Like what do you want under user actions. Save is not to save changes in a form, but like to rescue a person in the game, because this is like a game translation. So the translations are supposed to be different, but the translator needs to know that this is different just from reading the keys, which is not very intuitive.

Third one is duplicate wordings. It's easy to accidentally make duplicate translations across different keys. As an example here, we can have like a model button, form action, user profile. We would make different keys like if we group them by features, we would make different keys that basically need the exact same word. And in this case, we can see that there are a lot of duplications, but some people might say, we can just extract them to a common group. Like cancel is a very commonly used word. And that 's true, but that 's more cognitive. Like when do you start extracting these words? Do you start when you find the first duplication? Is it too much work? Do you start from the second duplication? You have to manually check and extract them. You have to remember to use the common key next time. If you forget that you have already extracted, you might accidentally use the existing key.

The fourth thing is the obsolete translations. Let's say like after you developed your project for one year, the key structure has changed. You want to refactor the key from one to another.

You want to refactor the key from one to another. What happens to the old key? Some people would say it's too risky to try to remove the old key. The app can break. It can be accidentally used somewhere that it wasn't supposed to be. Removing the key will also mean like we're removing the translation. You might actually want to keep it just as a reference for future translation, or you can just leave it there, which is like you're accumulating technical debt. You get more work for the translators, more cognitive load, and also adds to the final bundle size.

Also, what if a newcomer joins the team? They have to learn about the key structure. They have to check these massive dictionary files that you have. And they find conflicting structures. Like one page uses this methodology. Another page uses another one. It's pretty confusing. Some of these keys are not even used anymore. Like some pages are already deleted, but the keys still linger in the file. What if we could avoid all these pains? Stop inventing key names. We just write English in the code. That would be ideal, right? Like just write English, and you can, when you're coding, when you're reading the code, you know exactly what word is displayed in that element.

What if the tool can also extract messages automatically so that you don't have to fill in the JSON by hand? And then you can let the translators work with actual messages instead of with abstract keys that we define as programmers. So there is a tool that does exactly that. And this is not of the tool. The tool is called GNU GetTex. GetTex is one of the tools that is used in GNU, a set of apps that is commonly used in Linux operating systems. This is the icon of GNU itself. GetTex is a very, very popular package that is used like from as early as 1995 in the GNU project. It is powering the entire Linux ecosystem translations from the core CLI utils to the desktop environments. It's a standard not only on Linux environment, but also on web frameworks and languages like Python, PHP, WordPress. We all commonly use GetTex for translations. The core idea here is that your source code is the single source of truth. What does that mean is that in this case, if you write a translated message as an English string in your code, it will be extracted as a dictionary.

And this is how the dictionary looks like. You get the message ID, which is like the source language, translated message and then message string, which is what the translation will be for the respective language that you want. Then extracting the tool can also let you know where in the code base it is used. In this case, it's only used in one file, app.js on line 3. It can also list all of the files that are using this specific string. It can also automatically extract comments for translators. It can also mark strings as obsolete if it's no longer used anywhere in the project. This shouldn't have any reference here if it's obsolete.

GetTex workflow is like this. So first of all, we have the source code. In this case, it's C. We use xGetTex to create a portable object template file. From the template file, we use message init to create different languages files. And then if we have any new changes to the template file, we will use message merge, which is like merging the new template with the existing TO file. And then we use message format to generate the machine object from our portable object. And this machine object is the low-level code that is being used in our app based on the language.

When you see this workflow, you might think, hey, that looks very complicated. The good news is we don't have to use those complicated workflow in our project. So there are many tools that can allow us to use GetTex workflow in frontend. But I think like Lingui is one of the things that I've used. And I think it provides a very good DX. So Lingui is a very popular GetTex implementation, not only for React but also for vanilla JavaScript. It's a CLI tool that simplifies extracting messages, managing message catalogs, compiling translations. We don't need to deal with each of the GetTex tools that I explained previously in order to go with the entire flow. With Lingui, we can do something like this. Instead of writing the key name in our code and then providing the variables as necessary, we can just say, let's just use the English word here. So in this case, use hello instead of the key greeting. We write hi, user. Hi, username. You have count new messages.

Instead of just writing a key here, and then you have the reference to the dictionary, what should this key represent? You can just write English here and get the full context of what this h2 element is supposed to show. And this is the workflow with Lingui.

First, we define messages, either by calling it with the T function as earlier or with the trans component. We run the CLI extract command to generate the catalog, the dictionary. And then we add translations to this file, either by doing it manually, using AI or AI tools or services to edit or send it to our translators.

The PO file is a commonly known standard that translators should be familiar with. Also, then we run compile after we've been in the translations. And after generating the final bundle file, we deploy it together with our app.

So this is an example of how we define messages. We use the T helper function, and this is the tag template literal function. So it's like calling the T function with one parameter string called hello world. But this is just slightly more concise. If we need to use a variable name, we can just use the tag template literal, the parameter of the tag template literal. So we just parameterize this and use the variable name. Very simple.

One thing to keep in mind is that Lingui statically analyzes our code and tries to find which strings need to be translated. So it couldn't find strings that are evaluated as a result of an expression. In this case, we have a function that returns string even or odd. If we just use it here inside T, then Lingui wouldn't know that we need to extract even and odd because it doesn't run your code and find what strings are being imported. What we should do instead is we should mark each string that needs to be translated beforehand with the helper function called msg here for message. And then we just use T on the evaluated function. And with this, the extraction process works and the translation on runtime also works as expected.

This is an example of the result that we will see when we run the extraction command. When we call Lingui extract, it will go through all of our configuration, generate the files as necessary. In this case, I have two languages configured, English and Japanese. It shows me that it found like 22 messages that need to be translated. And since English is the source, there is no missing translation. The key is actually the translation message. But for Japanese, it says, I have seven missing translations. That means I need to go to the code and see what is missing and add those translations or request to translate or use AI. So it's very simple, it's very intuitive. It shows us exactly what we need in a lot of detail. And this is how the message catalog looks like. As mentioned a bit previously, we have the ID HelloWorld, myName is name. It's smart enough to tell like we're using a variable called name here. If it's a more complex expression, it wouldn't work. But for simple, just variable calling, it will be able to infer the name here. We just need to fill the translations in this message string field.

As mentioned, we can just edit the file, just a translation or just send it to the translator. And this is how the compiled output looks like. For example, in this case, I'm exporting a JavaScript file. Sorry, a TypeScript file. There's an import type here. And as you can see here, we are not really using the actual English word, the key, the message as key in the compiled output to save on the bundle size. Instead of using a potentially very long string, Linguin will hash the string that we use, make it very small, make it very simple so that it doesn't, so that the bundle size is kept small.

This is an ambiguously better DX. We eliminated the need to manage keys, to determine keys, no duplicated entries, because the string itself is the key. We have the full context for translators. They don't need to guess what the data structure means. They can just see from the dictionary that, oh, this is the word that I need to translate. And it's always in sync with extract and compile. Extract will always refer to your source code as the single source of truth.

So your extraction, your PO file will never be out of date as long as you extract them regularly. For translators, it's also nice you don't have to understand the arbitrary key structure. The PO file is an industry standard. Translators know how to deal with it, and they can focus on what matters most, which is the source text and making the translations. You might think, what about the complex stuff? Does Linguin do it well? Yes. As an example here, if I have the word in English right, that can have at least two different meanings.

Right, as in turn right or turn left. Or right, as in correct, you're correct. In this case, we can use the T function, passing an object, message, and context. The context here will let the translator know which right we're referring to. And inside the catalog, it will show as the message context here. So when translators see this, they will know, oh, this is the translation that I should make for this particular word.

There's also comments, if you want to write, explain what in what context this message is being used. You can just write the comment here, and it will show up in the dictionary as well. Plurals is also very common, very simply handled. And the nice thing about Linguin is that not only does it use getTextPlurals, which is slightly less flexible, it supports the ICU message syntax, which is much more flexible. For example, English has a very simple rule for plurals, one or anything other than one.

So one item, two items, zero items, and so on. So in the dictionary file, this is also what you write, called plural. You define for each variation what translation it should be. If you're translating to Japanese, they don't have plural, so it's fine. If you're translating to other languages that have like more complex one, two, three, four different plural rules, you just define it here. So it's very flexible. It can handle all languages that have different plural rules. There are also many other features like select for enums, you can interpolate HTML or component inside of it, a lot of stuff Linguin has handled.

So your next question would be like, why isn't everyone using this? If it's so great, why is it not popular? And I think this is because the default, like React-IoT-Next, React-Intel, is like the default libraries that many people know. It's like the de facto standard. When you do Google search, how do I add IoT and do a React application? When you do LLM, Azure, SGPT, that's also the first answer that comes up. So there's no incentive to actually look for a better way when the in common approach works well enough. And some teams are also like fear of adding a build step. Instead of adding a build step, let me just do it manually. I don't want things to be too complex. But front-end itself inherently has build step. So this is just adding a little bit more for a lot of developer convenience. Also, some people are just not aware that this is a thing. That this approach is a thing. If they start with web, they have no prior experience with new Unix. I think they are not aware that this is a thing. So this is something that more exposure will help. Or articles like success stories, people who have successfully used it will be able to use it well. So my call to action is that if you are working on a new project, consider using Lingui, try it out, see how it works. I believe the DX is much better than the existing alternatives. If you're maintaining an existing project, you can start discussing whether it's worth to migrate to this approach. It might not be worth it, depending on the size of the project and the resource. But it could be something to consider. Share the existence of this tool, discuss with your community, discuss with your teams if this is actually going to be a good fit for you. And yes, that's it for me. Thank you for listening. Here are my slides and my socials. Thank you very much. Enjoy the rest of the React Summit US.