How to make our CLIs safer with types?

Hi everyone, and welcome to this talk about making our CLI safer with TypeScript. My name is Mayr Neonaysan. I work at Datadog as part of the front-end dev team, where we are building all sorts of tools in order to make sure our front-end engineers can be as efficient as they want to be. You can find me on Twitter and GitHub, as Arkalis, and yeah, hope to see you there.

So first, about this talk, we are going to split it into four different parts. First, we are going to talk about what are CLIs, what are the problems that we face when building them. Then we are going to see some techniques that we can use in TypeScript in order to make some better inference on the type of the options. And finally, we are going to go over some of the frameworks that already do that for us before jumping into a conclusion and recapitulating everything that we learned. Does it sound good?

They are very simple to read on about and very simple to implement usually. We use them in order to parameterize actions, which can be as complicated as a full CLI application like YARN, or something much simpler like a script that you are using inside your build pipeline. The one thing they have in common is that they are typically very simple. Compared to full UI applications, they require very few boiler plates, and that makes them a very good thing to use if you are just trying to make a couple of behaviors configurable.

However, they can also be tricky. For instance, let's take npm run eslint ____ version. If you don't use the ____ token to separate the eslint and the ____ version token, you are going to end up running the version option on npm itself and not eslint. However, if you are running YARN, you don't have to do this ____ token. But in order to do that, as we are going to see later, things start to get more complicated.

The cp command looks simple. However, unlike many commands where the amount of parameters of variadic arguments is at the end of the command, in the case of cp, it's at the beginning of the command. You have slc1, slc2, any number of other sources, but you must have one destination at the end. So the required position of arguments is at the end rather than the beginning. You have the arm command that has the preserve-root option, but it also has the "-no-preserve-root option". And if you're implementing something like this in your scripts, you usually want to declare both options into one, so that you don't have to duplicate them. And finally, let's take the case of the vlc-v. If you look at this command and you don't know anything else about it, you would think that "-v", is actually a boolean flag. But it's not. If you run vlc-vvv, you will not get the same behavior as if you were running vlc-v. Because vvv is actually a counter. It counts the number of times that you're passing it to the command.

Now, let's talk about the Yarn CLI itself. It's very interesting, because the Yarn CLI is one of the biggest CLI interfaces that we all use in the JavaScript ecosystem. Each has more than twenty commands, and each of them accepts options, and we have some very specific behaviors like we previously saw. At first, it started to use Commander JS, which is a CLI framework for JavaScript that supports subcommands, and that's something that is very important for Yarn, because Yarn add, Yarn remove, Yarn upgrade to interact with you, all those kind of things are commands from the main application. However, we decided to eventually make the run keyword optional, so that you could run Yarn eslint instead of Yarn run-eslint, and that wasn't something that Commander JS actually supported out of the box. So we had to implement our own code in order to support that. Then, later on, we decided to also make the dash dash token optional, so that if you run Yarn eslint dash dash version, then the dash dash version is applied on eslint and not on Yarn itself. That's super handy. However, it required to implement something else on top of Commander JS.

However, it required to implement something else on top of Commander JS. In the end, the code ended up fairly complicated, and it was difficult to reason about it and to maintain it. This is a picture of me every time I had to do that. CLI is on top of that, or can also be error-prone. You can forget many things while writing a comment. You can forget to declare an option. You can forget to declare its default value. You can forget to validate it if it has a specific shape that you're expecting. You can even forget to use the option, and you can forget to remove it once you no longer need it.

For instance, in a couple of cases when working on Jan it happened that at some point of the development of a feature I added an option, worked with it for a while, then decided to remove it before managing the pull request. However, I may have forgot to remove the code that was relying on that. Or I could remove the code that was relying on this option, but forgot to remove the option from the actual declaration of the command, meaning that someone running the help command would see this option exists but where it wouldn't actually be there.

So in 2018 when we started with a large scale project in order to rewrite Jan and make it fully compatible with TypeScript, we decided to ditch several of our legacy systems, and one of them was the CLI implementation. So we started to look at what existed, and unfortunately we didn't really find anything that was solving all of our requirements. We needed something with good readability because we had a lot of contributors that would jump into the codebase without prior knowledge of anything that we were using. We needed something that was fully type-checked because we wanted to prevent as many errors as possible without having to rely on manual code reviews because we noticed that at every code review there's always something that lives by us undetected. And finally we wanted to really protect against mistakes. We wanted the pit of success to be there. So if you're not familiar with the concept, the pit of success is that whatever you're doing, you will fall into a command behavior. And then if you can improve it later, but by default everything is sane and works as you expect. And you really have to get out of your way, in order to make something that is broken.

So we talked about CLIs in abstract terms. And in the case of YARN what we needed to build. So we decided to build our own framework and by doing that we learned a lot of things. So that's what we are going to make together. Let's build a fully typed CLI framework from scratch. Of course we are going only to focus on the type-check aspect and not the implementation of how to parse the arguments because that would be much more than a 25 minutes talk. But as for the type-check, we have three ways of achieving these effects. First we can use metachaining, we can use decorators, and we could use what I call the third way. We are going to keep this one for the end because it's quite interesting but first we need to go over the two first in order to better understand why it's so good. So first let's talk about metachaining. It's the simplest one, in fact that's the exact same syntax as Commander.js uses and that's a good thing because it means that it's familiar to anyone that used to make CLI interfaces a couple of years ago. You have a command and then you declare the options by compounding the results of the command function and once you have declared all the options you can declare what the command does. And then the CLI framework when the command is called will parse the options and call the action by passing it an OptionBug that contains all the options that got parsed. For this talk, we are going to pretend that the options are always set because otherwise all the things like obd.check would be boolean or undefined rather than just boolean and it would make the code harder to read. But here what we want to reach is a state where obd.check exists and is a boolean, obd.require exists and is a string and obd.foo is properly detected by TypeScript as not existing and that's the important part. We can't have obd be any. That wouldn't work for us. In terms of TypeScript, what would be the type of the resultant value of the command function? First, we would want something that returns a command with no option, because it doesn't declare any option.

Then, as soon as we call boolean, this time the command is a generic with a check property that is a boolean. And then, once we call the string function, the command is returned as a generic with both check and require. And finally, when we have action, it just takes a callback that accepts the option bag as parameter and returns void.

By doing that, we allow TypeScript to properly infer when we are passing the action callback that the OPITIZ type is composed of the option that we declared earlier. To do that, it's very simple, really. This is all that we need.

So we have our command interface with its generic type, which defaults on void, and we declare both Boolean and string as a function that takes a literal string as parameter. We even use some kind of string concatenation as types, because a recent version of TypeScript supports it. And each time we call either Boolean or string, we return the same command type, except that we are extending it to now include the newly declared option. And finally, we have the action function that, as we mentioned, just lets TypeScript infer the type of OPDs based on the generic itself.

This approach has pros and cons. It's easy enough to write. Clearly, there's not a lot of code involved, as you can see. It does feel a bit like 2017, because most of the CLI framework of the time were written like this by chaining functions, one after the other. It means that if you're already familiar with this kind of framework, then it's exactly what you would expect. However, it's a bit variable. You have to use a lot of functions in order to declare your options. It's not very idiomatic. You're using method chaining, which is something that you don't find in a lot of JavaScript interfaces. So people coming into your code may find this a little difficult to read. And it has poor tooling integration. And this is the main one I want to talk to you about. Let's take method chaining here. As you can see, we are declaring a command with a string option. And we have an action here, which prints a string with the name of the person running the command. But does it really? Actually, no, it doesn't. We forgot to use opent.name. But TypeScript is not able to see that, because from its perspective, perhaps the option bag that is passed from parameter to the action is also used somewhere else. So it's not a local. So the no unused locals check will not be able to see that we forgot to actually use that.

So that's what I mean by tuning integration. Since we are bypassing a little declaration of local variable and this kind of thing, we don't get to as easily check that our code is correct. So we have a second option to do that.

Decorators are interesting because they are using a new syntax that has appeared very recently. It's now stage three. Until then, it was kind of difficult to know whether we could use them or not. With decorators, you're attaching annotations above each property of your class. And by doing that, you can attribute special metadata, special logic to those properties.

So in theory, we could imagine something like this, where we're declaring a class for our command. Then we are declaring the options. And then we are attaching an annotation, which declares whether the check, for instance, is a boolean option or a string option, like for require. And that's much better than the meta-chaining in terms of threading integration. Because this time, we have just regular class properties, which are declared private. So TypeScript knows that they are not going to be used anywhere in this file. So if they are not used in this file, it means that there is a problem and it will warn you about it.

Additionally, yeah, see, that's the example that I was mentioning about. If you're not using name, then TypeScript emits an error. However, decorators have a couple of issues that we need to talk about. First, they have a complicated history. You have two different kinds of decorators. You have the old decorators and the new decorators, because decorators went through various stages and have various designs. So if you have a library that relies on old decorators in your code base, then you cannot use the new decorators. And if you have a library that use new decorators, then you cannot use libraries with all the decorators support. It also means that you need to have a compatible transpilation steps. And as we saw, there are both old transpilation steps and new transpilation steps. So it's a bit difficult to reconcile. And perhaps the biggest issue is that they cannot affect types. So if we take, again, the example that we wrote, here, we have the check option that is a Boolean and the string option and the require option that is a string. However, TypeScript is not actually able to infer that the type of check is Boolean and the type of require is string, despite the annotation.

However, TypeScript is not actually able to infer that the type of check is Boolean and the type of require is string, despite the annotation. What we need to do is to explicitly set that check is a Boolean and require is a string. Which means that we have some kind of duplication, because we have to write Boolean twice, string twice. It's not great. Additionally, what if we make a mistake and set Boolean instead of string when we are declaring the require option? Turns out we have some way to prevent that, but there's still the duplication issue. And it requires a lot of code just to make sure that TypeScript is happy.

Ideally, we would want something where we don't have to duplicate both the declaration and the types. So how can we do this? That's what I call the third way. With the third way, instead of assigning the annotation as decorators, we are just assigning them to the property. So here, we have our command class. We have our check and require options. And for each of them, we are just assigning the result of a Boolean and string value. And once the action is called, it's magic. Everything just works. TypeScript is able to infer the type of check and require, and you never had to declare them yourself. It seems the best of both worlds, right?

So you would have your argv array, you would instantiate your command, you would inject options into the command, and then it just works. You can call the action method as you want. But how does that actually work, right? Because it does seem too great to actually work. You don't have complex transpilation. It's directly in the AST, so things like TypeScript or ESLint can operate very easily on it, and check that everything matches your requirements. And you have no type duplication as we see in the decorators. That's weird.

Now let's dig a bit to understand how that's possible. First, we are going to focus on the implementation of the Boolean and String functions. What do they look like? Well, they're very simple, they're just functions that return a Boolean and a String. But are they? No, they are actually functions that return a set of metadata that masquerades as the type that we want in the end. So for instance, here we have Boolean that returns the type Boolean object. But that says to TypeScript, no, it's actually a Boolean, I promise you. We are using as any to lighten it to the time checker. That might seem weird.

No, in the setup command, in the setup code, sorry, we saw that we have an inject options into command. How does it work? This is the general code of this function. For each argument, we are going to strip the double slash, and we are retrieving the metadata that are associated to the option. As we saw earlier when we are doing check equal Boolean or require equal string, we are assigning an object that contains a type property. Here, we are just retrieving this object. Since we typed command as any inside the parameter of inject options into command, TypeScript will let us do this. And now that we have the type, all we have to do is to actually assign the real value to the option, which matches the type that the metadata promised that it would be. And by doing that, it works. So in other words, we lied to the type checker, but it's for a good cause, so that's fine. It even allows us to make more complex things, like format validation. Let's take an option function that returns a string, just like the string annotation that we declared earlier. We can also add it overload so that it accepts validators. In TypeScript, you have different validators that allow you to check at runtime that a specific value has a specific type, and then for the rest of your code, treat it as the proper type that you validate. You have two libraries that are doing that. So you have a Tpanion and Zod that are both TypeScript compatible.

So you have a Tpanion and Zod that are both TypeScript compatible. And we can integrate them with the syntax that we described. Since both of those libraries offer a type that has a generic on the type that has been type-checked, all you have to do is to tell TypeScript that your option function accepts a validator that returns something. And in returns, once the option is parsed, it will be this same type, and it will just work. For instance, here with T-Banyon, we are declaring an option that has a validator that checks that the argument is a number. And then inside the action, TypeScript properly infers that count is a number.

This third way has pros and cons like the others. First, it doesn't require any special syntax. It's just classes and properties, so it requires the bare minimum of ES features. It also supports generics as we saw. It's fairly easy to write some code that infers the resulting type based on the parameter provided to the annotation. It works very well with ESLint and Prettier. The syntax is pleasant to read. And finally, all of the complexity is completely hidden away within the core. You don't have to deal with anything special in order to understand how you're declaring your CLI options. You don't have to worry about anything, even if something is required. For instance, if you want to declare a required positional argument, that's something that is going to be abstracted inside the declaration of the positional arguments, but users won't have to worry about it in order to have the type work out of the box.

The cons is that we are aligning to TypeScript a bit, so it may come as a surprise to people who are more power users of TypeScript that may expect something and not understand why it doesn't work as they would expect. Additionally, composition is a little more difficult because then you have to wrap the annotation inside one another. But usually, from my experience, composition is fairly rare inside CLIs, so this has never been a problem on the CLI I worked on.

So, now let's talk about the options that you have when writing CLI tools. When I talk about YARN and say that we improved it, we ended up writing our own CLI framework. So first, I'm going to talk to you about this one. It's ClipAnion, and it allows you to do something very much like the third way I described earlier. You have a command class, you have an option namespace that contains a bunch of annotations, and you have the runExit() function that lets you run your actual program. Here you're declaring a command that contains a couple of options, so check and require. And you're assigning them to an option that is either a boolean or a string, with all the option names that you want. And finally, TypeScript is able to infer their types thanks to all the overloads that ClipAllian declares and make sure that it follows during the CLI-paired parsing. In fact, here you can see that this time we added the or-undefined() because an option may be set or may not be set. If it's not set, then it needs to be either boolean, in case it's set, or undefined.

It also supports validators through ClipAllian, through, sorry, through TipAllian, which is the type valid, runtime type validator that I mentioned before. So if you have a string option, you can also validate that its format is actually a number, in which case TypeScript will properly refine the number to be number, will refine the type to be number instead of being string. ClipAllian supports many things. It's the framework that we are using in the Yarn CLI, which does a lot of different things. It supports optional positional arguments, required positional arguments, will card positional arguments. For instance, when you are running Yarn ESLint and accepting options, that's a will card because we are accepting anything. It supports boolean, strings, arrays, tables, counters. It implements the parsing of the CLI tokens through a state machine, which is something very unique. It means that we are just generating a full state machine with little room for bugs. A lot of CLI frameworks nowadays try to parse the token one-by-one through a for loop, basically. But you have the risk of having a lot of bugs. It also enables us to support overloads. For instance, you can have different commands with the same path but different options or different requirements. KeepOnion will be able to figure out what is the one that it should use, depending on what the user provided. Native error handling, low runtime override and, of course, it's fully type safe. So, for instance, given the following command declaration, it will be able to properly refine the types depending on what you declare. There is, of course, a regular option that has Boolean, that is Boolean, or undefined. But you can also assign a default value, in which case KeepOnion will see that since there is a default value, it cannot be undefined anymore. You can also declare optional option that accepts a string as argument, in which case a string or undefined, but you can also mark them as required, in which case they will become a string, because they are required and if they are not there, KeepOnion will throw an error. You can also declare tuples using the ret option. You have a lot of different choices and each time KeepOnion does the right thing and properly type-checks it. But you have also other tools that are doing the same thing with slightly different syntaxes. First, you have Common.js, of course, which is the oldest one and actually supports types really well using the extra typings package. So it works just like metachaining that we saw earlier. And you also have Oclif that is perhaps a little more verbose but works sensibly the same way as KeepOnion.