How TypeScript is integrated in your editor

Hello, everyone, and thank you for joining me today. I'm Maria Solano, and I'm a software engineer at Microsoft. Quickly introducing myself, I focus on TypeScript editor tooling, but sometimes I also dive into LSP extensibility, Visual Studio's JavaScript Project System, or I just stare at the title TypeScript checker for a couple of hours, trying to absorb its wisdom, as I'm sure all of you have done.

Today we'll explore how TypeScript is integrated into your editor, and who does what, and when. For this, we'll use TypeScript inline variable refactor as an example, which is my favorite, mainly because I implemented it. Say that you have this print greeting function, and your cursor is in the first line of its body. In an editor like VS Code, you'll see a lightbulb. Where is that lightbulb coming from? Spoiler alert, if you click on it, a list appears. Who constructs a list and how are items inserted into that list? Not only VS Code has this functionality, Visual Studio, NeoVim, Emacs, Zed, and other obscure editors also have a TypeScript lightbulb experience that doesn't differ much from the one in VS Code. How is that done?

As a side note, during this talk I'll be using code action and refactoring as synonyms, despite that they're not exactly the same. Although the difference is a technicality, I'll explain it anyway just so that you can understand when your next code editor trivia game. A code action is tied to a diagnostic, and hence represents something that you can do to potentially fix those errors. Refactoring, on the other hand, aren't corrections, they're just smart recommendations for writing cleaner or nicer code, but it doesn't mean that there's something wrong with what you wrote.

Going back to PrintGreeting, the first thing that happens is that the editor will recognize a trigger event. This can be a passive action, such as your cursor being placed in a position where a refactoring could be applied. The trigger could also be explicit, such as one that you configure with a keybinding. Note that results could differ based on the trigger kind. Having a constant lightbulb popping up everywhere on your screen could be annoying, and so TypeScript might decide to show you certain refactorings only if you really want to see them. In this example, TypeScript will show a lightbulb besides the identifier of a variable that could be inlined, but you'll need to explicitly request the refactor in references of such variables.

Now say that we are no-beam gurus and use the mouse to click on the lightbulb. The editor doesn't actually know which refactorings to display here. It needs someone who can understand the code.

To communicate with that someone, the editor will record the position as well as the content type. Note that the content type isn't exactly a 1-to-1 mapping with the type coming from the file extension. A TypeScript snippet inside a script block of an HTML file is also assigned to the TypeScript content type. This is how you can still get code action, completions and the correct syntax highlighting in embedded code.

An extension announces that it can handle the code action request. There are different mechanisms to do so. In VS Code, you would register a code action provider for the TypeScript content type. In Visual Studio, you could use Roslyn APIs or the language service protocol. Most of the time, this basically involves registering a callback and telling the editor when to invoke it. Note that this is unlimited to language servers. Extensions like ESLint can also hook up into the code action list to enhance the fixes provided by the language server. This also means that there can be multiple code action providers coming from different extensions. The editor will then combine all of the results into a single list.

But OK, I'll back off now because I already mentioned the TypeScript server before properly introducing it. As the name suggests, TS-Server is the JavaScript and TypeScript language server, an executable which encapsulates the language compiler together with other features that are represented with a JSON protocol. This protocol defines a set of commands which represent each message you can send to TS-Server together with the request and response format. This is how different editors can provide the same TypeScript experience, they are all just following the same protocol.

For example, when sending a completion request with a completion request payload, you'll get a completion response back. When I first joined this team I remember being confused with the difference between TS-Server and the TypeScript compiler. TSC can indeed show you TypeScript errors, but these are only the ones that prevented successful TypeScript compilation. Other features revolve around the editing experience. Think of how it doesn't make sense to figure out refactors if you're just transpiling to javascript code.

Before returning to the event handling, remember that the editor just had the position and file information, and so first it needs to translate that into something the TS-Server understands. Here I show a simplified snippet of what the TypeScript VS Code extension does, but Visual Studio has a basically identical C-Sharp implementation of the same logic. About that first line of the arguments, that's because VS Code uses zero-based lines and columns, but TypeScript requires 1-based positions for these. You can imagine the off-by-one errors we've had because of that. You might also wonder, where is that client variable coming from? In this context we use client and editor interchangeably. Since TypeScript is the server, the client is whatever is sending messages on the other side. TS server is listening on the other side, and the state and delegation of this communication is mostly encapsulated in session.ts. In this file, there's a handler for each command defined in the protocol, delegating to each language service and performing some processing of the arguments and responses before putting them back on the wire. Here we also keep track of the TypeScript project, which is defined by your tsconfig.json file or by performing some discovery heuristics in the case that you have a set of TypeScript files in a random folder. It's up to this layer to pick up the results from one project, or combine them across multiple ones.

Moving on to the session handler of interest, we arrive to the getApplicableRefactors function. Here we extract each argument, get the project, and then delegate to the assigned service for that request. Notice the getPreferences call. Since refactorings can change your code, we try to ensure that the applied modifications have the indentation, code type and other style needs that you have configured. Each refactoring has its own handler and the one for inline variable does something like this. Given the arguments from the request, it will ask the type checker for their respective typescript node. And by the way, if you want to learn more about typescript's inner workings, you might have fun looking at that 50,000 line long beast. It then performs a bunch of checks to make sure that such a node corresponds to a variable initialized to a value. It also tries to ensure that your code doesn't break when applying the refactor, and so we check that you are not exporting the variable, among other obscure verifications that my more knowledgeable peers told me to consider. After that, we piggyback on findAllReferences to find all the references we would need to modify. Next we ensure that all references' locations could correctly handle the value replacement.

And if any of these checks fail, we just return on defy to indicate that the refactor cannot be applied here. The response is then sent back to the editor, which performs an analogous translation to add those refactorings to the lightbulb list.

And fantastic! The editor now unfolded the code action list and showed the user how they could improve their code. They picked the best action ever, of inline variable, and we sent another request to tserver.

At this point you might be wondering, why do we need to talk to tserver again? This is because code actions are an example of a tserver command that happens in two stages. In getApplicableRefactors we just notified the editors which refactors could be applied, but we didn't tell it how. This is because we don't want to compute all the necessary file modifications for refactorings if you won't even click on them. Only when the refactoring is actually selected, the editor will ask tsserver to apply the chosen item.

Refactorings might seem special here. But there are other operations with two steps that you've used a lot already. Completions are another example, you might have a completion item with an exported object from another file. If selected, not only the completed line will be modified, TypeScript will also update your imports to make sure the new reference is valid. You know the drill, the editor registers a selection, which later the TypeScript client will map back to the refactor it received in the previous request. From that, it constructs the commands arguments following the TypeScript protocol. And when it receives the result, it will return the workspace edits, which include file modifications, additions and removals that the refactor implies.

Let's go back to the handler for this refactor. When asked to apply the action, it will again ask the compiler for the node that's being inline, the references to it, as well as the expression we'll replace the variable with. It then updates all the references to use the value instead of the variable identifier. And finally, it gets rid of the now on use variable. And ta-da! After the editor reads and translates the response, the refactor is finally applied.

Now that you're TS Server experts, impress editor and TypeScript maintainers the next time you encounter a bug and create a GitHub issue. Say that when applying the code action, the modified line is off by one. Without further context, I would dare to say that this is a language server bug, since at the end of the day the editor is blindly applying the change that the server told it to do. Also, if it's a server bug you would be able to reproduce the issue in another editor, so you can also try that. As another scenario, say that when clicking on the light bulb, the displayed list has duplicate items. This is a bit harder to say, but from my experience, I would guess that this is an editor bug, since it is the one responsible for querying the code action providers, combining the results and displaying them in the UI. As a funny story, one of the first bugs that I fixed at Microsoft was duplicate completion entries in Visual Studio. It was both a fascinating, but slightly traumatizing experience. If you really want to look like a professional, you can even take a look at the TS server logs, or by setting TypeScript trace level to verbose in VS Code, to see what exactly the server is receiving and returning. That will help pinpoint where things go downhill.

Now that we went over how the TypeScript protocol works, you might ask if it was always designed like this. The answer is a bit vague, because TS server kind of always existed, despite not being formally specified. In the early days, TS server ran in processing Visual Studio. And yes, I mean IDE and not VS Code, because VS Code didn't even exist back then.

Later on, TS server was moved to its own node process, as an effort to improve the performance of the IDE. Fortunately, that transition was smooth, because with the existing JSON protocol, going to an inter-process communication had basically no impact. From the snippets we just looked at, you might feel like translating the arguments and response might be annoying, or even redundant. However, this allows the same server to communicate with anyone that follows the TypeScript protocol. The language the client is implemented in doesn't matter, neither does the UI used to interact with the user. TS server can just care about TypeScript stuff, and the editor about everything else.

Thanks to this design, we can have a TypeScript client in VS Code, a Lua implementation for NeoVim, and a Python handler for Sublime. In fact, TS server was even an inspiration for LSP, the protocol that defines a standard over how editor tools and language servers communicate. With it, a single server can be reused in multiple development tools, and editors can support languages with minimal effort. LSP uses language-neutral types, like file URIs and document positions. This is because it is much easier to standardize the text document URI or a cursor position than it is to defining a common ground between abstract syntax trees and compiler symbols across multiple programming languages.

So why don't we just use LSP? Why stick to this custom TypeScript protocol instead of the more flexible and general language service protocol? After all, by doing that transition, there would be no need to translate between TypeScript and Editor types. However, such a change would require a major rewrite of not only the server but also all of the clients that have been following this communication handshake for years. Moreover, such a change might not mean a huge win anyway, since in order to provide rich Editor interactions, we often need to override LSP methods. Remember that LSP remains language neutral. Enhanced TypeScript-specific features require custom extensions to the protocol. An example of this is the new Move to File TypeScript refactor. With it, you can move a block of code to an existing file, which requires asking the user for the destination file name. LSP doesn't have a standard for code actions that require user input, while TS Server can evolve to do so.

That being said, LSP is still a powerful extensibility solution, and even complex and popular languages like Rust, you sell SP for their language service interface. And that's all that I had for you today. I hope that you enjoyed and learned more about how you're getting all of those type goodies. And the next time that you click on a lightbulb, you'll remember this talk.