Deep Dive into Undici

Hi everyone, I'm Matteo Collina, Platformatic Co-Founder and CTO, and today I'm going to dig deep into Undici, the new HTTP library built by the Node Core team. A few words about me, I am part of the Node.js Technical Steering Committee as well as a board member of the OpenJS Foundation. I created some libraries that you might use. I spoke about them at Node Congress a few times, so you might want to check them out, both Fastify and Pino if you like them. I worked eight years as a consultant, then decided to do a startup. Here is Platformatic. I have a newsletter at node.landofdev, a YouTube channel, and so on and so forth. You can check it out. But in all of this, the important part is you're likely using my stuff, because my modules have been downloaded 22 billion times last year, which probably includes you as well, that you're watching this video.

Let's talk a little bit about Undici. Undici is a relatively new or modern HTTP client written from scratch for Node.js. What does it mean from scratch? It means it's not based on the internal HTTP and HTTPS module, but we created it from nothing. Undici, not of color here, means 11 in Italian, and comes from HTTP 1.1. It's also a Stranger Things reference because I was watching Stranger Things when this started to come into my mind.

How does it work? It's very easy to start using, to be honest. You can just import requests from Undici and just put your URL in, and you get all the things that you would expect. You get the status code, headers, trailers, and body. You get all the things. Note that it has implemented the same body mixing of fetch, so you can await body.json, for example. Something very important to remember is whenever you get a body, please consume the body. There are certain logic internally that will make sure that things are handled correctly, but clean it up if possible.

Undici is also the home of fetch for Node.js. When you use fetch inside Node.js, in reality, you're using fetch from Undici and Node.js bundle Undici internally. It's a spec-compliant fetch that you like and love. That's more or less all the things that need to be said. We are working very hard to have as much higher spec compatibility as possible, so this should in theory work as close to as possible as the fetch that you are running in your browser and also the other runtimes. If you need to have spec compatibility, if you want to work on spec compatibility for fetch, Undici is the right place.

What are the good things about Undici? First of all, Undici has improved performance. We'll go through that later. I have a cool few slides. It also creates a complete split between the API, the public API, the developer experience, and how connection management is handled. Something that was not present in HTTP.request and HTTP and HTTPS module. It has also multiple implementation. I said pool. Yeah, I said pool. HTTP is a connection-based protocol, so you probably want to do connection pooling. Despite all the literature says that HTTP is a stateless protocol, but it's stateless because you're talking about a stateful... How is this possible? Well, yes, of course, because it's stateless in the sense of semantics, but it's implemented on top of a stateful connection, TCP and TLS, of course. I will talk about what interceptors are and how to integrate with them.

Also, it's maintained by the Node Core team, HTTP team, and in fact, most of us think that HTTP.request is almost unfixable and therefore that's what we are working on instead. At the beginning, it started from support for HTTP 1.1 only, then we also recently added HTTP 2.0 support, and now it's... You know, the Undici name might not be the best reference anymore. Anyway, so... Also, we have HTTP 1.1. It also has HTTP 1.1 pipelining support. So... It also... Something that is a little bit... There are some things that are different in it, so you might experience a few errors that you might need to deal with, essentially. So, why? Well, the problem is the HTTP, HTTP modules are a mess, okay? Mostly because they use the same API structure for both the client and the server of HTTP 1.0, and both of them are tied with the connection management with the public API. So, how can you see it? Well, you have a request.socket, okay? The fact that you have a request.socket property will tell you that there is this kind of connection, because if your system is not being on socket anymore, like on HTTP 2.0, how would you implement that? And this is the reason why you cannot support both HTTP 1.1 and HTTP 2.0 on the same API.

Finally, last but not least, we cannot change those classes without breaking express. So, yeah, well, that's the situation. So, we created undici.request to be as fast as possible, because making HTTP request should not be the bottleneck of your app. It also provides a good enough DX, a very good DX, from my point of view. It's highly configurable, it uses Node.js stream, supports HTTP 1.1 pipeline. It's great. Just use this. This is probably the code that you want to be using, okay? Undici Fetch is built to be spec compliant, okay? So, it's not putting performance as the top priority. It's spec compliance that it is. And right now, 88% of the WPT tests that we are running for Fetch, we are passing them. So, I think it's pretty good compatibility overall. However, internally, it uses WhatWGstream, so you can use Pipe2, WhatWGstream, all the things, the Redible stream, like the browser. So, all the code that uses Undici Fetch and Fetch Node could be isomorphic. Again, the protocol handling and the API are completely split, so all the things that we are going to talk about, Undici and connection management and so on, all apply to both. And this is the reason why Undici Fetch was built on top of Undici. The Fetching Node was built on top of Undici. It also supports HTTP2, HTTP pipelining, because of this separation.

Okay, let's talk about performance, because this is the critical stuff for it, okay? So, which, you know, it's the numbers are really significant here, okay? Like, Undici is pretty much unconstrained, okay? You can use, on this testing system, you can actually use Undici, one of the lowest level APIs, which is Undici.Dispatch, but to be honest, this is not what you should be using, okay? It's the underlining bit, or Undici.Stream, which is, you know, even faster, but again, probably too low a level.

You can get those up to a huge amount of requests going out from a single node process, okay? Getting 20,000, 21,000 requests per second out, it's a lot of requests that you're going to send, okay? So, essentially, our goal is making HTTP requests should not be the bottleneck of your app with Undici.Request. And you can see how the other APIs are being set up. So, we have SuperAgent, we have HTTP from Node Core, we have Axios, how Axios performs, we have the old school request module, how that performs down to Undici.Fetch, which is actually slower, okay? As I said, there are a lot of things that can be optimized still, which is just a tad bit faster than Node.Fetch, which is great, and then we have got at the end.

Okay, just to clarify, all of this is actually very important to note, that if you are experiencing any performance problems doing HTTP requests, you probably should switch to Undici. I talk a lot about HTTP 1.1 pipelining, and I need to, I really want to explain it, okay? Because otherwise you probably won't know what I'm talking about. So, in a normal HTTP request, when you open a client, a request from a client to a server, okay? First of all, there is a ping pong, okay, of opening up the HTTP request, okay? On top of that, you need to add TLS if there is TLS. So, yeah, a lot of exchanges. Once you have this connection set up, okay, the client sends one request into that socket, okay? Once the socket came back, it does the processing and gets it back to the beginning, and then it can send a second request on that socket. I told a lot about connection pooling. So, you have, essentially, if you want to talk to another microservices in your network, you have, I don't know, three, four, five, 10, 20, 50, whatever connections. So, you want to, you know, allocate one connection to a request up until you get the next second, the request, the next request after. So, if you have 50 connections, essentially, it means that there can only be 10 connections, sorry, 50 requests in flight overall, because each one connection, one request, okay? Pipelining changes that. So, with pipelining, we can send two requests at the same time, okay? We can send one request, we can send two requests at the same time, one after the other. Now, this is great because we can actually cut the response time significantly because we send two messages and we can save all the round trip time, okay? The servers then will need to parse those one after the other and then send us back the results, okay? Which is fine because if you slide in this example, it takes 172 milliseconds to get both responses while in the other one is 228. So, we are actually responding sooner and having data sent back sooner, but there is one catch, okay? What happens if the first request is delayed? If the first request is delayed, okay, as you can see in this example, we are actually losing a lot of processing, okay? The second one, the shorter one, needs to wait to be sent, okay? Because first, I need to send the first one and then I need to send the second one. This phenomenon is called head-of-line blocking. So, okay? So, great, not great, you need to take this into account, okay? But especially when calling microservices, this is probably fine or my experience is this is fine and it can help us achieve way, way, way more throughput because we can use our sockets way better, okay? So, let's take a look inside and how this works.

And first of all, we have, we want to provide the foundation for HTTP next. And, as I said, we separate the API from the API, the API that users use from the internal. So, there are two separate layers, completely independent from each other.

And all the internals are callback based. They're not based on error backs and they're not based on events, okay? So, they're based on lower level mechanism that is extremely fast. So, we are not having any overhead whatsoever.

Also, we manually manage our connection pool, okay? And you can do different things. And we are very picky about overhead and memory allocation. Last but not least, we minimize the transition between C, C++ and JavaScript.

So, the key part of the connection management and how you interact with internals is by the concept of a dispatcher. And there is an API in Undichi that's called SetGlobalDispatcher. So, if you want to change the global dispatcher or the global mechanism for handling the HTTP request, you can do it anytime via the Undichi library, calling SetGlobalDispatcher and passing either an agent or something else that we'll talk about sooner. You can configure a lot of things. You can put a lot of parameters in here. So, you check it out. Now, what's important to note is that calling SetGlobalDispatcher from Undichi will also change the one from fetching Node.js, as the two things are built to be completely interoperable.

So, what is the hierarchy of those dispatchers? So, we have a Dispatcher abstract class, which includes a bunch of things. And it includes a lot of the fancy methods. And then we have an abstract that implements the dispatcher for connection management. And from there, we have the client, which wraps a socket, a connection. Then we have the pool. And the pool, it's an array of clients, essentially. So, if the pool receives a request, it says, oh, give me a free client. Otherwise, it waits. The balanced pool is an alternative system that uses a recent algorithm that was published on a recent paper that allows us to choose the socket that will offer the best latency for that specific request. How does it work? Yeah, it's a complex algorithm. So, check it out. It uses a lot of weight and math to make it do its job. What does the agent do? Well, the agent allows you to work across a group of pools. One for each origin.

So, if you have a combination of domain plus port, or IP plus port, then the agent is the system that coordinates it and sends a request to the right pool. This system also allows us to implement a mock agent or a mock system for Undici, which is built in. So, you can actually configure a mock agent, and the mock agent can be set up to only intercept a few requests. So, if you configure this, you can intercept a request and you can wrap it via the mock agent. Seems fantastic, right? There's no monkey patching happening, nothing. This just works out of the box.

Why is it called dispatcher, though? Well, because a dispatcher has the most important methods for dispatcher is the dispatch method. And the dispatch method takes an option object, which describes the request, and a handler for that request. A handler is a complex object that implements a lot of callbacks for a lot of different states of the HTTP request. And if you're calling dispatcher.request internally to use a request handler, and stream, stream handler, fetch, fetch handler, pipeline, pipeline handler, yeah, seems about correct, okay? And this is how we separate between the lower level interface, which is dispatch, and the higher level, like request, fetch, stream, and so on and so forth.

This system allows us to create something like a retry agent. So, let's imagine that you want to retry your HTTP request because they are failing, for whatever reason. Well, the retry agent just does that, okay? It's built in, okay? So I don't need to do much. I can either configure it as a global agent, or I can just call it. Works great. Internally, this, it uses the, implements a retry handler. So the retry agent, when you receive the dispatch method, creates a retry handler, okay, that will, that essentially wraps the previous handler and adds some retry logic. Hey! That works, okay? Pretty neat, right? And it's endless all the way down. And one of the options in, inside the, for dispatcher are the dispatcher interceptor, okay, sorry, and the dispatch interceptors essentially are just objects that has a dispatch function. That's it, okay? Which offers the same API as the dispatcher class, okay? But an interceptor only, it's an object that only have dispatch. The other, dispatcher also have a lot of other methods in it, okay? And for example, what I have created, it's a system to automatically issue M2M, machine-to-machine, OIDC tokens for each request coming out to a certain destination. In this example, and you can check it out, it's a separate module called UndiciOIDCInterceptor. And we can, when you create an agent, you can configure your interceptors in the code, okay? It's pretty, it's pretty great. It's pretty neat and pretty great from my point of view. And you can check it out, in this slide on the right, you can see a big example for about what the provider does look like. It's a lot, okay? It's a lot of code, a lot of config, but it's compatible with Auth0, for example. It's compatible for anything that is OIDC compatible and supports machine-to-machine tokens. So pretty great from my point of view.

It's compatible for anything that is OIDC compatible and supports machine-to-machine tokens. So pretty great from my point of view. But be careful, because probably in the next release of Undici v7, we'll be changing these APIs. So we are working on something better.

So from the code, the example we did at the beginning, you wonder why fetch is low, okay? So I took the liberty to run this benchmark. Very simple, very, to be honest, straightforward. We are issuing a hundred thousand fetch requests locally and get the text. And that's it. And you will be much surprised at where we are spending the time. All the time, most of the time, 50 percent of the time is actually spent in initializing the WOTUG streams, okay? Not HTTP processing, not anything else, only initialization.

Something that does all of this means that you can use the dispatcher logic also to make some magic, okay? So, in fact, because of how this works, the new HTTP RFC 9110 just splits the semantics from the transports, from HTTP 1 and HTTP 2.2. So basically, the wiring format is now only an implementation detail. So basically, it means that we can implement the HTTP logic, okay? The HTTP semantics on essentially anything, which is fantastic.

So what you could do is essentially implement it with Fastify, the Fastify Undici dispatcher, which is another dispatcher from Undici that completely bypass the HTTP request logic and create essentially a mesh network inside your process to route your request to your Fastify server. It's pretty fantastic.

All of this, that I told, powers the new platformatic runtime, which is a system to run multiple microservices inside the same box, inside the same process. And I don't know, check it out. It's there. I don't think I have time for a demo, but stay tuned, because I will be publishing something sooner rather than later on this front. And thank you very much for watching, and bye-bye.