Instrumenting Node.js Internals

Hey, everyone. Before we continue, I wanted first to thank all the organizers and volunteers that made this happen, dedicate their time, and also gave me the opportunity to be presenting to you today. My talk is about troubleshooting Node.js applications.

Before that, a little introduction about myself. I am Alejandro. I'm from Buenos Aires, Argentina. I work as a back-end engineer at X-Team and I also contribute with a couple of communities.

So, into the topic of this talk, say you have a Node.js application that is running, but something is not working. There is no exception. You don't have logs. Where would you start? What I would try to do is to tear apart the parts of the application, so it's easier to pinpoint where exactly the problem is. Say it's a server, I would check the bootstrap part of the process. A breakpoint right after it initializes to see if the error happens before that or after that.

Oftentimes I would be in the process of troubleshooting an application, and I would think I have no idea why that is happening. I would be just like this dog. So a lot of times, this would also happen on my development environment, where I have full control, full access, and also should have a good observability about what is happening.

This last part was one of the motivations that I had to work on a tool that would practically no changes to your application could allow improved visibility and potentially turn yourself into Ironman, like in this scene. Viewing a lot of useful information at once. I will go over a few specific examples that I could think of on a short demo.

On the scenario number one, imagine you have a script or an application that is running through Node. We want to be able to tell if it's going over the network and if that's the case, to collect more information about what URLs it's using and possibly more information on that. So we'll go on to my demo and for this let's say we want to try to do that with NPM. And we will try to retrieve the latest version for a package, in this case Mocha. We get the result 802.1. Say we wanted to do that with this tool with NPM we can require instrument pass the same arguments that we were passing before and first of all we will see the same output that before 802.1 and and after that a short summary for the tool.

In this case for these two native modules that are enabled HTTP and HTTPS. Those two are specifically configured here in the modules list and you can add another ones like FS, Require, and others as well. This configuration is added on the instrument.config.js part. So we can see that this is going to registry.npmjs.org. We see that the method is get on the headers user agent npm in CI and whatnot. So all I had to do to require that tool is to add as a development dependency.

For another example, say you're a refactoring HTTP server, and you want to identify which files are importing module A.js. The module is only being loaded dynamically by a specific probe in your server. And in this case, it would be practically impossible to do this task with static analysis. You would have to instrument your application or your server to get that information. So by adding the required module in the list we previously saw, you could get that information using this tool as well.

A little caveat on this and a recommendation if you should be running these on other environments that are not your local environments and the answer is no, definitely no. There is a performance overhead on adding this instrumentation and I'm sure there are a few bugs that you will find if you test this on your own, so it's not recommended to do so. You can find this tool on github and see the code, see an open issue, pull request, and if you have further questions on how this is being done or other technical questions, we will be holding a Q&A session or feel free to send them over via social media, twitter. That's everything that I had for you today.