What Happens When an AI Has Access to a Node.js Environment? Spoiler: Wild Things

Hi folks. Thanks a lot for being here today. I'm Alfonso and today we're going to talk about something which is very exciting. So we're going to learn and we're going to see what happens when an AI, so an LLM, has access to a Node.js environment and therefore can write its own code and run its own code. Before we start, a quick introduction from my side. Hey, again, I'm Alfonso, probably from Italy, and I'm a tech lead in a company called NearForm and we're basically a consultancy company for especially Node.js and a few other technologies. But if you are interested about the topics that we're going to discuss today, if you want to learn more about the usage of AI agents in Node.js or AI in the software development lifecycle, please reach out on LinkedIn. I'm always more than happy to have chats and get feedback on these projects because they're very, very valuable. So again, if you want, let's have a chat on LinkedIn.

Let's get started. So everything that's going to be discussed in this talk is in this website, which is js.ai.com. And this project is the Node.js Sandbox MCP server. The idea behind this project is that we'll allow any LLM to run its own code. So it's going to write some code, run it, install dependencies, have access to the file system, and so much more, as you may guess. In case you want to give it a shot, in case you want to try it, you can take a look at this website. Or if you want, this project is one of the verified projects on the Docker Hub. So it's mcp slash Node.js Sandbox. You can easily download it if you are using Docker Desktop. It's super easy, it takes around 30 seconds. So let's get started.

And before we go into the technical bits and the technical details of the implementation, let's see what are the reasons of this project. And first of all, what is the MCP? So what is the Model Context Protocol? I'm not going to invest too much time in this section, because I'm sure that you already heard quite a bit of MCP, and you already know what it is and how it works. But I just want to bring you, I just want to bring what are the problems that MCP is supposed to solve. So before MCP, while creating connections between AI agents or AI applications and data sources and tools, we had one common problem, which was the amp-for-end problem. So the idea is that you have M applications, so like your calendar application, maybe your Gmail, maybe, I don't know, anything that you can think of, right? And then you have to connect those applications to the data sources, which might be like a database or a scheduler, whatever. The problem here is that you don't have a unified way of connecting those two things, right? So in the case of an AI agent, you don't have any standard way to connect your AI to a data source or to something which allows you to use tools. And that's where MCP comes into play. So the main idea is that you can just communicate, you can just use the protocol. So the LLM, it's going to implement the MCP client. So for example, Cloud Desktop is going to implement it, ShareGP recently implemented it.

So the idea is that you can just integrate with the, and then you can, and then you can implement your own tools. You can just connect to an MCP server. So it's that easy. We will see it in a bit. The way it works from a technical standpoint, it's relatively easy, but still interesting to see. So the idea is that we do have an MCP client, as I mentioned. In reality, it is called, for example, Cloud, it's the MCP host, because then each server is going to have its own client. But the way it works is that we simply do an MCP request to the server. Then the MCP server is going to act as a proxy and it's going to do whatever we need to. So it's going to do API calls, running code, I don't know, going through a database, whatever we have to. And then it's going to give the results back to the MCP server. And then the MCP server, of course, it's going to give back the results to the client.

Now, the question is, okay, we kind of understood what is MCP, but why should I build an MCP server? Well, there are a few reasons for that. The first reason is that I want my AI to be able to write and run its own code in a safe way locally. So the thing is that we already know that things like ChargePT, Gemini and Cloud, they already have their own tools, but they run on the cloud, right? They run on some things maybe like an edge runtime or something else. But we want something which runs locally because this allows us to do some cool things. So, for example, one thing that we can do is local data analysis and visualization. So let's say that we do have some personal finance data, some medical data, I don't know, anything that we want to keep private. Instead of uploading it to ChargePT, we want to keep it private on our machines. And we can also do, again, visualization and analysis using tools like ChargeJS, Danfo JS, and so on. So this is one cool thing that having it running locally unlocks. Of course, this has some improvements both in terms of security and privacy and so on. Next one is that DLLM thinks in text or tokens, if we want to be precise, but not bits. So as we can see on the left, we do have a relatively simple expression and the model, especially not state-of-the-art models, can easily hallucinate and give wrong results because they have not been trained to run a mathematical expression, but they have been trained to predict the next token. So how do we fix this? Well, as we can see on the right, we can just say verify with code. So the DLLM is going to write some code which is then going to run and the result is going to be, like in this case, three errors in strawberry. So there is going to be something like count the number of errors, and this code is going to be executed. Last but not least, the question is, okay, but why JavaScript? Why no JS? Well, the reason is that the JavaScript ecosystem is actually huge. We do have more than 3.1 million packages, which means that for every problem or for every idea that we do have, most probably there is at least one dependency that is going to solve that problem. All right.

Now that we saw what are the reasons, as you may guess, there are quite a bit of use cases that are unlocked thanks to this MCP server. So first of all, we can generate QR codes. We can search the web. We can generate PDFs. We can do a lot of things, from file manipulation to fetch. We can do, you know, even markdown slides generation. I experimented a bit and I found out a lot of interesting things.

But let's move now on, and let's see how to actually implement this stuff, right, which is the interesting part. So this is the flow. This is what we can manage in a radio, which is the LLM writes some code. The code is sent to the MCP server. Then the MCP server, it's going to run this code somehow safely with Node.js. And then the output of this code, which run is sent back to the LLM. We have to understand how to run this code safely and locally.

There are a few possible options. As we can see on the x-axis, we do have the implementation complexity, and on the y-axis, we do have the safety. The first thing that we can do is the eval. So we just take this code, we pass it through an eval, and it kind of works. Then we do have the via module, which is kind of similar, but it has more security in it. The main problem with it is that the code that the LLM generates is untrusted code, and we have to treat it as untrusted code. So we definitely don't want to use something like eval, but then we don't want to use something like child process, for example, because child process, of course, it's going to spawn a new process, but it's still on the context of execution of the user, right.

So we are still executing it in the user space, and we don't want to do that. Another option on the other side is to use things like microVMs, or even edge run times. As you may guess, the edge run time is very secure, because it's not on your machine, it's on someone else's machine. So it cannot cause any harm, hopefully. And kind of the same applies for the microVMs, given that they are very well isolated. But then there has to be a sweet spot here. And the sweet spot between complexity and safety, as you may guess, is Docker.

So we can spin up a new Docker container, we can run our code, and we DLLM-generated code, and we can get back the results. That's exactly what we are gonna do right now. The intuition is that the code that DLLM writes is then sent to the MCP server, and the MCP server takes care of spawning the Docker container with the Node.js image. Then the code is copied into the container and executed. The image that we're gonna use, it's gonna have Node.js pre-installed. And then the output of the code is sent back to DLLM.

Writing these kind of things is relatively easy, it's super easy, thanks to the official SDKs. We do have official SDKs for a lot of different technologies. But if, for example, you are using something like, I don't know, C++ or whatever, and there is no official MCP implementation for your own technology, you can just take a look at the community ones, there are quite a few of them. Or if you want to go wild, you can still reimplement the spec, which is relatively easy to be reimplemented. So you can give it a shot.

Let's see how it works with the TypeScript SDK in this case. In 20 lines of code, we can create our first MCP server and expose our first MCP tool. Through MCP, we can expose tools, resources, prompts, and more. Keeping it simple, I'll expose just one tool, naming the server, version, and description, utilizing the MCP TypeScript SDK. Describing the tool requires the name, description, and parameters in a JSON schema, using XOD for validation.

When invoking the tool, the MCP client interacts with the server, which then calls the function. The function, runJS ephemeral, processes the LLM-generated code and returns the output as an object containing content. Connecting the server to the SDDO transport for communication. Moving on to running code securely in Docker, setting up a container with customization options, executing the code, and finalizing with cleanup tasks to remove temporary files.

By generating a random container ID, running the container with a specified image, copying and executing the code within Docker, and handling cleanup tasks efficiently. This approach ensures secure code execution, leveraging Docker for isolated runtime environments. The process involves container setup, code transfer, execution, output retrieval, and cleanup for a complete and safe code execution environment.

And as we can see here in the function, I just do have one simple function which is the runJS ephemeral, and then passing the LLM generated code to this function, which we will see in a moment. I'm taking the output, and I'm returning the output as an object, which contains content, which is basically an array. Now, this array is how MCP is expecting the return. So I am passing, for example, the type, which is a text in this case, and then I'm passing the text itself. So in this case, I'm just passing the node.js process output to the LLM back. And then last but not least, I'm connecting the server to the SDDO transport.

Alright, let's see now how we can actually run the code safely in Docker. So as we can see, the function is relatively simple. The first thing that we can do is, first, we generate a container ID, like a random one. Then we create, now we run the container, and we run it with the node.lts.lim image, but of course we can customize the image. And then we use a simple trick, which is the tail, minus f, slash dev, slash null, in order to tell the system, okay, you cannot, you don't have to stop. So please stay idle and wait for me. Then what I'm going to do is, I'm going to copy the code into the index.js file, which I'm going to create in a template here. And then I'm going to copy this file into Docker. And then I'm going to execute this, I'm going to execute with node index.js this code, and I'm going to get the output from the Docker container. Last but not least, as you may guess, I'm just doing some cleanup. So I will be able to kill the Docker container and remove the temp directory. And that's it, that's the gist of the solution.

The cool thing about this solution is that, as you may guess, expanding it is relatively simple. In fact, we will see that expanding this solution, this runtime, to Python, it requires changing three lines of code at least. The first thing that I'm going to change is, of course, the name of the function, which is not anymore like run.js, but run Python ephemeral. Then I'm going to change how we structure the container ID string, but that's not really needed. Now the three changes that are needed are, well, first of all, instead of using Node.js, the node image, I'm going to use an image with Python pre-sold, so it's the Python 3.11 sling. Then when I write the file, it's going to be not index.js, but script.pi. And then instead of launching the script with node index.js, I'm just going to launch Python script.pi, and that's it. So as we can see, this kind of solution is very expandable to other technologies while keeping the same core idea. Of course, let's keep in mind something super important, which is the tool usage accuracy is not perfect. So function calling is one of the capabilities of LLMs, like summarization, rewriting, and so on. It's one of their capabilities, but it's not something which is perfect.

It's one of their capabilities, but it's not something which is perfect. Of course, there are some benchmarks, and especially with not state-of-the-art models, or especially with smaller models, we can expect that the model is going to hallucinate quite a bit. It's going to not call the right tool, or especially if we do have a lot of tools, it's going to have a hard time to define what tool has to be called and with what params. So there is something that we can do, which is we have to keep in mind that the kind of prompt engineering principles are still valid when we are writing some MCP server.

So the idea is that the name, the description of the tool, the parameters, and the description of the parameters are going to be passed to the LLM. So while we write the description, for example, we have to kind of keep in mind that the LLM is going to read that, and we have to guide the LLM into writing the code in the correct way, right? MCP servers are softer, like usual software, so we need to test them, and we need to test them as correctly as possible. LLMs can be leveraged to write tests for this kind of software, which works relatively well.

The next feature which is super useful is to have a working directory mounted in Docker so that the Node.js process can both read files and write files. Then we are going to notify the LLM via MCP what file has been dated. This setup allows for smooth execution in Docker while handling dependencies efficiently. Having the LLM define both the code and dependencies ensures a comprehensive approach to running and testing software, enhancing the overall development process.

Let's add a few more features right now. The first feature involves installing dependencies. The LLM defines the code and dependencies, writes the package.json, and performs npm install. While this may introduce latency, running code in Docker ensures smooth execution. Another useful feature is having a working directory in Docker for file operations. Notifications to the LLM via MCP help track file changes, enhancing code development and testing.

A practical example involves creating a QR code not natively supported in cloud desktop. The process includes snapshots of the working directory before and after script execution. By comparing the snapshots, the LLM is informed of file changes. The Node.js process output, including file updates and resources, is managed at the MCP layer. Additional features like generating charts may require customized Docker images.

Considering security, a crucial aspect, the setup focuses on ensuring secure code execution. Customized Docker images and managing dependencies play a vital role in maintaining security. The integration of various components, such as AI-generated charts and secure execution measures, enhances the overall functionality and reliability of the system.

So first of all, we have this working directory. We take a snapshot of the working directory, then the script runs. The script might update some files, create some files, and so on. Then I get another snapshot of the directory. I compare the two snapshots and then I just say to the LLM, okay, these are the files that when I saw the diff between the two snapshots are changed. This is the LLM output that we can see.

So for example, here I run the JS ephemeral tool and it just said, okay, the Node.js script successfully executed. These are the files that have been created and so on. And this is what happens at the MCP layer. If you recall, we have this contents array, which contains the type text. So with the Node.js process output, but it can contain more things, right? Like the list of files changed, the list of resources, so I can have the URLs to the files.

And then I can also pass the images formatted as base64. Of course, we can also allow the AI to generate some charts. This is going to require some more dependencies. Therefore, I also created some customized Docker images for it. All right, next one regarding security, which is super important. Let's keep in mind that MCP is not secure by default, and everything that goes into the MCP server has to be treated as entrusted code. So the idea here is that we'd want to check everything, sanitize everything. In fact, I also had a relatively high CVA on my repo because there was some sandbox escape via command injection.

The idea here is that we want to ensure that everything is fully secure. In fact, if you want to learn a bit more about security in MCP, you can just Google MCP horror stories. There are actually a lot of breaches and issues that have been reported so far with MCP in terms of security, isolation of the data, and so on. So I highly recommend you that if you're looking to write your own MCP server, you take some time and you study a bit what are the most common attacks for MCP so that you can study and implement your MCP server safely. That being said, thanks a lot for your time.