Creating a Test Runner: What Happens Behind the Tests?

Hey everyone, it's a pleasure to be here. And today I will show you how some of the most popular Cache runners work behind the scenes. And who is Wesley? Well, I am Wesley, nice to meet you. And who is our dinner pick? Well, choose both and you'll choose a little different cache runner of the usual.

So, why I create a test runner from scratch? Let's go back in the timing first. Before we had TypeScript and Node, but now the language involved in its ecosystem. We need to understand how our applications that come from Node 8 works with BAM, with Gnu, etc. And that is a good example.

I just checked if Gnu version is a springy, but using just Gnu and AR models, and it will fail. But, ok, you don't need to read it. And, but look at this error. Just installed an unexpected token. Look, this is a really simple test, and the error is really large. It's because it just needs to use an experimental feature of Node.js to run an AR model file.

While Gnu is an AR model by default. It will also happen with BAM. Now, let's run a test that will run, but not exactly work. A test really simple again. In this case, we are just checking for BAM version as a springy. But, the error is different now. As CommonJS is natural for gestures in this case, we can see a different reason now. BAM is not defining. Ok, but check the comment here.

BAM is calling gestures. How BAM is not defining? There are two main reasons for that. The first one is mishbang. Mishbang works like I need to define a CLI to run with a binary. For example, I want my JS file run with Node.js, and even if I don't call Node.js to execute this file. For example, the stage.node.nodemodels.gest is a line on JS, I can just run gest. This is the magic of Mishbang. But, it comes with a paradox.

BAM, gest. BAM, call, gest. Gest, being a, calls Node.js Mishbang. So, what is really wrong with this? Now, that is a superficial example of how gest works behind the scenes. It creates a virtual machine context to isolate the test. So, this isolation is a problem. Because, gest is in fact run via BAM or Dyna. But, when it passes the test to a VM, it does propagate the BAM.

As gest runs via Node.js Mishbang, it loses the context in the process of isolating the test. It brings a complex conflict because gest is running really with the runtime or colored it. But, test itself probably is run via Node.js. And, it is like, well, it's like defining the test. So, there are some conflicts in the test processing mode. So, I show gest because it is really popular. But, it happens mostly with all the CLI projects, not only for test runners. And, it also happens with workers. So, for me, it is a real problem because I maintain a lot of projects. And, I really want my project to work with all runtimes. And, I want to ensure it uses a test switch that runs across all of them using the same test switch. And, one more point. The dependency graphic is really hard for me.

The dependency graphic is really hard for me. Just to use a lot of frameworks. Dependence, it works like the dependence of the dependence of the dependence and again and again and again. So, it's enough. One of test dependence should break the behavior across different runtimes. And, how can we control so many dependence? And, it is the reason why our test runner here needs to be very dependent. And, to size it in specific is the size of both now.

So, one more motivation. Honestly, I really like this part of the philosophy of the test runner. Because, it's not about bad implementations or nothing like that. It's more like I would like to test it exactly as I write my code in JavaScript, you know. But, look. How beginners see a test for example, I want to see if one equals one. But, first you need to configure your workspace. You have to adjust. This example is complex because we are using a NGIS extension.

So, you just need to understand this extension. You need to disable a lot of things for AS models. And, let's remember. In this example, I am a junior developer. And, what is performing? What is this reject gadget? I probably don't know what is a reject. So, once I configure my workspace, I can finally create my test. And, that is okay. But, since I am using AS model, I need to enable the experimental feature of Node to Run. You see that it's not too simple.

So, a developer comes from tests. And, I know it is a specific behavior of Jest. A lot of runtime is just install and run. But, Jest in this case is present in practically all companies I work in. And, let's pause a moment. How JavaScript flow works? We just write. If it is a promise, we can wait. And, finally, it's done. So, for me, it is really simple. Even if I am a junior developer. But, that is a thing in my mind. If I know JavaScript, I know how to test JavaScript, right?

And, no, I don't know. Because tests don't follow the same flow of JavaScript. Description. If launchability is not shown, even a problem, it's expected behavior. But, for a beginner developer, it's confusing. Because a lot of devs take years to understand how promises work. And, even tests don't exactly work the same way. In the description, we have the expected behavior for a junior developer. And, 1, 2, 3, 4, 5. But, in reality, it will be 3, 1, 4, 5, 2.

And, okay, there is something interesting here. Console.log 5 should be the last one. But, it will be run before the test itself. And, it is a problem for users that don't know how to test yet. And, there is a proof of concept. So, you can see it's real and it runs in node 25. And, as I said, the last step of the test in the previous slide runs in the second point. Before the test. So, for tests, we need to create a boilerplate or a hook to explicitly indicate who will run before and after the test.

For example, we need to use a BeforeAllConsoleLogStarted and a AfterAll to a ConsoleLogDone. And, now we have two async tests and it works like a synchronous test. Now, you can see that for a senior developer it is like, okay, what's the problem? It should be like this, right? Okay, yes. But, for a junior developer, it's like a barrier.

And, alright, let's stop the preamble for now. I hope it shows you why, in the age of AI, I created a test runner instead of an amazing project using AI. And, please, it's not to criticize no one of the test runners here. It's to learn and understand, especially my personal motivations, to create my own test runner.

So, how I would create my test runner? The first step is to need to run across all runtimes and it needs to isolate the process and bring back the natural JavaScript code to the test. And, as a bonus, we can make it zero configs and use only matribs of the runtimes. So, how we can run the same test across all runtimes? Let's see how each runtime works one by one. And, it's interesting because the engine works with TypeScript by default. Node works with TypeScript after Node 22 and, in this case, I will use VSS because it works with previous versions of Node.

Once we know how to invoke a file using each runtime, we can create a subprocess, for example, using Spawn. In this way, we can use Node.file.file.genu.run. And, in options, we can use a default for everyone. And, the same idea for TypeScript, for example. And, it will run not exactly a test. It will run a JavaScript file or TypeScript. And, what about CacheBank paradox? Well, we can just use a really basic logic to get a runtime, properly propagate it across all subprocesses. And, it will ensure each test runs across the same runtime that trawls our test run.

Here is an example of the test executor. See, we have a file.js. We identify our runtime, create a subprocess based on who is the runtime that will trawl our test run. And, ok, you just got it. And, simply, we solved our problem with TypeAnd, our VMs, our workers. Node, for example, uses the fork approach. It's really similar, but, in our case, it fits better because we need to run across different runtimes. And, Node will run always using Node, for example.

So, let's talk about performance now. So, we can find and list all our test files, create a concurrency, understand what is the better resource. For example, the CPUs available. Choose a way we will prevent overload. And, plus, for each subprocess that ends, it will disappear of the memory. As you can see, this is the before using the default way, and this is the new way with our test runner. So, in this example, you can see the scribe 1, 2, 3, 4, 5. And, look, on the right, it is exactly as in JavaScript. And, in a real example, you can see, on the left, a Node example. We need to prepare the connection we used before to create the connection. So, after, to end the connection. So, we can test. And, the assert for Node in this case, only is displayed if the test fails. And, for our test runner, it's like JavaScript.

See, the scribe, connection, await to reach connection, our test, and await for the end of the connection. And, it's done. As you can see, our assert shows the message. So, for a real beginning user, it's really democratic to introduce in test, because they don't need to learn a lot of curves, for example. They just need to understand the assert. And, Poco uses the exact same way to use the native assert.

As you can see, even with two lines, the report is complete. And, to start is not really about testing. It's about developer experience, because when we see something that could be different, even if it's just for us, someone probably needs our solution, too. That's why I create a lot of projects. Choose one is my favorite.

But, in my mind, the people would hate my project, and they love it. So, here I show you how I turn problems into what you need to create something new, that new people can use, that has a purpose, and has a philosophy behind it. And, it's important because when we create complex projects, we are creating about here. So, really thank you. I hope you liked it. Please feel free to connect with me. Bye-bye.