MemLab: Automating Memory Leak Detection and Heap Analysis

Hi, everyone, my name is Liang. I work in the Meta, focusing on web application infrastructure and performance. Today, I'll give an introduction to MemoryLab, a tool I built on open-source automated JavaScript memory leak detection and heap analysis. At Meta, a larger scale web application has shown that increases in client-side memory usage can significantly impact web apps' speed, responsiveness, crash rate, user engagement, and in some cases, even revenue.

MemoryLab is an open-source framework from Meta that automates memory detection for JavaScript applications. It also exposes a powerful API for traversing heap snapshots, so you can build custom analysis tailored to your app. Out of the box, it shifts with CLI tools that test common memory and anti-patterns and help you debug issues quickly. More recently, we have been pushing into AI-assisted workflows with the MCP server that lets tools like Cloud Code reason directly over heap graphs. So the goal here is simple, take memory debugging from a manual expert-heavy process to something automated, scalable, and much more accessible.

To go deeper into each part, I'll start with automating memory leak detection. We first need to figure out what's a memory leak. Memory leak is essentially a subset chunk of memory that has been allocated, and it was supposed to be released at some point, but somehow it wasn't. So the broader class of these leaks can be detected using a three snapshot differentiation approach. MemoryLab automates the process. To better illustrate the idea here, I'll use some animations. Suppose we want to find memory leaks in route B of a real application. MemoryLab starts by visiting a different route, for example, route A, and keeps track of all objects allocated in route A, then navigates the browser to the target route B where we want to find memory leaks. During the navigation, some objects from route A are deallocated, and some new objects are allocated in route B. Finally, MemoryLab navigates away from the target route B. It can either navigate to a different route, for example, route C, or it can go back to route A again. Now, some of those objects creating route B will be deallocated, but some will remain alive. And of course, route C will create some new objects, but in this example, we don't really care, since we want to find memory leaks introduced in route B. And as highlighted in the orange section of the slides, the interesting part is the object allocated in target route B, but still remains alive even after we navigate away from it. So we have reduced the search space, but still, this is a superset of memory leaks. To get real memory leaks, we use some domain-specific rules to filter the superset. For example, most of the red fiber nodes allocated from a page should be deallocated after a few cleanup navigations. So MemoryLab identifies all fiber nodes and detached DOM elements from the superset. We further filter the list by looking for fiber nodes that keep detached DOM elements alive. So after refining the list by incorporating the domain-specific knowledge, we get a list of leaked objects. Now the memory detection is done, the final step is to ease the memory debugging process by generating actionable reports.

MemoryLab automatically traverses the heap and generates reports that explain why the object is still kept alive in memory. It decodes and scans the job to heap snapshots, searching for a reasonable path to the garbage collector route to make retainer trees readable and actionable. The retainer trees display reference chains from the garbage collector route to the leaked object. Despite the simplistic view in the graph, heap snapshots usually contain millions of heap objects and tens of millions of references, including VM internal optimization metadata and implementation-specific references.

MemoryLab uses heuristics to balance the trace searching algorithm, providing a readable and reasonable trace for developers to follow. It helps users understand why leaked objects are retained in memory by presenting reference chains step by step. To prevent overwhelming users with thousands of leaked traces, MemoryLab clusters similar memory leak traces using a hybrid model combining heuristics and machine learning algorithms. By showing representative traces from each cluster, MemoryLab reduces the debugging work for users.

The output in Terminal or similar tools like Chrome DevTools presents representative cluster retainer traces of each memory leak cluster. With AI capabilities, users can easily copy and paste traces for further analysis in tools like chat GVT or cloud code. MemoryLab's GitHub repository includes a skills file to explain memory leaks retainer trace. To utilize the MemLab API for defining interactions in applications to find memory leaks, users only need to create a MemLab test scenario file with three callbacks using PubTier's API, specifying the initial page loads URL in the first callback.

It will be overwhelming to show thousands of leaked traces to users. Ideally, we only want to show one representative leak trace from each root cause. For example, here are three leak traces caused by the same bug. As you can see here, the reference chain looks more or less the same, each with a little bit variation. So, how do we put similar leak traces together? This is where leak trace clustering could be a massive help.

We developed a hybrid model combining heuristics and machine learning algorithms to cluster memory leak traces. Memory leaks with similar retainer traces are clustered together. For each cluster, we only present one of the representative traces to reduce the amount of debugging work. MemoryLab shows the representative cluster retainer trace of each cluster of memory leaks. This is what the output looks like in Terminal and similar tools like Chrome DevTools.

With AI capabilities, users can easily copy and paste the trace into chat GVT or cloud code for a clear explanation of the reference chain. MemoryLab's GitHub repository includes a skills file for explaining memory leaks retainer traces. To utilize the MemLab API for defining interactions in applications to find memory leaks, users need to create a MemLab test scenario file with three callbacks using PubTier's API. The test scenario file guides MemLab in the memory heap analysis, reporting memory leaks and retainer traces for debugging purposes.

To achieve that, we only need to create a JavaScript file with three callbacks using PubTier's API, telling MemLab how to navigate your app and which interaction may trigger the leak. We call this file a MemLab test scenario file, and then we pass the test scenario file to MemLab command line tools. MemoryLab will do the memory heap analysis I showed earlier and report memory leaks and retainer traces for debugging.

The first callback in the test scenario file specifies the initial page loads URL. MemoryLab opens a web page in a headless browser and starts tracking objects allocated in the JavaScript heap. The second callback defines the interaction where we want to find memory leaks. Inside the callback, we specify that we want to click the hotel button by using PubTier's API and CSS selector. Those are all standard and well-documented APIs. Under the hood, MemLab will track objects allocated and released during the interaction.

Similarly, in the last callback, we specify that we want to click a button to navigate away from the target state. The second segment of memory contains memory leaks are highlighted in the orange section of the heap in the slides. Without building leak detector, MemoryLab will check and filter those unreleased objects that are allocated by the target interaction. After running the test scenario, MemoryLab will generate and cluster leak traces, which show why and how leaked objects are still kept alive in memory. MemoryLab's documentation sites have more information about the usage and interpretation of the results.

Now let's talk about the MemoryLab's API. In addition to the predefined leak detector process and callbacks, MemoryLab also provides different callbacks to customize the browser interaction and memory leak detection process. For example, you can use MemoryLab's API to redefine how to identify memory leaks by providing a leak filter callback implementation in the test scenario file definition. The callback will be called for each object allocated but not released in the target interaction. Your implementation of the callback can return true or false to decide if this object should be considered as a memory leak. For each one of the identified memory leak identified by your implementation, MemoryLab will generate the leak trace and put it in a candidate cluster representative trace for the final output.

MemoryLab will also pass in the heap graph reference as an argument to the callback. The reference is an intuitive object-oriented heap graph representation of the entire javascript heap including native objects and references added by the engine. The first argument of a callback is a reference to the potentially leaked object in the heap graph. With those arguments, the self-defined leak detector can traverse the heap using the API MemoryLab provides. As an example, our building leak detector follows the return chain of a ReaTaFiber tree to traverse the entire fiber tree and checks if the fiber node is detached from the ReaTaFiber tree. The program under test will likely have a memory issue and therefore its heap size could be non-trivial.

We want to be memory efficient while providing an easy-to-use object-oriented heap traversal API. So the graph nodes are designed to be virtual and these nodes are disconnected. When we traverse the heap, MemoryLab constructs a graph just in time. For example, when entering the callback, only the arguments corresponding graph nodes are constructed and when you try to query other nodes, for example, the referrers of the leaked objects, MemoryLab will create those inbound nodes on demand. MemoryLab also provides an interactive CLI tool for exploring heap snapshots. This could be useful when the snapshot is on a remote server, you are connected by SSH, and want to quickly inspect it remotely.

You can also take heap snapshots manually from Chrome DevTools, save it onto a disk, and use MemoryLab's npm package to programmatically load, parse, and analyze the saved heap snapshot. And again, you can use MemoryLab's heap graph API to traverse and define your own heap analyzer. MemoryLab also provides some predefined heap analysis via command-line tools. The MemoryLab CLI provides different heap analysis commands. For example, the unbounded object analysis shows which objects keep growing over time by analyzing a sequence of heap snapshots. And the example app under analysis shown on the slides has one object with a size increase from 12 kilobytes to 27 megabytes. This is something worth looking into when optimizing memory usage. And you can use the memory lab trace command to get the object retainer trace. There are other predefined MemoryLab CLI commands to find common antipaths. Here, I won't show all of them due to the time constraint.

MemoryLab also provides an interactive CLI tool for exploring heap snapshots. This could be useful when the snapshot is on a remote server, you are connected by SSH, and want to quickly inspect it remotely. You can integrate MemoryLab with your internal CICD pipeline and run it regularly to detect the memory leaks. As shared in Meta's engineering blog post, this is our internal UI showing a leak trace detected by our internal regular run. This is essentially a reference chain from the GC root at the top of the UI to the leaked object at the bottom of the UI. It tells developers why and how the leaked object is still kept alive in memory in an interactive UI. Each object and function scope on the reference chain can be interacted with and show its internal structure and content. The debugging process is straightforward. Just follow the reference chain one step at a time, and eventually we'll find the reference that should be set now to release the retained object. And you can also aggregate all of the representative leak trace to give a summary view. With MemoryLab, we are able to find memory optimization that leads to significant wins. Here is one example optimization we did with my Meta colleague Benoit Girard in the earlier version of React a few years ago. React builds a Fibre tree for rendering components. For this presentation, you can ignore the structure of the Fibre tree and just think of it as the inner data structure React uses for rendering. The components Fibre tree is referenced by the host root at the top. The React component instance and their corresponding Fibre nodes are referencing each other. Similarly, DOM elements and their corresponding Fibre nodes are also pointing to each other. And all Fibre nodes are connected in the React Fibre tree. Meaning if you get reference to any one of the nodes in the graph, you are able to reach or retain any other nodes in the graph.

Meaning if you get reference to any one of the nodes in the graph, you are able to reach or retain any other nodes in the graph. Ideally, other parts of the GSTP is only referencing the host root. So when the least components are mounted in earlier version of React, it just breaks the connection between the host root and the rest of the Fibre tree. So the entire Fibre tree can be garbage collected once the components are mounted. In this case, there is no memory leak. But if application code caches additional references to the component instance or to a detached DOM element, the amount of the Fibre tree cannot be garbage collected. Since this is a strongly connected graph, every node in the graph will also leak.

As an example, if a GS file exports the React component instance that was rendered in a previous route, since the export of the instance is at module level, it won't be released and therefore the entire amount of Fibre trees leaked. And this is just a simple example for illustration. There are many unexpected ways application logic can keep such references. At Meta, we found that singular React leak often causes the leak of the entire route. This means that as you navigate to other pages, we'll be keeping majority of previous pages live in memory, leading to huge memory leaks. And it's not just the React data structures that were kept alive.

By hooks and their closures, this would often keep alive all kinds of product objects. And in some situations, leaking the product code wasn't even required. React cleanup wasn't always immediate and therefore still leaks the entire Fibre tree of the previous routes. So we added the optimization in React by introducing a full traversal of Fibre tree on unmount to do more aggressive cleanup. This allows the garbage collector to do a better job at cleaning up unmounted Fibre tree. And also any accidental memory leaks, it's bonded to a smaller size. To measure all these improvements, we actually run A-B testing on Facebook.com. The number of auto memory crashes in browser dropped by 50% due to that optimization.

And surprisingly, the full traversal didn't slow down the page. The memory actually slightly improved the performance of the page. And we observed significant improvements in many other metrics.

MemoryLab also includes an in-browser tool called Memory Lens to visualize memory leaks directly on the web page's UI debugging layer. The tool is a GS module running in the application. It monitors the changes in DOM tree and React Fibre.

Memory Lens highlights the outline of leaked React components when their DOM elements get detached from the DOM tree but are still kept alive in memory. It also attaches a special memory ID to each DOM element, making it easy to link them back to the view in heap snapshots. You can also quickly search the memory ID in the Chrome DevTools when you see a leaked component outline and its ID in the UI debugging layer. We have more details in the MemoryLab documentation page for complete guidance for Memory Lens.

Finally, a large language model and AI coding are pretty powerful these days. MemoryLab with AI can make memory detection, memory analysis, and memory debugging even easier. In MemoryLab GitHub repo, we've added skills for teaching AI how to properly use MemoryLab tooling and MemoryLab APIs. MemoryLab MCP server package allows you to ask AI coding tools like Cloud Code to investigate memory leaks in JavaScript heap snapshots more accurately.