Long Frames and INP – Understanding the Post Load Performance

Hello, everyone. Welcome to my talk on long frames and INP, understanding the post-load performance. My name is Vinicius. I work as a tech lead for Gillian and I have worked for Volvo and Spotify and Kleiner before.

I have worked with performance for about six years and you see some similarities on every company you work for when you work for performance for this long. We are all somewhat familiar with at least some form of performance debugging and that be Lighthouse or DevTools. We are also all somewhat familiar with performance metrics and sometimes with some metrics more than others. So we are familiar with Barbados. But most times when we talk about performance, we are biased towards the load time and it has been like this for a long time. We focus on delivering a good loading experience and there is a good reason for that. Apps or websites that load slowly and has a bad loading experience can lead to frustration and high balance rates. But forgetting the post-load part is something that can also happen and that's where most of our users spend most of their times.

And for a long time, even responsiveness metrics were somewhat bound to the load time. So how long does it take for the application or the page to be interactive? How long does it take for the browser to respond to the first user input? Or how long did the JavaScript passing block the main thread during the loading time? All of those are very much focused on the loading experience. That's why we now have metrics such as INP and NAPI such as long animation frame. And they help us decipher and improve interactiveness. But I'm getting a little bit ahead of myself there. In order to truly appreciate those amazing additions to our toolkit, we need to get a bit of a history lesson on how performance metrics got here, why they evolved the way they did and how we got here on this long journey. You can only connect the dots looking backwards. And a bit of context can give a great perspective on the reasons why users, why we use the metrics the way that we do and why techniques got shaped the way they did nowadays. So let's take a look on those early days and see how the landscape looked like. Why tools like such as rail model were shaped the way they did and how they helped pave the way to interactivity metrics to come. All the way to INP and long animation frames.

First, let's take a look at the rail model. It was introduced in 2015 and it stands for response, animation, idle and load. Response being responsiveness, not HTTP response. It is a significant step forward on how we think about users' perception and interactions. Each part of the rail model comes with a set of recommendations. So let's focus on what we have for response. So it again stands for responsiveness and it started making more popular a set of timings for referencing on how we divide work in the main thread and how we should think and organize our JavaScript work around user input and how long should we take to respond to user input and why.

So the rail model, it kind of dictates this 100 millisecond window as the total amount of time to process a user input and ship the next frame. But since processing also involves the event loop queuing the user input and dispatching the listeners, tasks were left with a 50 millisecond budget to respond within a frame window that could serve a 60 frames per second experience. Any task that exceeds that budget may incur visual jank and delay the visual update, resulting on a frustrating user interaction which leads us to the early days of interactivity metrics.

Although those metrics were almost entirely focused and biased towards the loading experience, it was a great starting point. With metrics such as FCP, or first contentful paint. With metrics such as FMP, first meaningful paint, and LCP, largest contentful paint, it helped us measure the visual completeness and metrics that gravitated around CPU idleness after loading times, such as first interactive and first consistently interactive, which later got named to CPU idle, to first CPU idle, sorry, and time to interactive. But we lacked good ways to measure and understand performance problems around user interactions. Our first metrics to assess interactivity is still measured around events that happened during or right after the page loads time. Assessing how much time it took for the browser to download and pause assets and be able to start responding to user inputs.

There was another metric that came also around that time, after those, called the total blocking time. And although it was a lab metric and lab tools primarily assessed it in correlation to time to interactive, it was a step into the right direction on trying to create a metric around the impact of long tasks on the page responsiveness. So let's take a closer look at how those metrics were computed and understand some of their shortcomings. Let's look into TTI or time to interactive again. So it was first conceived as first consistently interactive, it tries to assess when the browser is able to respond to interactions reliably. I think that the original name conveys the concept really well, but it does not have a catchy acronym.

I think that the original name conveys the concept really well, but it does not have a catchy acronym. So the graph taken from the original Google Docs on the metric in the case study for the metric, it showcases a series of events that the browser would take in order to be able to compute time to interactive. And we also can use the official chart taken from the web.dev first input delay article to see how the time to interactive and first input delay are connected.

So the time to interactive, it measures backwards the total time for the browser that the browser took from the start of the response all the way to when the browser has a five second window of five second quiet window. And in order to assess and gauge how the browser is able to respond reliably to user interactions. First input delay measures the very first interaction and assesses any delay on handling input. The first input delay metric was a good start. In attempt to assess the disruption and jank on serving the first interaction, give you a number in milliseconds on how much time that interaction response got delayed with them with the metric at first input delay came with some eye and the metric also came with some recommendations on how to avoid slow interactions. With that, we also got total blocking time as a lab metric to help us measure the total impact on long tasks.

So this metric is not necessarily connected to user interactions, but instead it measures how much work is being done on the main thread which can impact the user's interactions and visual updates that need to happen during that time. This metric though used by lab tools such as Lighthouse to measure the impact on long tasks during the load time is not necessarily a load time metric, but instead measures a measurement of long tasks blocking the main thread over time. Since the rail model's inception, we try to understand the impact of a busy main thread on our users experience with fancy acronyms and mental models to help us divide work and better understand the impact of long tasks when trying to keep a responsive and delightful user experience as a key result. Speaking of long tasks, the long task API is another important milestone. If total blocking time is the total blocking time over a period of time, the long task API provides us granular access to all of the long tasks that happened over such a period of time, giving you some form of enumeration and basic attribution model over those entries. Although it was a step into the right direction, it did not solve our problems. The basic attribution model gives us more granularity and some information on each long task, but it fails to give us proper insight over which script or function it might have been attributed to, giving you not much more than a timestamp and a total blocking duration. This quote actually comes from the article on long animation frames on the shortcomings of the long task API, which is another point to another point is that if you only consider long tasks as a source of interactivity problems, you're eliminating an entire class of performance problems that has to do with styling and layout, that also will occupy the main thread preventing the browser from responding to interactions and slowing down the production of new frames.

And so with that knowledge, the Chrome team started investigating on how could they create a better user-centric responsiveness metric, one that could observe not only the load time, but post load time as well, as part of the user experience. And also encompasses all of the parts that might be causing slow interactions. This image is taken from the article towards a better responsive metrics that precedes the announcement of INP. And you can already see a lot of resemblance on how INP as a metric functions and identify some of its parts. You can see the input delay section, the processing time section, and the next frame being shaped as a full account of the interaction duration. This is also represented on DevTools, nowadays at least. On the interactions track on the performance panel, you see the input delay and the presentation delay as whiskers on each side of the of the input event. And the processing time as the solid bar, representing the totality of the interaction duration. So all of those three parts account for the interaction. And with all the call frames within the stack being displayed underneath on the main thread track, which now gets us back to INP and long animation frames. So let's start with INP. INP, as a Core Web Vitals, tries to assess the page overall responsiveness by giving you a score based on the longest interaction observed. INP, similar to CLS, is a session-wide metric.

And it replaced first input delay as a Core Web Vital on March 2024. One important caveat is that not to penalize highly interactive websites and applications, the highest interaction over every 50th interaction over the session duration will be dropped. So this is to avoid random glitches, random long tasks on the total session duration to be accounted for your interaction to NextPaint. Here, we have a great visualization on INP coming from the article on web.dev that showcases what is considered an interaction and showing all of the different parts of the interaction to NextPaint score as a timing. Every part of the interaction duration can be accounted as the attribution model for the INP score. So you can see the input delay, you can see the processing time, and the presentation delay accounting for all parts of the input or the interaction.

Which brings us to the long animation frame API. It introduces us to the concept of an animation frame, which is similar to the INP metric shifts our perception of how we section portions of work on the main thread. We can use the same model from the INP article to understand what an animation frame is. Same as the INP metric, it goes from solely focusing on the JavaScript part to focusing on all of the work needed to ship a new frame. From any delay in execution caused by a previous task, to the task execution time during the animation frame, to any delay caused by styling and layout. This helps us better understand any problem that might lead to slow interactions or jank. As it focuses on how the user perceives performance. Connecting visual updates and responsiveness to a single attribution model.

It also is important to notice that long frames, they can happen at any point in time. As a script execution comes from many different sources and trigger different side effects including visual updates without direct user input. So animation frames are really helpful to help us identifying all sorts of different problems connected to responsiveness. And as a model, the animation frame help us to better section parts of portions of work on the main thread, accounting for all things that might be delaying the next frame. Meaning that the long animation frame API and animation frames as a unit of work is a much better attribution model to work with. It improves where it provides us with a much, much richer data as an attribution model as well. Similar to the long task API, we have a time stamp and a duration. But also we have a breakdown of all the segments of the animation frame as a unit of work. So you can see we have the start time and duration. We have the render star and the style and layout star and blocking duration as well. Alongside with any input that might have happened during the animation frame. With that data provided by the animation frame entry, we can section all of the units of work that happened during the that interval or animation frame and understand where it got, where it spent the most time on. So you have the start time and end time that can be easily calculated, but also the total work duration, that is JavaScript execution time, the render duration, pre-layout duration and style layout duration. So this will really help you pinpoint problems during that animation frame very consistently. It also provides with a better attribution when it comes to any possible script that could have resulted on a long execution time. Giving you separate timing per source and details such as which script and function called it.

So you can see that we have the any different sections on who called and separate timing data source for a better attribution and also source details.

Here's a quick example of how the animation, sorry, here's a quick example on how the attribution data looks like when you put it all together on an animation frame entry. It's important to note that the interaction to next paint entry might have more than one long animation frame entry. Since the input delay attribution model would also be connected to its own long animation frame entry. It is also important to remember that long animation frame entries might have multiple scripts attributed to its execution time.

Here we have two examples of how you can visualize interaction to next paint in the wild. On the left you have the Vercel toolbar showing a collection of inp entries on dev mode and it's showing all the different parts of that of each interaction as a breakdown with the timings. And on the right you have the trace viewer on a tool that I'm creating called PerfLab. You can see the interaction to next paint entry highlighted on the trace displayed above and alongside it you can see report cards with different timings for the same trace. This is another trace visualized on PerfLab. And I wanted to showcase animation frames that can happen on any given time. So this long animation frame entry is not directly linked to any interactions, thus does not account for any inp. As this trace was done over a load time trace that I wanted to assess the impact of different third parties on the total loading duration for a trace session. And but long animation frames can help you identify long tasks and well long frames that happen at any time. So we can better understand the impact and different things that happens over any time.

But it's important to understand that long animation frames also have their shortcomings. Where a great addition as it is, is not free from some things that is very important to remember. So here we can see the all the scripts that happened during the timeline because this is a trace that I captured and I am highlighting the entry itself that I capture as long as a long animation frame or as an animation frame entry within the trace that I captured. But an important thing to remember, that if you're capturing animation frames or inp metrics in the wild without a trace, it's important to remember a few things. That script attribution can only come from scripts running in the main thread of the page. That includes same origin iFrames. As long animation frames can also be caused by third party scripts executed on your website or even Chrome extensions, it can impact your animation frame attribution and inp attribution by having a longer execution time but a missing script attribution. There is also a problem as of the time of writing of this presentation at least, where script without source information such as event handler callbacks and inline scripts also will be lacking attribution data within the attribution model, at least for the time of this.

We've come so far from the early days of performance tooling and metrics and now have such incredible tools at our disposal to help us better understand and improve experiences on the web. It is a truly incredible journey with INP and long animation frame as the latest entries to the toolkit and we can finally have a better understanding of user interactions and better deliver delightful post-load experiences to our users. Thank you very much. My name is Vinicius and I'm on X or Twitter as WebTwitter. Feel free to reach out and thank you. Have a great day.