You’re Probably Using Lighthouse Wrong: How We Got Tricked by a Single Magic Number

Rate this content
Bookmark

These days web performance is one of the most important things everyone wants to optimize on their apps, and it's clear to everyone how dramatic the impact of a poorly optimized website is on business. Yet we as an industry completely fail in recognizing its complexity and widely misuse the most common tool to measure it — Google Lighthouse. If you’re one of those people thinking that good performance equals a good Lighthouse score, you’ve also fallen into this trap and this talk is for you.

This talk has been presented at Vue.js London 2023, check out the latest edition of this JavaScript Conference.

FAQ

Vue Storefront is an open-source tool designed to help alleviate the challenges of building e-commerce storefronts by providing tools that save developers from the pain of long and often painful development processes. It aims to enhance e-commerce performance and is available on GitHub for further exploration and contributions.

Amazon's study found that every 100 milliseconds of added page load time costs 1% of revenue, highlighting the critical impact of website performance on financial outcomes.

Google Lighthouse is designed to measure the quality of web pages across four metrics: performance, accessibility, best practices, and SEO. It aims to provide a comprehensive view of page quality to predict the user experience.

The Google Lighthouse score is calculated using a variety of metrics, each with their own weight. The algorithm considers these weights to prioritize certain aspects of performance and quality, though the algorithm may change with each version of Lighthouse.

The most reliable way to measure Lighthouse audits is to use Lighthouse CI or PageSpeed Insights, as these tools provide more consistent results and minimize the variability seen in home dev tool audits due to external factors like network conditions and CPU variations.

The Chrome User Experience Report (CRUX) collects real-world performance metrics from users browsing with Chrome. It provides insights into how websites perform across different devices and conditions, offering valuable data for improving user experiences.

Real-world user experience data can be accessed through tools like PageSpeed Insights, which shows performance data at the top of each audit report, and the Chrome User Experience Report (CRUX) available in Google Data Studio. This data helps website owners understand how their sites perform in actual user environments.

Core Web Vitals are a set of metrics that Google considers important for a website's overall user experience. These include metrics like loading performance, interactivity, and visual stability. Having good scores in these metrics can positively impact a site's SEO performance.

Similar to how the Body Mass Index (BMI) might not provide accurate health assessments for all individual body types, Lighthouse scores might not accurately reflect the true user experience for every website. Both tools provide general guidance but require deeper analysis for specific cases.

Filip Rakowski
Filip Rakowski
29 min
12 May, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

The Talk discusses the importance of performance and mobile consumption in e-commerce, as well as the use and limitations of Google Lighthouse for measuring page quality. It highlights the challenges and considerations in using Lighthouse, including the difference between lab data and real-world data, and the need to understand user experience beyond a single score. The Talk also touches on the potential use of AI in evaluating website performance, optimizing third-party libraries, and setting a JavaScript budget for better performance.

1. Introduction to the Talk

Short description:

Hello. Hey, hey, how are you doing? Are you having fun? I'm happy, because I'm having a lot of fun. It's great to be here again. The title of my talk is, You're Probably Using Cloud House Wrong. My name is Filip Rakovski. I'm a chief developer experience officer and co-founder of Vue. I'm also a technology council member of Maha Alliance. Maha Alliance is an alliance of the biggest enterprise vendors that are modernizing the e-commerce landscape and I'm extremely proud to represent it. I work in the e-commerce industry. Building e-commerce storefront is harder than it seems. And I can guarantee that you will feel physical pain once you learned what faceting is.

Hello. Hey, hey, how are you doing? Are you having fun? Are you having fun? Okay. Good, good. I'm happy, because I'm having a lot of fun.

It's great to be here again. I think it's the first time I'm here in Vue London, in Vue.js Live, this time. And the title of my talk is, You're Probably Using Cloud House Wrong. And I know it sounds a little bit provoking. That was my intention. But I don't assume you're using it wrong. I really hope you're using it in the right way. But just in case you're using it wrong, here's my talk.

My name is Filip Rakovski. I'm a chief developer experience officer and co-founder of Vue. I was introduced as a CTO because I used to be a CTO, but we hired a better CTO. So, right now I can move to the things that I'm best at, and that I enjoy a little bit more. So, yeah. I'm also a technology council member of Maha Alliance. Who here heard about Maha Alliance? Okay. Okay. So, Maha Alliance is an alliance of the biggest enterprise vendors that are modernizing the e-commerce landscape and I'm extremely proud to represent it. And who heard about Vue Storefront? Please raise your hand. Nice. Nice. It's getting better every year.

So, you know, I work in the e-commerce industry. And I work in the e-commerce industry literally all my life. And building e-commerce storefront is harder than it seems. Like if you're powerful after displaying, you know, the first product, the first data on your website from one API endpoint, but the road from there to production is very long and it's often painful. And I can guarantee that you will feel physical pain once you learned what faceting is.

2. Importance of Performance and Mobile Consumption

Short description:

The goal of Vue Storefront is to provide tools that save you from the pain of building e-commerce storefronts. Performance is crucial in the e-commerce industry, and even small delays can cost millions of dollars. Check out WPO stats for insights on how performance optimization can boost revenue. Five years ago, frontend performance was not a significant concern, but with the rise of JavaScript frameworks, the importance of performance became apparent. The growing size of websites and the shift to mobile consumption highlighted the need for better mobile performance.

So, the goal of Vue Storefront is actually to provide tools that save you from this pain. And Vue Storefront is open source, so you can check it on GitHub and give a star if you like it. I'm not encouraging, but, you know, it would be nice.

And in the e-commerce industry, performance is one of the most important things to look at really. The fact that the way how people look at this is often completely wrong is another topic, but that's what I'm going to address in this talk. So, Amazon did a study on that topic. And what they learned is that every 100 milliseconds in added page load costs 1% of revenue. For Amazon, it's millions of dollars, really. 100 milliseconds.

And speaking about numbers, if you need a good source of arguments for your boss, for example, to take care of performance because you know it's important but you need the argument, check out this website, WPO stats, which stands for performance optimization stats. And it will give you great, great insights on how optimizing performance helps other companies to grow their revenue.

And it feels quite crazy from today's perspective, but five years ago, when we were writing the first lines of code for vstorefront, the topic of frontend performance was almost nonexistent in the web developer space. At that time, JavaScript frameworks were just gaining the traction. Angular, JS, React. They were already well-established tools, gaining popularity every day. Vue, it was just getting the attention of the broader developer community, fighting with frameworks like this one.

Do you know those frameworks? Raise your hands if you know all the frameworks, all the logos from this picture. Oh, really? It could have been Vue, but luckily, Vue made it to the third place. So almost no one cared about how fast the website built with those frameworks are. And of course, now everyone say that, you know, putting so much JavaScript on the front end, it was a terrible idea. But honestly, it wasn't that clear at that time. It wasn't that clear because the reason why we are having performance issues with single-page applications these days is because of the ecosystem and how much JavaScript you're adding through the ecosystem, not the frameworks themselves.

And, you know, as long as we're using PCs or laptops as our primary machines, which believe me, like seven years ago was a normal thing to consume the web, no one seemed to be concerned with the growing size of websites. Both CPU and internet bandwidth, they were growing faster and websites were growing in their size. It all changed when mobile phones started to become the preferred way of consuming the web. And according to Google Research in 2017, into it's took on average 15 seconds to fully load a webpage on a mobile phone. Imagine 15 seconds. If I wouldn't have only 20 minutes, I would just wait to give you, you know, this perception. At that time, the awareness about the impact of this poor mobile performance on their business started to emerge. But we're still lacking an easy way to actually link those two components.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

A Guide to React Rendering Behavior
React Advanced Conference 2022React Advanced Conference 2022
25 min
A Guide to React Rendering Behavior
Top Content
This transcription provides a brief guide to React rendering behavior. It explains the process of rendering, comparing new and old elements, and the importance of pure rendering without side effects. It also covers topics such as batching and double rendering, optimizing rendering and using context and Redux in React. Overall, it offers valuable insights for developers looking to understand and optimize React rendering.
Speeding Up Your React App With Less JavaScript
React Summit 2023React Summit 2023
32 min
Speeding Up Your React App With Less JavaScript
Top Content
Watch video: Speeding Up Your React App With Less JavaScript
Mishko, the creator of Angular and AngularJS, discusses the challenges of website performance and JavaScript hydration. He explains the differences between client-side and server-side rendering and introduces Quik as a solution for efficient component hydration. Mishko demonstrates examples of state management and intercommunication using Quik. He highlights the performance benefits of using Quik with React and emphasizes the importance of reducing JavaScript size for better performance. Finally, he mentions the use of QUIC in both MPA and SPA applications for improved startup performance.
React Concurrency, Explained
React Summit 2023React Summit 2023
23 min
React Concurrency, Explained
Top Content
Watch video: React Concurrency, Explained
React 18's concurrent rendering, specifically the useTransition hook, optimizes app performance by allowing non-urgent updates to be processed without freezing the UI. However, there are drawbacks such as longer processing time for non-urgent updates and increased CPU usage. The useTransition hook works similarly to throttling or bouncing, making it useful for addressing performance issues caused by multiple small components. Libraries like React Query may require the use of alternative APIs to handle urgent and non-urgent updates effectively.
Managing React State: 10 Years of Lessons Learned
React Day Berlin 2023React Day Berlin 2023
16 min
Managing React State: 10 Years of Lessons Learned
Top Content
Watch video: Managing React State: 10 Years of Lessons Learned
This Talk focuses on effective React state management and lessons learned over the past 10 years. Key points include separating related state, utilizing UseReducer for protecting state and updating multiple pieces of state simultaneously, avoiding unnecessary state syncing with useEffect, using abstractions like React Query or SWR for fetching data, simplifying state management with custom hooks, and leveraging refs and third-party libraries for managing state. Additional resources and services are also provided for further learning and support.
The Future of Performance Tooling
JSNation 2022JSNation 2022
21 min
The Future of Performance Tooling
Top Content
Today's Talk discusses the future of performance tooling, focusing on user-centric, actionable, and contextual approaches. The introduction highlights Adi Osmani's expertise in performance tools and his passion for DevTools features. The Talk explores the integration of user flows into DevTools and Lighthouse, enabling performance measurement and optimization. It also showcases the import/export feature for user flows and the collaboration potential with Lighthouse. The Talk further delves into the use of flows with other tools like web page test and Cypress, offering cross-browser testing capabilities. The actionable aspect emphasizes the importance of metrics like Interaction to Next Paint and Total Blocking Time, as well as the improvements in Lighthouse and performance debugging tools. Lastly, the Talk emphasizes the iterative nature of performance improvement and the user-centric, actionable, and contextual future of performance tooling.
Optimizing HTML5 Games: 10 Years of Learnings
JS GameDev Summit 2022JS GameDev Summit 2022
33 min
Optimizing HTML5 Games: 10 Years of Learnings
Top Content
PlayCanvas is an open-source game engine used by game developers worldwide. Optimization is crucial for HTML5 games, focusing on load times and frame rate. Texture and mesh optimization can significantly reduce download sizes. GLTF and GLB formats offer smaller file sizes and faster parsing times. Compressing game resources and using efficient file formats can improve load times. Framerate optimization and resolution scaling are important for better performance. Managing draw calls and using batching techniques can optimize performance. Browser DevTools, such as Chrome and Firefox, are useful for debugging and profiling. Detecting device performance and optimizing based on specific devices can improve game performance. Apple is making progress with WebGPU implementation. HTML5 games can be shipped to the App Store using Cordova.

Workshops on related topic

React Performance Debugging Masterclass
React Summit 2023React Summit 2023
170 min
React Performance Debugging Masterclass
Top Content
Featured WorkshopFree
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
Building WebApps That Light Up the Internet with QwikCity
JSNation 2023JSNation 2023
170 min
Building WebApps That Light Up the Internet with QwikCity
Featured WorkshopFree
Miško Hevery
Miško Hevery
Building instant-on web applications at scale have been elusive. Real-world sites need tracking, analytics, and complex user interfaces and interactions. We always start with the best intentions but end up with a less-than-ideal site.
QwikCity is a new meta-framework that allows you to build large-scale applications with constant startup-up performance. We will look at how to build a QwikCity application and what makes it unique. The workshop will show you how to set up a QwikCitp project. How routing works with layout. The demo application will fetch data and present it to the user in an editable form. And finally, how one can use authentication. All of the basic parts for any large-scale applications.
Along the way, we will also look at what makes Qwik unique, and how resumability enables constant startup performance no matter the application complexity.
Next.js 13: Data Fetching Strategies
React Day Berlin 2022React Day Berlin 2022
53 min
Next.js 13: Data Fetching Strategies
Top Content
WorkshopFree
Alice De Mauro
Alice De Mauro
- Introduction- Prerequisites for the workshop- Fetching strategies: fundamentals- Fetching strategies – hands-on: fetch API, cache (static VS dynamic), revalidate, suspense (parallel data fetching)- Test your build and serve it on Vercel- Future: Server components VS Client components- Workshop easter egg (unrelated to the topic, calling out accessibility)- Wrapping up
React Performance Debugging
React Advanced Conference 2023React Advanced Conference 2023
148 min
React Performance Debugging
Workshop
Ivan Akulov
Ivan Akulov
Ivan’s first attempts at performance debugging were chaotic. He would see a slow interaction, try a random optimization, see that it didn't help, and keep trying other optimizations until he found the right one (or gave up).
Back then, Ivan didn’t know how to use performance devtools well. He would do a recording in Chrome DevTools or React Profiler, poke around it, try clicking random things, and then close it in frustration a few minutes later. Now, Ivan knows exactly where and what to look for. And in this workshop, Ivan will teach you that too.
Here’s how this is going to work. We’ll take a slow app → debug it (using tools like Chrome DevTools, React Profiler, and why-did-you-render) → pinpoint the bottleneck → and then repeat, several times more. We won’t talk about the solutions (in 90% of the cases, it’s just the ol’ regular useMemo() or memo()). But we’ll talk about everything that comes before – and learn how to analyze any React performance problem, step by step.
(Note: This workshop is best suited for engineers who are already familiar with how useMemo() and memo() work – but want to get better at using the performance tools around React. Also, we’ll be covering interaction performance, not load speed, so you won’t hear a word about Lighthouse 🤐)
High-performance Next.js
React Summit 2022React Summit 2022
50 min
High-performance Next.js
Workshop
Michele Riva
Michele Riva
Next.js is a compelling framework that makes many tasks effortless by providing many out-of-the-box solutions. But as soon as our app needs to scale, it is essential to maintain high performance without compromising maintenance and server costs. In this workshop, we will see how to analyze Next.js performances, resources usage, how to scale it, and how to make the right decisions while writing the application architecture.
Maximize App Performance by Optimizing Web Fonts
Vue.js London 2023Vue.js London 2023
49 min
Maximize App Performance by Optimizing Web Fonts
WorkshopFree
Lazar Nikolov
Lazar Nikolov
You've just landed on a web page and you try to click a certain element, but just before you do, an ad loads on top of it and you end up clicking that thing instead.
That…that’s a layout shift. Everyone, developers and users alike, know that layout shifts are bad. And the later they happen, the more disruptive they are to users. In this workshop we're going to look into how web fonts cause layout shifts and explore a few strategies of loading web fonts without causing big layout shifts.
Table of Contents:What’s CLS and how it’s calculated?How fonts can cause CLS?Font loading strategies for minimizing CLSRecap and conclusion