Holistic Web Performance With Grafana and K6

Rate this content
Bookmark

When writing web performance tests, the tests you create from these tools come from lab data collected from pre-defined environments, devices, and network settings. Lab data allows you to reproduce performance results repeatably, making it useful for detecting and fixing performance issues early. However, lab data ignores a very important aspect: your users’ real experience of using your applications. Field data, collected from real users under various conditions, reflect how your application is being used in the real world and track the performance your users experience and even the errors they encounter. Real users also have different user behaviors, which cannot be all simulated realistically via test scripts or test cases. Real users use different devices, network conditions, caching mechanisms, and even system fonts that can impact how a site loads. Therefore, you must still complement your performance tools that are using lab data with field data or real-user monitoring (RUM) solutions to achieve a holistic approach to web performance testing. One way to do this is by complementing Grafana k6 browser with Grafana Faro.

This talk has been presented at TestJS Summit 2023, check out the latest edition of this JavaScript Conference.

FAQ

Using both lab and field data provides a comprehensive view of web performance. Lab data helps detect issues early in a controlled environment, while field data monitors real-world performance, capturing issues that lab data might miss.

Grafana Faro can monitor errors and performance in a live environment, while the K6 Browser module can reproduce and test these issues in a lab setting, ensuring fixes are effective before deployment.

Users can start with GrafanaFaro by signing up for a free GrafanaCloud account, using the Front End Observability product, and following the setup instructions for their applications.

Grafana Faro tracks Web Vital metrics such as First Contentful Paint, Largest Contentful Paint, and Cumulative Layout Shift to understand real user experiences.

The speaker is Marie Cruz, a developer advocate at Grafana Labs.

The main focus of the talk is achieving a holistic web performance approach using tools like Grafana and K6.

QuickPizza is a sample application that generates random pizza combinations. It is used to demonstrate setting up GrafanaFaro and creating browser tests with the K6 Browser module.

GrafanaFaro is a web SDK that automatically collects logs, errors, and performance metrics. It helps in monitoring real user interactions and provides insights into web performance through predefined dashboards.

The Grafana K6 Browser module, inspired by Playwright, is used for browser automation and end-to-end web testing. It helps simulate browser actions and collect web performance metrics as part of tests.

Lab data is collected in a controlled setting to simulate user interactions, useful for detecting performance issues early. Field data reflects real user interactions under varying conditions, providing an overall picture of how a website functions in the real world.

Marie Cruz
Marie Cruz
21 min
07 Dec, 2023

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Today's Talk focused on achieving a holistic web performance approach using Grafana and K6. The importance of complementing lab data with field data for realistic user experience testing was highlighted. The use of GrafanaFaro for collecting performance metrics and integrating with other Grafana OSS products was discussed. Detailed instructions on setting up GrafanaFaro with the QuickPizza sample application were provided. The benefits of using K6 for reproducing errors, improving testing, and demonstrating browser testing capabilities were explained.

1. Introduction to Holistic Web Performance Testing

Short description:

Today, I will share how to achieve a holistic web performance approach with Grafana and K6. Lab data alone is not enough for a realistic user experience. Complementing lab data with field data is crucial for web performance testing.

Hi, everyone. My name is Marie Cruz and I'm a developer advocate here at Grafana Labs. Today, I would like to share with you how you can achieve a holistic web performance approach with tools such as Grafana and K6. But first off, if you want a copy of these slides, feel free to scan the QR code that's displayed here on my screen.

So let's explore the problem statement that I want to focus on in my talk today. Performance testing tools are awesome. But what about real users' experience? For example, performance tests that are created with the Grafana K6 browser module use Lab data collected from predefined environments, devices, and network settings. Lab data allows you to reproduce performance results repeatedly, making it useful for detecting and fixing performance issues earlier on. Lab data, however, doesn't account for a very important thing, which is the real user experience. Real users have different user behaviors, which can't be simulated realistically via test scripts or test cases. This is why it's best to complement lab data with field data to achieve a holistic approach to web performance testing. But I think first it's very important to know the difference between lab and field data to understand why both are essential for web performance testing.

2. Lab Data vs Field Data in Web Performance

Short description:

Lab data is collected in a controlled setting, simulating user interactions on a website, while field data reflects real-world conditions. Metrics may differ between the two due to factors like query parameters and targeted ads. Lab data is useful for early development and testing, allowing for detection and fixing of performance issues. It provides insights into specific settings without external factors.

Here's a car analogy that can help us illustrate the difference. Imagine that you're testing the speed of a new car model in a controlled environment such as a racetrack. You have a stopwatch, and you measure how quickly the car accelerates, you measure how it brakes, and you measure how it can take turns. You record these measurements consistently with no traffic, perfect road conditions, and there's a very skilled driver behind the wheel.

Now imagine a car's performance on railroads like a busy city street or highways. The road conditions will vary, there's gonna be unavoidable traffic, and different drivers may handle the car differently. The lab data for web performance is similar to our car performance under an idealized setting. Lab data are collected in a controlled setting, often with specialized tools or scripts that simulate user interactions on a website and are useful for detecting performance issues earlier on. On the other hand, field data is similar to our car performance under real road conditions. Field data reflects how your users access your site from different devices and locations and under different network conditions. These users also might have various browsers and connection speeds just as real drivers on railroads will have different skills and behaviors.

Now, when you work with tools that use lab data, you might notice that the web performance metrics could differ from those generated by tools that use field data. As an example, when you run a web performance test in a lab setting, you normally just include the base URL of the web application that you are testing. However, in real world, different users will access a website using different query parameters or different text fragments, which can also impact the web performance metrics. When your users access your website in real life, there will also be different targeted ads depending on your users browsing history, so that can also impact the web performance metrics. So, you've now understood the difference between lab and field data. When should you use them when it comes to web performance? So, both types of data are valuable for different purposes. Lab data excels when it's used during the early development and testing stages. Because of the controlled environment, performance issues can be detected and fixed earlier on. So, as an example, if you want to understand your performance metrics in a specific setting such as a particular device or a particular screen size, then in that case, you'd be able to set this up in a controlled manner without really worrying about other external factors.