The State of the Web: Key Insights From the HTTP Almanac 2024

Yeah, last time you saw me, I was in costume. So give me a bit of context. My company, we host React Berlin Meetup and we did the Halloween edition of it. And actually I had my Gandalf costume, but you saved the day because you sacrificed your lipstick so that people can have costumes, right? It was a nice, nice of you.

Hey everyone. Thanks for joining. Today I will be talking about Almanac 2024. First, can I ask how many of you know Almanac? Did you ever hear about it or are you going to get into it? Okay. Okay. Not much. Perfect. Yesterday.

So let me start about myself. I'm Burak. I'm a developer based in Berlin. I'm originally Turkish. I moved here four years ago. I've been here for four years. I work in a company called Überall and I love cold weather. It's a great match for me, the city. Here you can see me stupidly jumping in Admiralbrücke. If you're not from Berlin, I would recommend. It's a great place. It's a bit cold now, but still worth to see. I have a toxic relationship with German. I'm trying to learn German, but here on top you can see my Duolingo streak. I left my Duolingo streak. It was 643 days.

I left my Duolingo streak. It was 643 days. Now I left it because I was procrastinating. Now I'm in the expat level. I just know like Genau, Mittkarte, Schönentag and some weird words from board games like Wachterin. It means guardian. Useless. I hope you won't need it anyways, like any of us.

Start with good wishes. And there is one tradition that I like here. It's Wegbia. It's basically the city is flat. It's kind of weird for me. I'm from Turkey. It's super hilly over there. You basically grab a beer, grab a friend and then you walk. It's a quite nice thing. It's an alternative therapy if you ask me.

And enough of me. Let's dive in what I will talk about today. So today I will be talking about Webalmanak. For the people who didn't hear about Webalmanak, it's basically... So all of us, like most of us got our Spotify wrapped, right? Like the best artists, best songs that you had. It's basically internet wrapped for 2024. And we do test nearly 17 million webpages. We process 83 terabytes of data monthly. And then at the end of the year, we release our report about each year. And 2024 edition just got released last month. And I've been volunteering for that project. I will be talking about this one.

So let me get into the chapter that I will be talking about. It's a huge report. It has like, I think 21 chapters. So I cannot talk about all of them in 20 minutes. It's kind of impossible.

But Sustainability chapter is the chapter that I got most involved. And I was an author and data analyst over there, according to the initial plan. I was going to be just author. I don't like SQL. I'm frontend engineer. It's a volunteer project, so I need to cover up some parts of it.

And here in this screenshot, you can see my anxiety, because I was using BigQuery. And I was doing some data processing over there. It's like 5.2 billion records. And I said, okay, I hope Google is not going to charge me, but HTTP archive. So far, I didn't get any invoice. So I'm happy about it.

I need to do a shout out to my team. I'm part of them. We are nine people. We have authors, data analysts, and editors. It's a great team. We wrote a huge chapter. And it got the best chapter award of the HTTP archive this year. And we will be talking about the web sustainable question a lot today. I will be giving you some statistics and some recommendations, how we can improve it. And let's dig in. We will start with the statistics. But before all of that, I will start with a disclaimer. I'm not a person to judge you, for sure.

And so I'm not the person who is the most climate friendly. I'm doing lots of crimes in terms of climate. I'm taking lots of flights. This is... You can see my fake PNG flying in a plane. And basically, I was... There was a package called with air all you can fly. I don't know if you heard about it. It released two or three months ago. I bought it. Because I was planning... I will get my computer. I will take my backpack and I will fly anywhere. And I will be enjoying my life working from there. And then I canceled it. Because they changed the terms. It was kind of a scam. But I need to be honest with you. I canceled it because of my environmental concerns. We will talk honest today. I canceled it because it was a scam.

And I met with thousands of people. Like great people. I met with some of them. Some are using Fairphone. Some of them are not flying. Some of them are just using second hand items to reduce their carbon emissions. So I'm not the person who can even reach to their level. But yeah. I will be talking about the sustainability of the web.

This is my interest. Yeah. I hope Vizier is not a sponsor. Because I just called them. Scam. Grammarly, I saw it's great. Porsche. It's amazing. Okay. Sorry. If it is, it's my first and last React Day Berlin talk. Enjoy it. I hope not.

Okay. Let's start with page weight. According to our data analysis, we found out in... Okay. Let me actually give you an introduction about these slides because it's a bit complicated. In most of the slides, in the left side, you will see a 2024 chart and 2022 chart at the right side. So this chart is basically showing the desktop and mobile statistics. And there is a percentile over there. So the 10th percentile is the best performing websites. 19th percentile is the average of all of the websites that we process. Like 17 million. Nearly 17 million of websites. And I will be mostly talking about the 19th percentile because I will be talking about the average.

And according to this year's results, averagely we are shipping 8 megabytes of data to the client's computer or device whenever they open our website. And it's huge. It's definitely huge. According to our researches, ideally it should be 500 kilobytes and the maximum amount that we can accept should be 1 megabyte.

And now we are doing 8 times more than the maximum amount which is quite bad. And there's also... There's just one good thing about it. We just decreased our carbon emission of each page load by 1 megabyte. It was 9 megabytes before. It decreased to 8 megabytes. But in two years, the environment... It's like environmental knowledge is increasing and we are not following it up. We are in a really bad speed. We should speed it up more and we should decrease our page rate.

But okay. Let's see the CSS stats. So we... Like CSS is easy, right? It shouldn't be producing some carbon emission or something like that. It should be great because we have frameworks and stuff. But no, we are... As you can see on the left side, unused CSS, we are shipping 225 kilobytes of unused CSS. And if you compare it to 8 megabytes, okay, it's not that much. It's very small. Maybe it's not the first thing that we need to kill and we need to like tackle. There might be like bigger problems. It just increased 4 kilobytes more. It didn't decrease. We are shipping more unused CSS. It's not the worst, but it's not a good one.

Maybe we can check JavaScript because we have all the tools, we are advanced, right? We should be like in a good state. No. We are shipping 907 kilobytes of unused JavaScript, which is actually the upper limit that we set. It should be at most one megabyte and we are close to that limit that we recommend. And there is one concerning thing.

When you see the 2022 data, it was 645 kilobytes. Like we increased JavaScript for like 300 kilobytes, which is not cool. And there's even one worse thing about it. When you check the 2022 data, the best website, best performing websites, they were shipping 0 kilobytes of unused JavaScript. Now this year, 2024, they even start to ship unused JavaScript kilobytes, which is super, super concerning. And it's something that we should fix.

I don't know. Let's continue with the statistics. The greenhosting. Okay. We have unused JavaScript, we have unused CSS, but most probably we are using greenhosting, right? Like we are balancing it out, kind of like being concerned about sustainability. No, we are not. As you can see here, in the old part in 2024, the desktop is not visible, but I wrote it on top, just 12% of the websites are using greenhosting. And even got decreased from 2022, it was 13% before.

There is one... So in our report, we don't have such a saying about this one, but if you ask me, maybe AI can have some kind of like, its own effect on it. I will come to that part at some point. But yeah, greenhosting, it got decreased. So far, we don't have much good statistics. Let's see the image rate. In my opinion, it's the worst statistics that we have.

So in the right side, let's start with this graph. Here you can see in average, we are fetching that amount of kilobytes by person, by person type, by type of images, the blue ones, the sky blue ones. These are just kilobytes for images that we fetch. And this is more than 50% of the page rate that we are shipping, which is huge. It's like 4.4 megabytes of data that we are shipping in those 8 megabytes average. It's just images. And on the right side, you can see the, you can see the emissions that we calculate according to the file types. And the green one is images. The blue one, or like the dark gray one, that's JavaScript. And you can see in the average, in the average website, like 90% percentile, it's around like 70% of the emissions being caused by images. I need to give one note about this one. So here, you are seeing our emission calculation by type. But there is one thing. JavaScript is most probably emitting a bit more because our model that we calculate, carbon emissions, it's not, it cannot add the parameter of usage of the CPU or like the battery, like the energy that you use in your local device. Most probably it's a bit more, but still, images are huge.

It's not acceptable. We should decrease it. But maybe we are using good image formats, right? On the left side, we can see some good data, finally. So I don't know if you heard about aviv or webpy. These are like the new two formats that, which are more lower size, and they are better to use because they are, they have less size in, after all. But aviv increased 386%, which is cool because it's the most compressed and it's the lowest size of images. But it's a bit misleading because it was, in 2022, it was 0.3% usage. Now it's just 1.4% usage. Yeah, some people are discovering it. Webpy is increasing by 34%, which is super cool, actually. And JPEG decreased 20%. This is not super bad.

We have some progress here, at least, let's say. But we should be focusing more on aviv and webpy because they are, right now, being supported broadly, and even if they are not being supported, there are some ways to use them and fall back if we cannot do it. But maybe we are good at image optimization, so let's see.

We have the lazy loading statistics on the left side and 35% of the images that we have on those webpages, they have lazy loading. It's not a bad statistics, but personally, I would prefer it to be like, it speed up more heavily because it's not a hard thing. It's just like, generally, it's very easy implementation. It's just like an attribute and it loads lazily. If it's not in a viewport, it won't be even loaded. I think that's just something that it's a low-hanging fruit. We can definitely pick that up and decrease our page weight. And on the responsive image types, it's actually, this is a good graph if you ask me. Usage of source set we are seeing here. 42% of the webpages are using source set. Source set, if you don't know it, basically, let's say that you have a website, it's responsive. The desktop one is like, you are pushing a higher resolution photo image to the website because it will be huge and you need a higher resolution. It's understandable, but if you don't use source set or like an alternative image for that one for client sizes, you are also shipping a huge resolution size image to mobile clients and it's a waste. If you use a source set, basically, it will be just like switching to the low resolution image and it will be reducing some data transfer for you, which is quite cool. I think it's 42% is not bad, but maybe we can increase it a bit more in the next year.

Text compression, so if you ask me, this is the lowest thing hanging for it. It's super easy and we are missing 53% of the files. 53% of the files are not using text compression, which is, I think it's like the first thing that we should do. It's the start point of solving this page weight issue because basically, let's say that I'm sending my phone to my grandma. I'm grabbing a huge package, I'm going to Deutsche Post, DHL, I'm putting my phone inside of it and I just ship that huge package. It's just going with the air, it's just like there's a small phone inside of it. We don't need to ship a huge package, we can just compress it and we can ship it that way. It will save a lot of data transfer for us, up to 80%.

Caching. Caching is actually in good statistics, if you ask me. 25% of the pages are not using caching. Even if they are not using caching, at some level they are being cached without their control. If they don't specifically say that, no, there's no cache. 25% is not bad. You can say that maybe 25% is in purpose not caching at all, but I don't think 25% is in purpose not to cache it at all, maybe like 10%. It would be better to have it more, it's really good statistics if you ask me, but still caching is not something new. It's been there for ages, so maybe it would be better to have that to have it in the level of ideal level. Yeah, it's not a bad statistics.

And lastly, AI. So I'm not against AI, basically, but I need to tell this. In compared to traditional searches that you are doing, AI searches, according to research, it's consuming 30 times more energy. So maybe we should stop asking small questions to AI and ask it to Google, just like a traditional way, or other platforms, Bing, other ones, I don't know, whichever you prefer. And I like this, we had this phrase before, the juniors, interns, people were coming to us, they were saying, I have a stupid question. And at that point, it's like, they feel weak a bit, but it's the point that they have something that they cannot resolve in their minds. But once you resolve it, they are going to grow a lot. And now nobody's coming with, I have a stupid question phrase. Everybody's going to AI, unfortunately. And I really miss that phrase, I think it was super lovely. Please return back that question if you're a junior or an intern, I'm not going to judge. If the people who you ask will judge, come to me.

I will answer in front end, and if you're asking back end, I will find someone, no worries.

Okay, I need to be fast. Enough negativity, I don't have too much. Okay, let's go.

Image optimizations, I set AWIF and WebP. You can see here the support of the browsers, it's being widely supported. In Internet Explorer, it's not being supported, both of them. In Safari, in a very old version, AWIF is not being supported. But still, I know Internet Explorer is not being used a lot. I don't say ignore them, it will be an accessibility issue. Maybe some old people need it, we cannot kick them out. They also have right to use the internet. But still, you can use the picture tag. Picture tag is basically a tag. If you don't know, you can have some fallbacks. You can start with AWIF, you can start with WebP. And if those are not being supported, then fall back to JPEG and PNG. And then you will be saving a lot of data. Also, please your source set. Source set is like, it will decrease mobile data transfer a lot. And lazy load, it will help us a lot. These are the ways to fix it.

Text compression, it's super straightforward. I don't know the third option, it's in our report before. I don't want to talk about it without knowledge. But Gzip and Broadly. With Gzip, you can basically have like potentially up to 80% of compression, which is a super low hanging fruit. And you can grab it and you can decrease your page weight a lot. You will be saving more kilobytes of transfer. And plus, you will be reducing the costs of your company. You will get a raise or something.

Page weight. See, there's a video here. I took this video from Amazon two, three days ago. And you can see the coverage report of it. It's like desktop page, like the main page of Amazon. In the bottom part, when we go to the coverage, you can see that I think it was like 6.4 megabytes of unused JavaScript or unused files, file transfer. Here, you can see 6.3 megabytes. It's gone. But it's over there. You will see. Yeah.

So, it's not easy, I know. It's not easy to remove your unused JavaScript or CSS or other file things, but it's easy to detect. It's just one Chrome DevTools feature and then you detect it. And then you can use tree shaking. You can start code splitting. You can make it like smaller chunks. And with the smaller chunks, you will be saving a lot of data if you load them when they are necessary. Green hosting, it's so hard to define some of those hosting... So it's hard to define which hosting service is green, because there are lots of parameters.

So it's hard to define which hosting service is green because there are lots of parameters. If you are interested in it, you should check out this article that I shared on the right side. If you want a quick solution, you can use the third bullet option. It's like... There is a link over there. Over there. If you go there, you can find the green hosting services according to the countries because not every green hosting service... Not every hosting service is green in every country.

You can try to switch to green hosting. It's also a low hanging fruit. And plus, you can try to use more CDN because CDNs are like geographically placed, like better placed packages, data that you can grab. And it will be easier for you to not request something from the United States. If you use CDN, maybe it will be a super close destination. It will travel less. It will consume less energy, let's say.

Caching. I don't need to talk about caching. All of us know it. It's over there for ages. Maybe if you don't have any strict caching policy for some strict pages, if you don't need to fetch it all the time, maybe you can do caching for those ones. Last but not least. The final statistics. This is the... According to our report, we are getting this carbon emission. For each page in 2024, we are generating on average 1.47% of... 1.47 grams of carbon dioxide. And it went up unfortunately. In 2022, it was 1.29. It went up unfortunately. We should decrease it.

We should follow up with the other initiatives of environmental friendly initiatives. And I need to share this also. We also calculated our carbon footprint as WebAlmanac because I'm just... I cannot be just judging people. You need to decrease your image size and I should also put some sticks to our group of people, right? According to our researches, we are doing web crawl monthly. We are getting a huge data. It's in Google BigQuery. You can go there and you can basically run your queries. Just beware. If you run your queries in huge data sets, you will get a huge remorse. Just beware.

According to our crawlings, we are just generating monthly a standard old gasoline car to travel three times around the world. Or like 323 smartphones equal to 323 smartphones carbon emission when you manufacture them. It's our emission as WebAlmanac or HTTP archive. And we are trying to reduce it. It's not good. It should be less, but it's not the worst. But definitely we are working on it to reduce it.

Finally, it's the end. Here you can find my socials. Ask me anything. Ask me stupid questions. Ask me. I'm Turkish. You can ask me where should I eat doner in. Here. I will guide you. Even I will guide you with the vegan ones because I was pescetarian for a while. I didn't eat meat. So ask me anything.

Here we need a lot of people. Right now we have 20 chapters, I think, but we couldn't release some of the chapters because we don't have enough people. We couldn't assign any people to those chapters. Please help us in 2025 edition. Even if it's too far away, if you want to start right away right there, you can join us and you can start translating the WebAlmanac to your native language. It will help us a lot.

And there are some additional links. Sorry for the extra time. Right. Thank you very much Burak. That was an amazing talk about sustainability.

So among those websites being analyzed, is there any comparison about what web framework they use? Does React have any features to improve sustainability? So I don't know if we have specific data for React. We have data for CMS and e-commerce websites like Shopify. If you use Next.js, how much does it produce? I'm not sure if you have for React because it's a huge chapter and it just got released one month ago. I didn't have a chance to read all of it. I'm sorry, I don't want to give a false answer to that, but maybe after this talk we can meet and check it out together.

Definitely, thank you. So next question, couldn't GPT also cage stupid questions? Do you think they will work on optimizing? I think, yeah, eventually definitely they will work on optimizing. I heard something like this. They are planning to be working on nuclear power, which will be a huge improvement because I don't know, nuclear power, it's in debate in my opinion. I think it's super green option to produce energy, but I don't want to offend anyone. If they switch to nuclear power, it will be super super low emission. I would switch from Google to ChatGPT right away. Yeah, it will be fixed. I'm just saying for now, just be careful what you're doing. Don't ask like a food recipe. You can find it from Google or YouTube. Google, let's say Google is the best or the other directives. I don't know. Yeah, yeah. All right. Thank you. Next question.

We have, what can we do to shift the focus from heavily performance UI and UX oriented to more sustainable model? So if you're working in a company, basically the first thing that you should do is show the average image percentage of page weight to the persons who are paying the hosting prices. You just show them, okay, more than 50% of our prices are going to images. We should talk to designers or whoever is responsible from it, the UX engineers and we should decrease it. It will save us some money right now. I think that's the best approach to do that. You will definitely gain some enemies from designer team and UX team, but maybe it's worth it. Probably. Maybe it's worth it. Maybe it's worth it.

And then we have, how do you know what JavaScript is used or unused? How do we know? Yeah. Basically, I showed the, like it's not that different. I showed the coverage report from the DevTools. You can even find it for your website or any website.

You just go to DevTools. You say, check the coverage and then you just find out the JavaScript files which are not, which have unused bytes. And yeah, that's the way, easy way for the web almanac data crawl. I didn't work on that, like data crawl team. I just worked on writing some SQL queries to crawl data. So I'm not the person to ask, but if you find me, I will contact you some people who are from the main team of data crawling. Yeah. Thank you.

We have another one. How does unused CSS have an effect on carbon emission? So think about this way. Let's say Amazon has 200 kilobytes of unused CSS. Let's say, I don't know. We had this video. We can check it actually, but let's say 200 kilobytes. Amazon is getting billions of clicks, right? So you are basically multiplying 200 kilobytes with billions. And 200 kilobytes with billions, when you multiply, it's a huge amount of data transfer and it's a huge amount of energy. It depends on a lot of stuff. It depends on which country you are. It depends on the grid intensity, how much energy you're wasting while transferring some data through cables. It has lots of parameters. But basically at the end, in average, according to our latest calculations, a website has 200 kilobytes. And if you're multiplying it with billions, you're emitting lots of carbon, unfortunately, because it's not being used. If it was being used, okay, then emit, because maybe it's on purpose. It solves something, maybe. But if it's unused, it's a waste. Thank you.

We have time for one more question. Is there any tooling that can support moving to a green application? Oh. You are the tool. You are the- You are the tool.

We are the engineers, right? We are the engineers. We are the tool to move to greener. So you can basically switch to green hosting, okay. But it's just one thing to do. You need to also decrease the image size. Yeah, we are the tool. There is no tool. Or maybe as tools, we can build a tool. But I think it's too detailed to do that. I think it will be better for you to do that. Okay.

Yeah. We as developers, we often ask, is our code clean, sustainable? I know, not sustainable. Most of the time, not. And that is a very interesting topic to look into. Thank you very much for your answer. Thank you for your questions. And if you have more questions, he will be available at his Q&A spot. And for the folks at the distance, you can still ask questions on Discord and you will be answering them later. Sure. For sure. Thank you very much.