Automate the Browser With Workers Browser Rendering API
From Author:
In this talk, we will explore how the Browser Rendering API can automate browser tasks, freeing developers from repetitive manual work. We will begin with an overview of Cloudflare Workers and how they enable running JavaScript at the edge. Then, we will discuss browser automation in detail, covering how to interact with the DOM, fill out forms, and scrape data from web pages. We will showcase real-world examples of how browser automation with Cloudflare Workers can improve the user experience of web applications, increase productivity, and automate tasks, such as generating screenshots and PDFs of web pages, testing web applications, and running performance audits. At the end of this talk, attendees will gain a better understanding of how to use the Browser Rendering API to automate browser tasks and take their web development skills to the next level.
This talk has been presented at JSNation 2024, check out the latest edition of this JavaScript Conference.
FAQ
Cloudflare R2 is a blob storage service offered by Cloudflare. It allows users to store various types of data, including images and other files.
Cloudflare Workers supports JavaScript, TypeScript, and Python, allowing developers to create new applications or augment existing ones using languages they are already familiar with.
Common use cases include taking screenshots of websites, generating PDFs from web pages, testing web applications, gathering performance metrics, and automating tasks that require browser interaction.
The Workers Browser Rendering API is a fork of Puppeteer, a Node.js library from the Chrome DevTools team. Cloudflare patched Puppeteer to connect directly to the Workers Browser Rendering API, enabling its use within Cloudflare Workers.
The Workers Browser Rendering API allows developers to perform browser automation tasks within a headless browser instance using the Cloudflare Developer Platform. It can navigate to websites, take screenshots, create PDFs, and more.
The Workers Browser Rendering API can automate the process of checking apartment availability on websites. For example, it can navigate to a specific page at a set time, take a screenshot of the availability page, and even send an email notification if an apartment is available.
Yes, Cloudflare has added features like browser management and Durable Objects that allow developers to reuse browser sessions and extend session durations, making it easier to perform multiple tasks within a single worker.
Developers can integrate tools like Durable Objects and session management to extend browser sessions and improve performance when performing multiple tasks. Additionally, analytics and logs can help monitor the number of browsers spawned during worker integration.
Documentation and examples for the Workers Browser Rendering API can be found on the Cloudflare documentation website and the Cloudflare blog. These resources provide detailed guides and use cases to help developers get started.
Yes, the Workers Browser Rendering API is generally available and can be used by developers to automate browser tasks within their applications.
Video Transcription
1. Introduction to Browser Automation
I'm excited to be here at JS Nation, talking about automating the browser with the workers browser rendering API. I'll show you how to perform browser automation tasks using the Cloudflare Developer Platform and share use cases and examples. I'm Gift Iguenu, a developer advocate at Cloudflare. Let me tell you a short personal story about finding an apartment in the Netherlands. I had a specific apartment in mind and had to manually check the website every week.
Hi, everyone. I am really excited to be here at JS Nation. I'm going to be talking about how to automate the browser with the workers browser rendering API. Now, that's a mouthful. From now on, I'm going to refer to this as just rendering API or browser rendering. I'm really sad I couldn't make it to the conference in person, but I hope you still enjoy my session.
In this session, I'm going to show you how you can perform browser automation tasks using the Cloudflare Developer Platform. And at the end, I'll show you some cool use cases and examples of some tasks that you can automate with the browser. Again, I am Gift Iguenu. I work as a developer advocate at Cloudflare. I'm really excited about the Developer Platform at Cloudflare, and I talk about it on Twitter slash my personal blog. So, if you'd like to follow me there, that would be great.
All right. So, I have a short personal story to tell before going into this talk, and it's likely based on my experience as an expat living in the Netherlands. So, I feel like this is going to be relatable if you live in the Netherlands, or if you live in any major city that has housing crisis. It's super difficult to find an apartment. Usually takes around, what, one to three months, if you just relocated or moved to a new city, to get a nice apartment, right? And it's also quite expensive. In the Netherlands, last year, there was, like, house shortage over 390,000 homes were had shortage. So it was very difficult to find an apartment.
And at the time, I was looking for an apartment in the Netherlands, specifically in Amsterdam, because I was living in a different city, and I wanted to move to Amsterdam. And I had a specific apartment in mind that I wanted to move to. It's like a building complex with several types of apartments. Now the interesting bit about this specific place is called our domain. Basically how it works is you just need to go to the website, and once there is an available apartment, you pay for it, and it means that once you are able to pay, it means that you have the apartment.
Now, the problem is, for every apartment that is available in that building, it's essentially already booked, right? So every week, if maybe someone moves out, or there's an availability, they would update that every week on Wednesday at 12 p.m. So for the next one month, I would go to the website on Wednesday at 12 p.m. to check if there is an availability, right? So I was looking for a two-bedroom apartment. And of course, it took me a while to be able to find this. I was manually checking the websites every week, which was not nice. But thinking about the subject in question, browser automation, at the time, if I had known about this, this is something I would have done.
2. Automating Tasks with Browser Rendering API
The Worker's Browser Rendering API allows you to automate tasks within your browser, such as navigating to a website, taking screenshots, and creating PDFs. I used it to automate the process of finding an apartment, navigating to the website at the specific time, and paying for the available apartment. Browser automation can also be used for tasks like email automation. Cloudflare Workers is a serverless environment that supports multiple programming languages like JavaScript, TypeScript, and Python. The Cloudflare developer platform offers the browser rendering API, which allows developers to control and interact with a headless browser instance. This API is based on Puppeteer, a Node.js library from the Chrome DevTools team.
But just so if anyone has this problem, this is how I solved it, using browser automation. So the Worker's Browser Rendering API basically allows you to, you know, run automation within your browser. You could do things like, you know, navigate to a specific website, take a screenshot of the website, create a PDF out of the website, and so on.
So for my specific use case, instead of going to the website on Wednesday at 12 p.m. every single week, I wanted to automate this process, right? I wanted to allow my browser do that for me, without me having to do it myself. So what did I do? I essentially wrote the browser rendering worker that would navigate to the specific website at the specific time. Honestly, I just would run this every single time, just because I wanted to be sure that I could find an apartment, right? So what this does is, it navigates to the websites, takes a screenshot of the specific availability page. So if there is an availability at a time, the website is going to tell me that, oh, you're lucky. There is currently an apartment available. In fact, at this specific time, there was three different apartments available. So I was able to, you know, go to the website and pay for the apartment that I wanted to do.
Of course, this is just a cool example that, you know, browser automation can help you with. There's so much more that you can do. For example, with this demo, I could as well go one step further to make a Chrome job that I would make this worker run at a specific day every week. Or I could also add like email automation where if there's an apartment, instead of like returning the value of the apartment to the website, I could send myself an email and I would be automatically informed of the availability, right? So that's an actual use case that I think browser automation can help with.
So with that, I want to talk about underlying technology within that, you know, setup I just showed you, which is Cloudflare Workers. If you're not familiar, Cloudflare Workers is a serverless environment that allows you to create new applications or augment existing ones with, you know, languages you're already used to like JavaScript, TypeScript, or even Python. So this is an example Cloudflare Worker. Basically what this does is it's returning a string or an array of conference data, in this case, GSNation, and I'm returning that to my browser. It's essentially written in JavaScript. This example I'm showing you, but you can as well write it in TypeScript or in Python if you want.
Now moving forward, just a few slides. Yeah. Browser rendering API is one of the services that we offer in the Cloudflare developer platform. Essentially what this is, is it allows you as a developer to control and interact with a headless browser instance. And from that, you can then create automatic flows, for example, navigate to a website, take a screenshot, and so much more. And the underlying technology here, if you ever explored the topic of browser automation, you possibly have heard of Puppeteer. So Puppeteer is a Node.js library from Chrome DevTools team. It essentially provides an abstraction of the DevTools protocol to help you control Chrome or Chromium. Now the worker's browser rendering API is a fork of Puppeteer.
Comments