Durable Objects - Everything Everywhere All At Once For Not Very Much Money

Let's get going. Thank you, Kevin, for this wonderful introduction. It's a thankless job to be an MC. Can we have a round of applause for Kevin and the MCs as well?

Yes. I do love being a part of these conferences. I've been part of almost every React Debrieve in the past. And it's my first time speaking. And I don't speak very often because I hate making slides. So today we're not going to do slides. We're going to do live coding. But because I did make the mistake of naming this talk everything, everywhere, all at once. I thought there might be some movie buffs in there. So I did put in one slide just in the beginning just so people don't get angry about me not, you know, putting in the slides. So this is the slide. It's a keynote animation. Look at it. It's beautiful. There. Wow. Look at that. Wow. Okay. Nice.

So today I want to talk to you about something called Durable Objects, which is a bad name for a really cool thing. So this year, like every year at the end of the year, I reflect what is it that I've been working on differently this year. What are some of the sort of new tools and new practices that have sort of, you know, improved my ability to do my job as an engineer. And Durable Objects is one of those discovered from this year I really want to share with you. Can I have a show of hands? Who here before coming to this talk has even heard of what Durable Objects are? Okay. We have about 10 hands. And I'll keep the hand up if you've actually run Durable Objects in production.

Okay. So like two or three people. Great. This is exactly what I was hoping for. And don't worry, this is not your fault. This is Cloudflare's fault.

So first, a little preamble. Durable Objects is a technology by a company called Cloudflare. I do not work for Cloudflare. I'm just an enthusiastic fan. I am not paid by them. They didn't even review my slides, so everything could be incorrect. But then on the other hand, I get to say anything I want about Cloudflare. And the thing about Cloudflare is that it's a really cool technology company, that there's really cool technology, and they're not really good at telling people how to use it or selling it or anything like that. So I thought instead of trying to explain it to you, I'm just going to show you how this works.

Okay, so this is Cloudflare. If you haven't heard of this company, it's one of the biggest sort of internet infrastructure companies. You might know them as a CDN provider, as a cache provider, proxy provider, bot defense, stuff like that. But more than that, they have a perfectly wonderful developer platform called Workers, where you can run serverless code, just like Lambda. But instead of like Lambda that usually runs in only one location or one availability zone, depending on how you configure it, Cloudflare's workers run in every one of their 300 server locations that they have around the globe. And they automatically in their network, basically just route your request to the closest one. So this means that thanks to these things being close, usually these services are extremely fast.

But so if we go to this developer platform website, behind here, below this Workers, there's this kind of well buried section called durable objects. And under here, if we go, we get to find out what durable objects are and what they do. And remember, I don't work for Cloudflare. And they tell us that we can build collaborative applications, like chat rooms, games, whiteboards. This is really cool. I love playing games, like building games and chat rooms and whiteboards. But in reality, most of us don't really get to build this type of software in day to day thing. And this is, I think, the biggest underselling of what durable objects are, is that they are really versatile programming paradigm, which I will soon demonstrate to you.

But when you do this, when we go through this, keep in mind, you know, what else you could use this technology for? And in the end, we'll discuss that. But, you know, don't get fixated on this.

But let's scroll down. So you get to maintain states. Well, that's something. Scale and demand sounds marketingy. Embedded compute, no idea what they're trying to tell me here. So I figure what we'll do is that we'll build a little app. We'll all get to play with it together. And then at the end, we'll come back to the same page. And then we'll see if we actually understand what this thing is. So good? All right. Let's do this.

So I have prepared us a little app. If you want to join us on this thing, you can scan this QR code. And you can basically play along on your phones. If you're joining us via live stream, you can also just go to do-it.pages.dev on your computer or whatever to join.

Okay. So let's get started. Here at the bottom, I am running two processes. One of them is my local Wrangler development environment. This is the CloudFlight development environment that allows me to run serverless workers, stuff like that. And then running a background deploy. Not logged in. That doesn't sound good. Let's run that again. And so every time I save any code, it will also deploy it to CloudFlare. So you will also get to enjoy the same code a few seconds later, maybe five seconds later. So let's start. I think the first technology demo that we always need to build for any app is a counter, a number that goes up.

That's also the point of most crypto scams. But then the number keeps going down. I don't know what's wrong with that. But here we do like an honest to god number go up where we just say, let's say value is zero. And then if I save it here, I see that this app turns into a button. And in a few seconds, your app should also turn into a button. So I'm kind of controlling your phones from here. And what we see also that we're returning this colo. This is the location where the code runs. So if I go back to this CloudFlare end, we see that this code is currently running in Frankfurt.

So let's start. Let's make this counter application. So if I remember how to program, we have some kind of a variable. And let's say that is a number. And then let's say that if these requests up, that is not where that goes. If this request, for example, is a POST request, then let's increment that number like so. And then let's return that counter. And now, here in my end, I can hit this button and this number will go up. And on your phones, when in a few seconds, if you hit the button, the number will also go up. Can somebody give me a whoop when their number starts going up? All right, great. Sounds good. I can, by the way, see that there's 50 people currently on the app connected. I don't know if it's all in the room or if some of them are on livestream, but thanks for joining us.

Now, number go up is really cool. But if you compare the phone of the person next to you, you see a different number. Why is that? You are most likely probably connected to Frankfurt as well, but we are seeing a different number. Why is this and why is this a problem? Well, because the way serverless execution environments typically work is that for every request, we basically spin up a new, very short lived environment. In the case of durable objects or CloudFlight workers, it's a V8 isolate, similar to the WebWorker, where we can run JavaScript or Wasm code or stuff like that. But the problem here is that we all get a different execution environment, which is good for security and isolation, but not really great for collaborative applications. And you also see the number went zero here. Why did that happen? Well, this number happens to go up just by accident because CloudFlare optimizes resources and keeps the thing in memory for a while. But after 10 or 15 seconds of me yammering over here, they will spin it down. So how do we make it so that we can all see the same number? Well, let's start with that. So this is where durable objects come in. So instead of just having a function that takes in a request and a response, we're able to basically provision these short lived or long lived, for that matter, server environments that are stateful, then uniquely addressable, and we can all connect to them at the same time.

So let's start with that. So let's make a counter class. And if we remember our JavaScript, we can have a instance variable over here. And then instead of using that variable, we'll just increment that instance. And just before we go any further, let me also add this constructor here because we're going to need it in a little bit.

Now we have implemented a class. I haven't saved it yet, so nothing will happen. This looks like pretty much normal JavaScript, except that you can make requests to this function. That's pretty sweet. But in order to instantiate this class, we also need to do something else, which is that we need to basically decide which instance of this class do we all connect to. And the way that we can do that is we can have like a normal standard... Oh, thanks, co-pilot. You got it correct. That's cool. So we can create an instance of this by a unique ID. So if we read this code line by line, we basically create an ID of an object. In this case, it's just a string that says counter. Then we get an instance of that durable object. If that durable object previously exists, we get an instance of it. If it doesn't exist, it gets created for us. And then we forward the request that comes in from the app to that durable object, and then that object will handle it. So now if I save, here on my end, number goes up. Well, we saw that already. That's not too exciting. But in a moment, hopefully, deployment has gone correctly. Great. Now, oh, look at that. That number keeps going up by quite a lot. I think that's because all of you are hammering it. And every time you hit it, you're incrementing the same number as the next person is, and then you're getting the increments of the previous person. So that's pretty sweet. That's cool. I think we can make this a lot better. And before we do that, let's make this more exciting. I think counter example, I think, is good for simple things.

But I think we, as adult developers, I think we can do something a lot more fancy. So what is cooler than incrementing a number? I think I would posit that it's at least twice as cool to increment two numbers. So let's just do that. And while we're at it, let's make it interesting. Let's make those numbers mean something. So let's say we're a React conference, let's say a remix, or in Next.js. Let's make this into a popularity contest. And then here, let's see. We will take the request that comes in, and the app here will send an ID of the button or like a number of the button. And then let me just add this question here as well. Great. So now on my machine, we see this little voting application. And in just a second, you should be able to do that as well. That's great. All right, so we have built an application that increments two numbers. But by attaching names to it, now we're having an argument. Funny how that works. Okay, so that's okay. But before we go further, I think at this point, it would be good for me to explain what is it that is actually happening here. Because we're going to add a lot more functionality. And before we do that, I don't want anybody to be confused. So if we look at this project that I have here, this is a completely normal standard Vite React app that's running in a browser. And all of the application is really doing is it makes an HTTP fetch to this deployed code that we have. And then when you click it, it makes another request with a method post. So we increment the number. And so the reason the number goes up, oh, 2000 times you've pressed 4000 times. Great. Somebody's having their fingers busy, or somebody wrote a little script in their browser to hammer the button. Please don't do that. But if you must, go ahead, I think we can scale for that.

So this is all that's all that's happening here. And then we render a button for each of the values that's coming in. So in this case, you know, two numbers, and you know, they get two buttons. But I'm also doing one more thing here, which is that I'm attempting to make a WebSocket connection to that same address. And if you don't know WebSockets, it's basically like imagine a tin can with a string in between them. You know, I hold one end of it, you hold one end of it, and then I can yell into it. And you know, I can send a message to you and back and forth. That over TCP is basically what WebSockets are.

So that allows us to build push based systems. So let's do that since our client is already prepared for it. So let me just go back to this contract here. And let's just do this. So if request headers, oh, thanks, Copilot. And then, oh, nice. Okay, so, so basically, what we're happening here is that the WebSocket constructor, the library that we're using is just a standard React wrapper over a WebSocket constructor. It sends this upgrade request. And in return, we give them, we hold one end of the tin can and we give the other end of the tin can to you. And then we can push messages to you.

And now, every time the messages, sort of the number is incremented, let me also then, oh, nice. We will, you know, loop over all of the currently connected WebSockets because we are, you know, having in-memory states. So we can basically just loop over them. And we can, you know, send the same state to you over that socket. And now I've saved. My end seems to be working. And so maybe in a second after your app refreshes, oh, we see the number go up live. And I don't know why that is changing the location between. Oh, I just made a bug. So that's basically the location through the first worker that connects you to this durable object. I am actually just made a bug here, which sends, you know, the location of each of those people to you. So we see where everybody is.

Everybody is in Frankfurt, Vienna, Sheldagol, etc. But it's moving a little fast. OK, so does everybody understand more or less what's happening here? We've created a little WebSocket server that is able to hold the connection between me and all of you. Currently, 45 people, but this would scale up to 32000 if we wanted to be really ambitious about it. And we're able to broadcast messages to all of you. So we've created a little collaborative application. So far so good. Nice.

OK, but one thing that you might have noticed, and this is easier for me to demonstrate when I go here on my localhost. When I make any code change and I save, the number goes back to zero. Boo. That's not very useful. How do we how will we ever know which one will win? Remix our next JS if we keep resetting the number every time we deploy code. Well, the reason why that happens is that, you know, we are using in-memory state here. If we just kind of like recap really quickly. You know, this is all of the state management that we have. We have a counter in memory. Kind of cool, but at the same time, we can't really guarantee that an object will stay in memory forever. That's not really realistic. What we can instead do is we can store that information on disk. And so when this object comes back alive, in this case, it keeps resetting when I deploy new code. We can use some kind of storage system. Now, I could go to PlanetScale and I could provision myself a, you know, like a SQL database and I could contact that. But I think that would kind of defeat the purpose here. And thankfully, DurableObjects, you know, is actually a storage product. So we have a built in key value store for every object. So what we can do here is we can say, let me just write this code. So let's see. Let's make some kind of function that loads the data from... That's cool.

So we have storage and that has, you know, a method called get, from which we can get data that we have previously put into it. In this case, we use the key value and we also type that the data is just the array of numbers. And then similarly, we can have a function to save that data into it at some point. So let's save the data every time we increment. And let's also load the data when we come to, when the object constructor runs, meaning that when we are loading the object after redeploy. And now we've basically implemented a sort of like storage backed system. So next time that this goes, redeploys, now it went to zero. But then if I know where to deploy again, hopefully let's say, save, I'll walk off and you keep pushing the button and hopefully when the redeploy happens, the number doesn't actually reset. All right. So this is interesting, right? Like if we recap again, we have just a normal JavaScript class that we can hold state in memory, we can basically, without serializing or without having to do any fancy things, we can basically store any JavaScript object or data structure without having to convert it to JSON. And CloudFly will make sure that that gets stored somewhere and then we can read it back when the object comes back. So we are essentially creating a fully functioning database-backed system in just a single JavaScript file that exists in our project right here among my frontend components. So that's pretty sweet, but it's not sweet enough. This I could have done very easily. I just made a node server and deployed that to fly.io or Heroku or something. But what's really extraordinarily interesting about this is that we can create any amount of these instantly, freely, anywhere in the world. So here, if we go back to where we create this object, we create a counter. And if I change this ID, for example, the counter two, and then in a few seconds now, the number will for you reset, because we're actually talking to a different object, right? Like we have provisioned a new database. There we go. New object, new database, new memory isolation layer where we can run this code. But you don't need to use static IDs. We can create objects dynamically. So for example, in this case, let's say that we send a path, you know, like a normal HTTP path into this request and we create an object for it for that. And to make that demo a little bit cooler, let me also change the question for every room so we kind of see what's happening here. So I've prepared a little code here. What is a good question from room ID? And this just basically picks like based on the room's ID, picks a random question from, you know, from a list of questions. And now, hopefully. All right. Semicolons or no semicolons. Do you see that on your phones also? Semicolons or no semicolons? Great.

Awesome. Let's make a different room. Let's say room Berlin. All right. Stickers or socks? This is, you know, classic conference question. Let's say a new room. Let's go to wall room. Office or remote? Ooh, remote. Employers take notes.

And then if we go back to the original room over here, you know. Oh, sorry. This is a bug on my client code. What was the one Berlin we had before? Great. So all of that data that we had in the previous room is still stored. You know, we are coming back to the thing. So we were essentially able to spin up an infinite amount of these rooms, you know, by just essentially coming up with a new ID. So that's pretty useful, I think.

One final thing I will do before we jump into the explanation part of this is I don't actually want to store this data forever. I think right now what we're doing is that for these rooms that we've created and any new rooms that we would create, we would basically keep this data around forever. But we don't actually need to invoke a durable object in order to run code there. We can also run code there in a scheduled manner. Durable objects has this thing called alarms where we can say, you know, this state storage set alarm. Let's say, OK, so for example, if my math is correct, in one week we want to run some code and then we'll make this function called alarm. There we go. And then this alarm will run basically one week from the most recent invocation of set alarm. And all we do here is we basically just delete the storage that we're no longer using because we don't need to store this data for longer than a week. If you wanted to, you know, save this somewhere else at this point, it would be a good time to send it to maybe some kind of like backup system somewhere like some long term storage R2 or S2 or S3 or something. But until then, the storage is all really storage that we need. In order to demonstrate this, I'm just going to shorten this time to a second just so that we don't have to wait here a week. And now the next time that we come to deploy, which will happen about now.

Now if I can all get you to co-operate and not push that button for one second. Then the next time we push this button, it will go to zero. Oh, you did it. I was expecting some more resistance from you there, but this is good. All right. So I think that's all code I wanted to show. And I think that went pretty well. Can we have a round of applause for me? Thank you. Thank you. And Co-Pilot was behaving really well today. So thanks Co-Pilot as well.

All right. Now the timer is... How are we for time? We're already over, are we? Great. Well, I want to say just two things, and then I will go to Q&A and I'll let you go. So building collaborative applications is cool and fun, but hopefully you can sort of squint a little bit and think about like, so if you can create essentially compute capacity instantly, you can co-locate storage with it. You can push data from web sockets. You can open TCP connections to databases. You can call these as webhook endpoints from external systems. It's really interesting how you can like really think about computation differently. Because one of the big mistakes I think we've made in AIRA, and I happened to see a tweet about it just today, so I will just let this tweet speak for it. But like, I think we as an industry have been trying to reinvent distributed computing basically by just creating these separating compute and state. We create these little pieces of computation and then we create these really complicated hell machines where not only do we need to use a database for what database is good for, which is storage, but we also need to use database for coordination. Like if we wanted to make a number that goes up and we didn't have the ability to use in-memory state, we would have to have like a Redis database where that number keeps being incremented. And then you get into all kinds of like database isolation level issues, which you just don't want to deal with. And then if you think about where do you want to run your serverless workloads? Like either you want to run them close to your user, which makes the initial hop really, really fast, or you want to run it near your database, which makes your data fetching hop, but you can't have both. So at some point you're essentially saying if your database lives in one place, you are going to build a slow application if it depends on data. So like co-locating your compute and data at the edge, I think is a really powerful new sort of paradigm that we as industry should really explore more. And thankfully, it's not just CloudFlare that is doing this. So Azure has this durable functions, which is very different from an API perspective, but basically does the same thing.

Deno has this KV, which is a slightly different product in the sense that instead of just keeping the data in one place, they actually seek to distribute that data eventually, in an eventually consistent way globally, which gives you different constraints. But in this case, we don't need that. And also just this week, fly.io released this paper or this blog post called Flame, which is a new type of architecture where you can just wrap any code in a function and they will make sure that it runs exactly the same way as we've just demonstrated. So I think this is kind of like a path that has helped me simplify the kind of programs that I write, because I don't need to think about provisioning databases, provisioning compute, provisioning, like running a Kubernetes cluster. I can essentially just have the compute that I need available to me when I need it.

And yeah, I think that's it. If we now go back to the sort of the slides, I think maybe we've explained the title, everything, everywhere all at once. So everything as in, you know, it's compute, storage, everything you need to build something. It's everywhere. It's globally distributed. And all at once, you know, you can basically spin up any amount of instances. This scales to millions very, very quickly. I haven't really touched the for not much money. We can talk about that in the QA or over time. But basically, these things are also kind of cheap to run.

And for movie buffs, I did want to have one last slide here, you know, just to have something to look at. That's it. That was my movie reference. Thank you very much. My name is Janne. And let's do a Q&A.

What the limitation of durable objects? 128 MB of RAM, which is not a huge amount. But then again, I haven't really have a lot of use case where you exceed that. Storage as much as you want, but each value can only be 128 kilobytes. So if you have like massive singular values, you might need to chunk those up a little bit. Yeah, those are the big ones.

How are they built? This is the really cool thing. And now we can get to talk about costs. So billing is basically on three factors. So it's the compute. So every time we send a message, you're billed just like a normal function in a worker stack. Then you pay for the storage, of course. So that's why we want to clear, for example, the storage of seven days not to pay for something that we need. And then the memory used of the object while it runs.

And then the memory used of the object while it runs. But the WebSocket API that Cloudflare offers here is really interesting because what they do is that they on the platform stack level network level, they hold on the WebSocket connections, but they can unload the object from memory when it's not actively processing connections. So if you have, for example, a room that stays active for a day, but it only gets a few messages during the day, you only really pay for the brief millisecond that the object is alive while it's processing that message. So it's actually really, really affordable as well.

Nice. How do they handle authorization? So you handle authorization, right? So it's HTTP endpoint. So, you know, like you can send over a token, you can authorize that with any third party service or your own auth endpoint. It's completely this thing that I have is not authorized, but you could add a auth sort of ping back for just a few lines of code.

Do you know of any commercial apps or I may just say large applications running on durable objects? So there are a lot of platforms actually built on durable objects. So for example, Reflect, which is a sort of like a multiplayer state sync sort of package. That is it's an amazing platform. It really solves like a lot of the common sort of state syncs of problems that users might have. And it's built on durable objects. And, you know, basically it's cheap as well because the billing model is the same as the durable object one. But these things are actually used a lot of in a back office capacity. So if you think of, for example, a session that lives for a while, let's say you're building Uber and you have a taxi booking app. You need a piece of compute that is able to not only, you know, communicate to multiple different back-end services, but hold its state for the duration of the ride and then no longer. Like this is a perfect use case for it. I'm not saying that Uber is using it, but these are the kinds of like, you know, industrial back-office use cases where I feel like the pitch of collaborative apps and whiteboards and games is kind of underselling it. You can really build serious production scale things. And I will say this one thing. I don't work at Cloudflare. I am not trying to sell this shit in particular. You can also use the Azure one or you can use, you know, whatever. But the good thing about Cloudflare is that if you work in a decent sized company, you already have a billing relationship with them because you use it for, you know, DNS and caching and proxy and all of that. So it's not actually a hard sell to get this into into corporates as well. And if you're a startup, they have this program that you get it for free for a year. So. And I suppose the other interesting thing is the Uber example, I think, is a fantastic use case for when you would take part of that state and store it permanently. You don't need all of it.

You don't need to be storing every step of the synchronization. Just these key points, we store that at the end. Exactly. And yeah, exactly. Yeah. And then you just send the metadata over that you needed to. But like, Uber now has this feature where you can, for example, share the ride with your friend, you know, safety feature. Right. So imagine building that, you know, without something like this, you need to provision a massive amount of WebSocket servers. You need to create this sort of like abstraction, like socket IO kind of thing. It gets really, really messy. But here, like making pushing into one client is as easy as pushing into all registered clients. So you don't really have to do any software engineering as such. Nice.