P2P React: Local-First State, Shared Truth

First, I want you to close your eyes and imagine a user in your app. Can you believe it? It happened all this time building. Suddenly they showed up. This is the dream. This is the dream you spent so long. Now they're finally here. The graphs are graphing. The users are using. Are they users? Are they bots? Who even knows? Who cares? It's happening. The dream is real. You finally put IndieHacker on your Twitter profile and expect to wake up to peer levels of wealth and fame.

And then, the next day, you do wake up and suddenly the users are there, plenty of users, you build an app, it's sort of maybe a chat thing. Maybe it's like sharing media files. There's a lot of media storage. The bills start showing up as well. You get a bill from your hosting provider called Triangle. It's like 20k out of nowhere. Like, what? How? Was it just images? Then your database provider, SuperMaze, another bill, Dork, your auth service thing charges by the seat. Do bots need seats? Do bots have bots? Who knows? But all these users using your app, this is the dream. Shouldn't this be a good thing?

Suddenly it starts weighing on you. How am I going to pay for this traffic and all this usage? And this is where peer-to-peer can come in. Because what if all those users were actually a good thing that would help you? But before that, the problem here is that everything runs through the servers. But what even is a server? The traditional sense is more or less three things. Let's just say it is three things at least. So there's all the post requests coming in from the users. They go to the server. The server validates them, makes sure everything's all right, and it stores the data. And then of course on the other end, there's all the get requests, people reading the data. So it stores the data, makes sure the data is valid, and then it delivers the data, I guess. It holds the truth and the data.

But it's fine. We can trust this. I mean, at least we do. We do trust this. Because yeah, we do. But also this becomes a single point of failure. If the server has a bad day, this happy server guy goes down, GitHub. What happens then? Everything goes down. Nothing works. So but then you remember in your app building thing here that you once went to a conference and there was this handsome guy, smart guy, brilliant guy, talking about peer-to-peer. And then the next day, there was another guy also talking about peer-to-peer. And he mentioned how this would solve some of these problems.

So peer-to-peer, just the same things. You can't remove these things. You just redistribute it. And we do this with just plain cryptography. So the first thing we have is what even is a user in this space? In a traditional sense, it's the table row, HTML table row on doc, your auth provider, or it's a row in your database. What's cracking? Is it my beard? A row in a database. So that's the traditional user that we think. But in reality, a user in this sense is like the keys. The keys that you use to sign the things that go into this shared system. So before you have a user, you have a set of keys.

And then we can make it so that we make up this thing called a profile. We're used to having profiles, it's like an avatar, and a description, and a username or whatever. And we can make it so that these keys are the only keys that can control this profile in a sense that the use becomes that profile, and the keys determine who can modify it. And you can have multiple keys controlling the same profile, like your mobile and your desktop. Anyway, we have users again now. And then all the users, of course, gather around things. So in the chat app example is probably the room, for example. We're building a chat app at Tether called Keet.

You should use it. I'll bring up a QR code later. In a documents app, it would be the document. In imagine a DJ app, it would be the DJ set. Whatever it is, you probably always have something to gather around. This is the space in this sense. And we can make it so that only users that are invited to a given space will have access to it. And we can even make it so that there's roles and permissions and all that. You have to be a moderator to, I don't know, you define it. Key is that you can sort of build any app that you can imagine, almost, in this shape where it's distributed instead of server-based.

And so what happens is that all users... Is it me? Is the cable on the ground? No. No? It's just the echoes of UNICEF audio effects. Anyway. So we have users participating by sending messages. Everything that happens in our system we've determined is like messages. We know this from, I don't know, Redux or something. You have the actions and then they go through a reducer or anything. And then we have to say, coming out at the other end.

We can do the same thing but distribute it. So every user uses their keys to sign the messages and we can encrypt it all and everything. And then as long as we share the same truth... How do we get to that truth? We have the logs of all the user messages. All the things that have happened in our app is like a giant log of things. And then if we want to do the same function here called apply, which is sort of... If this smells a little bit like blockchain, it's because it is kind of like blockchain. If you run the same messages through the same function, you can do it however many times you want. As long as it's deterministic, you'll get the same results.

And so if we run the same messages through the same function, we'll get the same result. We can trust each other as long as we get to the same result. Yes. That, of course, means it's deterministic. We can't use dates. For example, in the apply function, we can't use randomness and all that. Yes. And this is, of course, stuff that you don't probably think about so much when you're building against servers. But it becomes then important when you do it for peer-to-peer. But in a sense... What was I thinking about? Yeah. So, I think I want to go back actually to this one. So just a point that I maybe forgot to make, but it's your profile. So you can hold the profile. In a sense, anyone can hold the data. It doesn't matter who holds the data, because it's encrypted likely, and you hold the keys to it. So say my profile, or my data, or my notes are in a tiny safe. I can give you the safe, and I can give another one a cup. It doesn't match completely to the physical realm, but still, if I'm the only one with keys to the safe, it doesn't matter really who carries it. And this is the same with here, with the profile. My profile can be shared between anyone.

We have stuff even in Keap, like an inbox. So when you get a friend request, a user puts a friend request in your inbox, and then the inbox can live wherever, but you're the only one with access to it, was the point that I think I forgot to make. And all of this exists already, is what we are using to keep the chat app. It's kind of done, it kind of works. Keap does everything that you would want from a Messenger app. And what we're now starting to slowly go into is to take all the learnings that we have learned from building Keap, and then putting them into sort of a framework-ish SDK kind of thing, to make it easy for anyone to build peer-to-peer driven apps. So you don't have to think too much about the apply methods, apply functions, and everything, but give you some good conventions for building stuff, so you can stand on the shoulders of tether, I guess. So this isn't final, this isn't what it's necessarily going to look like, but this is sort of the direction that I want to go in, where you have a space manager, so the space in this case is the thing that you gather around, the room for the chat app, the document for the documents app.

So this isn't final, this isn't what it's necessarily going to look like, but this is sort of the direction that I want to go in, where you have a space manager, so the space in this case is the thing that you gather around, the room for the chat app, the document for the documents app. And you can add commands and you can add collections and all that. So for a chat app, for example, we define a schema for what things look like, it's a chat, it has collections, one of them are messages, they have feels like anything you've seen before, and there's commands as well. So it sort of looks like a Redux thing, we have actions and they resolve to the same state. You could also, so this is key to the chat app, but you could also imagine, in this case I'll call it bunk docs, this is a Google docs competitor, where we have roles too, so an owner can do anything, an editor can do these things. I've toyed with a Yjs, do you know Yjs? It's like a multiplayer library thing, so you can use tip tap, but have multiple editors on the same thing, so I've written an adapter layer that puts Yjs onto this, and of course a document has metadata, we can add that, and we have commands again. And this is a demo of a test app of bunk docs running on the simulator and electron here. You get the drift. But just to say, anything's kind of possible, it's just a matter of structuring it in this special way. Do we see it to the end, I guess? Alright, that's enough. We get a little bit of tuna action. And emojis don't work.

Okay, well, so this is imagining Dropbox, the good parts, I guess. How it used to be, you just share a folder and anything you put in that folder you can have on all your machines and you can invite people to it. It's the same kind of thing. The space here is the individual folder, then use is a keyword for, sort of a middleware layer where you can use all the building blocks that we have built for our app as well. So we have support for like blob storage, and in this case, drive, which is hyper-drive, one of our modules, where you can get a drive and it's automatically shared between the participants. So we have files, we have commands, so on and so on. Here's a demo again of Dropbunk running on my machine and VPS. So we edit in the folder, we get an invite, we press the trackpad too hard, and then you can probably imagine what happens. It almost sounds like fun when you don't do it, when you show it.

OK, yeah, we join it, we go to the thing, click randomly. What's going on? Keys? All right, yeah, what happens next is I create files and they sync, you can imagine. You know files. Yes. So I'm calling this peer SDK, we don't know actually what to call it yet, because it isn't really a thing yet, but it's an effort that we want to do. Peer in this case is what you heard Dave maybe talk about yesterday, which is sort of the distribution layer and the runtime for much of this. Even down to deploying your app, it happens on peer-to-peer, meaning that it's unrestricted by firewalls and all that. We have some hole-punching technology, it's called, that finds a way through all of the common routers and everything. So that's really what peer-to-peer shines, when there's a case for if the Internet goes down, but people are still together, they turn on Bluetooth, they can still share the messages on Bluetooth, because this is the transport layer, all of the math behind it still works.

Just to end it, peer is the platform to build this on, it exists, you can go read the docs, going through a bit of a transition, but it's there, it exists, it works. Tether is the one paying the bills, because we like this idea that apps should be able to do this and work like this, and be unstoppable, is the thing I think. If you want to follow me on Mecha, if you want to try Keet, you should. That's the code. And then that's the actual... Thank you. Thank you, everyone. Great talk, I love some of the jokes in it as well. And there was a lot of things that I think it really made me think about, about use cases and about some of the engineering challenges that I think this could breed if I was building something with it. And I think a lot of other people in the chat were also thinking the same.

One of the first things is oftentimes we're thinking about performance. Sorry, we're thinking about where data lives and making sure it stays safe. One of the lines you said is, it doesn't matter where a place of data lives, but isn't there a risk of data getting lost if it lives on the hardware of a peer? Maybe that hardware gets compromised, someone drops their laptop and a bus runs over it. Like, what happens then? Yeah, so you'd have to... Like for Keet, for example, you can have to use the same account on different devices. So that's just how we structured the thing. And then, I mean, you can work without servers, but we do add some to be just like the relays between stuff. And like when you send a message, you don't want it so that your mom also has to have her phone turned on for her to receive the message. So we do have some servers in between to just make it work as you expect.

But what's important is that the server doesn't hold... It doesn't have a role in a database. It just holds some encrypted data that it has no idea what it is. Just that it has to go there, I guess. No, not even the metadata of that, but just that it exists. And so that actually triggers a follow-up and this is from me, which means like when you were giving your talk, I was thinking of every single peer being a user. But in this case, a peer could be a server that's just standing in, helping the whole network as well. Yeah, if you have a fairly active app, you would want a server to be a relay just for like the usability of things. But like I said before, it's important to know that it isn't a server holding a cache, like unencrypted or holding a database or anything. It's just like holding encrypted data that it doesn't even know what it is. It's just like in between for usability.

That makes sense. And then this leads to a follow-up question, when we talked about the peers, there's so many different devices as well. Does P2P have a negative effect on power consumption when a device becomes both the client and the server? And that's a question from Lars, especially when you think like you showed an example of having React Native, sorry, having an app, I think it was powered by React Native doing this. Does it have an effect on power consumption or is it just pretty much the same as data transfer from anywhere? I mean, if it was running in the background always, of course, there would be an effect of that, but it doesn't. So does the same rules apply like everyone else on the iOS devices, I guess. So in that case, no, I don't think there's any, there's no worries there, I guess. I don't think so. I don't know of any precautions around battery use. That makes sense, that makes sense.

And you talked about relay service as well, and I think relay service kind of answers this next question, but I think I'm still going to ask it, which is from Alex Gale. What if all the users are offline, where does the data live then? Yeah, so when you write a message, if you're offline, it just stays on your device. Still goes through the same checks so we can make sure, if we require a field to be text, we can still make sure it stays before we save it locally, but it stays on your device. When you then connect, if you have the relay peers in your network, it's probably going to go there. And otherwise it just sits there until another peer shows up and then it transfers, like save from both of them and syncs it. So from what I'm hearing is people have to think about their use cases. What are they building for? Do they need, how many relay servers are they going to need? Do they want to make it enabled that someone can have the disconnection? And it all depends on the use case and what they're doing that for in that sense. Yeah, I guess, imagine a node server, if it was just you, you wouldn't need any relays. Perhaps you would want, like if you wanted to, you could have your own server be a client or a peer as well and then it would sync to the server and all. But all of this is local first, which was all of the hype a few years ago. And all of this is local first by default, because that's just how it works. Like you save locally, then it syncs with all there. That makes sense. I think because for a lot of us, we spend so much time in the server client paradigm, seeing a whole different model, a whole different way of thinking. There are so many questions that server client has just been designed to answer and we're trying to use the same model to answer questions on peer to peer when it's very, very different.

There are so many questions that server client has just been designed to answer and we're trying to use the same model to answer questions on peer to peer when it's very, very different.

The next question is quite specific, it's about glob storage. How does that work? Do you have limits on file size? And is there a limit on the P2P network throughputs? So across the peer to peer network, and maybe if I'm building an app that's on top of your SDK or maybe using that tool, are there things that I should be thinking about, about throughput and about maybe configuring some of these?

It's really as well, I like the way you thought about, we'll get you miked up in a second, I like the way that you explain it, and you talk about the fact that as the network gets bigger, a lot of these questions kind of get answered themselves as well. Because if you've got like a million, 10 million peers, the chance is that enough are going to be suddenly turned off or suddenly disappear. So a fetching network is unlikely. And I think that's where like having smart relay service strategy and then understanding how to do that could be very, very interesting.

And it's the same, the more seeders, the greater the bandwidth. So imagine people watching the World Cup over peer to peer or whatever. The combined bandwidth of all those people is gigantic, right? And so in the server sense, we're all restrained about the speed of the transmission to that server, to and from that server. But in this case, it's just the more, the merrier. The more people joining in, the faster the stream will actually be.

Awesome, no, that makes sense. I like the way you thought about. I like the way that you explain it, and you talk about the fact that as the network gets bigger, a lot of these questions kind of get answered themselves as well. Because if you've got like a million, 10 million peers, the chance is that enough are going to be suddenly turned off or suddenly disappear. So a fetching network is unlikely. And I think that's where like having smart relay service strategy and then understanding how to do that could be very, very interesting.

But it's like, yeah, yeah, yeah, yeah. Go ahead. I'd love to hear you deep dive in. Yeah, it's just like, I don't want to put too much on the relay service. It's like an implementation detail, whether you want that or not in your app. So it's just like an optional thing you can do if you want the always on, always transmitting thing. You could just as well choose not to use a relay server. That makes sense. That makes sense. And, I mean, for a lot of JavaScript and TypeScript developers, we always cared about security, but this year we've been caring about security even more.

So there's no server carrying metadata like there is with Signal, for example. They still know who you send the messages to and when and stuff like that. They don't know the content, but still there's something living on their servers. And that's the case for all the messaging apps, but Keed. We proudly don't want your phone number when you sign up, for example. You can just start using Keed and then go. So first of all, it solves a lot of security questions from like the other angle. And then the ones that are left are like still the social engineering things, like someone can still trick you into clicking the wrong link or stuff like that.

Like we're in a room with 800 people or something. Someone's always going to be online, carrying the data for everyone else. Yeah. Now that makes sense. That makes sense. We've got another one coming as well, which is, how do client devices ensure that they're in sync with each other? Like you gave us that nice demo of, it was like a Google Docs-esque type typing, and you saw the cursors and some rules are the same. But if you get a lot of clients, and I'm sure we've all been in there, like it's so funny, one of my favorite things that my team does is, if we're online and everyone's in the same doc, we have a cursor parking space for you to just pick your cursor when you're not moving around or when it gets so busy.

Right. Yeah. Dave nods. Yes. Is it odd? Okay. Yeah. Yes. I mean, you could construct a system that would reuse everyone's devices as relays, but not by default, I guess. But imagine like you're in a room with 800 people or something. Someone's always going to be online, carrying the data for everyone else. So yeah. Now that makes sense. That makes sense.

We've got another one coming as well, which is, how do client devices ensure that they're in sync with each other? Like you gave us that nice demo of, it was like a Google Docs-esque type typing, and you saw the cursors and some rules are the same. But if you get a lot of clients, and I'm sure we've all been in there, like it's so funny, one of my favorite things that my team does is, if we're online and everyone's in the same doc, we have a cursor parking space for you to just pick your cursor when you're not moving around or when it gets so busy.

Just a little heads up to anyone else. It was like it could get suspicious. Like, why is there a cursor on that word? Why is he... Yeah, exactly. Exactly. But yeah, so how do you ensure that these clients are in sync with each other? Are there any specific rules or ways to make sure that the history, especially with changes and diffs, doesn't get too messy? Yeah, so by default, everything's a write ahead log, it's called. It's like everything that happens goes into the same list of messages. And when we go through them, we end at the same point.

So for the docs example, for example, you just use an algorithm that's based around not the content of the document, but the changes in the document. So it's almost like nested change algorithms, almost in that case. And I don't know actually what Yjs uses internally, if it's operational transformation or what it's called, which is the one that I think Google based the original Google docs on. Anyway, it's just a matter of not storing the text, but storing the changes that lead to the text. Storing the changes. Well, it has been a pleasure. And also I want to say a big thank you and a huge shout out. You handled the speaker problems really, really well. Thanks so much. Can we give him one more round of applause? Thanks.