No Servers, No Cloud, No Masters: Make P2P Apps

For the past four years or so, we've been working on making something that works point-to-point, device-to-device, with no infrastructure at all. And we've made a platform, we've made an app, and more apps on top of it, and we've been dog-fooding it, we've been iterating, so much iterating.

But we've gone to a point where we've distilled down what it is that we need to do to the smallest possible touch points, in order to enable any JavaScript application to be peer-to-peer, and to be deployed peer-to-peer.

So what that means is you don't need servers. You think you do, because of the availability problem, but that's a solvable problem. You don't need the cloud. You think you do. There's been immeasurable budget spent on making you think that you do. And you don't need masters. All you need is some ability to make some front-end-y stuff, and the ability to step into a different paradigm, a different way of thinking about technology, and how technology can interact, and you too can make peer-to-peer apps, with no infrastructure.

Where the data lives on the user devices, which means you can't exploit the users by their data any more. But, while that does create a fire hose of money, no question, you don't need to direct that fire hose of money into OPEX paying for infrastructure. So if you take both sides of the equation, it turns out you can actually monetize apps in different ways. There's not just one business model.

So like I say, we've been refining this down for four years now. The whole thing has been in the works for about 15 years. But we've been working on it intensely for the last four years. Tether partnered with us, Tether is a 100 billion plus cap stablecoin that is investing very heavily in decentralised technology and technology that empowers users. And it's really just a pragmatic thing. No one else is taking control and power away from users. We want to give it back. That's an excellent USP.

So here's the distillation. Use a module. Deploy peer-to-peer. That's it. No cloud needed. Just device-to-device. So the app that we've built on this is called Keet. It's like meat with a K or key with an ET and not a Y. Or like Parakeet. Or actually the thing we've built is called Pear and the app we've built is called Keet. So if you say Pear and Keet a lot it's like Pear and Keet, Pear and Keet, Pear and Keet. When the Twitter bird died, Keet was born, literally around the same time. We are the reincarnation of the bird.

Recently we've been getting a big uptick in users because the secret source that allows us to allow devices to connect to each other is something called hole punching. So what hole punching is is where it's a statistically guaranteed algorithm for punching through nat traversal policies on home routers that stop users connecting to each other. It also punches through the firewall of China. So we have about a million new users per month at the moment on Keet and growing. Really, really large user bases in countries where the internet's being shut down a lot. And users beyond that as well. You don't need a phone number to use Keet Messenger. For every other app you need to give them your phone number. We don't want to know your phone number.

We don't want to know your data. We don't want to know. We just want you to use the app and benefit from it. You don't need to provide an email, nothing like that.

So this is built using the PAIRS stack. Which stands for peer to peer embedded application releases stack. If there's no cloud, there's no fees. If there's no cloud, there's no outages. This is a more robust way to build technology. And it's a way to build with lower cost. And it's a way to build with different incentives.

If you're on Keet already or if you'd like to get Keet, Keet.io. This QR code gives you access to the PAIRS development group. Which isn't publicly online or anything. This is just for developers. It's currently about 7,000 plus people in there. Members. Fairly active crowd. A lot of people building a lot of different apps on this. A lot of experimentation, a lot of research. And then sometimes things emerge out where it's like, yo, that's a full app you've built right there. And there's many in the community. And more and more coming all of the time. This QR code will be on the slides down in the bottom corner so you can keep on getting Keet and scan it. So here's the stack. We have a module called PAIR Runtime. You NPM install that into, say, an Electron app. You give it a key, actually a link with a key in it. So it's like a PAIR link and then just a string. You put that in the upgrade field of your package JSON.

And then it listens over a DHT using something called HyperSWARM on that topic for update events. And you use PAIR, the command line interface, to then deploy to your application. And when you deploy, it creates an update event among all of the peers. And then the peers receive that update event and there's a swap on disk where they press a button. It calls apply update, it swaps on disk, and they have the update. It's really as simple as that.

There's no servers in between. On top of that, there's BEAR. That's right, it sounds like pear. It starts with a plosive, with a B, and the logo for it is a bear, but it's spelled B-A-R-E. And the reason it's spelled B-A-R-E is because it's very bare. There's no standard library like in Node. It's a Node-like runtime that runs on mobile, that runs on desktop, that runs on embedded, that's specifically built for peer-to-peer applications as a first-class citizen. So it had support for UDP and UDP+, which we use for the transport, before it had support for TCP and HTTP.

It has that now too. But support for any of these things comes through as a module. So if you need to use the file system in Node, you just use the FS module. But in BEAR, you install bear-fs and then you use it. So there's sort of a natural sandboxing thing here too. Like, if you don't need it, you don't install it, and then there's no chance of it being exploited in some kind of supply chain attack if the primitives aren't even there to do it, right? There's more sandboxing for us to do on that, but I like that it's naturally as minimal as possible. There's not even a process object in BEAR. You have to install that separately too.

Then there's the BEAR module ecosystem, which is all of those things like bear-fs and bear-udp or whatever, but we also have things like bear-ffmpeg. So there's add-ons that go beyond what the Node core library would provide also. The BEAR module ecosystem is basically like the core library for Node, but more.

And then there's the hyper ecosystem, which is the peer-to-peer capabilities. This is things like hyper-swarm, which is for connecting two peers over a distributed hash table via a topic link. There's hyper-core, which is the persistence layer, and there's a bunch of things that can be built on top of that, like hyper-drive, which is a peer-to-peer file system, hyper-B, which is a peer-to-peer key-value store, hyper-DB, which is a peer-to-peer database, autobase, which is a multi-writer peer-to-peer linearization library, which is what we use for Keet because one of the big problems for messaging or anything that's eventually consistent, which all peer-to-peer things are, has this problem of conflict resolution.

You have an apply function, and your apply function determines how things should resolve. So the module is for desktop, mobile, and terminal applications. You just install it into your project. You receive and apply over-the-air updates. It also comes with peer-to-peer storage management because if you're doing something peer-to-peer, you need to use disk. And it comes with a method called peer.run. And what that does is it starts a bear runtime worker.

And what that does is it starts a bear runtime worker. So you don't need to install bear separately on your system or anything like that. It comes as a native dependency, but then it starts as its own runtime. The reason we do that is because, architecturally, we found that we can write the exact same back-end, what we call a back-end, but it's actually local. We call it a local back-end. So it's on the mobile device and on the desktop device. You have the exact same back-end code, which runs in this worker, and then you just talk to that over IPC.

So then you can have a React Native app that's talking to that on mobile, and you can have an Electron app that's talking to that on the desktop. But you have all of your peer-to-peer and business logic lives in your back-end, and then all of your view materialization logic lives where it should in the view. So it means it's really easy also to add new things on top. So we could add a terminal app that's IRC-style keyed or something like that, very easily, by just reusing the same back-end. So what the PeerRuntime module gives you is the peer-to-peer updates, but also an architectural set of very simple primitives. That's literally all of the primitives to make peer-to-peer apps across platforms and devices.

The CLI, you can install it right now with npx pair. You don't need Node to run pair. It's its own build of Bear. But we use npm because everyone's got npm, right? Just to get it bootstrapped. So you've got your PeerRuntime module in an app, and then you basically use these pair commands to deploy it. This is literally all of the commands that you need to have a full end-to-end deployment flow with a staging environment, with a pre-release environment, and with a production environment. Pair touch creates a pair link. That creates a key. That's a topic on the DHT that I was talking about. Pair seed exposes that link. It's just the same as with BitTorrent. If you're seeding a file and someone else has taken that file, it's literally the same. It's next generation of that technology called Cademlia that this is all based on. Pair build creates a multi-arc deployment folder. So you make your build on Windows. You make your build on Mac. You make your build on Linux.

And then pair build brings them all together in this single format. Pair stage, you stage your build folder where it basically synchronizes from disk into a hyper-core structure, which is seeded by that pair link. Once you're doing that, immediately you can now take that application, put it on another machine, run it, stage again with an update, and your update flow works. That's your internal staging environment. You can just stage a key any time, send it to another developer and say, hey, can you check this out? And you can all have in one team a single stage key that you're like, this is the staging environment key.

Once you're happy with the way things are on the staging environment, you go to prerelease. So pair provision creates a prerelease link. All that's doing is stripping out intermediate operations. A hyper-core is an append-only log. So everything that happens, even deletes, is appended to the log. What a provision does is it just takes a snapshot of the state of the data and block syncs it across. So then you have a much smaller deploy, which is going to go onto users' machines. So that's a prerelease key. That's when you're getting really serious. You go to QA. You make sure it's really, really good, and once everyone's happy with that, then you go to pair multi-sig.

And what multi-sig does is you have, say, we've got five people that are stakeholders of an application, and you need three out of the five people to say, yes, I'm going to put my career on the line and say this application is ready to go, and I'm going to sign it with my public key to say I have verified that this application is ready to go. You gather the signatures, and then you do a multi-sig commit. Boom, you've gone to production. That's the entire end-to-end flow that you need. That's what we use for deploying Keats. And as I said, like, we've got a million users coming in a month. It's getting very stable at this point. There's always, like, little UX bugs and stuff, but, like, it's getting very, very firm at this point. So we're very happy with this, the process that has shaken out from us actually doing this in practice over the years.

So this is the dual-distribution model. As you can see, you have vendor-signed distributables. What that means is if you're on Mac or Windows, you need an Apple key, you need a Microsoft key, license key, to actually build applications that you can then give to users, right? That's non-negotiable. So you can do that, and then you can put it out in the app stores. You can put it on your website for download. But then once the user is running the application, they're connected to your pair seed.

And then any updates that you make go through the peer-to-peer deployment flow and then go over to your users over-the-air peer-to-peer updates. Also coming soon is a pair install command, so you can just install peer-to-peer, as well. But this means that, like, as far as your user's concerned, they don't care. Like, they're just installing an app as any other app. But for you, you've eliminated all of your infrastructure costs.

I've talked about Bear quite a lot already. I think I've covered a lot of this. You can play around with it with npmig-bear, but that's just for, like, the REPL and, like, trying stuff out, running tests and things like that. The whole point of Bear is for it to be embedded in something else, like in the Electron app or on mobile or IoT stuff. Here's a bunch of the Bear ecosystem modules, just to give you an idea of where we are. The green ones are supposed to be, like, this is, like, your Node core equivalent and then some more stuff beyond that, as well. This is the Hyper ecosystem. So you have, as I've mentioned already, Hypercore. You have HyperDHT, which is direct peer-to-peer connections, and then you have HyperSWARM, which is topic discovery over the DHT. Data structures I've mentioned. Hyper B is the key value store. HyperDrive is the file system. Autobase is multi-writer linearization. And then these utility modules, core storage for managing multiple Hypercores. Local drive is the file system equivalent to HyperDrive. So you can mirror from a peer-to-peer file system to a local file system and vice versa, which is also what MirrorDrive provides. Here's the deployment layers. So you version your app. Like, you bump the package version just the way you would with a module or anything, same there.

And then you make a new build, and then you use pair-build to create the deployment directory. And then you stage that deployment directory, and that's your staging environment that we talked about earlier, which is this middle tier here. Once you're ready, you go to prerelease. That's provision, and then once that's ready, you go to production, and that's multi-sig. So we find this to be very effective.

Then there's the release cycle. Again, you version, you make the distributables, you build a deployment folder, you stage, you provision, repair the request, get the signatures, verify the signatures, commit, go to production. The reason I keep saying the same thing is because once you get into the pattern, it's like, oh, this is really easy. Once you set it up, it's like, ba-ba-ba-ba-ba, ba-ba-ba-ba-ba, and it's like a flow state, and we've been in it for a while. And it feels great.

Don't have to deal with servers being down, staging servers down, or someone's pushed to it, and now it's broken and stuff. You don't have to deal with any of that. If you'd like to play around with this stuff, we've got some boilerplates. The Hello Power Electron boilerplate shows you... You clone it. It's a repo. You just clone it, and then you can just follow the instructions and quickly get a PIDs-peer-enabled Electron application set up straight out of the gate. If you already have an Electron application, take a look at this, and you can see how to put it in there. It's like a couple of steps. Get it in there, and you've got PIDs-peer updates. We've just recently... There's a blog post on pairs.com, forward slash news, about this. We've just released the Hello Pair Bear boilerplate, and what that does is it's peer-to-peer over-their updates for terminal program processes. So like a CLI, a REPL, a daemon service. All of these things can be written as standalone executables. No peer dependencies, no node needed, nothing else needed that you can just give to your user through website download, app get sources, brew, whatever you want. And then from there, they've got instant peer-to-peer updates. They never have to update again through another channel, another mechanism. And what we also have coming soon, we have this working internally. It's just a case of applying rigor and making sure it's good and checking it all works out.

React Native boilerplate, peer-to-peer over-the-air updates for mobile applications. So they get their app from the App Store. They put it on their phone. You do a release to the mobile app, you know that can take a couple of weeks to go out and stuff, right, because it's got a whole review process. That goes out onto the App Store, and you've got that update there. But at the same time, you can peer-to-peer over-the-air update your users to the latest thing, so they don't have to go through that App Store flow. And at that point, especially when you're using a shared backend, it's fantastic because you've got one backend, and then you just over-the-air deploy that same backend to your desktop application, your mobile application, and if you want, your terminal application as well.

Other apps we built in production, we've got PearPass. PearPass is a key password manager. So if you're sick and tired of keyword password managers leaking clear text passwords, try PearPass, because PearPass can't leak your passwords unless you leak them yourself. The data is on your device. Another one in production is QVAC. This is local LLMs with a focus for decentralized usage. So it's more of an SDK, and it's something that we're starting to integrate into Keat as well for things like translations and stuff. But the point is, our whole ethos is, you should own your data, the LLMs should work for you, not for someone else. That's really where it's at. So QVAC is doing some amazing work in making high-quality local models that can be models of higher parameters on-prem somewhere. Check it out.

And then, of course, there's Keat, which I believe I've mentioned. Really, really good app. One other thing I'll say about Keat is, when you go on a video call in Keat and you have a video call with someone, you're kind of like, wow, I didn't realize I've been living my life at such low resolution. It's sort of like when you have Italian food for the first time in Italy, you know? But if you have a server and you have people connecting for a video call through that server, what are you going to do? You're going to start constraining your costs by constraining your resources, by compressing the video feed. If you have a video call on Keat and you compare it to a video call on Google or on anything else, it's night and day, because if there's no server, there's no need for compression. And that's a thinker, right? Like, what else is like that? Join us on Keat in the Pear Dev Group. Pears.com forward slash news, check things out. Docs.pears.com, everything you need to know. We are hiring. So if you're into this, if you're into this paradigm, if you're into this kind of area, talking about P2P, working on P2P stuff, great. Come talk to us.

Come talk to us. Come talk to me. Connect with me on Keat. But also, if you're like traditional stack, no senior software architects kind of style, we have projects within Tether that we're helping out with as well. So we've got a whole range of needs for really good talent to come and join us. Frontenders as well. We have a lot of frontenders working on Keat because it's Electron, right? So that's web API stuff. Mobile's React Native. And UX. Look, if you work, if you're in this room, we'll hire you, okay? Thank you.

A bunch of questions here. Let's see how many we get through. One thing that is up on the top that has been... By the way, as you're leaving, if you could just keep it down a little bit, that would be nice. Yeah. Thank you. Thank you. Yeah. So there's a risk that this would become maybe a popular tool for bad actors in the world. So with less centralized systems, then people could run amok. Right. This was one of the first things that we actually discussed when we came up with the concept. Basically, it's about tradeoffs. And if you focus in on one type of bad actor, you can forget about the other types of bad actors.

The centralized space has been captured by profit-based interests. When you're operating on a profit-based model, as everyone knows in this room, then you make decisions sometimes that isn't in your user's best interest. And the more you monopolize, the more you centralize, the more the power converges on this single point, the more chance there is for mass-scale abuse. When it comes to predators online, I don't think that mass surveillance is actually the solution. I think that you should be putting that money instead. All the money we're putting into mass surveillance and AI surveillance and all that kind of stuff could be going to cyber detectives.

And ultimately, I think the way that predators are generally taken down is actually people infiltrating and taking them down. And there's nothing about peer-to-peer that stops that. What peer-to-peer stops is the situation where you have an ultra-power-leverage situation where an organization or a government gets to say what's okay and what's not okay and then monitor you about that. Now, you might be like, I've got nothing to hide. But you don't. But if the government changes in some way or that power changes to make something that you are, in your identity, illegal — we're seeing that all over the world. But in the West, more so than ever, it's in our faces. You can be made illegal in a way. Do you really want people who are up to that kind of stuff to have that kind of power leverage, to have that kind of control over our lives? These are questions that we should ask, for sure. I think it's a worthwhile tradeoff.

One specific thing about security, just staying on that topic, about the supply chain attacks, because you were mentioning these bear modules, which is kind of an interesting thing, is that you have a very small core and then everything is installed on top of it. But I guess the question is saying the Keet, Pear, Bear, all of these are still vulnerable to normal supply chain attacks and through updates through peer-to-peer. Is that possible? Absolutely. The supply chain attack problem is universal. Everyone is vulnerable to the supply chain attack problem, not just in programming, but in food, in absolutely everything. We control the supply chain for the bear modules. It's open source, but we control it.

The big thing is provenance. We actually have some interesting stuff coming down the pipeline also to do with distributed package registries and stuff like that. Because it's all cryptographically based, you can create a system that's peer-to-peer with much better provenance primitives than with a centralized system. Because a centralized system is a black box. Once it goes in the black box, you don't know. But if it's open source and decentralized, you do know. So that can reduce the attack surface and we're moving towards that. That's why we do multi-sig as well. When you run the pair stage command, or the other ones, it gives you a diff of the files that are changing. And it's up to you to verify every line of that diff and be sure. When you're signing on a multi-sig, you're putting your career on the line saying, I'm sure that there's no supply chain attack here. And that's how it's distributed through the peer-to-peer network, with signatures on each update that goes out.

I get that part, but once you've downloaded it, how does your local thing that's running on your desktop or on your mobile device, how does it then get updates? It turns it into a peer. And that peer connects with another peer that's seeding. Okay, so your peers or whoever is creating software. We could get other peers to seed too. Got it, got it.

Okay, yeah, I think I got that now. Yeah, one thing about the peer-to-peer via a browser, is there some way to do this or? I don't think that's ever gonna be a situation that's gonna happen. Like technically, there's no reason why not, but look, the browsers are controlled by specs, the specs, people that work on the specs, a lot of them are hired by companies that have interests. And so things get implemented and don't get implemented based on those interests and agendas. It's just a natural phenomenon. It's a naturally occurring thing.

The browser doesn't have the primitives. We use UDP with a little layer on top of it for the transport and the browser's just never gonna have UDP. It's just not gonna happen. So we think apps are the way to go. There's been people in the community making peer-to-peer browsers and different things like that, all very interesting stuff. But we do like the web technologies because it's so available and so many people know it. So building on things like Electron is where the sweet spot is for us. Yeah, okay.