Building End-to-End Encrypted Apps (Web & React Native)

So, hello, everybody. Let's get started. Building end-to-end encrypted apps. What do I mean by that? What do I understand as an end-to-end encrypted app? Basically, an application where you can have multiple clients, can be your devices, can be devices from other users, and you can collaborate. And the content is only known to these participants. So, everyone in between, every third party, every ISP, the people who are running the service, managing the database, they cannot read the content.

And that might bring up the next question, why actually do this? Well, for me, I was really intrigued by when I first used messaging apps like Signal and so on and realized, wow, it's actually me, I can be assured that the content can only be read by me if the code is solid. And I was intrigued from two perspectives, because as a user, I can decide who can actually read my data. The admins of the service, they cannot read the data. But also, as someone running services and databases, I was really intrigued by the idea of some kind of data, like sensible data, I don't want to even have access to. And with end-to-end encryption, you basically can make that happen.

So, I went down that road a couple of years back and started building end-to-end encrypted apps. And with that, I started working on tools. And obviously, they came with a couple of challenges in terms of UX and architecture, because a lot of things are different. And today, I want to share these lessons. And to do that, I felt like I want to build something really hands-on or show something hands-on. And to procrastinate, I built this app called Linny, which is on GitHub, it's open source, and you can even try it out on linny.app. And the thing about it is, it's an end-to-end encrypted to-do app.

So let me quickly show you. It's built with Expo. So it's React and React Native. It compiles the web, compiles to iOS, and I haven't set it up yet, but can do Android as well. And let's quickly see here, we can sign up, username, password, and we can log in. We have here our to-do list, we can add items. If you look at the right side, it syncs directly to the other device, to the mobile app. And everything's end-to-end encrypted. But what's special about it, we can also create invitation links, copy the link, and go to another user. The user can accept the application, the invitation, and we share this end-to-end encrypted to-do list and can collaborate in it. And what I want to do now with this talk is basically walk you through how I got there. So let's get started with the core of it, end-to-end encryption.

What we want to do is, we want to get data from A to B, and nobody in between should read it. And that's a solved problem. You just define a key or generate a key, you encrypt the data. So here, for example, my to-do, with the key, you get a ciphertext, you send it over your server. And if you don't care about the metadata that a server knows, then you can just send the ciphertext and the ciphertext doesn't reveal anything about the data except for its length, which is metadata in that sense again.

And on the other end, if I have the key, I can simply decrypt it and have the data. And while there are a lot of details, all of these are solved, and you just have to write and pick the right algorithms and hint, hint. You can just use libraries to do that. We'll get there in a bit.

But I want to dive a bit more into what we actually encrypt. Because compared to a system where the database is the single source of truth and you have all the data in there, it's a bit more tricky when you only get ciphertext and encrypted blocks. Because if you have a to-do list, and you get an API request to add a to-do, you actually don't get that. You just get information that there's a new ciphertext as a server. So how do you manage that? And one very simple and easy way to do that is to just encrypt with a key, with a to-do list key, the entire content of the to-do list and send it across. And that works. And for some applications, that's definitely good enough. If you just need to do that, awesome.

But if you want to build something that works collaboratively and real-time, that will be not good enough. Because you quickly run into conflicts. What do you do if different devices create pretty much at the same time changes on the same object? Let me illustrate with an example. Let's say you have two timelines, two clients. One adds a to-do, you sync it over. And then basically before syncing again, each of these devices, you make a change, you encrypt it, and you want to sync it. And that becomes really tricky because how do you resolve that? These different... when you sync it. And fortunately, this is also a solved problem. You just have to use CRDDs, conflict-free replicated data types. Sounds scary. And definitely worth a talk on their own. But in a nutshell, they're just data structures that allow you to sync without ever getting a conflict.

And if you use these, you solve the problem of conflicts and you can build something real-time collaboratively. Let me illustrate it by our example with the events. Let's say you have an event-based CRDD. What the CRDD has to do is just resolve and merge your data in a way that it doesn't matter in which order the events come in. They always will resolve to the same state. And if you have that, you basically have a CRDD. And that's really, really great for our end-to-end encryption because what we can do then is we can just encrypt every change separately. We can sync them. It doesn't matter in which order they arrive at every different client. But as long as every client for the same to-do list, for example, gets the same events, they all end up in the same state. And that's why end-to-end encryption and CRDDs are really a match made in heaven. It's fantastic. This unlocks a lot of really great usability with end-to-end encrypted apps. And of course, you don't have to build these CRDDs by yourself because a lot of people did that already. And we have libraries that do this. I just want to show you one of them, Yjs, for example. You cannot just operate with JSON on it, but it's pretty much the same. You have a bit of a different API. You import it. You can create a document. Document is this. Think of it as like a GitHub repository, and you can make changes to it. And then they basically all sync up. And then we can see in this document, I want this to-do space. That should be like an object, like a map. You can create a new to-do, set the properties and add it to the to-do list. And a great thing for each of these changes now, we can get updates. And this is, when we think about our end-to-end encrypted system, exactly what we want. Because we can get updates, and we can also apply updates. And you have really cool things like, you can even take the whole document, you know.

Updates. If you have an event-based system, that could become a long list of changes. You can take one document and encode it as one update in this library. Which is really, really useful.

And that means, like- So we got the CADD part. You've seen it's not that complicated. You just handle different APIs and manage data, and you can get that to work. Let's move on to the encryption part. And this is the part that actually is pretty generic.

And that's why the team that I worked with, we just built SecSync. It's a library that syncs CLDTs in an end-to-end encrypted fashion. And it comes with a specification and an example backend, so you can adapt it by yourself. And you can check it out. But we really wanted to make the front-end part especially really, really simple. You just have hooks, like use YGS sync or use AutoMerge sync. And you plug it in. And if you're using one of these CLDT libraries, you should be ready to go and automatically merge and sync your end-to-end encrypted documents. In reality, it became a bit more complex than we wanted.

I mean, it was built for a specific use case, because we also wanted to have permissions and snapshots. But when I built Linny for this presentation, I realized it actually should be this API. And I promise, well, I intend to simplify the API to this. So let me walk you through what it probably will be in a couple of weeks. Just use YGS sync. And then you pass in the document. You pass in the ID, what it should be on the server. You pass in the key. So the key that you only know on the client, the web socket endpoint. And you should be good to go. So let me prove this. Here we have Linny open.

But we have not connected it to real-time sync of the list. You can see this. If I add this added to-do here and refresh, it will be gone. Because it was never stored anywhere. And this is the whole to-do list component. This completely isolated component. The only difference compared to the other components that we just managed to state through a YGS CDT. And what we can do now is we have this Y document. We can plug in Y sync. And out of the box, it syncs, it sets up the web socket, it does the automatic sync, and you can see the messages. It encrypts everything. Because with your key, with proper modern cyphers, basically using live SORIM under the hood. And what we can do now is we can add a to-do, and you can even refresh. So it not only relays the data but also has this database integration where it stores it. So the only thing you have to do is plug it in and you're ready to go to have your end-to-end encrypted data. And every time you do a change, you basically send a new message with new ciphertext. And that basically solved our problem.

If you have this and if you use this, you already can build end-to-end encrypted applications. If you have the key. And that brings us to the next big topic of this talk is like how do we manage this key? Key management is a tricky problem in end-to-end encryption.

You cannot ask users to just remember these. How to create the key. You could create it by itself. Usually what you do is you just generate a bunch of random bytes. And we don't want users to remember these. This is pretty terrible. And in fact, the users should not only remember one of these.

If you want to collaborate with others, we want to make sure every to-do list has its own key so we don't expose the data actually to other users. So we need to manage a bunch of keys. Here we only have two. But how do we do this? Well, it's actually pretty simple. We can just build a locker.

So what we want to do is build a locker. It's a concept of you have one key where you encrypt other keys, and these keys you can use to access your documents. So there is no special magic behind this. This is pretty simple. You just create a key and encrypt your locker. Locker means just you encrypt a bunch of a blob, a stringified blob, with your locker key, and that's it. And if you're using TRPC or GraphQL or whatnot, you can even build a hook that just really boils it down to these two functions. Or not even functions. Like the add to-do function. You can add a new key. And then you can access a key from the content. And it basically syncs up. Let me demonstrate this again.

So here we have, for example, every time you boot up the application, it loads, you get the ciphertext, and it does this all automatically, and what you get out is the keys. The content.

It decrypts it, and you only have to provide the key. And if you add a new to-do list, we run the add item key, and basically a new structure will be created, a new locker with the new key in there, and you just upload the whole thing. And that means it's still everything is encrypted. Because what you only send is ciphertext, and the server basically for each user stores exactly one locker.

Yeah. That's pretty cool. So we managed to boil down our key management problem to remembering one key, which is still not good enough. Because what we actually want is people should be able to just log in with username and password. They should not need to remember this key. And let me introduce one solution that can help you to do this, which is Opaque.

So Opaque is a password-based client-server authentication with the server never ever obtaining knowledge about the password. That was a paper a couple of years back, and now it's an RFC by the IETF. And Facebook actually built a Rust library to do what? To build end-to-end encrypted password-based backups for WhatsApp. And the same thing, pin-based, is now also used in the Facebook Messenger to do backups of your messages that are end-to-end encrypted. And what we did is we built these TypeScript bindings so you could actually access it through TypeScript. So you have the Rust library, compile it to WebAssembly, and then type it and so on and so forth. So we wanted to make Opaque really, really accessible for our own use cases but also for everybody else.

But what does this give us? What does client-server authentication give us? Of course, it gives us a good basis for a shared session key between the client and server in a secure way, and that's awesome. But there's one interesting fact about Opaque. It also gives us an export key on the registration. So this is a good, solid key that you can use for encryption or even for signing. And a great thing about it is that every time you log in again with the same username and password, the server never obtains the knowledge about the password and how to reconstruct it, but you also get the same export key back. And that means we can use this key to lock up our locker.

And to quickly run through this, of course there's some interesting math under the hood. It uses OPRF, so Oblivious Pseudo Random Functions, but on a high level you can build an API, and this is what we did in LINI, and you can check this out in the open source repository, you just have to call the function register, pass in the username and password, and what you get out is the export key and the result. Under the hood, there's like two requests going back and forth in this process. It's probably not worth looking into this in detail, but it's like it's abstracted in a way that every developer can actually use this. And under the hood, if you look at it, so for example if we go into this process, we log out the export key, because I really want to show you that it's exactly the same. So if I log in as Alice, Google will complain about my not so strong password, but then if you look at the network requests, there's a bunch of stuff, it's all Base64 encoded, it has to do with elliptic curves under the hood, because what actually is happening, your password is run through Argon2, which is a password hashing mechanism, which you probably use on the server usually, but in this case we use it on the client, then it's through a hash to curve function converted to elliptic point, and then every login, you get a new salt basically, or a new factor, before it's sent to the server, and this is why every login is randomized, and it's really hard to attack this unless you have a quantum computer. And what you get out is this export key, in this case it starts with C24q, and this was the registration, but we can go through the same process, more or less, it's a bit different, but very similar in that sense, and you get the export key out again, and it will be exactly the same export key, and this is why it's so great for building these lockers, because we can just lock them up.

Yeah, Google complains about our password again, and then again, a bunch of stuff is sent around, these elliptic curves, points, but at no point, and it does a Diffie-Hellman exchange, it's all fancy stuff, but what you get out again is this export key, C24q and yada, yada, yada.

And as mentioned, this basically allowed us to do the whole key management, solve the whole key management for our application in a couple hundred lines of code, and it's all, you don't have to know any math or anything, you can just build it, or go into the application and copy what you need.

But there's one last thing that we are missing, because so far, this works really well for building an application where you can collaborate with your own devices. You can sign in from different devices, different browsers, and you always get the same open key, and with that, you can fetch the locker, unlock it, and get access to your documents. But we said we want to build something collaboratively, and this is what Lini also supports.

You can share the invitation. So we basically want to go to a point where someone else with their own locker and maybe different access can also access the same document. How do we do this? Well, if you have an invitation system, you just extend it. For example, if you have link invitations. Link invitation can be just the URL with slash invitation, and then you have a token that the server understands to give you access to that data.

One interesting fact about URLs is that we could actually hide a key in the hash part. Because fun fact, did you know the hash part is never sent to the server? If you take this URL, you put it in the browser, and you hit enter, the hash part only stays on the client. And this is really, really nice to basically build this link sharing UX with end-to-end encrypted application.

And then basically on the other client, in this case, the one accepting the invitation, we can just get out the hash param, in this case the key. We can accept the document invitation, in this case using TRPC with the original token. Not the key, but the token to accept the invitation. And then once we know it was accepted, we just add it to our own locker. Voila! And this is how you can manage invitations on an end-to-end encrypted system.

There's one thing though. You probably don't want to put the actual key in a URL that you share, maybe in Slack or an email or so on. You can come up with schemes that are a bit smarter. And I actually did this in Linne, so if you check out the source, it's described and it's documented. But unfortunately, I'm out of time, so this has to be a story for another time. Watch the follow-up talk. I actually intend to blog a lot about this in the next coming months, because I was deep in the rabbit hole in the last two years and now I actually want to share all these lessons learned. But what I wanted to show you with this talk is that there's actually tools already, or systems, that we can use, libraries that we can use, like Opaque and Zexion and so on, that make it really, really easy to build end-to-end encrypted applications. They might be simple. They just have authentication and invitation links. But it's already quite powerful. And if you have a subset of data that you care about that should be end-to-end encrypted in your application, you might be fine just doing that.

And yes, of course, it can go a lot more advanced. But it's really a rabbit hole of, if you think about account recovery, how to do that, and key rotations and permissions that are cryptographically secured, you can go very deep. But what I wanted to show is that it can be also very simple and powerful. And I hope I could manage to get you excited about this a bit and you dig in a bit more. And if you want to, I will be at the Q&A. Talk to me. Ask me questions. Ping me on Twitter, on BlueSky, on Macedon. Happy to talk about it. And yeah, if you build it and you need help, I'm also doing consulting in this area. This is definitely most exciting work that I'm doing. And yeah, I hope this was interesting. I hope you enjoyed the talk. And at least you learned something. Thank you very much for listening.

Instead of asking you this first question on its own, I'm going to bundle it with a couple others. Which is basically, the people want to know about the concept of encryption. When should I do it? Should I do it for all of my app data? When is it too much? Well, so I think it really depends on... What are the benefits? I think it really depends on the threat model that you want to set up.

For example, let me actually do a real-world example. If I have an application where my users communicate with their doctor, it might be completely fine to share the patients that are related to a doctor and what's their relationship when they have an appointment. But if they are sharing medical information, this might be data that is absolutely worthwhile to encrypt it. Because in Austria, you even have legal requirements to make sure this is sent in a secure way. And while it might not need to be encrypted, end-to-end encrypted, you can do that. And that will save you from actually leaking the data if you have a text and the database leaks. And this maybe comes to the threat model.

It's like, I don't care too much about NSA and so on. They can probably zero knowledge. No, not zero knowledge. There's these attacks. They can attack my phone and I have probably no chance defending against it. But what I care about is like leaking my own database. This is yeah, this can happen and we see it. We have plenty of breaches and it happens all the time. So even with a leak, encrypted data is protected. Yes. From prying eyes.

The next question is about CRDTs and how they interact with state management. How would you integrate it into an app using like a Redux, for example? That's very tricky. I would say, well, you could, you could combine actually Redux to do the, so you have, I mean, Redux in a high level, is just an event based system. So you can, where you push an action. So you could in the Reduxer basically do the updates to your state management. And yeah, that might be working out. I just, Yjs has this Mutable API, so just mutate the document and be done with it.

I just, Yjs has this Mutable API, so just mutate the document and be done with it. AutoMerge by itself is immutable, designed. So if you like this pattern, you might be actually way more happy with AutoMerge. Yeah. Makes sense. I really like this one.

Can a developer without a computer science degree confidently create end-to-end encrypted solutions? Yes and no. I mean, I think you can, if you go high level enough and you trust library authors, yes, you can definitely do it. I've shown it here and there are some other frameworks like Chess or so that allows you to do that on a high level API. But yeah, you really need to go high level enough and basically trust these authors to do the right thing. I guess people shouldn't be writing their own encryption by hand a lot of the time. Yeah. Probably. Probably. Yeah.

I mean, there's this thing, don't roll your own crypto. But then, if nobody does it, we don't get forward and we don't get more applications getting there. So there was also this quote from one very famous cryptograph. He said it on Twitter. It's like, he would be happy if we have more broken end-to-end encrypted systems rather than not have them at all. Interesting. Like introduce competition a little bit. Yeah. I mean, the more you have them, the better they get usually. Yeah. And it's not just one point of failure.

A little bit sort of off the back of that, you mentioned YJS and you showed a couple logos. What are some alternatives that people should check out? Well, there's YJS, AutoMerge, LoRa. LoRa is, I think, still in alpha.

LoRa is, I think, still in alpha. These are per se CRDT libraries, but then there's frameworks like Chess or Eberloo, which manages CRDTs and the whole encryption and local first part, which I haven't even talked about yet, in one framework. And if you want to build applications, yeah, they might be a really good fit on a high level.

There is actually a question about local first, and are you aware of something called P2P Panda and just in general, how that integrates? Yes. I know P2P Panda. So, I know local first. Local first is the idea that you basically can run everything local and you sync it. And you also have this concept of leveraging end-to-end encryption optionally.

I mean, it's more of an idea and we have a lot of people building towards it. So, two weeks ago, there was the local first conference in Berlin. I was there and I'm part of this community. Yeah. And P2P Panda is one very exciting project, but I have not seen it used in production anywhere. Interesting. Not yet. But they're working on it and they have a couple of very interesting concepts.

It's like, the fascinating thing is that basically local first, end-to-end encryption and CRDTs, they go very well hand-in-hand. So, a lot of people just build everything. There's a lot of overlap. Yeah. But you don't have to build a local first app to be end-to-end encrypted and you don't have to be end-to-end encrypted to be local first. It just fits so well together that a lot of people go towards that direction. Especially with a lot more people being aware of their own data privacy. Nowadays, a lot of regulation coming up.

We have an interesting sort of use case question, which is, how do you deal with filter query data? So, for example, for searching, do you have to send all the data to the client so it can be decrypted and filtered? Yeah. So, yes. The simplest approach nowadays probably is just have the data on the client and then becomes local first application. And then you just have, for example, SQLite in the client and do full text search. But of course you could come up with schemes where you use... I mean, there's homomorphic encryption, which is still very slow.

But you could come up with indices. It's not an easy problem if you want to really tackle it. Except for the easiest one is just have to do data locally and then allow to search for it.

Fair enough. We have a ton more questions. But I am going to let you all go, because we are almost at lunch and I have to make a few more announcements.

One last thing is, are there any resources you'd recommend for people to learn more about either the libraries you talked about or end-to-end encryption in general?

Good question. The Signal blog, for example, is amazing. The metrics work. Signal. Signal. Yeah. They have a blog and they often share their lessons learned. They are on the forefront of it. It feels like they are five, ten years ahead of the rest. Keep an eye on Signal. Good signal.

But yeah. Actually, just ping me on Mastodon, Blue Sky, or Twitter.

Oh, what's your Mastodon handle? It's everywhere. Nick Graf. Nick Graf. Yeah. Just go to nickgraf.com and then you can find it. Nickgraf.com.

Thank you so much for your time, Nick. It's been lovely having you. Please give it up for Nick.