Local-First: A Crazy New Way to Build Apps

Hi everyone, my name is Anselm and today I'd like to tell you about LocalThirst, a crazy new way to build apps. If we want to build an app, you're like, I've got a new idea, I want to build this app, right? Usually what we think about what we need to do is something like this, right? We'll have a front end, a backend, some database, and maybe we'll have to interact with some Excel APIs.

But unfortunately, the reality today looks more like this. It's become really, really complicated and we're like a far way away from the simple traditional server client model. And it gets worse these days because of high fidelity apps like Figma and Notion, or even Google Docs users these days expect all these kind of advanced features like real-time multiplayer, cross-device sync. More and more people are starting to expect apps to work offline. You want to be able to share content in an app and kind of define complex permissions. And if you're starting to think about how to add these even more complicated things to this already complicated stack, it gets really difficult. Like what if we wanted real-time multiplayer, where do we even need to add that? It will affect the front end, it will affect the backend, it will affect the database. None of the traditional technologies here are even really made for that. Similarly, if we want to add offline support, is that kind of just a front end thing? Will it affect the UI state? We kind of also have to think about that in the backend. It's really unclear. Cross-device sync also gets really complicated. And if we want to do some sharing between users, that actually affects all parts of our app. And it just becomes more and more complicated. And my favorite one, something as simple as user file uploads, which is like radical, right? Even just implementing that, if you've had to implement that in apps, it kind of touches everything and it's way more complicated than it should be. So all of these things that users now take for granted are either like awkward up to really hard to build.

So if you look at this diagram as well, the code that's specific to your app is actually all over the place. And it becomes this really complicated thing. You have to make all these tech choices, like how are you going to represent UI state? What about, how do you talk to your backend? Do you have some query library, RPCs? What do you use for auth? Do you talk to the database directly or do you want to use some ORM? Maybe for performance you need a cache. Probably you need something like a message queue for certain kinds of tasks, particularly for talking to external APIs. And you kind of need to deploy all of that. And of course there are simplified solutions that give you any combination of these as a package, but internally they still look just the same, just as complicated.

And if we zoom out a little bit here and stop thinking about in terms of the patterns that we're used to that we take for granted that this is how you build apps and you need all of these things, what does a stack really do? And I would say that every stack really just implements shared state between users and devices. And for some reason you have to build that by hand each time, again and again. And note here that I don't mean like, yeah, new apps with these cool features like real-time multiplayer, even boring stacks for boring like web 2.0, SaaS type apps. All they really do is implement shared state between users and devices. So maybe a solution for helping us out of this mess is what if we could build general tools with shared state between users in mind from the beginning.

I've built a lot of web apps. In my experience, I've been like a full-stack consultant for something like 13, 14 years, depending on how you count it for every app that I've built, I've had to build this stack just to get shared state. And only then could I start building the app specific parts. And I never really knew how to solve this until I discovered this new way of building apps that answered my question like what if we could do it a different way. And this way of building apps is called local first. This is what a local first infrastructure looks like, way simpler. You basically have your front end, which just talks to local state and you get this experience of building like a good old, old school offline app. But then your local state gets synced to other devices, to other users, and even to back end workers in a generalized way through what I would call a sync and storage infrastructure.

Disclaimer here, a lot of the ways in which I'm going to explain this stuff are slightly biased because we are building a framework for building local first apps. So this will be opinionated and you might ask, who is we? Well, it's me and my team at Garden Computing, and we're building Jazz, which is a framework for building local first apps. And we're also running JazzCloud, which is sync and storage infrastructure as a service for apps built with Jazz. So this basically means that the sync and storage infrastructure part up there is either JazzCloud or self hosted. And I really want to underline that because I don't want you to think that this talk is super biased because I'm a vendor for this sort of stuff. It's all open source. You can run it yourself. I think it's just convenient to use something like JazzCloud. But the general picture is this. And the most important and powerful part for me compared to the traditional stack is this part that, by saying that you work with local state, which gets synced a generalized way, it means that your app specific code actually just lives in this part down here. You only have to build your front end and maybe some back end workers to talk to external APIs.

In fact, there are a lot of kinds of app that you can build with front end code only. You now only kind of have to make three tech choices or even less. And you have very little to deploy because again, you might only need to deploy a static front end and maybe some lightweight back end workers, not a whole stack.

How can this be possible? The first ingredient that I had to discover that really gave me the idea to build a framework like this and that made me think in this different way is something called CRDTs. You might or might not have heard of them. The abbreviation stands for conflict-free replicated data types. And a very quick, simple way to think about them is that they are Git for data.

To be more precise, it's super fast Git for lots of tiny data. And you can run that on the client. And that's really what makes hard things like I talked about earlier, really easy suddenly. For example, real-time multiplayer. Well, and if you've got a super fast Git for lots of tiny data, I think you can imagine how you could build real-time multiplayer experiences on top of that. You just share the edits and everyone will end up with the same data and you can do that in real time. You might be asking like, okay, part of real-time multiplayer is having user presence, so you can see the other user's curses and so on. How would you add that? Well, the curses are really just more tiny data, so you just also put them in the CRDT and it's super fast, so you can see other people's curses or other kinds of user presence in real time. So, real-time multiplayer and presence solved.

What about cross-device sync? Well, if you think about how you are using Git, you could work on the same repo on different devices and then just use explicit pushes and pulls to get to the same state everywhere. And the same is true for CRDTs, except it kind of happens automatically in real time. Offline support, same story. Of course you can use Git offline. You can work without any internet, do some commits, push it up and it all works and CRDTs are exactly the same. Again, because it's super fast, Git for lots of tiny data.

If you think about it, if you work offline on multiple devices concurrently or if different people work offline on the same data, you inevitably will run into conflicts. And this is where the last really important property of CRDTs comes in, which is that they basically have reasonable auto merging built in. So, even if you go offline, go back online, your changes are synced, most of the conflicts will just be solved in a way that you would expect as a user. And even this radical idea of user file uploads also becomes super simple because you can't just store lots of tiny data in CRDTs, you can actually also store large data in it. And then that large data becomes synced and shared between users just like everything else. So, user file uploads are also very easy. And you might start to see that while the architecture is very different and it might seem like this crazy new way to build apps, I really want to convince you today that it's actually the only sane way to build apps these days.

You might be wondering next about, well, how does sharing work and how do permissions work in this kind of context? And traditionally you do this by kind of centrally enforcing them in the backend at the API boundary.".

Here we don't really have an API boundary, so how does that work? Which brings me to the second ingredient, public key cryptography. A simple way to describe public key cryptography is that it means that you can encrypt data specifically for someone and only they can decrypt it. And you can sign data as someone specific, so you get kind of verifiable authorship. And you can also do this on the client.

So this lets us combine our two ingredients, CRDTs and public key cryptography, which means that instead of just encrypting static data and signing static data, we can encrypt diffs and sign diffs. Encrypting the diffs means that only people who we want to see our changes or our objects can see them, so we effectively implemented read access. And by signing them, other people can verify that edits that are being synced in are by people who are actually allowed to write to this object. So this is how sharing and complex permissions looks like in this world.

A lot of these things, as you saw, that used to be awkward to really hard are really easy now. And there's even one thing that I hadn't mentioned before, end-to-end encryption, that was just plain impossible on a traditional architecture. A lot of these, and that's possible now because, again, we're using public key cryptography and have this like verifiable form of access control that lets us do end-to-end encryption and verifiable edits. This means that if we look at our infrastructure graph for a local first stack again, actually only the front-end and the back-end workers see our users data in plain text, and the whole sync and storage infrastructure is untrusted, sorry, is trust-free, which means that even if you choose to use Jazz Cloud instead of using your self hosted syncing server, you don't have to trust me with your users data. Your users don't have to trust me with your users data. And if you don't have any back-end workers, because you don't need to talk to external APIs, you can actually build fully end-to-end encrypted apps where even you can't see your users plain text data. A lot of these, and the reason I built this into Jazz is because I really believe that local first is best if user identity and permissions are also local first, because that's what public key cryptography really is. It's local first user identity and permissions.

Now if you talk to some other local first folks, they might be like, yeah, well, that's just like your opinion. And that's actually a great way to divide the local first space. Kind of on the one hand, we have frameworks like Jazz or Evolu or DXOS, where they like a lot of these permissions and local first user identity based on cryptography really belong together with the shared state of local first. On the other side, you have frameworks like Electric or Zero Sync or InstantDB, who use CRDTs, but then rely on a more traditional centralized infrastructure for enforcing permissions at the API boundary. And then a lot of the open source libraries with all the service attached to them, like AutoMerge, Yjs and Laurel, they really only implement the CRDTs and leave the permissions and auth up to you. They usually even leave the sync up to you.

So you might be curious now, well, okay, I kind of get the general stack architecture. What does it actually look like to build apps this way? And again, disclaimer, I'll show you how it looks like to build apps with Jazz. It might look slightly different with other frameworks because another great way to divide the emerging local first space is by what does it look like to build apps with it? And roughly I would say on the left, you have these frameworks where it looks like you're just editing JSON state and it very much feels like a React use state that you can write to and then it gets magically synced to other devices. And on the right hand side, you have tools that are more like bringing the database into the client and you would do mutations and queries on the client. Jazz is on the left hand side. So that's what the examples here will look like.

Final disclaimer, I'll show you like the React version of Jazz. We already support using it with Vanilla JS and Vue and Suave are coming soon. But it's very simple. Step one is you set up a context and sync, which is just a simple provider here where you can choose from different auth methods that are built into Jazz. Here we're using demo auth because that very quickly lets you log in as different users and you can quickly try out multiplayer and sharing features. You can see that basically the only really important thing here is that we specify a WebSocket connection to the syncing peer that we want and we do a bit of state handling for the auth, right? We wrap our whole app in this provider and then we define a schema, which is really the most important part of building any Jazz app where you define the data model that really makes your app, your app. So let's say we want to build like a little chat app. So we only really need two classes here, one for message that just has a text field that is a string. And then we say that the chat is just a co-list or a collaborative list of messages, right? Both the message and the list of CRDTs, the message is one that looks like a map with fields that people can collaboratively edit. The chat is a list where people can collaboratively insert messages.

Once we have that, we can just create a chat object and it looks like a simple local object. But as soon as you do this chat.create, it actually persists this chat object locally in your browser and it gets synced to the sync and storage infrastructure where it gets stored like cloud storage and then other people can reference it and subscribe to it. So once you have a chat, we can use this use costate hook to subscribe to changes to it. In vanilla JavaScript, there would be a sample like chat.subscribe and you get a callback that gets called. In this case, this component will get re-rendered every time chat changes. And the cool thing here is that it will get re-rendered or your subscription will be called not just when chat changes because of local changes, but also when you receive changes from other devices or other people that are being synced in. So it becomes very, very easy to build apps. You basically just build UI around these collaborative values and subscribe to changes on them.

Chat actually just looks like plain JSON so we can map over the messages in it and then just render the message in a component. And in order to like, now you might ask yourself, what does it look like to edit something like a collaborative list? Well, you can see on this chat input, when the user hits enter on their new message, we literally just create a new message and then push it into the chat. Finally, to show you what this like complicated sounding public key cryptography looks like to use in Jazz, it's really simple. There's this abstraction called groups where you can add accounts or in this case, everyone with a specific role. And then when you create a collaborative value like the chat here, you specify a group as an owner. And then that way you determine the access rights on that object based on the roles, different accounts or everyone has in the group. That's all you need to do. It's basically Dropbox like permissions for any object that you create. You can define all of that locally. You can see this create chat function here. I'm literally calling that on a button on click. So you can set all of that up, the groups just like the objects in them will get synced and persisted in the cloud. And you can just handle them like local objects.

The last bit more complicated topic is, well, probably your app will change over time. So you might need to change the schema. And here we adopt an idea from databases called migrations, except they're a little bit more curse. Because if you think about it, clients might be offline. Their allegiance might run simultaneously on different devices now instead of in one centralized database. So this is probably the hardest and newest and most complicated part of Local First. But in our experience so far, it's okay. You just need to be careful. And we're kind of discovering the patterns that you can follow to make sure that works well. And then in the future, we might introduce guardrails to really make sure you can't do anything wrong with them. So probably for me, the motto for Local First is not just that hard things are easy now. Some things are a little bit curse now, but it's totally worth it, basically.

You might be wondering, well, you kind of explained the front end part, what that looks like. The nice thing is that the backend workers work just the same way. Because in the Local First model, if you have a backend worker that you want to do stuff like talk to an external API, they just become another client to the sync and storage infrastructure. So they can run wherever you want. They just subscribe to the same state that you have in your front end. You don't even have to do explicit API calls. You just have this shared fabric of data that you can edit and subscribe to.

And in the case of Jazz in particular, the server workers actually have little accounts just like users have. So you can precisely give them access to certain kinds of data. And it's again, cryptographically enforced that they can only see and only edit the data that you give them access to. So that becomes much more precise and verifiable.

Let me quickly walk you through some apps that people are already building with Jazz, just so you get an idea of how versatile it is. This is an app called Invoice Rater. They're building a tool that logs into different cloud providers like AWS, CloudFlare with your credentials, downloads all your invoices and collects them for you so you can easily submit them to tax authorities, for example. This is a great example of an app that you just couldn't build with traditional infrastructure because you have to trust it with your user credentials for all these platforms. But because it's built with Jazz in a local first way using local first permissions, it's actually all end to end encrypted and you get all the nice benefits of an offline app where you own the data and you know exactly what happens. But you also get cloud storage and sync across devices.

Completely different kind of app called Learn Anything. They're actually already launched. You can go check it out. It's this really cool learning platform where if you're interested in some topic, like you want to learn front end development or how to play the guitar, you just go to Learn Anything, enter your topic and then you get a list of users submitted and nicely curated links and material to learn that topic. You can then kind of track your progress as you go through different steps and progress in that topic. But you can also submit your own material. In this way, it's basically like a social network with lots of users interacting. They get a ton of first-time visitors each week, I think several millions. So because they chose to use Jazz for their whole stack, it means that Jazz, in their case, works much more like a traditional database and has to serve all these first-time users with lots of lots of objects to sync on every load. And we were able to make it stretch to that, which kind of really validated my hope that Local First isn't just a niche way to build certain kinds of apps where it's a particularly good fit, but it actually makes it really easy to build any kind of app, even something that needs to be as scalable as a social network.

Another app that kind of stretches what Local First apps can do is this app that's in development called Succulent, which is a social media scheduling tool. So you can plan your Instagram posts in there. The nice thing here is this is a good example of the file uploading.

It handles all the image data as well. And you just can, even while you're offline, put images in there, plan your posts, and it has a little server worker that will actually talk to methods API to post your posts. But the user experience is completely offline first and local first. And you can use it across devices from your laptop, from your phone. And it was just super, super simple to build because of this architecture.

There are a couple of ones that I would love to show you, but they're still currently in stealth. Someone is building a real-time CI dashboard where they have an existing API for their CI deployment runs, but it's kind of clunky and hard to build a nice dashboard around. So they're using Jazz as this kind of translation layer that makes it really easy to build a real-time UI. Someone is building an infinite canvas app, and they're now trying to build it with Jazz because it solves basically all the hard parts of the infrastructure. And they can really just focus on the UI and the user experience. And finally, another example where it becomes really important to have this cryptographic permissions and user identity is an app for sharing know-your-customer information, which is usually very sensitive and personal, with specific people and organizations. Yeah, all of their hard challenges are again solved by Jazz just having user identity and permissions built in in this secure way and in this generalized way. So they can focus, again, just on the UI and the UX of it.

If this sounds cool and you're curious and you want to try building apps in a local-first way, the best resource there is going to be is not quite there yet, but is hopefully coming soon, where Jazz and a couple of other vendors of sync engines and open-source local-first frameworks, we want to create this comparison page where you can really look at a lot of different dimensions and see what technology and what service is the best fit for your app. That's not quite out yet, so stay tuned. But if you want to start building apps in a local-first way right away, I really encourage you to check out jazz.tools, which is of course the homepage for Jazz and Jazz Cloud. And just have a look at the documentation and the guides and give it a go. Hop on our Discord if you have any questions. I hope I was able to give you like a quick taste of what local-first is like. Thank you very much for your attention. And this is where you can find Jazz on Twitter and me on Twitter. Yeah, hope you will join me and build very cool new apps much quicker in a local-first way. Thank you.