Jazz: Build Real-Time, Local-First React Apps With Sync & Secure Collaborative Data

Thanks so much for the intro. I'm Anson. I'm here today to tell you about JAS, my new framework for building apps around sync and secure collaborative data. Before I get into it, this is kind of the first time I'm talking about this publicly. So, if this is interesting to you guys, you could actually really help me out by checking out the homepage and joining the Discord, especially if you want to play around with JAS or have some questions. All feedback is just really helping me at this early stage.

Today, I'll talk about these things. Why did I start making JAS? What is it really? We'll have a look at a small demo app and do a detailed code walkthrough to give you a quick impression of what it feels like to build an app with JAS. If we have some time, we'll do a little bonus demo app and I'll talk some more about what's next for JAS.

So, why did I decide to start building JAS? I build a lot of apps and basically every time I'm like, hey, let's build an app, but then you have to worry about like all of this stuff, like choose a database, deploy it, how do you build the backend and especially if you want to do multiplayer in real time, it just adds more complexity. Suddenly you need message queues, web sockets, and I'm like, why is everything so difficult? And more interestingly, what if it didn't have to be? What if we could just kind of get rid of almost all of that and be left with only the things that really make our app, our app, right? Wouldn't that be nice? That's kind of the promise of JAS, sounds nice, right? And it gets even better. What if I told you that you also get cross-device sync, real-time multiplayer, automatic granular data fetching that you don't have to worry about, local and cloud persistence, offline support, and really fast UI for free? That sounds great, right? That's kind of roughly the dream that I had when I started with that, but how can we get there?

And I think to answer that question, it really helps. And that's what I started with, is to ask ourselves, how did we get into this mess in the first place? So really quick history, let's start from first principles. This is like kind of my mental model of a computer, right? And it turns out it's not my mental model of a computer, it's actually this guy's mental model, and he just willed it into existence, and that's what we have now. And it's pretty much unchanged since, except in the 60s and 70s, these guys added one box to it, which is the internet, and we're still kind of recovering from that. For the web itself, this guy was super influential. He basically saw this computer architecture and was like, hey, you know what, we should kind of build hypertext on that, and he created some of the first graphical user interfaces, but hypertext didn't really catch on until much later when this guy was like, research collaboration at CERN is kind of a mess, maybe we should build hypertext, and the problem with his implementation of hypertext, of course, is that it was good enough, and that's what we're stuck with now, that's the web. But later, in 95, this guy was like, hey, that's some nice hypertext you got there, how about we build a computer architecture on top of that? And also my boss gave me ten days to do so. And similarly, the problem with JavaScript was that it was good enough, so that's what we have now, right? So we're now in a situation where basically this is the most popular computer architecture, the JavaScript VM in the browser as we know it. But actually, there's nothing really wrong with that. The complexity is really inherent in the more generic model of the computer here because if we want to build an app, we have to worry about way too many of these boxes. Basically the reason we have all of these questions is because every time we want to build an app, we need to think about these arrows and shuffling data between the memory, the disk, the internet, other computers. That's where I think most of the complexity comes from. When I first heard about CRDTs, I got really excited because CRDTs promise an alternative model to that. I don't really have much time to explain CRDTs. There's a really good blog post called Data Laced with History. That's a very good mental model to think about CRDTs as well. So definitely check that out. A simple way of thinking about CRDTs that I like is, instead of trying so hard to put your data onto the internet and having that in different places, you basically put the internet into your data and your data is now aware that it exists in different places. Many people are editing it at once.

When I started looking into CRDTs, I realized that on their own, they're not really enough. What you really want in addition to them is permissions and you want to have sync and storage solved for you as well. If you have this package of things, you actually have a really powerful abstraction. So powerful that it lets you go from this quite complicated situation to a much, much simpler one that looks like this.

And this new abstraction is what I'm calling collaborative data. I think that explains my motivation pretty well. So how do we actually build this? And this is where I'll start talking about what Jazz actually is. We want to build this somehow and how do we implement this in today's world? My particular take on it I'm calling CoJson, which stands for collaborative JSON. And basically you just hook up some JavaScript to that and you render stuff into the DOM and that's all you need to do to build an app. To be more precise, CoJson implements multi-device co-editing, user identities and accounts as a first-class concept. Permissions and roles, sync and caching and persistence for you so you don't have to worry about any of that when building an app. Most importantly, it basically makes collaboration and secure access control feel like inherent properties of your app's data. And that means that you can build your app as if all you had was local state. You don't have to worry about anything else anymore.

What does CoJson look like? What does it provide you with? It gives you good old JSON as immutable values, so you've got your objects, arrays and so on right? And it adds to that a new world of collaborative values or CoValues for short.

There's collaborative maps and collaborative lists. These are quite self-explanatory. Soon, there will be collaborative strings, plaintext and collaborative text, which is rich text. And then there's something a bit weirder, which I'm calling a CoStream, which is basically a stream of values per participant. And we'll see in the example later that that's really useful for social features or anything where you want to actually not have people work on the same object with potentially conflicting edits, but separate them out. And there's a special version of that for binary data. But this is basically, these are building blocks that CoJSON gives you, and that's all you need to build your app around. CoValues keep, always keep their full edit history. They automatically resolve conflicts. And they're eventually consistent. And this is exactly what makes them a CRDT, right?

In addition to that, CoJSON offers, like I said, permissions and user accounts as a first class concept. And the way you interact with that as a developer is that you create groups, because CoValues always belong to a group. And a group is just a set of accounts with roles, like reader, writer, or admin. And then the account roles determine access rights to all objects that are owned by a certain group. And the interesting thing is that this access control is not implemented by some central authority or a backend, but globally by encryption signatures. And this is the main reason that you don't really need a backend for most things if you work with CoJSON, if you work with this new abstraction of collaborative data.

So now that you kind of have an idea of what CoJSON is, let's get into what Jazz is. Basically, Jazz is just an open source framework that provides idiomatic bindings for CoJSON for different environments with batteries included. So we want to have this, right? But it needs to exist in the browser. So what Jazz does, the Jazz browser package does is it hooks up CoJSON to external auth, persists stuff locally in IndexedDB, and does all the syncing over WebSockets, right? Now, obviously, we're not like cave people. We don't want to interact with the DOM directly. We want to use React. So what you really want to use is Jazz React. And if you do that, that's kind of like my flagship library. That's what I'm focusing on and really want to get the DX right for. All you have to worry about is this bit. And yeah, we'll see in a second how nice that actually is. In particular, Jazz provides this idea that I'm calling autosub for automatic subscriptions, which means that you can reactively subscribe to a tree of core values and they will update whenever you edit the core values locally or whenever another participant edits them. So you basically get a use state that magically has persistence and multiplayer built in.

And that's super, super powerful. In addition to that, like I said, it provides a couple of providers and it also lets you do file upload and download really easily. So you don't need external blob storage. You can actually stream data directly into core values and have them sync just like a structured data. Remember I said earlier we want not just CRDTs and permissions, we also need a solution for sync and storage.

So in addition to just the open source library, I'm also running something called global mesh, which is basically global sync and storage infrastructure that I'm providing as a service. You can think of it as basically S3 plus a CDN for collaborative data. It's free right now while I'm building all of this and it will be really cheap later.

Let's have a look at an actual app built with Jeth. Let's start really simple, just to make it easy to understand let's build a little chat app. I'll first show you the app. So this is the app. I can create an account, create a new chat, and then let's open this just in an incognitive tab as if it's another device, create another account. Ooh, and this is where we'll see if the wifi works okay. But yeah, you can see, I can just type here, and if the connection was a bit better, okay, it's reasonably fast. Let's see how the other demo will go later. Simply enough makes sense. It's probably not too hard to build this yourself using other tech, but let me show you just how simple it is to build it with Jazz.

So let's have a look at the code to build this app. So as before, I promised you all you have to do is kind of define your data model and build your app. So what does the data model look like? Basically a chat is nothing more than a collaborative list of messages, right? And then messages are just a collaborative map with a single property text. That's it. That's the whole data model for our app. Everything else is just UI. So like let's keep that. Let's have a look at our like top-level boilerplate. You can see just standard React stuff. with jazz provider where we can plug in an auth provider and an API key and that's it. So what does our app look like? You saw on the homepage, there was just this, basically we have the homepage and then we have the view for an individual chat window. That's just simple hash based routing on the homepage with the button that creates a new chat before the first time really interact with jazz.

So let's look at that in a bit more detail. We use this use jazz hook that gives us this object called me, which represents the current user. And then as the current user, we can start doing stuff in directly in the on click handler of the button. So it can be like, let's create a group with me as the admin, and then we can add members to it for simplicity's sake. We're just adding literally everyone to be a writer. So we can just share the link and other people can contribute within that group.

And then we create an object that belongs to that group in this case, a list. And we can be nice and precise with the types here using our data model from earlier. And that's the chat. And that's it. That's all I have to do. I don't have to make any requests or anything. I just create the object locally and in the background, it will be synced and persisted.

So now all we need to do is to navigate to the view for the chat, right? Slightly more complex, rendering the chat window itself and adding messages to it. Again, let's go through it bit by bit. I told you earlier about this thing called auto-sub for automatic subscriptions. That's the other hook. And that's really what gives you access to all of the power of Code Jason. You just say you want to automatically subscribe to this chat by its ID. And it gives you this object chat that always contains the most up-to-date version of the chat and any objects nested within there that you're trying to access.

So we can just map over that since it's kind of just a list. And for each message, render a chat bubble with the message. You'll notice the optional chaining here. Basically by accessing each message and the text within the message, that internally triggers auto-sub to fetch these objects if they aren't loaded yet. And by using the optional chaining, we can write that really quickly. Obviously, if you want to show loading indicators or something, you can kind of imagine what that would look like, but to keep it simple, it looks like that. More interestingly, like you saw in the chat app, we're rendering who posted the message and when it was posted. For collaborative features like that, we don't need to write any extra code because all of that information exists within the edit metadata of the co-values, and they're really easily exposed from auto-sub so we can just use the special .meta property and basically ask Codejson, Hey, who inserted the message at the index i, get me their profile and get the name in that profile. There's also this helper that shows me whether that was me so I can easily render it on the right or left side. And of course, I also get access to the time of that edit.

Last thing is we want to submit a new message to the chat, what does that look like? Remember we had our group that determines who has access to the chat and obviously we want to create the chat message within that same group. So we can again use the chat metadata to get the group and then we create a map of type message with our text and we append it to the chat by ID.

That's it. That's all that needs to be done. And the crazy thing is because it's so simple, this is our entire app, it fits on one screen for completeness sake, here's like the components for the chat bubble, but that's just a dumb that just renders stuff and the input that just drops like a native input element, right?

So that's now actually completely the whole app, 82 lines of code to implement chat. If I just put it like this, it would look a bit ugly. So let's spice it up with some Tailwind. And now that's actually the app as you saw it, yes, the Tailwind. Cool. I hope that made sense. You want to see one more app? Something slightly more complicated? How about Twitter? Let's rebuild all of Twitter with Jazz quickly. Well, maybe some of it. What should we call it? Shop the Arts, Cleaner, Twitt. I was considering getting the domain but thought it was a bit decadent for a small demo. This demo you can actually try yourself and we'll try it together. And I've never tried it with this many people before. I have no idea how the WiFi will handle it, but let's just go for it. I'll do it again here. So it's twitt.jazz.tools, a login as myself. And then I can just post stuff here. That's doing it again. Yes, there we go. Okay. I can reply to myself. Let me know if it shows up for you guys at all. Oh yeah. Nice. Seeing messages are coming in. Cool. And you see like it reacts quite quickly.

Everything I do is instant. I can like go to someone's profile. I can, let's maybe go to my own profile. I can upload a profile picture. As I said earlier, you can do file upload into code Jason. Boom. Got a profile picture. That should show up for everyone. And you're kind of seeing some like partial loading states here. That's probably the wifi not being quite up for it. I tested this with like thousands of bots just to make sure, oh, there we go. But the cool thing is, yeah, everything loads eventually. And for the stuff that's available, you have instant interaction. This will even work offline. I'm not going to do that now, but I could turn off my wifi, post some stuff, turn the wifi back on. And it would just sync to you guys.

Let's try one last thing. Uploading picture of a cat because that's what the Internet is for, isn't it? Cool. Yeah. Yeah. And you can see like with the pictures, even there's like progressive streaming and they slowly become available. Sick. Hey, that's actually working really well. Cool. Yeah. You can keep playing with this.

I'll just quickly finish my presentation. I don't really have time to explain the code for the Twitter example. You can find that on GitHub. But just really quickly, as you saw earlier for the chat, it was basically entirely determined by the data model and then the UI is kind of obvious around it. So for Twitter, all that a Tweet is, is a collaborative map that has a text. It can have an array of images. Oh, sorry. Like a list of images. And then for the likes and replies, that's the first time where we see co-stream. So I'll just quickly explain that. You can see like a like stream is a co-stream of either heart or null. And what that means is that for each participant, the state of whether they liked a Tweet or not is separate. And we can count how many people currently like it or not. And that's really simple to use like that. And then we have our profile. That's also quite obvious.

Actually, since we have a little bit of time, let me show you an Easter egg in the app. Because if you go to your profile, you have a bio, right? And I think what's something that's kind of boring about Twitter is that it doesn't really let us express our individuality as much as like, remember in the old days forums, when you could have like custom footers and shit. So something that I did here is if you put as your bio, something that is valid, kind of react CSS, then all your tweets will be affected by that. Oh, I see other people posting cats.

Perfect. This is working wonderful. Even Elan made it here. Cool. Finishing up very quickly, what's next for jazz? I mentioned earlier that there will be collaborative plaintext and rich text. That's obviously what a lot of apps need for jazz itself. There will be more auth providers I want to integrate with external ones like Auth0. There's actually another one which you can already use based on touch ID, where the user's encryption keys are stored with like biometric data, which is quite cool. Obviously you want user presence, so I'll make some ready-made libraries for cursors and text cards. Something that's really interesting, like you can kind of imagine how to build a new app from scratch with jazz, but I also want to allow you to adopt it incrementally with existing apps. So the way that that works is basically that jazz syncs your multiplayer data back into your existing database, and then you can build cool new multiplayer features that can still talk to your existing old school features using the API or whatever used to build that. And since we can do files and media, we could also do video and audio calls all within the same model. That's kind of my plan. And as I said, we're supporting react and kind of browser agnostic JavaScript right now. Wreck native obviously would be super cool to have. And then on the service side, no jazz, Dino and bon and later on other languages unlocking native apps all speaking the same protocol and just being able to talk to each other. Cool. Thanks so much for listening. Again, take a picture of this. Funny enough, I think you kind of almost covered off the first question that we had in. That was the top rated. All right. With your kind of explanation of what's next for jazz. So we've got here one of our highest rated questions is how does this data get stored on the server? Is there a database behind it and you kind of went into that a little bit. Flif notes for where you currently are. So if you're curious like global mesh does the cloud persistence and it's, it's called mesh because it's literally a mesh of service and they, they store and cache data locally. And then it's backed up into like blob storage. But it basically exists close on the edge depending on where users access it. And it gets synced between all the different nodes of the message as well as needed. I hope that kind of answers that question.

I think that, I think that answers that. There's a similar one from here, from, Oh, we've got some movement. Here we go. So can global mesh be self-hosted? Yes. So since it's open source, you could totally run your own syncing server, and there is even if you go to the repository, there's something called code, Jason, simple sync that you can run as a local sync server, or you could run it on a single server. Obviously if you want to scale it, like I'm trying to provide the best possible infrastructure and it has a free tier, but it's really important for me that there's no lock-in. So that's, that's the situation there.

Nice. We've got here is Jazz built on websockets? Socket.io also uses their own type of web sockets, but you can't connect it by regular web sockets and have to use their own system. I think that. Yeah. So it does use websockets and it has like its own protocol for syncing on top of that. Um, in the long run, I don't really want to be completely married to websockets because there's more interesting things coming up like web transport or quick that helps if you need to sync lots of values independently, because with web sockets is a single TCP connections, so you can get something called head of line blocking, where if you sync a huge value, everything else has to wait for that to be synced. Whereas with web transport and quick, you can sync stuff in parallel, prioritize it. And that gives you a much smoother experience for like complicated rich apps. The nice thing is, if you use jazz, you don't have to worry about any of that. You just interact with the core values and at some point in the future, I'll plop in the like web transport or quick support, and it will just run better. So yeah.

Nice. I'm just planning a project in my head. So is code Jason something that you came up with or is that something that already existed as a standard when you went to go build jazz? So code Jason, as my invention is basically and it will be a standard and it's it's like a certain implementation of CRDTs plus the way that access rights work based on encryption and signatures and like that whole package is what, what code Jason is. And that's what I've been building for the last roughly three years. Nice. Congrats. Love in the crowd. We've got no, I'm really liking how you're kind of responsibly voting up different questions because we've got six votes on this next one. How does Jazz handle different app versions? So that's that's the best and the most difficult question. Basically Jazz has a simple version of something like migrations where you can migrate the user account and the route of data you store in there. It's kind of tricky because it's in a distributed system. So it kind of happens on the client and it might happen at different times in CRDT research.

I think that's one of the biggest unsolved problems. And basically what I want to do is just provide some very basic tools for doing that. And as people are using it, see which kinds of patterns emerge and then support those patterns with built-in solutions. So I'm thinking something like database migrations plus something like Zot data validation and being able to define your schema in a way that remains future compatible and making that as smooth as possible for app developers. But yeah, that's something I'm really excited to explore together with you guys.

Exciting. And we've got, I think time for maybe a couple more. We've got here, how's concurrency handled by Mesh? What are some of the, is this bottlenecks here? I'm like, I'm not quite sure what you mean with that. It's definitely hard to scale because you've got lots of clients at once and the idea of code. Jason is that the individual objects can be quite small and you can sync them granularly. I actually, to prepare for this demo, like I said, I ran some stress tests and on a single CPU core of the Mesh, I can handle about 50,000 transactions per second. And in the tweet example that corresponds to about like 400 users at once. And again, you get benefits of scale because people usually interact with the same values. And if they are all cached on the same server, that just makes things a lot easier. I don't know who asked the question. If, if you have a more detailed question, just feel free to come up to me later and I'm happy to, to go into more detail on that.

Yeah. And then we'll be heading over to the Q&A speaker area. So make sure to head there when we break in about two minutes. So we've got a couple more, I guess we've got, we can go with, I mean, this question's front loaded with the compliments. It was very cool project. So props there. How does JAZZ support same user multi devices syncing? So what I showed you is multi-player, but multi devices actually kind of the more boring simpler case where, um, if you, what you can do is try the to-do example on the GitHub that uses the touch ID based authentication. And what that means is like, if you create an account on your laptop and, uh, it will, if you use Safari or Chrome, this account will be synced onto your mobile device and you can use face ID there and you'll see that you are locked into the same account and the same way everything is synced between multiple users. Everything is also synced between multiple devices. So you literally, you don't need to do anything. You just get multi-device support for free if you build an app with JAZZ. I think. It got some last minute breaks. I was gonna, I was gonna like ask you to just pick one to kind of break the draw, but I feel like this one is quite rightly one.

So right here, we've got, how do you handle conflict resolution, like in Google Docs with like conflicting edits? So that's the big, uh, that's the big thing that CRDTs are kind of for. And there's a bit of a misconception where, um, some people promise that CRDTs will solve all your conflicts automatically. And they kind of do. Um, CRDTs mean that eventually everyone will see the same state. And I would say in like 90% of the cases, the automatic conflict resolution is kind of what you want and that works just fine.

But if you build more complicated apps, there might be app specific like meanings that the CRDT doesn't really understand. And in that case, it doesn't really help you directly, but what it gives you is it gives you access to the full edit history. You can see exactly what everyone was trying to do. And then you can build on top of that, something that makes sense in your particular world of your app.

Interesting. Giving a bit of freedom to developers. Thank you so much, Hans-Simon. Appreciate it.