Local First – Collaboration Beyond Cloud

So, if you're someone building apps with collaborative features in them, hold tight. Today, we're going to fundamentally rethink how we build web apps, especially those where users actively work rather than just consume. Many of the apps we use daily are primarily for consumption. Think of a news site or a streaming service. Their core function is to deliver information to you. But then there are others where multiple people are actively creating and modifying shared content. This is where users are truly working together on a shared item while also getting the benefit of updated information.

While a prime example of this is Figma, where designers and developers collaborate in real-time on a single canvas. Some of the other popular ones are Trello, Miro, Canva, and Google Docs. When we think of these collaborative features in these apps, our default is to lean heavily on a big centralized server to power them. And for good reason, these specialized servers in the cloud offer power and scalability. It's a no-brainer. But this reliance comes with a cost.

Have you ever experienced a collaborative app that stops working when you lose connectivity, leaving you unable to continue your work? Well, the purpose of this talk today is to explore an extension to our collaborative apps such that they can deliver amazing experiences without being held back by connectivity and the synchronization challenges that come with not being connected to a server. I'm here to introduce a paradigm that allows us to build multiplayer experiences that thrive offline, sync smoothly when online, and harness the value of localized state, allowing the users to own their own data. I'm here to introduce LocalFirst. Well, before we get into solution mode, let me show you how a normal app would behave under unpredictable network conditions.

I'll take the example of Trello, a great product that I love and I personally use regularly. And on this board is my plan for the conference. With normal internet, as you can see, I can switch between swim lanes and cards are just moving fine. But say I'm working on my way in a cab and I go through a tunnel, which means patchy internet or slow for that matter. And I hit refresh. It takes a minute, but it ends up loading and it's fine. I'm still able to work.

But say I lose connectivity completely. What happens then? As you can see, as I go off, you can see the little toast message on the left bottom that tells me that I cannot work anymore. And you might have already noticed my ability to make any changes is hindered. But nothing an offline mode cannot fix, right? Just make the app a progressive web app to make it available offline and let the work continue. But then what about drift?

The longer I keep working offline, there is a high chance that the colleague who has been helping me prepare would have made some changes to the board. While I take the example of this board, a more realistic scenario is that of Git, where we inherently keep a local copy of our code and then have to resolve conflicts manually as they appear. Simply put, it's not quite that straightforward. Well, primarily because for the last couple of decades, we've built our apps around a simple premise.

There's a central server somewhere that holds the truth. Our clients, be it mobile, web or desktop, are essentially just windows into that truth. Every interaction, every change, every piece of data usually has to go back and forth to that single source of truth. Well, this approach gives us a sense of control, ensures consistency across users and provides that much desired single source of truth for our data. And it works for a lot of apps until we start building right heavy collaborative features in our apps.

And then if a central source of truth or the central truth is slow to respond or unreachable, the app grinds to a halt. The instant responsiveness we expect vanishes, and it may sound a bit dramatic, but our data is held for a ransom in a server never to be seen again. Well, this is where we have an opportunity to provide a better user experience, where the local first paradigm offers a powerful extension to the capabilities of our apps. Local first shifts control of the data directly to the user instead of the cloud.

Your data resides on your device, giving you immediate access and ownership. Regarding consistency across users, it completely reimagines it. Through synchronization mechanisms and conflict resolution strategies, often working in the background, local first applications ensure that while you have immediate access to your data, your changes are eventually and reliably propagated to other users. This maintains a cohesive shared state without relying on a constant live connection to a central server.

And finally, the most impactful of them all, being able to work offline. Because your data is local, your applications are instantly responsive. You're no longer waiting for network round trips or server processing, whether you're on Apache Wi-Fi connection, deep in a tunnel just as we were, or even completely without Internet. Your collaborative experiences continue to function seamlessly.

All right, so let's kick things off by seeing a small demo of an app that I put together. This is an app that uses the local first approach. And right here, there's two instances of a to-do list app running in two different clients mimicked by two browsers. We can imagine two different users using this, you and your colleague. As Panda on the left adds a to-do list item to the list, it just seamlessly shows up on the other client and vice versa as well, which is on the other end, adding Find Eucalyptus, which once clicked on add, shows up very easily. This initial glance shows you the powerful user experience we can achieve.

Now let's talk about the underlying technology that makes it all work. Well, this local first app and many others like it use Yjs, a powerful library to enable collaborative features while maintaining offline functionality. At the core is the Yjs package. This provides the Y doc, which is essentially a shared real-time data structure that lives on each client's device. The beauty of this shared type, like Y map or Y array or Y text, is that they are very familiar to their JavaScript counterparts in the APIs that they expose. To illustrate the difference between the two, let me just show you what a standard React component that would have a to-do list look like.

It's pretty much this. There'd be some sort of a local state. There would be an input field to be able to accept that value, input value, and then a bunch of operations on that value. So add, edit, toggle, and delete to push things in out of the to-do list state. With Yjs, we would start a step above, which is by instantiating a Y document. This document is where all of the collateral that you want to put into the Y doc would be stored. Every client has its own complete and authoritative copy of the document, saved directly within these special Yjs data structures. The server's role, though, has been fundamentally reduced. It now acts as a lightweight sync server, simply relaying updates between clients, rather than being the central authority that holds the one true version of the data.

With Yjs, we would start a step above, which is by instantiating a Y doc. This document is where all of the collaborative data lives locally. We then define our shared structures within it, like a Y array named Ytodo's to hold our list of to-do item objects.

Every client has its own complete and authoritative copy of the document, saved directly within these special Yjs data structures. The server's role has been fundamentally reduced to a lightweight sync server, relaying updates between clients rather than being the central authority holding the data.

By using Y todos and the Y useEffect hook, we bridge the Yjs shared state with the local React component's state. This allows any CRUD operation to directly modify the shared type Y todos, enhancing the collaborative power of Yjs for synchronization and offline support.

The server's role, though, has been fundamentally reduced. It now acts as a lightweight sync server, simply relaying updates between clients, rather than being the central authority that holds the one true version of the data. This is a powerful shift. While I'm using a simplified example, a natural extension of this server could be one that also maintains its own snapshot of the data, perhaps for backup or some server processing or user experience and analytics. But crucially, the user continues to possess their local copy and directly on their machine, ensuring they own their data and their work remains instantly accessible.

Back to the code. Here's what the local first to-do list component looks like, which leverages Ytodos. Notice the Y useEffect hook in the component. This hook is crucial for bridging the Yjs shared state with your React component's local state. This line sets up an observer or an event listener on the Ytodos shared array. Whenever Ytodos changes, say due to a local edit or a remote sync, the updateTodos function is called. Inside updateTodos, we convert the Y array into a standard JavaScript array, which then updates our local React state using setTodos.

UpdateTodos is called once initially to populate the local state with the current contents of Ytodos. And finally, we unobserved on the Ytodos array to ensure the observer is cleaned up when the component unmounts and preventing memory leaks. Well, with this setup, any CRUD operation, so add, edit, toggle and delete, would directly modify the shared type Ytodos rather than the local to-do state, which would have been the case if we were not using a local first approach. That was easy, right? So, so far, we have created a local Y document and swapped out the use of local state and instead are now using a Y array.

So basically just saving our list in a different area. Notice that we did not have to change much of the component's logic itself. The key adjustment was simply using the Yjs shared type instead of the local use state. The operations like adding or deleting still feel very familiar, but now they're backed by Yjs's collaborative power. This sets us up to now extend our app's ability to leverage Yjs for synchronization and support and offline support.

And for that, we introduce Yjs providers. These are modular pieces that connect your local Y doc to various external systems for persistence or real-time communication. The first one we look at for real-time collaboration is Y WebSocket or WebSocket. These are independent. To make these independent copies talk to each other in real-time, especially when we want to collaborate live across different devices, we use Y WebSocket. A Yjs provider that handles the synchronization of document updates over a WebSocket connection to a lightweight server.

Here's how our Y provider DS file on the client gets updated to include the WebSocket provider package. And just like that, as you can see, all we're doing is importing a new class, instantiating it, and that's about it. What we're doing here is that we're letting Y WebSocket handle all incoming and outgoing changes for us. When changes happen on one client's Y doc, for instance, in our case, the to-do item, Y WebSocket sends those small incremental updates to the configured WebSocket server. The server then acts as a simple messenger, broadcasting those updates to all clients currently connected to the same room name.

Crucially, the server is a simple relay. It doesn't perform complex conflict resolution or store the definitive truth. That heavy lifting, the merging, and ensuring consistency is within each Y doc on the client themselves. This design keeps the server light, scalable, and focused purely on data transmission. Now to truly thrive offline, we need a way to process the data directly on the user's device so that even if they close their browser or lose Internet connectivity, their work is safe and immediately available. That's where IndexedDB comes into the picture.

Yjs leverages a package called y-index-db-provider to automatically save the entire Y doc to a local IndexedDB. This means that even if the user closes their browser, loses Internet connection, or reboots their computer, their local copy of data is safe and immediately available at the moment they reopen the app. Pretty much what local storage would do in this case, right? Only just a little more powerful because it's being managed in incremental snapshots. This is the core of enabling the instant responsiveness and seamless offline work we discussed earlier. Your app now always has access to data regardless of network conditions. This diagram here illustrates the simplified flow. Each client, as you can see, has its own application powered by a local Yjs document. This document is where all of the data lives.

For offline support, the y-index-db-provider automatically persists the document to IndexedDB on the client's device. When changes occur, the y-web-socket-provider sends these compact incremental updates to a lightweight sync server. Since this doesn't manage, state, or resolve conflict to the server, it simply acts as a dump pipe, relaying updates to all of the other connected clients if they are connected to the same room.

Each receiving client then applies these updates to their local Y-doc managed by CRGTs, and more on this later, to ensure conflict-free merging. Now that we have an app which has all of the providers set up and is all perfectly local first, let's look at some of the common collaborative scenarios, focusing on how Yjs handles additions and deletions within an app, especially when users are working concurrently and offline. This will show how Yjs and its underlying concept of CRGTs, more on this, ensure lists remain consistent and ordered.

In this scenario, we've got two collaborators, they're the same Kuala and Panda, working on the same to-do list. First, let's establish a baseline. So, let's get Kuala to delete an item, and as you can see, just as easily, the Panda's item list also gets updated, and Panda, as it deletes Kuala's list, also gets updated. So far, so good. Simple deletions, instantly synchronized, the list is now consistent. Now for the real test. Both Kuala and Panda intentionally go offline.

As you can see, the indicators indicate up top that they are indeed offline. Now, while offline, Panda adds a new item to the list, and obviously because they are offline, it does not instantly update on the other screen. Shortly after, still offline, Kuala also adds a new item to the list. At this point, while they're both offline, they each see their own list. While offline, they each see their own newly added items appearing on the top of their respective lists. Their local copies of the document reflect their latest changes.

Now for the moment of truth. Wi-Fi getting turned back on, and their clients reconnect to the server. As you can see, Panda's item is first, and Kuala's is second. This isn't random. Yjs, for its Y array shared type, uses conflict-free replicated data types algorithms for lists like replicated global arrays or simple order preserving CRDTs. These algorithms have a deterministic way of resolving concurrent insertions and maintaining a consistent order across all replicas, regardless of network latency or the precise wall clock time of an operation. If you see, both of those lists look identical, even if they started off with both Kuala and Panda seeing their item as the first one on the list, and that's what we're talking about.

Now, while the exact algorithm can be a bit complex, the key principle that every operation gets a unique identifier that includes its origin, which is the client, and a logical timestamp. When two concurrent additions try to occupy the same logical position in the list, which is what was happening just then when both Kuala and Panda wanted to add an item to the top position, the algorithm uses a tie-breaking rule, usually based on their unique identifiers or specific properties of the logical timestamps, to consistently decide their final relative order for all clients. In our demo, Panda's item simply had a unique identifier that caused it to be ordered before Kuala's when the two concurrent offline additions were merged. The important takeaway is that the outcome is guaranteed to be the same across all clients. You don't get a drift state where Kuala sees their item first and Panda sees theirs. Yjs ensures that despite independent, concurrent offline work, your shared data converges to a single, consistent, and logically ordered state across everyone.

Cool. Let's level this up a little bit. What if both clients were editing the same item? Let's see that in action. So here now we've got, again, the same state of Panda and Kuala. And in this instance, only Kuala goes offline. And that can be seen by the indicator up top. Panda, while still online, is now editing the first item on the to-do list. And Kuala, on the left, now edits their first item on that list, on the same list as well. As Kuala comes back up online, we notice that both Panda and Kuala sees Kuala's edits while it was made offline.

In this specific case, Kuala's change won. And this happens because Yjs for individual property updates typically uses a type of CRDT called last write wins register. It is designed to ensure that the last recorded operation for a given piece of data based on their unique timestamp, which is internally handled by Yjs, is the one that prevails. It's a very common and intuitive strategy for many types of conflicts.

For another scenario, what if we took it to another level? Which is, Kuala goes offline, and Panda, frustrated by the fact that it lost the last time, has actually now deleted the first item from the list altogether. Kuala, because offline, is oblivious of that and is now going ahead and editing it another time, trying its luck. As you can see in this case, Pandas delete one, and the item remains deleted. Even while Kuala's edit was later. This demonstrates a subtle but powerful aspect of CRDTs.

While the last win register generally picks the last edit for operations involving additions and removals within a set or list, Yjs often implements a strategy that favors removals when an item can be concurrently edited and deleted. In many collaborative contexts, a deliberate delete operation is often considered definitive. Even if an edit occurred concurrently such as ourselves. This is the fundamental problem that Yjs solves. With an incredibly elegant and powerful solution called conflict-free replicated data types, conflict CRDTs are special data structures that are designed to be merged automatically and correctly without requiring complex conflict resolution logic, the kind that we're used to with Git.