Local-first Apps with ElectricSQL and React

Local-first is a new paradigm for developing apps with instant reactivity, realtime multi-user and conflict-free offline built in. This talk shows how to develop local-first systems with React using ElectricSQL, a open source software platform from the inventors of CRDTs.

Rate this content
Bookmark
Slides
Video Summary and Transcription
ElectricSQL is a local-first sync layer that integrates seamlessly with React, allowing developers to build modern applications directly on top of Postgres. This approach prioritizes local-first architecture, which enhances user experience by minimizing latency and enabling offline functionality. ElectricSQL employs a unique active-active replication model, supporting real-time multi-user sync across platforms even when offline. Developers can define Postgres schemas and use shapes to control data syncing, optimizing performance and resource usage. The system ensures transactional causal plus consistency, preventing conflicts and maintaining data integrity. ElectricSQL eliminates boilerplate code and simplifies operational processes, providing cost savings and a high-quality user experience. More information can be found on ElectricSQL's website and community platforms like Discord and GitHub.

FAQ

Electric SQL is a local-first sync layer that integrates with React to build modern applications directly on top of Postgres. It enables instant reactivity, offline capabilities, and multi-user collaboration by embedding a local database in the client.

James Arthur is the CEO and one of the co-founders of Electric SQL.

Local-first systems offer instant reactivity, offline capabilities, and eliminate network latency by embedding a local database within the client, which allows applications to function even if the server goes offline.

Electric SQL uses an Elixir sync service that runs in front of Postgres to handle bi-directional active-active replication, ensuring that changes in the local database are synchronized with the server and vice versa.

The CAP theorem states that a distributed system can only provide two out of three guarantees: Consistency, Availability, and Partition Tolerance. Electric SQL focuses on the AP (Availability and Partition Tolerance) side of the theorem to enhance application performance and resilience.

CRDTs (Conflict-free Replicated Data Types) are data structures that automatically resolve conflicts in distributed systems. Electric SQL's team includes two of the inventors of CRDTs.

Electric SQL achieves transactional causal plus consistency, the strongest consistency mode for offline-capable applications, ensuring no conflicts during offline writes and preserving relational data integrity.

Shapes in Electric SQL define what data is synchronized to the local device, enabling dynamic partial replication. Developers can specify subsets of data to sync, which helps in optimizing performance and resource usage.

Developers can start by defining their schema using Postgres tools, running the Electric sync service, and wrapping their SQL Driver with Electric SQL's API. Detailed guides and demo information are available on Electric SQL's website.

Electric SQL can be used to build applications like Figma, Linear, and other modern consumer or prosumer apps that require instant reactivity, offline functionality, and multi-user collaboration.

1. Introduction to Electric SQL#

Short description:

Hi, I'm James Arthur, co-founder and CEO of Electric SQL. We are a Sinclair for building modern apps. Our local-first Sinclair allows you to build applications directly on top of Postgres using React. We have a team of distributed systems geeks who have been researching and strengthening programming on the AP side of the cap theorem. This has led to a shift in application architecture, with local-first systems becoming more popular. These systems allow direct communication with a local database and provide instant results, offline functionality, and built-in multi-user collaboration. Let's now move on to a demo to see this in action.

So, hi, I'm James Arthur. I'm one of the co-founders and I'm CEO at Electric SQL. So this is basically Electric. We are a Sinclair for building modern apps. So, specifically, it's a local first Sinclair, which I'll talk about, and it allows you to build applications like Figma, Linear, directly on top of Postgres, using React.

Just to give you a bit of context on who we are, we're basically a bunch of distributed systems geeks. And so, the company comes out of a bunch of research into the AP side of the cap theorem, and we have a number of the people who've pioneered a bunch of stuff in that area on the team, including two of the inventors of CRDTs, etc. And basically, these academics, over the last probably couple of decades or more, have been working to basically strengthen what you can do with programming on the AP side of the cap theorem. And I won't go into the details here, but you can kind of look into a bunch of the kind of research.

And what that's now being used for is to change the way that you build applications today, and particularly, it's about targeting the state transfer piece of application architecture. So traditional cloud-first systems, you run on the server, they talk to web services over the network. Whereas what you have now is you have local-first systems, where you talk directly to a local database that's embedded inside the application, and then data syncs in the background. And this kind of architecture is now being used by a whole range of successful applications. So I mentioned things like Figma, Linear, you have the new Facebook Messenger, new Google Workspace applications, you have SuperHuman, for example.

And just to give you a very quick sort of sense of actually what's happening there, with a traditional cloud-first system, you have the network on the interaction path. So the user clicks a button, it sends a request to the server, the server sends something back to the user, and then you see the results. And so you have latency from going over the network, the server needs to be online, you get network errors that you have to code for every time you go over the network, and the user's out there watching a loading spinner or waiting for the page to load. Whereas, with local-first, you basically move the database, or a subset of the database, into the client, the user clicks the button, everything is instant, so there's no latency because you don't have the network on the interaction path, the user sees the result immediately, applications default to working offline, if the server goes offline, the application still works. Then you introduce this kind of live sync layer between the database and the client and on the server. And together, what this gives you is this model where you get a kind of holy trinity of modern consumer or prosumer great user experience, which combines instant reactivity so that apps just feel instant to use with built-in multi-user collaboration and also applications that can work offline and handle conflicts for you.

I'm just going to jump out at this point and let's hop over to a kind of demo just to sort of see that in action. I'll just change windows. So this is coming over to electric. Like first, I'm just going to jump in to give you a sense of sort of some of the code if that's useful as an orientation. So for example, like how do you get this thing set up? Right. So so we have an architecture where you have a Postgres database in the background. You run an electric sync service. So this as it happens is an elixir sync service. It's stateless. It runs in front of PostgreSQL and it handles that replication for you.

2. Data Access and Syncing#

Short description:

We generate a type safe data access library from the Postgres data schema. Define your schema as normal and use rules to control data exposure and permissions. Authenticate the local app using a JWT and wrap the SQL light driver with electrify to get a reactive live version of the database connection. The API uses shapes to control data syncing on the local device. Define shapes for dynamic partial replication and bind live queries for real-time updates. Write data directly to the local database and keep everything live and reactive.

What we then do is we generate out a type safe data access library from the Postgres data schema from the DDL schema. And so the way it works is you just define your schema as normal using whichever tooling you would use to work with Postgres. So something like for example, you then, we, we then provide a set of rules to be able to control what data is exposed to the system. And also who can, who has what permissions on what data. It's a bit like row level security, but it's a bit different because it's designed to be portable so that you can run the rules in the sync service and in the client that allows you to achieve something called finality of local rights. So you don't have to code for rollbacks and error every time you do it.

Right. You then authenticate the local app using a JWT, and then inside the client, you basically take the standard SQL light driver for your environment. So whether that's a web browser using a Wasm build of SQL lights, or if you're making a mobile app, it's like a react native driver or an expo or a capacitor driver. And then you wrap that with this call to electrify it. And that gives you a kind of reactive live version of the database connection that's also aware of your schema, and sort of knows where to connect to replication.

We then provide an API using a primitive called shapes, which allows you to control what data syncs on and off the local device. So if you imagine like you start with, say, a large postgres database, and you want to sync just a subset of the data onto the local device. You define a shape which could be like project 1234. So for instance, here, like, we can say a project and include all of the issues and comments and the comment authors kind of underneath that. So it's a bit like doing a kind of ORM query with an association graph. And then you can define multiple shapes as you sort of go along. And you can update them at runtime. And they will get aggregated into a kind of combined shape subscription for a particular client application. So that's an API that controls basically dynamic partial replication, what data syncs onto the local device.

And then once you have the local, once you have the data in the database for the local device, you bind live queries. So in this case, this is an example of like results as a react state variable, and use a use live query hook. And then you have a prisma inspired query builder for defining queries for how you want to access the data in the database. Or you can drop down to raw SQL. And then at any time, you can just write data directly to the local database. And basically everything stays live and reactive. So because you have this model of bi-directional active, active replication, kind of underneath the components controlling what data syncs on and off the device, instead of just doing sort of static queries or static fetches against the kind of backend service, what you do is you keep everything live so that if another user also changes the data somewhere else, your components also just naturally update. So here's an example of a simple React component. You get a handle on the electrified database. You set up a live query for all of the items.

3. Functionality and Syncing#

Short description:

You can create new items and pass them through a standard component, keeping everything reactive and working offline. Real-time multi-user sync is supported, allowing for cross-platform usage and resilient offline functionality. The system ensures transactional causal plus consistency, preventing conflicts and preserving integrity guarantees. Active-active replication enables updates across multiple instances of the app.

And then you have a function here to say, create a new item. And you just pass those through into a standard component. And everything stays reactive, everything works, the application works offline. When it comes online, data syncs. And if the component re-renders, whether you change the data, whether anybody else changes the data.

So just to kind of show you some of this in action. So for example, we talked about things being instant and super snappy. So this gives you an example of the type of latency you get to do a full re-rendering loop through the local database. It's in sort of single or low two-digit milliseconds. Whereas when you go over the cloud, this is talking to a backend service for the same application. A, the latency is much higher, but also just your availability and the quality of your user experience is at the mercy of the user's connectivity.

In terms of like real-time multi-user sync, you can see the same demo app. And now we have two instances of the same demo app where we're syncing via the server. So if I add an item on the left hand side, it syncs to the right hand side. If I move the slider, it syncs to the right-hand side. So it gives you this sort of built-in multi-user sync. It's also multi-platform. It works cross-web and mobile, and this real-time sync importantly works resiliently offline, right? So here's the same application, but now we have a connectivity hook. So for instance, say I drop user one offline, create two items here and clear the items here. When user one comes online, everything eventually syncs to the same state. So it's actually transactional causal plus consistency, which is the strongest possible consistency mode for this type of application. And it means that basically you can do offline rights and you don't get conflicts. And we also preserve a bunch of integrity guarantees around relational data, like referential integrity, which I probably don't have time to jump into now.

But just to show you one more very neat thing. I've mentioned it's this model of active-active replication. So here, for example, is the same demo app. Now, if I just bring up a terminal to follow the instructions. So what we have here then is I can grab a connection string to basically connect to the Postgres that is underpinning this example. And then this is gonna give me a command here, for instance, to update the slider value. And then if you come and see the application, as I hit this, you can see I'm just moving the slider around.

4. Syncing and Benefits#

Short description:

As you write content into the Postgres database, it sinks into the application. You can monitor the value of the same slider with a watch query. Local First Software provides a high-quality user experience, eliminates boilerplate code, and allows for cost savings and operational simplicity. For more information, visit the Local First Web Community and our website. ElectricSQL is a Local First Sync layer with React integration, providing a type safe client, APIs for dynamic replication, live query primitives, and framework integrations. Find us at electric-sql.com, Discord, and GitHub.

So basically, as you write content into the Postgres database, it sinks into the application, and as you edit data in the application, it sinks back into the Postgres. So just one last one, for instance, you can monitor the value of the same slider if we do that with a watch query. And then as you move the slider around in the browser, you see the value changing in the bottom left. And you can get some very interesting patterns around unifying state management between UE State and your application data in the same local database.

Cool. Well, I hope that gave you a very quick sense of the system. I mean, just to recap, so Local First Software, it gives you this really high-quality, resilient user experience. As the developer, it just eliminates the kind of crud layer and all the boilerplate around interacting with web services and handling network areas. And actually, it allows you to replace a diversity of back-end services with just a standardized replication protocol. So actually, with a lot of the applications that are running on this, they report back that it provides a whole bunch of cost savings and operational simplicity.

For more information about Local First, there's a really great community, which is the Local First Web Community, if you Google for that. There's a manifesto where a research group called IncanSwitch's gonna define the term Local First Software. And on our website, there's a bunch of things where you can go through some of those demo guides and kind of dive into the model. So, and again, just ElectricSQL. So what are we? So we're a Local First Sync layer with React integration. We basically do the Local First Sync properly between Postgres and React in your application. Give you a type safe client. We give you these APIs for dynamic partial replication based on shapes. Give you live query primitives, the reactivity model and a bunch of framework integrations. Everything's open source and it's designed for simple self host. For more information about ElectricSQL we're electric-sql.com. We have a Discord community if you have any questions or want to keep track of the project and you can see us on GitHub at github.com slash electric-sql.

James Arthur
James Arthur
12 min
12 Dec, 2023

Comments

Sign in or register to post your comment.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Everything Beyond State Management in Stores with Pinia
Vue.js London Live 2021Vue.js London Live 2021
34 min
Everything Beyond State Management in Stores with Pinia
Top Content
State management is not limited to complex applications and transitioning to a store offers significant benefits. Pinia is a centralized state management solution compatible with Vue 2 and Vue 3, providing advanced devtools support and extensibility with plugins. The core API of Pinia is similar to Vuex, but with a less verbose version of stores and powerful plugins. Pinia allows for easy state inspection, error handling, and testing. It is recommended to create one file per store for better organization and Pinia offers a more efficient performance compared to V-rex.
Using useEffect Effectively
React Advanced 2022React Advanced 2022
30 min
Using useEffect Effectively
Top Content
Today's Talk explores the use of the useEffect hook in React development, covering topics such as fetching data, handling race conditions and cleanup, and optimizing performance. It also discusses the correct use of useEffect in React 18, the distinction between Activity Effects and Action Effects, and the potential misuse of useEffect. The Talk highlights the benefits of using useQuery or SWR for data fetching, the problems with using useEffect for initializing global singletons, and the use of state machines for handling effects. The speaker also recommends exploring the beta React docs and using tools like the stately.ai editor for visualizing state machines.
React Query: It’s Time to Break up with your "Global State”!
React Summit Remote Edition 2020React Summit Remote Edition 2020
30 min
React Query: It’s Time to Break up with your "Global State”!
Top Content
Global state management and the challenges of placing server state in global state are discussed. React Query is introduced as a solution for handling asynchronous server state. The Talk demonstrates the process of extracting logic into custom hooks and fixing issues with state and fetching logic. Optimistic updates with mutation are showcased, along with the benefits of using React Query for data fetching and mutations. The future of global state management is discussed, along with user feedback on React Query. The Talk concludes with an invitation to explore React Query for server state management.
Jotai Atoms Are Just Functions
React Day Berlin 2022React Day Berlin 2022
22 min
Jotai Atoms Are Just Functions
Top Content
State management in React is a highly discussed topic with many libraries and solutions. Jotai is a new library based on atoms, which represent pieces of state. Atoms in Jotai are used to define state without holding values and can be used for global, semi-global, or local states. Jotai atoms are reusable definitions that are independent from React and can be used without React in an experimental library called Jotajsx.
Announcing Starbeam: Universal Reactivity
JSNation 2022JSNation 2022
27 min
Announcing Starbeam: Universal Reactivity
Starbeam is a library for building reactive user interfaces with JavaScript, similar to Svelte, Vue, and Ember. It provides a data structure and query feature for filtering and sorting. The useStarBeam function ensures JSX reconciliation only occurs when reactive dependencies change. Starbeam tracks every read and write operation to update the component accordingly. It can be used with React and other frameworks, and offers debugging tools and locale integration.
Thinking in React Query
React Summit 2023React Summit 2023
22 min
Thinking in React Query
Top Content
Watch video: Thinking in React Query
React Query is not a data fetching library, but an Asian state manager. It helps keep data up to date and manage agent life cycles efficiently. React Query provides fine-grained subscriptions and allows for adjusting stale time to control data fetching behavior. Defining stale time and managing dependencies are important aspects of working with React Query. Using the URL as a state manager and Zustand for managing filters in React Query can be powerful.

Workshops on related topic

Rethinking Server State with React Query
React Summit 2020React Summit 2020
96 min
Rethinking Server State with React Query
Top Content
Featured Workshop
Tanner Linsley
Tanner Linsley
The distinction between server state and client state in our applications might be a new concept for some, but it is very important to understand when delivering a top-notch user experience. Server state comes with unique problems that often sneak into our applications surprise like:
- Sharing Data across apps- Caching & Persistence- Deduping Requests- Background Updates- Managing “Stale” Data- Pagination & Incremental fetching- Memory & Garbage Collection- Optimistic Updates
Traditional “Global State” managers pretend these challenges don’t exist and this ultimately results in developers building their own on-the-fly attempts to mitigate them.
In this workshop, we will build an application that exposes these issues, allows us to understand them better, and finally turn them from challenges into features using a library designed for managing server-state called React Query.
By the end of the workshop, you will have a better understanding of server state, client state, syncing asynchronous data (mouthful, I know), and React Query.
State Management in React with Context and Hooks
React Summit Remote Edition 2021React Summit Remote Edition 2021
71 min
State Management in React with Context and Hooks
WorkshopFree
Roy Derks
Roy Derks
A lot has changed in the world of state management in React the last few years. Where Redux used to be the main library for this, the introduction of the React Context and Hook APIs has shaken things up. No longer do you need external libraries to handle both component and global state in your applications. In this workshop you'll learn the different approaches to state management in the post-Redux era of React, all based on Hooks! And as a bonus, we'll explore two upcoming state management libraries in the React ecosystem.