Supercharge Your Full-stack App with a Reactive Database

My name is Tom. I am the head of developer experience at Convex. This is kind of a fast talk. There's not a lot of time, and so I'm going to try to get through at least something that I found super interesting. I personally came across Convex about nine months ago. And it kind of changed my life in a lot of different ways.

One, I work there now. But two, I spent many years just building. I love to build. I built lots of things up and down the stack. I tried different technologies.

Convex was something that I came across, and the more I dug into it, the more I was relieved that I no longer had to make so many decisions that I find myself having to make every single time I want to fire something up. And I was excited, and then I was... I'm not sure what the emotion was. I wanted to get more developers to know that this thing just exists. It was really something. It was quite an experience for me.

And so that led me to researching the company, reaching out to the founders. We had a bunch of conversations, and ultimately, I am here now talking to you. So I suppose in some ways I am fulfilling that vision that I had.

So Convex is your friendly neighborhood sync engine. I'm gonna speed through a little bit of the technical stuff here. We have tons of blog posts about how it all works. The technical parts are super, super interesting.

Convex itself is a sync engine that includes database, file storage, vector embeddings... Other things I'm missing. Full text search. It's written from the ground up in 200,000 lines of Rust. It is hosted on our platform, which comes with an admin dashboard, and team member support, and usage data. But it's also open source. So if you choose, you can self-host.

So in general, the way that it works is that you write your Convex code right alongside the rest of your code. In this example that we'll go through today, it's React and TypeScript. We have clients for a whole bunch of different languages. But you put it all in the same... You don't even... NX is amazing. If you saw that talk, they do monorepo management. In this case, you don't need that.

They just live together in perfect harmony. The difference is that the code in your Convex folder gets compiled every time you hit save. It gets compiled. It's TypeScript. It gets compiled and then it gets deployed to your Convex instance, of which every project has at least two. There's a development instance, and then there's a production instance, with more redundancy. Your typical web app works just like normal. And then the Convex portion of it is this database and literally your code that's been shipped there that runs in a V8 isolate on the same machine as your data, which makes it wicked fast and effectively absolves you of any caching concerns whatsoever.

So when a client, like a web browser, hits your site, all the normal stuff happens and then anything to do with Convex, so your data or your state management, and we'll get to that, is attached to your Convex deployment through a web socket. Also wicked fast. This one, I'm sure I can make the slides available.

This is also available through a really good blog post by one of our co-founders called How Convex Works. This is diving into the web socket side of things.

So this is basically your instance. This is your machine that's running. It does a bunch of fancy stuff. There's a function runner. There's three types of ways to really... Three main ways, I should say, that you interact with Convex. And those are through functions. And a function can be an action, which is sort of like an arbitrary set of code that you could use to hit third party APIs. That also has access to your data and your database as you allow it. Actions, queries, and mutations.

Queries just what they sound like. Mutations are anything that writes to the database. Those are the three main channels that you would really ever use to interact with Convex. And then it has the database and it does all kind of cool stuff. I was telling you about the first time that I experienced Convex. And my immediate thought was, like, holy crap. I literally don't need to use useState anymore. When I started writing this code, I was like... I started replacing all my instances of useState with useQuery. And useQuery is actually this like active web socket hook to your Convex data. So you can store what used to be your entire local app state globally. And it's super fast. There's a whole bunch of optimizations, including optimistic updates and things like that. So it feels... I mean... Like, technically it is exactly as fast or, you know, exactly as you can be. And so I was like... Well, I'm just gonna kind of use this...

I'm gonna use this everywhere. Now, I do use it everywhere, except for the most, like, minute local state things. But even things like... Is the sidebar collapsed or not? And that's pretty much the most trivial example of using this.

We have customers who have built large scale applications with thousands and tens of thousands of customers on it. So this is what I want to do. With only half the time left to talk about it, I want you to go to this website, dxinit. We're gonna watch it in action. So if you go to... You can go to dnx... There we go. All right. We already got some folks. Let me see if I can double screen this here. It should work decently on your phone. CSS is... I mean, I love it. I'm not that great at it. But here, what I want to show you is... So there's an anonymous log in right here. No log in. I give you a fake random name. If you refresh, you'll be a new person. Hey. That's kind of deep.

Over here, we have the data. We're watching the data roll in. Roll in, roll in. We can see exactly what's happening. Everybody's device is going to be showing the same thing right now.

And if I were to be like... I don't know. I don't know about this app. I'm just gonna delete it all. Everybody's app also just closed out. Oh, Ashley's back. We're all back.

Sync engine. So let me show you how this works in our chat application. So here we have a component. This is the chat component. So this is doing most of the list view and the send button and the input and all that good stuff. And to list the messages, we just use this. We use this use query call right here. And what use query is referencing the API.messages.list. That is the name of a very simple query that we've written in our convex folder. And we've done so just by using the query function. This one takes no arguments. It's reading the messages from the table in descending order, which by default is the created at date. And it's taking the first 200 of those. And so this code gets compiled and is running on your deployment.

When we go back over to the component here, use query is connecting to that via a WebSocket. And so any time any of the underlying data that would be returned by that query changes, every client that's subscribed to that, just by that line, will get those updates instantly in real time. And the same holds true for mutations. So to show you one of these mutations, here's the send mutation. So we use it. Let me just show you in the component, in the react side, where we're using it. So we have handle submit. We got to prevent the default. And then we have our send message.

And you just pass in the arguments that you have defined that are also validated all the way through. So one of the most beautiful things about this is that it's type script all the way down. So you have this type safety that crosses the chasm from your back end to your front end to your components. And the IDEs are all happy. VS Code knows what's going on. You get the type completion. It's very satisfying. It's very ergonomic. It brought a lot of joy back to programming for me.

So here, we're calling it with the body, the author. And then we're using this hook called use mutation, which is, again, it's a mutation, but it's connected to your instance, just like every other client that's running your code. And it's going to call this function send with those arguments. And here's what it looks like. I mean, it's pretty straightforward. We have a mutation, which means we are writing it. There's some funny things here, though. Sarcasticize. That's cool. Add sparkles. Huh. I wonder if we can get those to work. And then we insert it into the database. And that's it. So that's like those four things, I suppose.

The consumption of the query, the use of the mutation, the query definition, and the mutation definition were everything that was needed for all of you to be able to use that chat application with each other right now and get all of that data in real time. So the last thing, with just a couple minutes left, I'd love to do a little bit of a pressure test. One of the philosophies that we have is around removing as much of the burden or onus from developers and offloading that onto the platform. I believe if you can imagine this perfect world in which developers can spend their time coding their ideas, the novel things, the interesting parts of their application that are not necessarily related to infrastructure or the fact that somebody ultimately has to use it somehow to stick to the ideas. So I subscribe to this. I mean, it's an aspirational, lofty, potentially impossible goal, but it's not the point.

I would love for our tooling to keep moving along that spectrum, removing the burden from the developer from being able to do anything but the interesting parts of development.

Let's jump back into, so we have Sarcasticize. All I have to do is remove that, remove that, and remove that. So where I was going... Wait, I got to deploy this real quick. Shout out to Fly. Where I was going with that is that we've spent a lot of energy designing Convex to be super ergonomic on day one and effortlessly scalable to year two, year three, year four. And what I mean by that is that you don't have to understand machine provisioning. You truly don't have to worry about your scale. We have customers that have a lot of scale and we're working our way up. I don't think we could probably onboard a large enterprise right now. But we've had a lot of success by removing those barriers to success, basically. And part of that is managing all the complex, deeply mathematical challenges that go along with scaling distributed systems like databases and managing database contention.

And so let's see. Once this is done, let's see if we can blast the bejesus out of this server. We shall see. Okay. Checking. Okay. I'm sure it's all good. What could possibly go wrong here? So if we refresh our DX init. Yeah, there we go. So now we have the option of sending things sarcastically. So you can just type your message and be like, this is not sarcastic at all. There we go. That's right. So we can do that. The other thing that we can do is if somebody's feeling like not quite so sprightly today and they're like lay off the sarcasm. I don't need that right now. You can hit the make serious button and it will remove all that sarcasm and just feel nice and calm.

There's a little sparkle button there, too. The make serious button and the make sparkly button, every time it's pushed, is pulling in every single message in the database, rewriting it, and putting it back out there. And so you can imagine as there's people in the room all doing the same action, writing to the same rows in the database at the same time, and we have gone deep down the concurrency rabbit hole to make sure that this remains smooth and scalable forever. Again, I never have to think about it again. It just happens.

And this was just, because this was a relatively short talk, I just wanted to pick a demo to try to convey that same feeling to you that I felt the first time I used Convex. It absolved me from having to worry about what database to choose. Am I going to get roasted on Twitter for making this choice because, I don't know, MySQL got canceled last week for some reason. I don't know. I can't keep up with all of that.

And so every time, what is it, every two or three months I go to start a new project, I'm like, well, Twitter, X, what's the database today, huh? And then I'll go and I'll learn it and I'll be like, this is cool. And then three months later, I'll have to do that again. It's not just the database, it's the deployment, it's the frameworks, it's the stack. So that was like this really key thing for me that I was like, oh, now every two or three months I've reduced all the friction that it takes for me to put my ideas into action. It's been just very enjoyable.

And I reached out to the founders to be part of this because this is, well, there we go. Cool. So here, it says, break this web page. I'm just going to delete that message. It should be gone. Not enough information is being collected. I'm going to delete that one, too. I don't even know what that means. Oh, it's okay, Daniel. It's okay. All right. But this is what I mean. I can go in here and I'm just editing the table. And it's affecting everybody. Which is freakishly powerful.

So yeah, thank you very much. I have, yeah, down for some system of a down.

Okay. I think I probably had a last slide that says, oh, yeah, make some friends, add some sparkles, contention, contention, contention. You know how it goes.

Anyway, I really, really appreciate you all taking the time to choose this talk with me. I know it's late in the day. It's a lot of coffee. I just I really do appreciate it. And thank you so much for your time. If you want to learn anything more, come chat with me at the Q&A. I'm going to talk here and then over there. And yeah, hit up convex.dev and start building your dreams.

Why store so much client state in a server instead of keeping it local? Does that scale well?

Yeah, for sure. So that is I mean, 1000% application specific. There's a lot of great reasons not to store any number of pieces of the client state on the server. However, for anything that you would want the user to experience consistency within for effectively free, if they were to switch between sessions or devices, like assume this person is off has has like logged in has a session. If there was anything at all that you wanted to maintain between those two experiences, that would be the opportunity to to put something in the in the database. And yes, it I mean, it absolutely scales well. And then yeah, I mean, follow it'd be logical to, to do certain things in the client that kind of just makes sense there. Yeah, I think you can use your use your best judgment for that. But it makes it so easy to make everything synced.

But it makes it so easy to make everything synced. Cool, I'm gonna get to the next question real quick.

I do want to ask you and I was just kind of going through the the convex dev site. You do have a few case studies. I mean, there are a couple that you want to or one in particular you'd sort of like want to discuss in like one or two minutes and you know, as it's like a success. I'm not allowed. I'm not sure what I'm who I'm allowed to talk about.

I have on the who do we have on the website? I mean, I don't want to get you fired. Well, if it's on the website, it's all good. Okay, let me check. Like, I have no separation between like, conversations that we have inside and Okay, how about tunnel the two week migration story? That's, that's a good one. I'm not totally sure and I don't want to get it wrong. Okay, cool. We'll get back to the q&a. Give me a second.

What are the performance limits and implications of handling 1000s of concurrent clients since it's using Websockets? That's a very good question. That is something that we ultimately end up contending with quite a bit. So to the there's some really cool computer science stuff going on in the back, that I try my best to ask a million questions about and understand, but frankly, I kind of only have a surface level understanding of how that works.

However, overall, the way that convex instances work is that we have a massive pool of resources that we dynamically allocate in real time as required. And so what that means is that if you have 1000s of users connected to Websockets, those Websockets are not always connected to the exact same like physical instance, it will appear to be the same for all intents and purposes. But from a scalability like a horizontal scalability point of view, we will automatically provision deeper vision and do load balancing between all of these physical instances. That is how we ultimately end up managing.

We have a handful of, oh, here's a case study. We have a handful of customers that go viral every once in a while. So there was like a video AI generator about dogs or something on Reddit a couple of weeks ago that was built on on convex. And so that was a really cool load test because like it went it went crazy. So it was a really interesting technical kind of systems load test for us to watch. And it went really it went smoothly. We're all happy and suspicious.

Awesome. What's the DB? Is it defined by you or you just connect to whatever?

Oh, yeah. So I love this. The database is written from scratch in Rust from the ground up. It's not Postgres, it's not MySQL, and it's nothing in between. One of the reasons I was so compelled after doing some research into the company was that the founders were the actual boots on the ground team that built Dropbox Sync, define the protocol, implemented it, and made it work at that scale. And so this was a team with really, really impressive chops when it comes to distributed systems. And they took everything that they know about distributed systems and databases and built what they thought needed to exist. So it is a novel, fresh, from scratch database built in Rust. It's open source. Check it out.

Awesome. Yeah. I like this question. WebSockets uses persistent TCP connections, and we are limited by the number of open connections on a single server. Any scaling concerns? So as far as I understand, and I might be mistaken, that is resolved by what we're doing this orchestration that we're doing in the background where we will spin up as many instances as required to safely handle any kind of load and then transparently manage all of the load balancing. It might go deeper than that, but I can find out. But that's how I understand it. Word, word, word.

Do you have any examples of more complex queries, e.g. an example with joins, etc? Yeah, yeah, for sure. So we have a whole suite of example repositories that show everything from the trivial, like I showed here today, to much more sophisticated examples and full-scale production-ready, testable applications. So I can't show you anything here, but yes, absolutely. So there's indexes. You can define your schemas and relationships between tables. Now, the schema is optional. So this is what I mean when I say we try to make it really easy on day one to kind of like create and prototype your idea. We know that like data models and your systems evolve, but then you can at any point enforce schema validation. Yes, and then you can do all kinds of complex queries, performant queries, joining objects, all that stuff. We have tons of example repos.

Awesome. Last question, and quick one, lightning answer. Tell me about Convex for Startups.

Yeah. Convex for Startups is this awesome program run by our head of community, Wayne. And it's been a resounding success so far. So what it is is we offer, there's a whole bunch of perks. There's like free access to Convex. There's like founder Q&As. There are, we do like code walkthroughs. And so it is a startup program that anybody can apply to for Convex. And we'll take a look at it. And then we've seen some of the most interesting projects come through. And that's really what we're looking for, like really interesting kind of like novel applications.

Awesome. Thank you very much for your time, sir. Thank you, sir. Give it up for my man, Mr. Redmond. Mr. Tom Redmond.