Exploring Modern Databases in React Applications

I'm going to be talking about modern databases in the React ecosystem. And first, a few more words about me. I do run developer experience at HDB currently. I was previously working a lot with open source at Bleach.js and also around GraphQL at The Guild and before at Hasura. You can find me on Twitter as Alexandra says. So if you have any questions or you want to talk about anything databases-related, don't hesitate to reach out. Or you can check out my personal website, alexandra.codes.

So not too long time ago, the development work was simple. So we had front-end developers in the world of UI, and we had backend developers dealing with databases, servers, and so on. And there was this wall between them. And when I wanted to build a feature, the flow was like the design team would design the application, then the backend team would design the API, then everyone would have to argue about the API, and then front-end developers would have to wait maybe a few weeks for the backend team to be done, then they can start their work. And if there are any inconsistencies, the work would start all over.

But then something interesting happened in 2009 with the development of Node.js, because suddenly JavaScript developers were able to write server code as well. And that slightly changed the ecosystem. The mean stack was very popular, so Mongo, Express, Angular, and Node. And then we also had Next.js appearing in 2016. And in 2019, Next.js introduced API routes, which made it a full stack framework. And now I also want you to take a look at some of the recent years. So in 2020, that was also a point where a lot changed in the development ecosystem. So we had Prisma beta that kind of led to development of a few tools, for example, Bit.js, where you were able to import server code directly into your client components and run it without fearing that it will leak to the browser. And then we also had a few frameworks like Redwood, T3 with trpc-remix. And in 2022, Next.js 14 announced the integration with React server components. So with all of those changes of the ecosystem changing and frameworks evolving into full stack ones, databases landscape also changed. So now that we are able to jump in and out of our servers within our front end, within our client components, we can move this network boundary as it suits us. So with that, we can now explore some of the modern solutions and their strengths when it comes to databases.

And that's especially important because front-end developers are more often writing back-end code nowadays, and React is now a full stack framework. So what is the most important component of back-end? It's a database. And that's the foundation of our application where we store our data and that data powers our features. And I'm not going to give you a recipe on how to pick the best database for your project, because that decision can be a bit more nuanced than just looking at the features. There are some decisions like relational, non-relational, OLAP versus OLTP.

You see, it's hard to pronounce, so that's a lot of decisions. And those decisions should be done, like, very early, because they are costly to go back on. But instead of that, I hope to give you an overview of what you can expect in 2024 and what are some of the features that you can take into account when looking at databases. Because the truth is that, not only in the frameworks world, a lot has changed, but also in the databases world. And if you're not, like, following the databases world, it might be, like, you know, there are a few traditional ones, like Postgres or MySQL. But the truth is that we have a lot of them. Like, there are jokes that there is a new database every week. And the thing is that those traditional databases, they can't really keep up to how React or other frameworks built on top of React changed. Because a lot of things got easier. And now we can, for example, in this code with React Server Components, we can access the database directly in our client component. And that looks fun, that looks easy. But then there is a question, like, what is this DB? Where does it come from? And how to set it up? So, those decisions and solutions that make those decisions easier is something that I want to talk about. Because databases are quite hard, and there is a lot beneath the surface that we don't really need to be bothered with when we are building applications. And there are solutions that kind of take this burden away from us.

So, here comes the first thing. And it's serverless databases. So, with serverless databases, they do for databases what serverless does for servers. They leave the complexity away. They also do it, like, the degree of lifting the complexity depends on the chosen platform. But basically, they abstract some of the difficult stuff away from us. And there are some examples, like Superbase or Firebase, that will not only, they not only give us serverless databases, but also an SDK on top of it that we can use, and other features, like additional storage, authentication, or AI. And there are others, like AWS Aurora, or PlanetScale, or Neon. And what their focus is, is to make Postgres or MySQL to scale easier. There's also a third category. And in this category, I would put Xeta, for example, which abstracts the database completely. They call themselves, like, Postgres++, because they not only provide a serverless database, but they also wrap Postgres to have additional functionalities. And there are, of course, some advantages and disadvantages. I have a typo in the first advantage. So, the autoscaling is obviously a huge advantage of serverless databases, full maintenance, built-in security features. But there are also things like vendor lock-in that you might want to consider, or limited customization options.

But here comes another thing that is very important nowadays, when we are talking about databases, it's all the things related to AI. So, it's important, because, like, databases with vectors they promise to provide deeper searches on data, and especially unstructured data. Because instead of, like, searching for exact keywords, we can search, like, a broader term and still get relevant results. So, we have kind of two directions here. The first one is databases strictly focused on AI, like, in other words, vector databases. And another one is databases, regular ones, with vector indexes.

So, the first one examples are Pinecone or ChromaDB, and they are optimized for AI and only for AI, and they very often offer SDK, specifically for dealing with vectors. Here's a short example with Pinecone, where you can set up a new connection, and then use Pinecone's SDK to work with indexes. And the other direction is regular databases, as we were using before, that added AI functionalities, because with LLMs becoming, like, mainstream with chatGPT, I think in late 2022, a bunch of databases added PGVector extension as something that is built in, like, SuperBase, Neon, Timescale.

And that's something that, these days, is kind of expected to have when you have a modern database. So, you can, for example, do this, you can create extension, you can run create extension vector on your Postgres, like, already Postgres database that you already have. Or another approach that HDB took was to have this using extension AI, which is SQL and gives vectors and also additional SDK. Here is a brief example. So, you basically create an AI client, and you specify the model. You can use OpenAI's models or Anthropx's models, however you wish. And then you can use the SDK to do certain things, like, in the example on the slide, I call withContext to provide context to my AI, to my astronomy AI that I'm going to use to search for certain terms. And then I use queryRug function to ask a question based on the data I have in the database. And now, a quick note about, like, what's the case for AI databases? If we have vectors being added to regular databases, do we still need separate databases as Pinecone or FromDb? So, I think that they basically have two directions. One is that maybe they will also add regular database features, so you will be able to use them not only as your vector database, but also as your regular database. And another direction is so that you are using them as kind of like a separate service. This is something that you might have seen before with Elasticsearch. A bunch of applications are using Elasticsearch as a separate database for just search services. And maybe something similar will happen with Pinecones. And then another very important trend is branching. And that's something that makes, like, our day-to-day development much, much easier. So, what is branching? It's basically whenever you're working on a feature and you create a new Git branch, you would like to have a preview deployment, right? But if your application also has a database, then you would need to have a copy of the database that the preview deployment runs against. So, this is what some of the databases make out of the box. And they nicely integrate with GitHub, so whenever you make a PR, there's a new branch. Then the database creates the branch, the copy of your current schema. And they are also really nicely integrated with your hosting solutions, like Verso.

So, here is an example where whenever I make a feature branch, there are new environmental variables added to my Verso that connect to the correct database branch.

Another trend among modern databases is multimodal databases. One example here is SurrealDB. They let you have different kinds of data in your storage without having to make a decision like key value versus document or relational versus non-relational. You can have it all inside of one database and decide which one to use when working with your data.

In terms of good type safety and developer experience, having everything in the database properly typed in either the server or front-end code is crucial. Prisma is a popular tool that generates a new Prisma client based on the schema whenever it changes, providing full type safety. Drizzle is another tool that infers the type based on the schema and comes with migrations tooling. EdgeDB also allows you to declare a schema and generate a query builder for running functions on your data.

EdgeDB introduces the concept of graph relational databases, similar to Xata. It doesn't have tables but objects, making querying relations easier by combining the SQL and GraphQL world together.

But we do not want to write those types manually. So, something that we are all probably familiar with is Prisma. And in Prisma, you do the setup once, and then every time a schema changes, the new Prisma client is generated based on the schema. And you have an ORM that knows what kind of data there is, what is the shape, what are the relationships, so you can use it with full type safety.

Another thing, slightly newer, is Drizzle, which is kind of a similar thing. You declare a schema, and then it infers the type based on that schema that you can use to safely run functions on your database. What's important is that it also comes with migrations tooling, both Prisma and Drizzle, so you don't have to worry about using a separate tool for that.

Another example in the DX and type safety world is EdgeDB. So, you also declare a schema with what types there are in the database, what are the relationships, and you run the generate command to generate the query builder. Then that you can use that way. So, you import the query builder and you run the functions that are available on your data.

And speaking about EdgeDB, I wanted to cover one more trend, and that is graph relational databases. So, something that is new that EdgeDB did, which is kind of similar to Xata that I mentioned before, because EdgeDB is also a layer on Postgres. So, in EdgeDB, there is no concept of tables. There are objects. So, the focus is to make querying relations much easier. So, it's kind of like combining the SQL and GraphQL world together.

Here is an example, a SQL query that I would run in order to have some information for the database, as long with an aggregate query. So, an aggregate query, for example, calculates some values, for example, minimum value of each of the rows that I have or an average value. And normally, in SQL, they do not exist together. I can either have data or a scalar value as with the aggregate. And this is an equivalent EdgeDB query where I can have them both in the same place.

I also have an example with an ORM, how a similar query would look like if I wanted to have data. And here I aggregate reviews when I take an average rating. And also, a query built with EdgeDB's query builder for the same thing. And now it's almost the end, and I wanted to leave you with something else. So, we're thinking a while ago, like, how can we blend the best of GraphQL and SQL? And there's one thing that is still relevant with React Server Components is a data fetching waterfall. So, imagine you have an application, and in the root of your application, you call, for example, like, GetSession function, and then, based on whether the user is authenticated or not, you render the components. And if the user is, you render, like, a user profile and user posts, but both of them need to wait for the GetSession to complete.

There are three approaches. We can have collocation, probes reading, or a custom compiler. With collocation, we run the queries against our server or database directly inside of the particular components, and that causes the waterfall, and it also might be hard to optimize. Then there's another way. We can fetch all the data in one place, but that may be more difficult to scale, because you have to pass all the probes down, and if your app grows, that can be overwhelming. So, here is something that I want to talk about. It's work being done by Terence Besman. So, he came up with this idea. What if we combined Relay from GraphQL and EdgeDB?

Because what Relay did nicely was to solve this fetch waterfall problem by having a custom compiler and extracting all the queries you had in the child components into one place and run them once. So, he came up with this idea that we could declare fragments on the database queries in HQL, and this is a language that is being used by EdgeDB, and then we can, in this example, he has a postcard fragment. There's also an optional defer, which means that you can defer the execution of that only if this component is being used, and in the relevant child component, we declare this function. So, we declare what data is needed by that component.

So, if this is something that is interesting to you, you can scan this QR code and create his blog post. There are more details also, and also a source code and a few more links. So, some takeaways from this. Something that I want to leave you with is to stay adaptable and remember that database selection is not permanent. Yes, this early decision is very important because it influences a lot of other things, but it's also possible to change that decision if it doesn't fit you anymore.

Then, also remember about your user first, so take into account how many users you're going to have, whether you need real-time, and some other features that might be required to have in your database solution. Then, also stay informed by knowing more or less what are some of the trends in the modern databases, and the last one to keep building, because this is also the way to pick the best solution.

And there was the... there's a case for something like polyglot programming, which means using different programming languages to do certain things, and there might also be a case for polyglot databases, which means using a different database for a specific need to optimize for that need. So, thank you very much. You can scan this QR code and that will take you to the slides.

I wanted to sort of like just say, watching this sort of space, you know, being a developer in the space, and then really like seeing the amount of complexity that is actually every developer needs to sort of decide upon in order to decide a data store, is, you know, like, you know, on one hand you look at like a SDK integration, you know, like integration to your UI framework, but then it's also the type of data and performance and all those kinds of things, is like, what do you think is a rubric that people can use to sort of like decide what's important to them? Is there, you know, a good sort of general rule of thumb? Well, like one general rule of thumb is that you might have some requirements that are like must-have and you don't want to compromise on, so you know that, for example, you're, I don't know, your project is going to be data-intensive, so you probably don't want to go with something that lacks performance, and maybe your project is solely focused on AI, then this is also your requirement. So, basically, like, list down all the must-haves and make sure that the database solution has them. Yeah, that makes sense.

All right, let's jump into audience questions. The top rated question here is really a question that I always have to think about myself as well, which is, what are your thoughts of the adoption of cutting-edge databases? Like, who are the early adopters outside of side projects? Because the database is the last thing that a person, like any developer, wants to disappear because of. Yeah, yeah, I mean, this is true, but also, I mean, if you're thinking about new databases, and yes, there is a risk that they will disappear, the risk of vendor lock, or you know, maybe they won't disappear, but simply like people working on it will stop working on it, and then you will have to fix the bugs yourself. I know that that's a risk, but also, by being an early adopter, you get almost full-time support. So that's the advantage, because, like, I think the like people creating the database will make sure to help, and to shape the tool however, you know, the early users want. But yes, that's also true, that most of the new databases are meant for side projects first, because they first need like a vast user base to shape the API surface, and so on. Yeah, it seems that, you know, I don't know any of the market data or market trends, but it seems like the databases that are experiencing a lot of adoptions are the ones that are based on, you know, standard SQL engines, like, you know, PlanetScale or Neon, and I guess it's the old, you know, nobody ever got fired for choosing Postgres, you know, kind of thing. But do you feel like with a more bespoke database, such as HDB or something like that, like, is there a sort of like a realistic migration approach for people, you know, if you sort of like find that maybe the choice wasn't right, or maybe the database is evolving to a direction that doesn't? Yeah, I think that's a good question, and that's a question that a lot of people have, and we also have that mentioned in the docs. Like, there is a way out, of course, if you decide you don't want it, and a good thing is that it's built on Postgres, so it's not like we're, like, starting from scratch and making everything ourselves. But yes, there is a certain vendor lock that you have to take into account. Not impossible to, you know, go back from. Nice. The next top-rated question here is, how would you sell, you know, convince your colleagues, maybe, or your boss at, you know, traditional SQL Server shop to try something like HDB? Like, what's the pitch to the people who already have an old-fashioned tool and they're happy with it? I think I would say that you can start shipping very fast. I know that's kind of like everybody says, that, like, you can ship fast, but you really can, because all the complexities of setting the database up are taken away. Like, if you want to have, like, a regular Postgres, how do you run it? In a Docker? That requires a Docker knowledge, something else, and something else. So that is completely taken away. And then you have all the features, like AI authentication, that is already there for you to just start building right away with no setup and no additional tools. Like, I know something for Elf, something else for the AI database. Yeah, that makes sense. Now let's go to a little bit more technical questions. Here is one. Why is it bad to have a server waterfall? Isn't it what GraphQL servers already do? Well, it's not inherently bad, but sometimes you may not want to wait for other components to render if you know that the data can run in parallel, for example, the request. I know, like, each browser has, like, a limit to how many requests can run at the same time, but still, you could avoid having to put, for example, like a loading state or use the suspense. Yeah, that makes sense. The next question I was looking at, like, maybe this was already sort of half covered, but I'm also maybe not sure the answer is, so let me ask it again. Oh, why would you use a bespoke vector database over a database with vector indexes? Why not just use Postgres for this? Like, what are sort of, like, the features or the qualities that you get? Well, to be honest, like, these days I would use a database with vector indexes because most of them already have them. For example, if you are using SuperBase, which is very popular, SuperBase added pgVector, and there was this time where, like, Pinecon or similar databases, like, had, you know, had their time, but now I think they are slightly being pushed away. However, if you want, if your focus is only on vectors, then you might want to optimize for that. That's the only use case. Like, if you know that you need strong optimization and you don't care about other types of data.

Yeah, and I think the last slide that you had with sort of, like, the multi-database world, you know, like, I think there is a generally a programmer adage of using the best tool for the job. So if specific type of data doesn't fit your existing data store, then maybe instead of trying to cram it in there, that's an opportunity to also try some new tricks.

All right, well, we are at time. There are a few more questions here, but I think you'll be doing the Q&A room after the panel.

Yes, exactly right now, I guess. Oh, after the panel. Oh, I don't actually know. When are you doing the Q&A room? I have no idea. Now, after the panel. Okay. So Alexandra will be joining us for the panel in 10 minutes here on the stage and on this stream. And then after that, you know, if she isn't already exhausted for speaking to us for an hour at that point, you know, you can ask her the remaining questions that we didn't get to here. So thank you very much, Alexandra. Thanks very much. And I'll see you in 10 minutes. All right. See you.