React Query API Design – Lessons Learned

♪ I'm super excited to be here today, because this is actually the first time that I'm giving a live talk at an in-person conference. Um, yeah, thank you. Um... So I thought I want to take a quick photo just to remember this forever. That's all right. Yeah, all right. Cool, thank you.

Anyway, my name's Dominik. I'm a software engineer from Vienna, where I work as a front-end tech leader at Verity. You can find me online as tkdodo almost everywhere. And for the last three and a half years, I've maintained the quite popular open-source library React Query. Sorry, PanStack React Query, as we call it these days.

Quick question. Please raise your hands if you've heard about that library before. Yeah, wow. Okay, that's a lot of hands. That's great, because it means you might actually know some of the APIs I'm going to talk about today. Because, you know, today, I want to walk you through some of the API design decisions that we've made in React Query over the last couple of years. Um, tell some stories about things that went well, but also highlight some of the trade-offs that we had to make and, you know, there were some mistakes that we made. And, yeah, maybe there are some lessons that we can all learn from those. And I want to talk about that mainly for two reasons.

One, I think API design is hard. If you don't believe me, that's not my quote. Julius said that. He's a really smart guy. He maintains TRPC. He also contributes to React Query from time to time. So if he says it, it's probably right. And the second reason is I think React Query has a really, really sweet API. And it's one of the reasons why it has become so successful over the last couple of years.

Now, of course, I can't take credit for that. Tener Linsley made the library, and he designed most of the APIs. And he actually has a very good tweet summarizing the goals of the library, where he says that the query API is actually medium sized when you unpack it all, but the most important part is that you can understand and learn how to use it by starting with a single function that provides 80% of the entire value proposition first try. From there, the rest of its API can be gradually learned if needed. And I think that's what it takes for a library to become popular. It needs to be both minimal and intuitive, as well as powerful and flexible.

But for any given API, you know, those two things are usually on the opposite sides of the same scale. If we take a look at array methods, for example, on the left-hand side, we would have something like array.join, right? A very simple method. It does one thing and does it very well, and there's no surprises there. And on the other end of the spectrum, we would have something like array.reduce, which is very powerful. We can implement all other array methods just with reduce. But you know, if this is the only thing we have available, it would probably also not be great, because it's also quite complicated to read from time to time.

Now, for libraries, I think the second scale is missing, and that should usually be application complexity, because as your app complexity grows, you actually want your APIs to become more powerful and flexible. And on that scale, I would put useQuery right about here, bottom left, if we call it with the minimal required arguments, which is basically just the query key and the query function. Now, that API, I think, is still quite simple and easy to use, but it gives us a lot of things already out of the box. We get things like caching, requested duplication, background updates, you know, automatic garbage collection, like the list goes on and on. There's a lot of things that we get from just this one function call. And then later on, we might add a useMutation call, right, to make an update and then link it back with invalidate queries. So, this is already a little bit more involved, but, you know, we can get really far with just those two functions.

So, I put that right about here. But as time goes on and your application becomes more complex, you might want to do more things. So, you're going to make an optimistic update, maybe from time to time, or you need an infinite query. And those APIs are certainly a bit more complex. And all the way up, we have, like, for example, our plugins or the cache subscriptions, which are really, really low-level. For example, our dev tools are built with the cache subscriptions. But, you know, once you get to a point where you need this complexity, you're probably happy that those exist as well, just like you are about using reduce from time to time. And that gets me right to the first learning that I had as an open-source maintainer, which is I'm no longer excited about major versions. And I think you probably shouldn't be either. API design is hard.

API design is hard. I think I said that already. But it's especially hard in open source because we can't easily revert the decisions that we've taken. As a different example, at work at Verity, we have a design system that we publish to a private NPM registry. And, you know, the latest version that this is on is actually 105.2.0, right? Those are huge numbers. But, you know, nobody cares because it's just a number going up. Most teams would just upgrade and, you know, see that nothing really changed that affected them or it was just like the default value that changed, and they just move on with it. So it's just not a big deal. But in open source, we can't do those breaking changes lightly because every new major version has to really be this kind of like marketing event where we have announcement tweets and announcement blog posts and even videos. Because if the users hear like major version, your major, I think it sounds huge and great. And the question that always comes then is like, what are the new features in this major version? The problem about that is that major versions aren't about features. They're about breaking existing things, and features usually go in minors. So if you remember React hooks, right, they came in version 16.8. Not 16 or 17, but specifically 16.8. And React router added the route loaders in version 6.4 and fun added Windows support in version 1.1. So those are just some examples where features usually go in minors. Of course, there are exceptions when you redesign something from the ground up, you know, and that enables you to do new features. You might have new features in a major version, but usually, they come in minors. So when I got asked what are the new features in React Query version 5, I actually started to sweat a little bit because we didn't have anything planned. We were planning on breaking like how you call a use query and basically everything, but we didn't have any new features planned. So what we did was we added some features to version 5 that we could have honestly also backported to version 4. And I don't think that's great because it means we're withholding features to the users just to have this kind of like marketing event or great new version. And I think what would be good would be to kind of like decouple these breaking changes from the marketing events. I think Anthony Fu had a great suggestion about the four-digit semantic versioning where you would have an epoch number in front of the major version that you could use for those overhauls and for marketing and we could do breaking changes then more often. I think it's a nice idea. I doubt it will happen anytime soon. It's just something to think about. And yeah, maybe when a new major version comes out, instead of asking what's new, maybe ask what is breaking in the library instead.

OK, so I'm no longer excited about major versions, but one thing I'm still excited about is TypeScript. Yeah, you too? Yeah, OK.

Don't worry, we're not going to go into library-level TypeScript in this talk. But I think if you're building something new when you're starting out, right, I think it helps tremendously to think about the types from the beginning and design your APIs with types in mind. Now, I know there are a lot of people who say, well, we can just make it work first and we can always add the types later. I don't think that's necessarily a good thing to do because when we're working with JavaScript, we can have all those cute and dynamic constructs that work at runtime, but that might be quite hard to type later on. I think everything is doable with enough magic in TypeScript, but you know, all magic comes with a price, and in this case, the price is usually application type complexity and maintenance burden. I really like this quote. Not sure who said it. If something is hard for a compiler to figure out, it's also hard for humans to understand. So if we're having troubles to express what we want to the compiler, you know, maybe the API that we've chosen isn't really the best. And one of those cute and dynamic constructs we had in React Query was actually useQuery because when we started out, you know, you could call it in three different ways with three different positional arguments. Now, there's not really a good way to make this work in TypeScript except with overloads, which is what we did. And overloads are problematic because they have, like, very weird over, very weird error messages from time to time because what the compiler does is it tries all different versions and then it displays the error message for the last overload that it tried, and that might be completely misleading. Also, we had to do some runtime checks to actually, you know, transform what the user inputs into a single object anyway. And we thought, like, really, who needs three different ways to do the same thing? So what we did was we, in version five, we removed all the other options to do it. Now you can just call useQuery with the options object and that reduced the lines of types that we had from 125 to just 25 lines of types. So great simplification for us as well and a better developer experience. And the thing is, I think had we started with types from the beginning, I think this is the API that we would come up with eventually.

Okay, now enough about TypeScript. One thing that comes up when the library gets popular every time is that users want more features, right? And to be honest, I think managing a demanding user base is one of the more tricky things in open source because, on the one hand, you really want to implement the features that users are suggesting and to listen to their problems and help them fix them. But on the other hand, the more we add to the library, like if you just add every feature request that comes in, the API becomes bloated and the library becomes unfocused and, you know, that might reduce adoption again for newcomers because they kind of get confused about what's actually going on. We have to balance this somehow. My advice here would be to just take your time before you add anything to a library. I think users can be very demanding sometimes, but, you know, in the relationship between user and maintainer, it is actually the user's job to tell you all about the deadlines and the use cases and how important it is that you add this feature, but it's the maintainer's job to keep the bigger picture in mind, like, will this API also work for other users, for other cases that the original reporter hasn't even thought about? Because remember, once we add an API, we can't really, like, change it easily without a new major version. Now, one of these things where I got this wrong was the RefetchPage API on our Infinite Queries. For context, Infinite Queries are our way to make these doom-scrolling pages that everyone hates relatively easy to implement. I'm really sorry about that. But, you know, technically, an Infinite Query is just one cache entry that is chunked up into multiple pages where each page depends upon the previous one. And that has the effect that when a refetch happens, you know, we refetch all the pages one after the other, and some users, you know, complain that they want it just to give me a method where I can refetch, like, a single page from these pages. And we thought, okay, that's not a bad idea. Sounds reasonable.

So we're gonna add to InvalidateQueries an option to refetch, like, a single page. That sounded good at first, but it actually had three problems. We found that out over time. For one, I think it's a bit weird and confusing because InvalidateQueries doesn't actually know anything about infinite or normal queries. So if you don't match an infinite query, this function actually does nothing. And for technical constraints, we could also only add it to the imperative versions that we had. And if there was an automatic refetch, it would still always refetch all the pages. And I think it's also one of the easiest ways to get, like, incorrect state on your screen because if you have a list of pages that depend on each other and you refetch one in the middle, you know, you might see duplicate results on the screen if someone else made an update to, like, the first page or something. So the UI can get, like, out of sync really easily.

So what we did was we took a step back and asked the users, like, what's the actual use case that you're wanting to use this for? What's the motivation behind this? And it was always the same. They said that, like, if I have 100 pages in the cache and the user scrolls down a lot, I don't want to spam my API server with a refetch. And I think that's fair. But we tried to solve this problem from a different perspective instead. And eventually, we settled on the much easier max pages option on infinite query, which simply allows you to limit how many pages are in the cache at any given time. And I think this API is a lot better because it solves the problem holistically, but from a different point of view. For all kinds of refetches, and it coincidentally can also speed up your rendering if you are navigating to a page that already has cached pages, it will render faster because it doesn't need to render as much. So my takeaway here is that we landed on a not-so-good API decision too quickly. And had we given ourselves more time, I think we would come up with a better solution instead. The only other alternative I can think of is prefixing everything with unstable or experimental until you know it's going to last. I don't know if that's better because I don't know if people actually use experimental APIs. And then, yeah, I get messages like this, and I don't know if that's really better.

Another API that gets requested often is to be able to debounce API calls. So we get that feature request a lot. And you might want to have that when you, for example, have a search field and you want to do auto-filtering. Unless you want to make an API request on every keystroke, you want some way of debouncing that. So the suggestion is just add a debounce number field to use query, right? But this is a very good example of an API that will not make it into React Query because it's just not the responsibility of React Query. There are a lot of different ways to do debouncing and do it correctly. It likely needs more than just a number as an option, right? So it can get complicated quite fast, and it will also increase the bundle size for users that don't use this. The good news here is that you can relatively easily implement this in Userland. You can use your favorite useDebounce implementation, you can write one yourself, or you can just use deferredValue from React.

The way this works is that the filter in your state will be used for displaying the current input to the user, but you will take the debounced filter and pass it to the query key and then to React Query to not make fetches that often. I think this inversion of control is a great way to give users the flexibility to implement certain features on their own and allows us to keep a small API surface.

Now, I think the query key is quite special here, but we can also get inversion of control on other options that we have by just making them a function. One example is a discussion I had with a user a while back where they felt that the refetch on window focus feature that we have that's turned on per default is great for most cases, but sometimes it's not, for example, when the query is in an error state. Now, we can't really add another option to that because that would be like, what is that refetch on window focus if error or something? It's kind of like, that's just weird. So what we did was is that we just made this function a callback function that accepts the query where the user can derive what they want from that in Userland. I think that's like a cheap trick to allow users to do that. And we've since made almost all of our options accept callback functions so that you have more flexibility.

Lastly, even if we keep all of these points in mind and no matter how well we try to design an API, I think there will come a time when some people will be unhappy with it. And those people can be quite loud about it. And I think open source maintainers are not immune to making mistakes. So the chances are that at one point, we might release an API that isn't very well received. I learned this lesson the hard way with React Query Version 4, where we made some changes to our primary state machine. Let's take a look at the example from before where we have this filter, and what happens when we try to handle the loading and error states with the derived isLoading and isError flags. Now, this code worked perfectly fine in Version 3, but what happened in Version 4 was that you would see like a loading spinner for all eternity. And that's because a query that starts out in a disabled state, because here we have the filter as an empty string default, so enabled is going to be false, so the query is going to be disabled. In Version 4, this would also put it into the isLoading state.

Now, of course, there are reasons for this, and it didn't sound as bad when we made these changes and implemented this API, but if we take a step back and objectively look at it, and if we have no knowledge about React Query, I think this behavior is just a very bad API. I think it's absolutely horrible. There are no excuses for that. And it turns out a lot of people feel the same way, and I agree that this is messed up. And this thumbs-up counter is still going up, by the way, every day, even though we've already released a fix in Version 5. And the thing is that this feedback would actually have been very good like a couple of days earlier because we got this right after the major release. So, yeah, what stuck with me is the user expectation that maintainers get everything right in their API designs, while at the same time, you know, they might not try out beta versions and report feedback. So if there's one thing that I would like you to take away from this talk, it's actually this. Please help out maintainers of open source libraries that you're using. Yeah, there are the other maintainers. So please help out maintainers of the libraries that you're using by trying out beta versions or release candidates and reported feedback, because I think it's the best time to be heard. Without those early feedback, you know, mistakes like the one that I made might make it into the stable release. But, you know, stable doesn't mean bug-free or battle-tested.

It just means that we can't change the API anymore without a new major version. So open source is a two-way street, and I think this is one of the best ways to help while also getting the most in return. Yeah, that's all I got. Thank you very much. Congratulations, Dominik, amazing talk.

How are you feeling? Tell us a little bit about how it's been going from having these kinds of parts of your day-to-day job and your life and these learnings that you've had and taking it into, first of all, an amazing first-time talk at a conference. Yeah, I'm relaxed now. This is just very relaxed. That was great. So all the tension is gone. That's awesome, and people really enjoyed it. People are vibing. I love it.

You do so much in the open source sort of community with your day job. You do so much work in the open source community, and you also have a day job. You're able to do it all, make some time speak at conferences. How do you balance it, especially when you have sort of a day job and contributing to the open source community and maintaining a React query? Yeah, I don't work full-time, so I work as a contractor. And I do the... My day job is a part-time job, so I have enough time to do open source. It doesn't really pay the bills. You know, it's a different problem. But I love doing it. I enjoy it a lot. And yeah, I think this has helped me, like, staying on it, yeah.

Awesome. So whoever asked that question, if you want to be able to balance, maybe trying to figure out ways that they overlap and taking some time to focus on each one, that is awesome. OK, so this is now more focused on the actual kind of product as well. Why did you start your use query with three signatures in the beginning? That was a question that came in. I think someone has a very specific example in their mind. Yeah, no, this actually predates my involvement, so I have to ask Tenor that.

I think when React query came out, the first version was actually in JavaScript without any types. This must have been before TypeScript got popular. And I think it was just, you know, something that you could do, where you just thought, OK, maybe users just want to pass in, like, a key and a function and no options, and then that's it. And, yeah, I think in JavaScript, it felt really, like, cute to do. Oh, you can do multiple things with the same signature. In TypeScript, it's not so good. And this maybe kind of raises a different question when you're sort of talking about developer experience and building a developer experience, because you are a developer, and how is it easy for you to put yourself into the shoes of the people who are going to be using your library, or do you have to kind of wait until things get into beta and people give you feedback? Maybe it's a combination of both. Yeah, it's definitely a combination of both. I've been using React query myself before. I became involved with the library, so I kind of, like, knew what I wanted from it. But, of course, I can't really prioritize my needs of it, so there's, like, feedback is always welcome. That's why I'm also all over the discussions and trying to understand what people want. For the first six months of my open source work, I did nothing but answer questions from people and look at the problems that they're having. I love that as well, how you started by listening before you turned around and started contributing. And that kind of links to the next question that is coming from Mark, which is how do you decide which features are worth adding? Obviously, everyone has their own opinions about what should be top of the priority list. How do you decide, or how does the team? I wrote a blog post about this, too, so you can go read that. Can people find the blog post? At tkdodo.eu. It is really good. I actually got myself lost into blog posts when I was researching yesterday. But, yeah. The short answer would be that I think if something is not very common, so it's not requested often, and it's easily doable in userland with a couple of lines of code, then we're not going to add it as an extra option. It needs to really, like, outweigh the negatives of widening the API surface. And it needs to be undoable, or very hard in userland to do it. Some things are just undoable because of how we integrate with React. So then we need to support them. And I'm going to link this to another question that came in, which is about API design. So you just talked a little bit about how you decide which ones, which features are going to be making it into the next versions and so on and so forth. But do you have just a process in general or framework that you use when you're thinking about API design in general? And I'm guessing that is a part of it. So I think it needs to be easy to type for the users. So I think having good TypeScript support kind of, like, guides me through how the API should be.

Other than that, there is a lot of trial and error. So I have to admit that. And talk about with other people who have done similar APIs. I talked to Mark yesterday about infinite queries and how they are doing it, how they want to do it in other libraries. So there's lots of exchange with prior art as well.

And then now this is more of a question about sort of your journey. Lots of people want to get more involved into the open source space, but maybe they're a bit nervous or maybe they're trying to figure out the best way to do it. Can you tell us a little bit about your journey into the open source space? And then I'll add to this by asking you to give any advice for someone who is maybe trying to get into or just getting started in open source.

I think the journey is totally different for everyone. There is no one single journey how it goes. The way it went for me is that, like during the pandemic, I didn't have a lot of things to do. There was nowhere to go. So I thought I'd join the TanStack Discord and try to help out with answering questions, try to give a little bit back to the community because we were already using, I think, three different libraries of the TanStack at the time. And I was just so focused about answering questions that sometimes I didn't know what the problem was. So I looked it up, looked at the source code and then got more and more into it until I made my first bug fix and then my first feature. And then, yeah, I kind of like knew the library quite well at that point. So then, yeah, that's how it went.

Awesome. Now I'm sure people want to ask more, but we are out of time. Where can people find you online? Yeah, TKdodo is my online handler. I'm on Twitter a lot. TanStack Discord is a good place to reach out. It's better than Twitter DMs. They'll suck. Yeah. And my blog, of course. Awesome. Thank you so much. Give him a round of applause one more time. Thank you.