From Websites to Games: The Future of React Three Fiber

Hi, my name is Chris Baumgartner, and this is a talk about the future of React 3 Fiber and how we intend to tackle video games on the web. If you don't know React 3 Fiber, it is a library for React that lets you render declared the 3D declaratively as if it were a browser API. Under the hood, it is a React render for 3JS, which is itself an open source 3D render. R3F is used everywhere from enterprise to hobbyists and, yes, even some games.

Some adopters are Vercel, Zillow, Ready Player Me, to name a few. We have an estimated install base of 300,000 and make up a large portion of 3's own installs at this point. We follow three design principles. Keep the API simple, pragmatic, and stable. But enough of that, back to me. Professionally, I am a web 3D developer at Chauncey Technologies. Also professionally, but much less lucratively, I am a member of Poimandris, where I maintain React 3 Fiber alongside Cody Bennett and Paul Henschel. It is also Poimandris where I'm cooking up some new technologies we'll be talking about today. And if you catch me on Discord, you probably recognize me from my avatar on the left.

Poimandris is an open source collective started by Paul Henschel back in 2017. If you work with React, you probably have heard of our libraries, whether it's Zoos Donned, React Spring, or even React 3 Fiber itself. We are an entirely horizontal collective, dedicated knowledge share and enabling creativity for developers. If you'd like to sound of that, come join us on our Discord. We have a pretty big community over there.

So you might be like me. I began as a web developer. Not even a very good one. More like a web designer who's proud of writing his own CSS. I never thought I would be good enough at programming to make a game. But I love games, so I would dream about it. My skills at programming steadily grew, and I was beginning to learn React when I came across React 3 Fiber. There were these beautiful demos, like the one screenshotted there. I think that was actually the first one I saw. And it just appeared so simple. And for the first time looking at something interactive in 3D, I thought, I can do this. And I realized later this was a moment of self recognition.

And those dreams suddenly felt a lot more real, and I suddenly got a new hobby. But as I soon learned, a demo is not a game. Even a 3D enterprise app is not a game. React 3 Fiber has inspired many web devs, but how can we take it further? How can we get to the moment of I can do this to make immersive 3D worlds? Even video games.

So the journey begins. And two years ago I started down this path, and today I'm going to share with you where it's led me. Ripple sponsored a lot of the research, so shout out to them, my company, Chauncey, and all of the community has helped along the way. So this starts with React. This is about building games with React. So let's start there. React is an abstraction over the DOM. Oops. I almost forgot what talk I was giving. React is an abstraction over document like renders. This is important because a 3D scene graph just happens to look a lot like a document. React is also functional, composable, and declarative. And all the traits that we know and love that make it scalable and testable.

React is also a framework. It makes you buy into its declarative approach and life cycle, but otherwise lets you bring your own tech stack. This is important because it allows progressive enhancement. And for the React ecosystem to move forward, you know, we need to be able to adopt whatever new technologies are coming out in the JS world in general. By contrast, game engines are almost always monolithic and require you to bring the whole kitchen sink with you everywhere, even if you refuse to use it. So React game engine, in my opinion, needs to embody all of these ideas. Needs to render like a document. It needs to be functional, be composable, be declarative, embrace progressive enhancement, and feel webified. And the last one is really important, but also impossible to define if you know it, if you see it as a web developer, you know if something feels webby. Let's see where we're at with React 3.5 today. And hey, looks like we did it. But it can't be that simple, right? But it does do all these things. It renders like a document.

It's functional. It's composable in all the ways that React is in that top down kind of way. It's declarative. It embraces progressive enhancement. 3 is just a render. It doesn't force you into anything else. And I think it's pretty webby. We have pointer events, you know, props look like webby props. It's got an image tag. What more could you want?

But we actually have a problem. Using React's model for functional and composable code runs us into a wall while trying to make immersive 3D apps. So this is what I'll be talking about a lot. But why? In a nutshell, React is render oriented, well, simulation driven apps like 3D worlds or not. If you don't know what render oriented means, don't worry, I made it up. In React, the flow or state or data, I'll probably be using these terms interchangeably, is oriented toward rendering a view. Which, aside, these kinds of diagrams are everywhere on the internet. But absolutely not true. The virtual DOM does not look exactly like the browser DOM. There are a lot more nodes there. And it is assumed that most, if not all, state updates and even interactivity gets digested for rendering. And so, really, what I mean here is that it's all about the return function. You're not running typically through a React app in order to just process some data. Because of this state, it's safely assumed to always flow top to bottom as we create the document top to bottom.

So by contrast, these 3D experiences we want to create are data oriented. That one I didn't make up. This is because, at their core, they are a realtime simulation paired with a view of that simulation. It is the same reason you can run a game headless on the server for multiplayer. The simulation doesn't require any rendering at all. And simulations, by their very nature, are data oriented. They're just transforming data from input into whatever it is that you're playing.

You cannot assume, as React does, that the same flow that creates a view can also run the simulation. And this image is evidence I didn't make it up, because Python is filled with people who hate anything that's not data-oriented programming. If you aren't convinced, let's look at this classic problem I found on the Internet, where the child has a state that needs to be set at the parent level. In this case, so that it can flow back down to other components.

So how do we solve this? What do we do? And so the canonical answer is, well, we just lift the state up. Whether it's prop Julian context, you're lifting and running all the logic at one domain-specific controller. So with a complex app like a simulation, we lift and lift and lift. And before we know it, we're at the top, orchestrating all of the relevant state in one place before dispatching it to the view.

This gives me an idea. What if we decoupled all simulation state updates from the rendering? We're already lifting the state all the way up to the top. So let's start with a frame loop. Real-time apps need to be real-time. They need to update every frame, and we definitely don't want to cause React rerender every frame. So we invent a frame loop. React ReFibre has long had this built in with the use frame hook. Now it runs independently from React's own loop, and it lets us execute code v-synced that doesn't stall or cause React to digest new life cycles.

But now we need to schedule all these callbacks, all these function calls in this loop. The most naive approach is schedule in order of render, render calls. But we already decided that this isn't going to work for us, even if it works for most apps that are rendering to DOM. Less naive but hard to use is sort by priority number. You quickly get into an issue where you have magic numbers. It's like if you ever mess with Z-indexes in CSS land, someone just starts putting 999999 in there just so it can always run at the end, and now you need to go after 99999, so now you're 1001. It's a mess.

Ideal is scheduled by dependency. So we notice that what we really want to do is have our data updated in the correct order, no matter how related the React components get shuffled around the render tree. This is done by telling our scheduler our intent, letting it solve for the order for us. So we say that I want this function to run before this one, I want this function run after that one, and then it figures out how to order them all for that to be true. So introducing directed. This is going to be our new scheduler library. It does exactly that.

Publish a point mandris. It is composable. It is functional. It uses a directed acyclic graph, or DAG, to solve for the order of update functions based on the intent we give it. It works with both Vanilla and React. And most importantly for us, it enables ecosystem integrations, because there's no magic numbers. And I'll explain how it does all these things.

All right. So we have functions A, B, and C that are going to run some updates. And we use our handy new hook, useSchedule, and we pass them in with absolutely nothing else. And our little buddy here, who's going to help us schedule, he just puts them in whatever order he wants, because we gave it no intent. But now we want a particular order. We want it to go A, B, C. So we tell the scheduler, okay, B comes after A, goes before C, C comes after B. And actually, we don't even have to put before C there. We just say after A, after B, and our little friend there will figure out the order it needs to go in.

But now let's do something a little more challenging. Here we want to make a typical video game, game loop. And usually what we're used to seeing in those cases is stages. So input, physics, default update, render. Instead of stages, we're using tags. This is going to give us a lot more flexibility. Because now we want to insert update camera. Update camera goes in the default, but it's also kind of its own stage. So we give it a camera and a default tag. And this lets us actually have tags, groups, I call them groups internally, overlap. And then also note here that we are targeting the move body function directly. We're not targeting a tag. And that's so that we can schedule linearly with move body, make sure update camera always comes after that. And then we give it its own before render, even though move body already has before render, which would be inherited by it through the default tag.

And that way, if we ever call this on its own, it has all the scheduling data it needs to contain itself, to run its own logic. It doesn't need to depend on another call from another scheduled function. And as a progressive enhancement here, you can see on the left, this looks very familiar to use frame, we're using closure, and we diff by ref. So if there is an arrow function generating a new function each time, memoized, of course, then each instance of moving box, or in this case, moving box, apparently, will create its own instance of this function update. On the right, though, we use a stable reference, because a stable reference, no matter how many times this component gets mounted, that update function will only get mounted once. This lets us do something very powerful, where you can, you don't need to know what a physics library is doing, you just need a schedule relative to something tagged with physics, or Rapier physics, for example. We will be rolling this library out over the next month, I think when this video lands, it'll be out, so keep your eyes peeled.

Okay, so data updates not fully composable, but what about data? Our little friend there has a good question, how do we access data if it isn't immediately available inside of the React component itself? Okay, but wait, don't say it, yes, we got here, you probably knew it was coming. We're talking about ECS. So before you panic, remember everything you see is progressive enhancement to help facilitate data-driven design. You can take it or leave it as much as you like. So I could fill an entire talk just talking about ECS, and it won't be this one. Instead, I will briefly motivate it, encourage you to look into it more if you're interested, maybe a follow-up talk.

So ECS is a natural design pattern for data-oriented programming. If you start going down this path, you'll almost always end up here. In essence, your app becomes like a live database. It actually just becomes a live database where components are composable global stores of data, entities are IDs that let us query collections of components, and systems are functions that update components, which we already just went over. And data is king. But this is JavaScript, so objects are never truly far behind.

Okay. So let's just get a little bit more familiar with it. Typically, if we're creating a body object, you would see it like on the left, the class where the position and mass are just defined in it. On the right is the ECS conception. So we create a position in the mass store here called components, not to be confused with React components, but it is confusing. And we add it to an entity. Importantly, what this means is that this entity's data is composable. We can add another store to it if we want, changing its signature. And then we just say that a body is any entity with position and mass. And then what we do is we query the world. We get all the bodies by saying, get me every entity that has position and mass, and then we iterate over those IDs, we pull the store, we pull the position store, and we move it. This can be done globally, anywhere, anytime. And you can query so much more. We can query for almost anything. We can get all the bodies. We can get all bodies that are moving. We can get all bodies that are moving towards a player. So here there's a relationship, a relationship between the body and its target.

We can get all players who are married. This is a player relationship now with the player. We got all players who are also frogs, and now we have inheritance. We can say, is the player a frog? And we can get as complicated as we want. All players who are members of the frog guild, sworn enemies of cats, but not descendant of frogs, right? So let your imagination run wild. Once you allow your app to be a living database, the freedom to be able to manipulate data, it becomes enticing.

And so I'm introducing here a very experimental library just to kind of get your head in the space that we're thinking called Sweet ECS. It is declarative ECS with deep integration with React, and what I mean by that is that automatically everything that is committed to the view in React becomes an entity that we can add components to. It is powered by the next evolution of Bit ECS. I've been working with Nate Bit on this. That should be out by the time of this talk. And to the right is an example of the API. I'm not going to go through it in detail, but you can see here that we're scheduling in our body component, and we're adding all of our components that create the body to the entity declaratively.

So let's look at how having composable data with composable updates helps us in a problem we were kind of looking at before. We have our MoveInBox component, and before we did a closure where we just got the ref and then we just moved the box. Here, we're going to pass in a update function that is defined stably, and we're going to query the box based off of position, and we gave it a position component below, and then we're going to pull the position store and move it. This is one function that will run for all instances of MoveInBox. And then on the right, and this is a classically annoying problem to do in React right now, we have a perspective camera. We give it the followCamera component. We query for it to get all follow cameras. We gave the box an isFollowTarget component. We query for that. We just take the first result because we assume there's only one target. We iterate over all follow cameras. We get their objects and then we just call the lookAt, and the three object here, again, an experimental API, automatically all entities, all view items get their objects, instances from the ref, added to an object component so we can pull them out and use them inside of the system. And there we go. We got full composability.

Finally, where this lands us is a React ecosystem where we can bring just enough game engine, and I think this is very unique. It's something that we need as web developers in order to move into game development, and it's something that we need PointMandrius, as a community, in order to build an ecosystem of first class physics libraries, UI libraries, got UIKit, we got Rapier, we got Jolt. There's all sorts of other parts going to this. You need character animations, you need controllers, you need camera controls. Being able to composably create all these things in a data-oriented way will let us build out that ecosystem. And importantly, run our games on a web server, because if you're building games on the web, it needs to be multiplayer.

So as I close up, the future React 3 Fiber, that's where we started. The future will have the scheduler built in for V10, V9, it's compatibility release for React 19, V10, we're going to have an elongated beta release where we roll out these new features. The scheduler will let us integrate with the ecosystem, anywhere, anytime. And then we're going to be looking at enabling data-oriented workflows as we move along. Thank you. Check us out on Discord if you want to learn more, and talk to you soon.