Multiplayer Games with React Three Fiber and WebSockets

♪♪ ♪♪ ♪♪ ♪♪ Hi! Thanks, everyone, for being here. Thank you, Sarah, for the energy. I absolutely need it. It's been a long day. We're going to be talking about making multiplayer games for the browser and WebSockets.

Who are we? First of all, who am I? My name is Maya. That's my dog, Api. Yes, that does stand for API. I'm a nerd. I went to art school and then I dropped out, so now I'm an art school dropout. I taught myself to program. Got several software engineering jobs for some reason, and currently I'm working at Linear, where the team and I are making software a little bit more magical. But most relevant to this talk is that I have a degree in game development. I do. Feel free to follow me on socials if you're so inclined.

A lot of you may be like this person that responded to the React Day Berlin tweet about announcing this talk. And this person says, Why in the name of all that is holy would anyone make a game in React? This is a great way to start a talk. I have to agree with them, because it's a legitimate question. I understand the bewilderment. There are so many better platforms for playing games than the browser. There are so many better technologies for building games in than React or JavaScript. So why would you want to do this? Accessibility.

I'm not talking about screen readers and ARIA tags, although those are really important things and they, of course, can come up in games. I'm talking about accessibility of game development to you, as the developer. So I say I have a degree in game development. People usually say something along the lines of, I have a really cool idea for a game, but I opened up Unity, it's really confusing, I closed it, never made the game. And these people are programmers. They know how to use React. They have these tools already in their hands. They just need a few extra skills and a shift in mindset to know that making games is possible.

The other reason why you might want to use React and make web games is to make them accessible to your players. So I've made a game for this conference, and I didn't expect any of you to bring your PlayStation 5 or Nintendo Switch. I just expect you to have a mobile phone with an internet connection and a web browser. And that's cool, because your players don't necessarily want to install anything. Maybe they want to play your game at work. Just saying.

So what are we talking about? We're talking about a game called React Battleship, and it's going to be a multiplayer game with the classic mechanics of Battleship that you probably played in elementary school when you were learning the coordinate system. So the idea is that you have a grid, you put your ships on that grid, your opponent does as well, and then you try to sink each other's ships by randomly firing onto their grid. Okay.

Game plan. Pun intended. Whenever I also talk to game developers, so indie or solo game devs, some of the practices they say are really useful for them are things that we already know in software engineering and they just kind of stumble upon it because they're not in that industry. And they tell me that you need to manage scope creep, first of all. You need to iterate quickly on your ideas to figure out if they work or not, and you need to fail fast. With that in mind, it would set our game up for success to do a little planning up front, and I propose we do that by dividing game development into four distinct phases. First we'll have a design phase where we figure out what we're making. What exactly is this idea of the game that we want? We put it down on paper and we agree on it, and we say, that's the game and nothing else. Next we make a prototype of it. Does this kind of game make any sense at all with the technologies that we want to use? Is it possible to have this kind of mechanic? Is it even fun to play? And we get to do this, we get to test these ideas at a very, very low resolution with very little time commitment. The next step is a block out phase, and this is because we're making a 3D game and this is where our development breaks away from traditional software development. This is the phase where we actually introduce 3D into our game. Does our mechanic still make sense if the game is 3D? And finally we have a production step where we make it magical. Make it nice for the player and actually make it a presentable game. So starting with design. In school they taught us, because they do have a degree in game development, that you need to start every game with a game design document, or a GDD. And these game design documents can be hundreds of pages. They are entire wikis. They're used by multiple teams, across multiple departments, they span years and are maintained. Or they can be a page. Which is our case, and it might not look like much, but defining these things that are defined in the GDD are super important constraints to what the game actually is.

So for example, we want to focus on mobile. That means I don't have to worry about larger resolutions, desktop resolutions, tablets. You're going to be playing it on your phones, right? I can constrain the platform that it's going to be on.

Next is the targeted audience. So you're, I'm assuming all adults, either way, you're here at a software conference, you probably aren't expecting a super flashy game, and my 11 year old nephew doesn't have to be a fan. He's a really, really strong critic. Especially with my games. So if he doesn't like it, I don't care.

Next, when we define the modes of play. So in this case, I have the idea of having kind of an open sea where everybody can explore and meet each other, and challenge each other to a specific game of battleship, and then have another mode of play that is the actual game of battleship. This helps me, in my mind, figure out what kind of scenes I need to set up for the game.

And finally, the unique selling points of the game are things that the game absolutely must have. Without these things, there's kind of no point to it. So I want the game to be something that's really accessible to you, in terms of not having to log in, not having to have any kind of authentication. You're not going to want to create an account, let's be honest. And the experience has to be pretty snappy. It has to be fun, because you're also not going to be investing hours and hours into playing it. You're probably just going to scan the QR code at the end of this presentation, play it a few times, if that, and hopefully, have a good time, and have a unique experience from this game.

So these are the mockups. The general idea of what this game is going to look like.

Next on, moving to architecture, as part of design still. So the shape of the game is technically in a monorepo, with a single server, that is Node.js, specifically Fastify, but we don't need to get into the details. And it will open a WebSocket connection using Socket.io. The client is multiple React clients, so all of your phones connecting to that same server. And each of the React clients will create a connection to the socket, the open socket on the server, which enables 2-way communication. So in case you don't know, WebSockets are TCP connection. It's a protocol that uses TCP connection to enable 2-way communication. So clients can send and receive data from the server. And what this enables us to do is to separate the responsibilities of our game to what is on the server and what is in the client. So the server can handle game logic and the game state.

The client doesn't have to know any of that. Basically, the source of truth is whatever is on the server. And then the client can just display the state, update whatever input comes in from the player, and we're good to go.

So disclaimer, this game does have a lot of code. And I will be going through some of it, but not a lot as you'll see. If you are interested in the rest of it, it's up there. And each of these phases of development are separate branches on the repo.

So, prototype. It makes sense because our client code is so dependent on whatever the server is telling it is the current state, to talk about how the server is actually implemented and how it communicates with the client.

So first of all, the server will send certain requests or not really requests, but certain messages to the client that tell the client what it as a player, what its state is. So, its ID, where it's located, and it will also send all of the information about other players to every single player that's connected. So when a player joins, when they disconnect, when they move, all the other players get updated.

In turn, the client will tell the server things like when it wants to join and when it's moving. There's also situations where the server needs to facilitate communication between two different clients. So let's say player one wants to challenge player two to a match. It will tell the server, I'm challenging player two. The server would say, hey player two, player one is challenging you. Player two says, I accept. And then the server starts a game and puts the two socket connections into a single room. So their communication is sequestered from the rest of the pool of players.

So this is cool. We have all this stuff on the server. The server knows everything that's going on, and the client just has to update itself. What's a good way to do that? Well, we have React context. And we can create a single React context which will open up the socket connection, connect to the server, listen to all the required events, and then just update the React state. And with this, we have just a nice reactive state that the rest of our application can use without necessarily having to listen to every single event and every single component.

And this is what the game looks like at the end of the prototype phase. It's very basic HTML, but it does actually show that this game is viable. You can actually play it. The technology we want to use can serve the purpose.

And even at this stage, I noticed something that was kind of critical to the mechanics of the game, or to the gameplay. And that was that if your opponent takes a really long time to make their turn, what are you going to do? You kind of don't have an option. You can either disconnect or forfeit. And that doesn't put you in a good place. So I added a timer to kind of prevent that from happening if you don't decide within seven seconds, your turn is made for you.

So the next step, now that we have this game, is the block-out phase where we actually introduce 3D into the game. And when we're talking about 3D in the terms of 3D technologies, we use words like a scene, a camera, lighting, objects. And this evokes the image of a movie set. And I think this metaphor is actually very appropriate because just like on a movie set where you have a physical world, you can walk through, those things exist. But when you watch a movie, it's from a single point of view. The same is true for a 3D scene. There is a virtual representation of things that actually exist, but you only get to see what's on the screen. And the way we represent this virtual 3D scene is through something called a scene graph. And this is from the 3JS website.

So 3JS is the technology underneath React 3 Fiber. And this kind of looks familiar to us, right? It looks a bit like a DOM tree or a component tree. That's cool. I think I can figure out how that would work within a React application. But because 3JS is the underlying technology and that is a scene graph from 3JS, this is the way you would traditionally write 3JS code. It's very procedural. You have to manually add things to the scene, manually compose things. And we don't like that. We want to write React, right? So that's where you would use React 3 Fiber. And these two code snippets are actually doing the same thing. Except one of them lets us write React and React components. So what that looks like within our game is that where we had a table and we would loop through all of the players to create table rows, now we loop through all the players and create player objects, which are 3D. We don't necessarily want them to just be sitting there. We want to add some movement to them. But they're there. So in order to add movement, the way I did that was creating kind of a plane that would represent the water.

And because our capabilities on mobile are kind of limited of what the player input is, you kind of get only two tap, right? And if this were a talk about just 3JS, I would be talking about ray casting and how you translate a tap into a 3D point that corresponds to a component. But this is a React 3 Fiber talk. And we just get a simple click handler that takes care of all of that for us and tells us where in 3D space we clicked. So this is the current state where we have movement. And because we already have the React context syncing with the server, our application is updating already. So our players are moving around on each other's screens.

The final part is to change the actual battleship game. And this requires a little bit of math. I used something called a grid helper from 3JS to create the grid and then figure it out based on another plane underneath the grid what coordinates were clicked.

Finally, to get the rest of the blockout phase done was to actually place our ships. And while it was easy in the prototype phase to just kind of say, this block represents a submarine, and this other block next to it also represents a submarine, that's not so simple when you're doing 3D. Because you want to add models. You want to add actual things that will kind of take up space. And positioning isn't so simple. Especially since some of them are vertical, some of them are horizontal. Some of them are an even number of spaces. Some of them are odd. And you're positioning all of them based on their very center. So it's just a little bit more math. But getting this done while we only have cylinders to work with makes it so much easier when we actually put in our models.

So speaking of the models, these be them. I made these in Blender. And I just want to remind you, I am an art school dropout. So be kind. And this is what they look like in-game. So, maybe not the best looking game. But it still has a charm to it, right? It's something that you can conceptually get your mind around about how to get from something that's just plain HTML, just plain UI, into something that's kind of 3D. And really this is just like artist skills that's failing at this moment. Finally, to make it kind of a real nautical game, you need to have a level of animation that causes motion sickness. So I did this in three different ways. Namely a helper called float from React 3.J, which is a set of helpers for React 3.Fiber. Whatever you wrap in this component, it will automatically kind of float, which is very appropriate for our ships. The next is when you're firing the cannonball. A library called React Spring with a callback called uSpring that kind of enables this nice fluid animation without having to really do much. And the final part is model animations. So the little fire is actually an asset that I found on Sketchfab and it had this animation within it.

Kind of makes your game look 3D. And finally I made a few tiny UI improvements just to make it feel more like a mobile game. So instead of having buttons, buttons with icons and emojis.

Okay. So. If you want to try it. I have never tested with this many people. So I'm really excited to see how quickly it crashes. But I will give you a moment. Still some phones up.

Finally I just want to give a shout out to some resources that have really helped me with React 3.Fiber. I forgot to put the actual documentation because like Eddie said, good documentation is worth its weight in gold. But yeah. Check out Wawa Sanjay's React 3.Fiber Ultimate Guide and the legend Bruno Simons 3.JS Journey. And also the Web Game Dev newsletter and Discord server. So I'm guessing you all are having fun. You're not listening to me anymore. I'm happy. This has been great. Thank you so much.

Okay. So the first thing that I want to say is that I think it's very funny that your name is Maya and you use Blender. And there were three people that got that joke and I love you all. Okay. So let's start with some questions. And the first thing that I think is... How viable is the latency in these games? I love it. This is always the first question. So for this game specifically, latency is not that important. What's more important is the reliability of getting the messages back and forth.

If you're building a game that's more like a fighting game or a racing game, anything that requires quick reactions, latency is much, much more important. You might want to look at something like WebRTC instead.

How long did it take you to make this? So this game specifically is a rebuild of a similar game that I made with just playing 3.JS. Building that one took, I want to say, like two weeks. And this one took maybe four days.

You made the first one in Vue.JS? Yes. And not even React for UI, I just did like document create element. That was so much code! Yeah, it was unnecessary. Okay.

And I think there is one question about asking how to pronounce your name basically. But it's the same as a 3D software. Yeah. The question of is it Maya with a J or Maya with a Y, it is Maya with a J in my passport. And Maya with a Y to the rest of you.

Oh yeah, the server crashed. There was a bunch of people saying they couldn't access it either. But to be fair, it told me that bit.li was offline. So I'm not sure if it was you or them that crashed. We crashed bit.ly, guys. I'm proud.

Someone is saying your design skills are awesome. Thank you. You're very welcome. I didn't do anything. Okay.

So. Graphic skills might be a huge blocker. Do you have any suggestions there? Try it? Maybe it would work. I think now with the advent of generative AI, I'm getting canceled for this, but you can make things look really nice with things like mid journey. And I would encourage you to try.

I will tweet out a tutorial that I found from somebody who entirely made a game from generative AI. All their assets are kind of from mid journey. So that might be a way forward if you don't want to actually learn how to get better at design.

There's also two websites that I genuinely use to get 3D models if I'm looking at something and don't want to do it. Sketchfab. Please do not forget to attribute people on Sketchfab and also BlendSwap. Most stuff on BlendSwap is actually CC0. And that only works on Blender, but let's be honest. Are you really going to buy Maya? You're not. You can also, like, yeah, if you're importing to technicalities, if you're doing like 3JS or React 3 Fiber, you're probably going to export to GLTF anyway. And they do automatic GLTF on Sketchfab, which is nice. So yes, this is getting way too technical now.

How would you go about writing end-to-end tests? That's a good one. It is, right? I love it. Is that even a thing? So for writing end-to-end tests, if you're doing something that's visual probably the way you would want to do it is with what are those tools that do screenshot analysis? Cypress. Yes, Cypress. I think Playwright does that as well. You would need something that would take into account the visual aspects of it if you're not testing just like the functionality of the game.

Someone is asking are games like GTA possible in the browser? Whoever asked this, I want to ask, are you talking about the underlying complication of building a game like GTA or are you talking about the graphics? Because I think the complication is, but I don't think the graphics are. I would agree. I think even the complication might be a bit much for what browser. It's gonna crash. But so does GTA, to be fair. That's true. I will say, GTA specifically, that level of graphics you have graphics developers or graphics cards developers and platform developers like PlayStation and Xbox, working with the game studios to kind of get that level of actual detail and get these specifications in there. We have WebGL which is like 30 years old in the browser. So, yeah. Graphics-wise, not that great. But I will say, there is a developer building a game called Hordes.io, I think that looks a lot like World of Warcraft in the browser. So, check that out if you want to be impressed.

Oh, yeah. There are people doing, like, I forgot the word, but like cycles rendering in the browser. What's the difference? I forgot the word. The way that rendering works most of the times in most game engines like Unreal and stuff is that every particle of light is emitted and then they have to calculate where it hits and where it hits and all the colors and all of that stuff. That's not what Three.js does. Three.js sounds like this, I'll do this. And someone is doing the other thing in the browser, and that will crash your computer. Yeah.

So, you're talking about ray marching? I thought it started with ray, but I didn't know where it ended. Ray tracing as well. Yeah, like Sarah said, it's basically calculating for every single pixel every single place on the screen, you calculate all the different light sources and where that light sort of bounced from. That's really computationally intensive. What Three.js does is called rasterization. It's a completely different pipeline and most of it is just smoke and mirrors. When you have something that looks good in Three.js or anything that just does rasterization, it's mostly just like, we have this plane that kind of has fog over it and that's why it looks cool. PS1 graphics. They never actually went away.

Someone is asking, can you elaborate on how you keep each game separated from each other in the WebSocket server? I'm using Socket.io for that. It basically has this feature called rooms, where you can group players together in a room. It's like an in-memory thing, it's not something more technical than that. It allows you to have a nice API for just emitting events to that room when players play the game and don't need to be isolated from the rest.

Cool. Now two questions that are going to be very easy for you to answer. The first question is, does API fetch? He is really lazy. He doesn't fetch. He runs to the ball, takes it, and lays down and chews it. Oh, that's what my cat does. Cool. And the last question that I have for you is, do you use Jira at Linear? So you prefer Linear other than Jira? Wow, madwoman up here. Linear all the way. Thank you so much, that was amazing. Thank you so much. You are free to go, Maya. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you.