How Not to Build a Video Game

I'm just here to do another raffle. Do you want to do another raffle? No.

Hey, this conference was amazing. There's so many people behind the stage, everywhere doing the MCs, there are the organizers, there's So let's just give a round of applause for all of the folks that made this happen.

So this talk is a bit different. I just want you guys all to relax, and I just want to entertain you. I'm not here to sell anything, I'm not here to convince you that what I'm doing is better than what everyone else is doing. Quite the opposite. Most likely, what I'm doing is much worse than what you should be doing. Today I'm going to talk about how not to build a video game and I'm going to show you technologies that you're using every day in your job as front end engineers, but in a context that you probably have never seen them used in. And I'm hoping to inspire you to do things maybe differently. To think more about domain specific solutions instead of building another 11 step form.

Let me just show you the video game that I'm building. Here's a video. It's called Athena Crisis. It's a modern retro 2D turn-based strategy game. Actually wait a minute, something's missing here. I think music is missing. Let me pull up VS code. Let me get the music in. You got music? No? Oh... Yeah, that's Athena Crisis. I guess you noticed this is all React. And it's not just React, this game is running inside of the slide deck which is written using MDX which is a fork of React. Sorry, which is a fork of Markdown which allows you to put React components into your presentations. So I have the game running inside of a slide deck which is all written in JavaScript, React and CSS. Let me just get rid of VS code again. Once I realize I can put one version of the game into my presentation, I could also put three in it. And I can have the AI playing against me and this is like sitting in Central Park in New York and playing speed chess, playing multiple games. So I can wait here until the AI is done.

Let's just wait for a second. The AI is trying to beat me. Now I can jump in here and I can actually just go and play the game simultaneously on all of these maps. Then I realized I can also just translate this game into any language and have the game running inside this light deck in any language that the game is translated in. Even more, I realized, let's do a different one, I realized I can put the map editor that I built into the presentation. So I can just jump in here and make a new map. And there's only two constraints here to make all of this work, to make all the maps automatically beautiful. One is, can you place this tile at this location? And two, depending on the tiles around me, what should this one look like?

So for example, you can put any combination of trees together. You can go into the scenario editor, you can pick the character you like, pick their profile picture, say. And then from there, you can jump directly into the game and test out the messages in the campaign mode. So this is Athena Crisis. Everything you've seen is built from scratch with JavaScript, React, and CSS. The stack is using Vite, PMPM, TypeScript, React, Relay, Emotion, Prisma, GraphQL, GraphQL Pothos, Socket.io, and persistent data structures.

I wrote a blog post. I took out the core pieces that I'm opinionated about, like Vite, PMPM, TypeScript, put them into a template, and explained how to put together a really fast front-end experience. The constraint I put on myself when I started this is I'm only allowed to use tools on the layer directly below me that I understand so much that I could build a very basic, compatible version of it if I had to. I don't understand game engines, so the only way for me to use one was to build one by myself. I've worked on test infrastructure in the past. How do you test a video game? If you're playing video games, they're triple A games, and they're released, the first thing you do is you download a 10 gigabyte patch because, since the time they shipped it and the time they released it, they found so many issues they had to patch it. There are not many tests, and then you start playing it and there's so many bugs, so many visual issues, all sorts of problems, right? And so, I like testing. I don't write a lot of tests, but I wanted to figure out how can I make sure when I'm changing the game, I'm not breaking something all the time. And we've recently exhibited this game in Tokyo at a game show and had people playing at two stations from 10 AM to 6 PM throughout the day and we didn't experience a single gameplay bug. Not saying there are no bugs, but on that day nobody found one. But anyway, what you do, you have this video game and you have game states. How do you verify that it works? You could write tests to verify, is this thing here, is this thing there? Does this one have a health of 50 and this one a health of 77? And then you change the balancing and everything just breaks, right? Or you change the graphics and stuff breaks. And so I thought, okay, maybe I'll just render my game state to text. I make a gest snapshot or something like that and I make a renderer that takes my game state and turns it into text. Then you end up having two implementations of your renderer. One for the actual game and one for your test. And if you're using text to represent your map state, you realize that it kind of sucks because it gets out of sync.

It doesn't have the same level of fidelity. You don't get the same information from it. So how about just snapshotting the game and putting that into images and having snapshots that are verified depending on what the game looks like. And if you're using iterm or similar terminals, you can just print the actual game map, the into the terminal while you're developing.

I'll just show you. I have the test suite running here right now. You can see that it verifies that the game state is correct at each stage and it just prints the game state at that state to the terminal. So while I'm developing, I can watch this to see if things look correct. And if something changes, like a unit loses an ability or a unit is just missing, I can point to the exact commit where something went wrong.

Just as an example, this is the test infrastructure that I built. You basically just execute a bunch of game states against the map. You receive a new state and then you capture that game state as a screenshot and you match it against an image snapshot. All right. I think that was a lot. Right? A lot of JavaScript. A lot of really poorly written JavaScript. And a lot more CSS. Why would anyone write a game with CSS, the DOM, and JavaScript? One of the things is I got instant load times. I've been playing this game the whole time while I'm on stage here. It's all live. I can reload this light deck. It's right there. I can tap this. I get these amazing smooth transitions. This is all CSS. This is just a CSS. There's a bunch of CSS variables and it automatically figures out how to lay out this circle. And can you see that smooth transition when I tap it? It jumps out a little bit to just look really polished. Accessibility. The web has great support for accessibility.

You can just hook into those APIs and make this game work for people with various degrees of color blindness, for example. Light and dark mode. One of the cool things is I don't know why you would do that. But you can support dark mode with the video game. So for some reason, because I build everything from scratch here, my entire slide deck is now in dark mode. And the game is also in dark mode. So I'll just switch back again.

The downside is, as you all know with JavaScript and DOM and CSS and React, is that you sometimes fight with performance. And earlier I was mentioning persistent data structures to declaratively describe the game state. JavaScript is well suited for that because you can move so fast and you get instant reloads and hard module reloading. And then you have persistent data structures to just evolve the game state over time. And if you have persistent data structures, the nice thing about it is it's very cheap. For example, I'm talking about Immutable.js. It's very cheap to copy the current game state, change just one little thing about it, and not use too much memory. So I can keep a lot of game states around. And then I take this game state, and I put it in at the top of my component tree to declaratively tell each component what they should render to. Let me just show you. Here I just loaded a map and told the AI to produce 30 game states and see what it does. And I got an FPS meter here. And the cool thing is I can jump in here and just play the game at any spot, depending on what I want to do. But you know, 30 maps isn't cool. How about 848 maps? Actually I don't think that's very cool either, so let's just see if we can get some more if we redo the whole thing. Might take a while. So the thing is this is not scripted, I have no idea what the AI does. It's not entirely deterministic. And I'm just telling two AIs to play against each other until one of them wins and then give me all the states. Let's stick with 1507. And again, I can just jump in here and play the game at any point in time. Let's just see who wins in the end. I have no idea.

Okay. It ended in a draw. Too much going on. Let's do one more. 1461. I cannot get to the end. I think red wins on this one. Okay. Anyway, that's Athena Crisis.

So now, I want to play a little game with you. It's really boring, because the game is called is-that-CSS and except for one time, it's always CSS. Let me prove it to you. Just tap this, go to the elements panel. These are all divs. All divs. The only thing here that's not CSS, actually, is this arrow. That one is using SVG. However, this marching ants animation is using a CSS animation. If you're looking at this background there, this blue radius where units can move to, those are all diffs, rectangles with a border around them and a rotated mask image on top of them that is like a striped pattern to mask out every other pixel.

To show you this infrastructure, let me pull up VS Code again. Go into the unit. Here you can see the units that I can interact with, they have this red outline, and it kind of animates. It comes out. Let me zoom in here. It comes out when you tap a unit. Do you see that? It's like a really smooth, small 200 millisecond transition because I want it to be as polished as possible. The nice thing with my infrastructure is I can just change it, and it immediately hot reloads. There's no delay. This is the infrastructure. If you look at the buildings here, this building, there's a unit behind it.

The building fades out towards the top so you can see what's behind it. That is also a mask image with a linear gradient that just progressively fades out that image. Let me just turn off that gradient and then it looks like this. Alright.

Fog, these sort of games they have a system called fog where you cannot see what's behind it until you move a unit nearby that has a certain range of visibility. If I move over here parts of this fog will be revealed. I went through a few iterations of it but the most fun one that I found is rendering a bunch of black rectangles in the DOM, adding a blur filter and then using Mix Blend Mode with the attribute saturation to merge it in the layer beneath it. I can also change it. This is what Mix Blend Mode looks like. This is what the blur filter looks like.

One of the cool things about Fog is that because this game is a hybrid experience it's not just a single player game. It's a hybrid online single and multiplayer game where other people might invade your worlds and you might have to invade them when the game comes out. But if you can just take this DOM node with a fog in the elements panel in your deaf tools and just remove it and it's like ha ha I got you, I see everything that's going on and I'm just going to go and cheat, that wouldn't be very fun, right? So here's the thing, in Athena Crisis, all of the game state that you're not allowed to see is on the server and the server decides what each client is allowed to see. It's kind of like when you go on a social network and you're posting something for only your friends and somebody loads you a news feed then the messages they're not allowed to see will be hidden from them from the server side. Except here, each field has that visibility rule applied to it. So here in this example there are two maps, they have the exact same game state, but the pink player here cannot see anything that's hidden here in the fog. Let me just have the AI make the same moves here. And so on the client you'll only see the game states that you're allowed to see. So for example down here there's a new unit that was created during the AI's or the enemy's turn. So I can take this one, move over here, and once I'm arriving here it will show me what's going on around here, depending on what I'm allowed to see I receive those game states. So that's how fog works.

Another thing is that I want this game to work really well on mobile, on IOS and Android, and with touch devices specifically. Have you ever typed on your phone and you're typing a word and the next character is an E and then you just slightly tap the W close to the E and it still comes out as an E? That's because the buttons that you see on your mobile phone keyboards are actually sized differently depending on what the mobile phone predicts you're likely typing next. And so I thought, how can I apply this to a video game where there's a grid and they're all fairly small and I want to make sure you're always typing the right thing? So I built a small predictive engine to figure out what is the player most likely going to interact with, and most likely you're going to interact with your own units. So if I'm the pink player here and I go onto this beach and I just go near that tile where the unit is on, where this jet is on, it will grab the focus immediately. So I can click here, even though it's not on that field, to focus that one. So once I focus that unit, it changes the touchable areas depending on what I'm likely going to do next. So I'm most likely going to want to go to this unit. So even if I'm here on this forest tile and I move just closer to that but I'm not on the helicopter field, it will still focus it. But as I said, there's some pitfalls around performance.

One of them is that the game is actually rendered like this. I can play it, but I don't think it would be a very nice experience if you have a large screen. So, how do we scale things up in the web? We just add transform scale three and then the game looks like this. You might have to squint a little because I'm presenting here on a big screen, but all the colors are really washed out here. It's not pixel perfect anymore when I'm zooming in. Do you see that? And I added like this small little 3D effect to the map so they look like they're kind of like floating in the air. And when you do that, it washes out the colors when you use transform scale three.

One of the great things is when you have pixel art and you want things to look pixel perfect, there's a CSS property called image rendering where you can tell it that it should look pixelated, so when it upscales, the crispness of the pixel art will stay the same. However, I still had washed out colors because of that issue that I mentioned about the 3D rotation. And I realized a 25-year-old nonstandard CSS property that was built for Internet Explorer is just the solution. And every pixel is just perfect.

I built an image pipeline. Whenever there's a sprite and I change it or my artist change it, it goes through the image pipeline to compress the image to the smallest possible size, and then I thought, I don't want to download 300 images. I want to download one file. And ideally I can just put my images into a CSS file and base 64 encode it.

And you've seen those all those little units that kind of move up and down. That's the idle animation. It's kind of like breathing. All of you are like moving while I'm staring at you. And so what I did is that I had my nice little image rendering pipeline. I had one CSS file with all of the sprites inside of them, and they worked great in Chrome, but then I opened Safari, and somehow it was 20 frames per second or 15 frames per second slower than Chrome. And I'm like, what is going on here? And so what I realized is that if you're changing your background images, like the positioning, the sizing, or how often it repeats, if you're doing that quite often and you use Base64 encoded images, which we often do for small icons or something like that, Safari will not have it cached. So when Safari looks at that URL with a Base64 image encoded directly in there, it's like every 180 milliseconds in the game, it would be like, I've never seen this art in my life, never seen this unit in my life, and then just goes and decodes that entire Base64 string again. So there's no caching there.

What I did as a solution was to delete my image rendering, image processing pipeline, and just fall back to use normal PNGs. Another one was that I have a five-year-old daughter, her name is Mia, and she comes to me quite often and she's like, hey, Papa, can I work on the game with you? And so I gave her the map editor. And so she was like, okay, I'm going to put a house on every single field. The map was more sparse. So I had about 150 of those entities on it. It's not that many. It's not a real scenario. But somehow she found a bug. The game rendered at seven frames per second after she did that. And I learned that there are bad filters and good filters. The CSS filter prop is supposed to be GPU-accelerated. Some of those are. Not all of them.

You saw earlier when I was explaining how that drop shadow around units are using fading in just slightly. The only way to achieve that is by setting a drop shadow on all of them that's transparent so that when you change the color, it will fade in. Because if you don't set it up front and then you set it to a specific color, it will not transition. You've all experienced that most likely if you're using CSS transitions on an element that just gets inserted into the DOM.

I put four of those drop shadows on each of the entities in the game because I wanted to be pixel perfect, not blurred like they normally are. One to the left side, one up, one to the right and one down. And they were transparent because they're just there in case I need to fade that outline in. And I realized that even though they were transparent, the GPU was doing a lot of work and instead of 60fps, I only got seven. So just be careful, even if somebody says this property's GPU accelerated, some parts of it might be, not all of the parts. So brightness, saturation and grayscale are always accelerated and fast.

One day, I didn't even have a performance problem, so I don't know how this happened, but I woke up and I saw this thing called will change. And the description of the property said, this is how you can tell the browser to skip some optimizations because you're telling the browser which CSS properties will change. And I thought, damn, I'm so much smarter than browsers, I know exactly what's going to change. And so I put that in, didn't benchmark it because I just trusted what I read there. Two days later, I realized, oh, I have a performance problem again. So I bisected through my history. And then I realized, okay, actually don't use will change. Browsers are really smart. When I read this documentation page, there's one part of that page that I didn't read and that's the red part.

Another learning I made is that you should, you know, or if you want to, please use an FPS Meter. I don't want to make any prescriptive statements here. I put in an FPS Meter. You saw it earlier and it ideally was almost always at 60 FPS. I can show you that this is live by opening and closing and causing reflows. Yes, it is live. What I realized is, I put my FPS Meter on the top level of my React tree and I was looking at my game and I was debugging something and I'm like, why is my game re-rendering every second? I'm not doing anything. And that's because my FPS Meter was at the top level of my component tree. So my recommendation is, use an FPS Meter. I know there's one inside of Chrome. You have to enable it.

You never look at it. If it's always there in development, maybe even in your staging environment, and it drops, you'll immediately know that something's wrong. And the pro tip here is put it into a separate React tree on your page so that it doesn't re-render the rest of your actual application.

One thing I've learned is to benchmark early but only optimize what matters. I built an AI from scratch for this game based on how I would play the game. I have absolutely no idea how to build an AI. I somehow built an AI. I built it using functional programming, immutable data structures, and I didn't benchmark it. It seemed fine, it seemed fast, but then somehow I was playing in the map editor where the AI runs locally, not on a server, and my game locked up for two seconds, and I thought, ah, I'll just put a web worker in it and just take this off of the main thread, but then I started just measuring performance, and I realized that if you're using functional programming for things that you do a million times during one turn of the AI, it's probably not a great idea with JavaScript. JavaScript is fast when it looks more like C and less like a functional programming language. So I benchmarked it, but only optimized — there's other slow parts, but I optimized the AI to be about 20 times faster.

In terms of tooling performance, you saw that some of these units, they're the same for different factions, different teams, but they have different colors, so you can distinguish them. I built a small palette swapping library, which is not at all a great implementation, but it works for me. And I realized — when I open-sourced it, first off, I have to say, I built that whole thing while carrying a baby. And then I realized it was very slow. And so I just looked at it and rewrote it to be 23 times faster. The learning here is just that don't trust someone doing open-source while they're carrying a sleeping baby. That's one thing I did. I'm trying to open-source as much as I can as I'm going on this journey. There's one more thing. As you saw, this entire slide deck was using MDX and built specifically for this presentation. So just earlier today, I open-sourced re-MDX, which is the exact code that I'm using for this slide deck. You can check it out. It's very minimal. Please fork it if there's a feature you'd like to add. But that's re-MDX. If you're okay with it, I'll bother you with yet another thing. So I started this project a year and a half ago to meditate, to just see how far can I go. To just bring balance to my day. So instead of sleeping, I spent every night coding for one hour, maybe two.

Sometimes maybe four. But it felt better than just sleeping, because it brought balance to what I was doing. But then over time, I felt like I have something. It's really fun. So if I gave my laptop to a friend, or somebody in my family, they would steal it for an hour and play the game. And then lose. And then want to play again. So I thought, some people want this game to exist. I want this game to exist. And this is how I started 22 years ago, when I wanted to learn programming to make video games.

So I built video games and social networks. I spent a bunch of time working at a social network. And so I thought, I want to go back to that. Maybe this is something fun to do. And maybe it won't work out. But when I'm old, I don't want to regret not doing it. And so last week I announced that I'm starting a company, it's called Nakazawa Tech. Which is an indie game studio based in Japan, but also an open source company. And I'm so excited to announce, there's a new video games publisher called Null Games, which is super developer friendly and is working with indie games. And we just signed a partnership. And they will be publishing Athena Crisis next year.

Can I just say Kristof, this is the second conference that I see you this year, and you are always such a goddamn delightful person to be around. I really, really enjoy you and everything that you do. And I just want to say, I appreciate you, man. I appreciate all the kisses I'm getting from you at the conferences. Yes, I know. All right, cool. So we have a couple of questions. One question, which I don't know if you know the answer to this. How much time did you spend so far on all of this? Yeah.

An hour and a half every day for the last 1.4 years. So if you do the math, you'll have an answer. Yeah. Yeah. You have to do the math yourself. This man is a riddle.

Do you have an art background? Did you create this pixel art yourself? Oh, well, we took away the slides on the last slide. I gave credit to all the artists. So Nakazawa Tech is working with six amazing artists that are working on the art. I learned I'm a good art director, not a good artist. I can modify what they make, but they are amazing. And I'm lucky to have them.

Yeah. So, one question is, why not build a game this way? I mean, obviously you've already, you know, like, you know, talked to a lot of the performance pitfalls and whatever, but, like, why is it a bad idea? Yeah. Yeah. I think it's really the performance that's, like, giving me the most headaches where I'm, like, genuinely, like, scared about. Where sometimes things are just suddenly slow and I'm like, okay, maybe it's not so smart to do what I'm doing. But you know, it's also just fun to call it talk like that, because I think I called it that before I started a company to do this. And as I said, it was just a meditation project. And now I have to switch to be much more pragmatic about actually shipping it, which is a bit of a change. So, if you had, I mean, obviously here, the journey was the goal itself. Yeah. But, like, if you had actually just set out to create a production grade game, what would you have done differently? Maybe ask me in a few years. I have to have some experience with, like, what I might do in a different way. You know, the great thing about this and using a stack that you know, is that you can learn so much by figuring out all the other things that you don't know. And if you jump into a new stack, you don't know anything, and then you don't ever get to that stuff where you're really learning something. So I think that was mainly why I wanted to try it this way and just see how far I can get.

Nice. What library did you use for the multiplayer interaction? You mentioned Socket.IO, but is there anything more like behind it? No, no smarts. Just my own code, which I wouldn't, which I don't like using.

I wish that was something I could just use. That's better. Yep. How heavy was that baby at the time? Six kilos. Oh wow. Nice. All right. These questions are dumb.

All right. OK, here's a question that sounds really smart but could be a dumb question. I don't know anything about game engines. I don't know if we have time for that. I mean, I can say ECS means entity component system. No, I didn't. I just use JavaScript and immutable map. Great. Well, I mean, these questions are just too dumb. Let's not waste our time here. Is there anything, do you see anything on the screen you want to actually?

Oh yeah. I mean, we, like, you know, the last one is good. Now, how could you publish this to other platforms? Oh, yeah. That's fun. Well, answer the question then. Yeah, first off, you can. Well, I have to read the questions, so we know what the question is. We could by swapping out the entire renderer to not use CSS and the DOM, which is maybe a lot of work but then, you know, there's upsides and downsides, right? What I said earlier, I think one of the real upsides and, sorry, I was maybe a bit tongue in cheek on how not to build a video game, but you just get so much by relying on web technologies and the DOM and React and CSS. You get all that for free, whereas other game engines, they have trouble supporting online play, where it's like, you know, if you're building frontend, you're like, I don't even know how to not make it online, you know? And so there's so much stuff that you get from the web platform. And if you wanted to support something outside of the web platform, you could take the entire engine and just replace the renderer to render to something else. You know, it's something I'm thinking about, but not for the first version of the game. All right. I think that was a great answer to finish on. Everyone, give one massive applause to Christoph.