Marrying WASM/WebGL Games with React UI

Hey everyone, it's Jonny, and today I'm talking about marrying WebAssembly and WebGL games with React, or in some other words, using the right technology for the right job. So, first about me, I work at a social media company as a web engineer, that media company has a blue logo, but it isn't Facebook. In my free time, I love to ride bicycles. That can be helping my brothers with bicycle advertisements, can be a road bike, it can be a mountain bike, as long as it's bicycle related I probably like it. And the other big hobby I have is games.

Together with my dad I have been making multiple board games, for example Lost Valley, Nord, Half-Band Heroes, they are usually funded via some crowdfunding campaigns. And since high school, I've also been developing multiple electronic games.

Initially, they were all iOS based, recently I've been branching out a bit, as you might notice from the talk, I talk about making games on the web. Why did I become an engineer? I became an engineer to save time, to save my time and to save other people's time, so they have more time available to play games. And that brings me to the first thing I want to talk about, picking the right tool for the job.

My previous games I largely made with Cocos2D, but Cocos2D is a very code driven game engine, so it's familiar for a developer to just write out the code, but it's not very time efficient because there's no tooling. I started using the Godot game engine, which is enabling me to save a lot of time on the web development side. I love to use React, which is also allowing me to save a lot of time, and I intentionally put it on the board game side here because there's one interesting similarity in board game development and web development with React, and that is being declarative.

If I come up with a new rule to play on the table with my other testers, I just roughly explain them the rule and declare my goal, my intention, what is this rule change supposed to do? Then they will interpret it and use their brain processing power to come up with an interpretation on how to actually move it into the game system, how to execute the rule. On the other hand, if I want to teach a new rule to the computer to play games with, I need to be very precise about everything.

Let's ask the question, what's the quickest way to make a game and why might React be a good choice for that, and why might Godot be a good choice for that? Also, very importantly, what's the most fun way to make a game? Because in the end, this is a talk with no commercial interest. This is just about having fun developing some games.

Okay, so why React? Some people might remember my talk from React Summit Remote Edition in 2020, where I talked about making Diagonal 4. The interesting thing about that game is, first, it's very declarative. I just have a very simple discrete board state. In the end, it's just a bitfield of eight by seven pieces. I can just go from that bitfield, and I can calculate all available moves for every player, and I can just calculate the view very directly because it's just a very simple state and very simple rendering. That maps well to React.

The other great advantage of using React is that it's very easy to make a responsive UI and to have some design systems, making that even easier on the web. Why might React be not the best choice to make a game or modern game on the web? Let's look at this example, Blood Fever. It's a game of a vegetarian vampire defending his coffin against zombies. But the important point is that there might be hundreds of enemies who all are running animations on their own. They don't have a very discrete position. It's more like a continuous state. Some functional Reactive Programming purists might say it's still a state and you can still do like very atomic calculations to come up with possible actions by the player. And then put that in a very big state machine. On the other hand, with animations, it's just not the best thing, I would say, to keep a nice performance running. So just the performance aspect, React might not be the best choice here. And also tooling. Gamedev gets a lot quicker if you use the correct tooling.

And React is really not well equipped there. The ecosystem works. So who might bring good tooling to the table? Godot. Why use Godot for game development? First thing first, it's free software. It's very popular software. If you look at this graph on OSS Insight on the right, this is the Pull Request Creators and you just see Godot becoming the biggest player with most contributors by far in the game engine space. It exports to web, iOS, Android, Windows, Linux, OSX, pretty much all platforms you could need. And it is a very mighty editor. If you look at the screenshot on the left, you see the scene graph. On the right, you see the scene in the editor and the node editor. So you can click simple things and you can move them around, which really makes it a lot quicker than doing all of that just via code. And lastly, it brings an integrated scripting language called Godot Script, which is very Pythonesque and might be simple to pick up for a lot of people. Some other people might still mistype script, but there are solutions for that. But that's a story for another time.

Why should you maybe not use Godot? Well, it doesn't have any FlexBlock or Grid things, so layouts might get a bit complicated. Why could that be an issue? Just look at this example from Age of Empires 3. You see a screenshot, the screen was very big, but the menu didn't scale so they do some acting background images. It's just not that great. It can get even worse in game if you have an even wider screen. And where the person would naturally just look in the middle of the screen and only see that. And the menus left and right are very far away, you might need to physically turn your head. So this is just very awful, UI-wise, because the tooling isn't that great, or made for it.

Another reason to not use Godot is there might be some external SDKs missing. One great thing about the web is that it's very easy to just go to npm and integrate some modules into your application. It's a bit more complicated with Godot. Also, on the web you could just easily integrate some authentication SDK from Twitter or from Facebook and then add some sharing functionality, whereas doing that natively in the game engine itself would be a lot more complicated. And lastly, I'm a web engineer most of the time, so I just might miss some web tooling if I just use the game engine. I also want to be up to date with web stuff, so it's a fun way to play around.

So, what's the solution? I mean, the obvious solution is combining React with Godot, but how would we approach that? Well, first, caveat, noncommercial. This talk, as I said, there's no commercial interest. This is just me in my hobby time playing around with games. So, if you want to make money with your games, you might have widely different considerations to think about. The other caveat is just a prototype. I just did start developing this like 2 months ago and was thinking about, hey, how can I combine my two favorite technologies? I came up with this, so I'm showing it at the moment and let's just jump into the example video. So, it's all running on a website. The website is just, in this case, host on localhost, it's on the Playground and there's another game running on the localhost. So there's my game called Kellergewurl und Lauf, which is just a dungeon runner. And you see the connection between the React UI and the game running itself. So I changed my name. Johnny is now popping up in the game itself. What is the game about? You're just running in that dungeon. You can push your adventurer up and down and you need to avoid all the dungeon dwellers and just not touch them. Now, I did touch the developers, so the game is restarting.

Now, I did touch the developers, so the game is restarting. I think that wasn't quick enough, so let's use the React UI to change the game speed. And, well, that might have been slightly too quick, so let's slow it down a bit again. And then speed it up again, just to see, like, the connection is instantious, so it's not waiting for the game to reload again to pick up the status changes from the React UI, but it's like a full connection.

And, well, let's talk about a very annoying thing, sharing scores on social media. We might have all been annoyed by the word craze and people spamming our Twitter feeds, but this game also can generate, like, a tweet for you, you are authenticated in the background on Twitter, you can press share and it will be automatically shared to your feed. It's just one possibility of the web integration. But let's look at one other feature, let's switch games dynamically on the same website. Now I'm switching to the remotely hosted version of Caligrelden Live, which has a different scaling mode activated, so now it's just using the design size of the game, so it's just 480 x 320. And lastly, it switched to yet another game, Blood Fever, to just show that there's also some other UI patterns possible, the game we've been using the keyboard and with Blood Fever all is mouse controlled now, and it's all still picking up the context nicely on the web. And that's it for a quick demo combining React with GoDo games on the web.

How do you get there? Well, the first step is, actually have a game in the GoDo engine. Caligrelden Live, as I talked about, is a quick 2D dungeon runner. It's all available on GitHub under GPL license if you want to check it out. So you just create that game in GoDo, you can debug it in GoDo. How do you export it? Well, GoDo has a very simple export dialog. You just say you want to export it to the web in the HTML5 version. You press export, and we'll generate some files. We'll actually generate this doc folder on the left. There's going to be the game.html. This is GoDo's native game wrapper, but that game wrapper really only supports running the game stand alone on the website. It doesn't really have a nice environment to add any further UI on. There's also a file called game.js, which contains some of the game code and some of the engine code, but we're going to look into details for that slightly later. Inside there is a game.wasm file and a game.pak file. The game.wasm file is the actual engine compiled down to WebAssembly and also some of your game code compiled down depending on how you configure the export. The game.pak file is basically all the resources you need for your game, which most of the time are going to be textures or audio files. So Godot exported that, but how can we use that? Let's take a step back initially. Let's quickly talk about WebAssembly and WebGL. That might be relevant for you. WebAssembly, there's this long description you can read if you want to, but roughly speaking it's just the binary instruction format which applications can compile to if they don't want to, for example, compile to JavaScript or some other transpilation, but it's more like a binary format.

And that's very common for native engines to now add another compilation target. So Godot engine can just choose WebAssembly as another compilation target and then can compile the C++ engine code to that. WebGL is a JavaScript API for rendering 3D and 2D graphics. I personally love to do 2D games because it's just easier as a single developer to just pick up some graphics from Open Game Art on the internet for free and use them in their own games. It's just a lot quicker than 3D, but you could also, with the technology presented here, just build a 3D game if you wanted to.

And WebGL also means you're rendering into this canvas. That means there's no further DOM elements for the game itself. It's all a pixel target. It's rendered into the texture, and the texture's rendered in the canvas. Okay, enough for that excuse. Let's jump back into the game.js file. The game.js file considers a large area of two bits. The first bit, which looks very horrifying, that's why I'm not zooming in, is in the end just mapping some of the JavaScript code to some of the WebAssembly code. The connection back and forth works out, but we can totally ignore that because it doesn't really matter at all for embedding our games. There also is some Engine code. That Engine code is loading the fetch files. That's a bit more relevant for us because the Godot wrapper I'm presenting is extracting some of that Engine code.

Now, you might ask, okay, so Godot is providing this like fully. Why can't we just use an iframe? Yes, you could not use an iframe, but you also can't use an iframe because you might have some problem with cores because you're going to run installations where you can't run the code. If you want to, you might run installation where you can't pass messages to the iframe because it's blocked. Also, it's just using iframe, isn't really the React approach, I would say. So, what's our local architecture going to be? Well, first things first, the user visits the website, the websites renders some UI, and that website UI also renders the gsgd host, which is provided in an on-prem package. That gsgd host then goes to some other web host, where you uploaded the game.js, game.pak wasm file which go to exported, and then the host is loading that JavaScript into like the local context. It's picking up the engine definition from the game.js file. It's instantiating that, and it's also providing a WebGL context, and then the engine starts running and is rendering into the WebGL context that the user can interact with.

What is the infrastructure for that? Because it sounds very horrible to set all of that up for all the games you want to publish. Also, again, I think a very convenient and fun solution for that, you just have an embedding website you want to work on, you have a Godot project you want to work on. The Godot project just pushes to a game repo on GitHub. In a YAML file, you can reuse the workflows I'm providing in the KS Godot CI package. These workflows will use a Docker image to compile the game on the CI with the web target, and they will then be published to the game page and you can run like your normal CI on your web repo as you're used to.

And then your web page is configured to fetch the game data files from the game page and then can run it from there.

Next step, you might ask, can we actually look at the code? Absolutely. It's all available online. I can also share a screen shot to show the very fundamentals, but I don't really have time to go into the details. So let's look at a higher-level abstraction here.

What is the actual communication happening between React and the GoDo game running on your website? So on the website component, you have a host component. That host component has props of a player name and a power-up. There's a callback of onReportScore to trigger that score-sharing pop-up. The host will share these properties through the window object, because there needs to be some back and forth in sharing the callbacks with the game. The GoDo game itself will look in the window object, see, okay, what's my initial player name? What's my initial power-up value? Then we'll also set these two functions onPlayerNameChanged and onPlayerPowerUpChanged on the window, If the props change on your component, these callbacks will be called. And then there are signals on the GoDo side, so PlayerNameChanged signal, which will trigger if you want to change the name of the power-up, and then the other game components can read the signals. How does it look? Very simple. In the end, you're just registering a callback to say, okay, when the power-up change, let's call that function. In this case, this is changing some of the particle effects slightly. That's actually most of it from the technical perspective.

What's working well at the moment? Working well is loading and caching games. It's pretty much feature-complete. It's quick. You'd go back to the website. You would get the cached version. It's all running fine. The communication between React and GoDo is working flawlessly, as you did see in the example. It's just going back and forth. Instantly, it could be abstracted slightly more, but more about that in a moment. In general, it's very easy to add new games to your own personal game playground. I would say the developer experience is very nice. I can just create a new game repo, I reuse my workflows, and it's pretty much running on GitHub instantly. That's all pretty nice. The most important thing, it's a lot of fun, because you can use your full game engine and you can still reuse the games in your personal React website. Take new technology and technology you're used to.