Run Games Within Your React Native Apps

Yeah, thanks for having me here. Really excited finally. Two years of not being at in-person conferences to finally be here, one of my favorite conferences, so like really excited to see the audience.

And today we're going to talk about some cool stuff. We'll talk about how you can run games within your React native apps. So, yeah. Clicker is working. Cool.

So, a bit about myself. So I'm Vladimir Novik, I'm software architect and consultant, and I do a bunch of stuff in all these fields, and beyond that. So if you're interested in like something within these fields, you can reach out on Twitter or my website.

So let's start with like the first statement that game development is pretty hard. So you need to take care of a lot of things there. Rendering, 3D, materials, lighting, only this kind of stuff. Physics, how objects interact. How they collide with each other. Is there a gravity, acceleration, all this kind of stuff. AI, how enemies attack your character. How the pathfinding algorithms, all of that. Networking, sound engine because it's not enough to just have like regular music in the game. It's all sounds of interactions, of like effects and even sounds of the footsteps when your character is walking.

And when you think of a game, you start with a concept art. So, you need to come up with the idea how to do that and you basically need to get lots and lots of creative assets, which is hours and hours of work of 3D artists. Or you need to purchase them. You get into like level design of things. How you structure each level, how they can be like inherently more complex than one another without actually impacting gameplay. Speaking of which, there is gameplay mechanics, which is different from game to game that you need to take account for. Animations, character animations, UI animations, all of that. And obviously post processing, because effects matter if you have like a bloom effect of like things blooming or HDR and stuff like that. And most of these things are taken care of by game engines.

So, we need to have one. There are multiple ones and the top ones in the market, one of them was actually mentioned in last talk. It's Unity.

Now, the other one is Unreal, but today we will talk about Unity game engine, and how we can actually create it, create a wrapper for Unity, and wrap it into our React Native app.

Now, Unity gives us lots and lots of capabilities, that is, from their website, has like 2D, 3D world building, characters, mobile, AR, VR, lots of stuff. And you can actually use all these capabilities and wrap them into React Native apps nowadays, which is pretty cool, I think.

So, features for mobile that Unity has, there is a lot of them, I will just briefly go through some of them. So, you have like a rendering pipeline with lightness systems and how you present realistic textures on objects and stuff like that. Native 2D tools if you want to create complex 2D games. You have visual scripting. Now, if let's say you don't want to write in C-sharp, which is the language Unity is using for development, you can actually use visual tools to really create your game, like full blown game using visual scripting. Harmonization solutions, AR, which is one of the main reasons to use Unity within React Native because AR within Unity is going like really low level, working with ARKit and ARCore and it's really, really performant. And you can check other features here on the link above on what Unity supplies.

But today, we will talk about one major feature that is game changer for React Native developers is using Unity as a library. So React Native and native code, they know how to interact with each other, right? So if you look at like really, really simplified version, so you have the JavaScript, you have the native code and we can use the Unity as a library to bridge into this native code from Unity side and vice versa, and do the same bridging from the native to JavaScript. And that's what we will be doing today.

Let's get to the demo. Because I want to show you what you can achieve with Unity and React Native. Focusing also on performance, we will see how performant it is. Load the startup though. Takes a little bit of time to load everything. Let's see.

Okay, cool. So we have regular React Native app. Just blank screen, right? And on the top you have performance profile, it is a little bit small to see, but you can see that we're running with 60 frames per second. Now if I go to Unity side of things, I have two paths. Yeah, sure, thanks. Yeah, thanks. And I will also make it bigger. Okay, that's better.

So I have Unity running within React Native app, and on the bottom is React Native. On the top is Unity. So if I change color, you see that immediately the color is changed within the Unity. Now if I write text, I will write, hi Berlin. We can see that this text is rendered within the Unity, and I can obviously clear that.

Now if I click on the message React Native, ta-da, I have the message within my React Native app. Now you can see it's pretty fast. So the whole like, and you can see the frames on the top, if you've noticed, they drop a little bit but not that much. Now I can even have post processing effects here to give some bloom look to the view. But let's get into like more complex games. So the character walking, jumping, and even running, a bunch of physics involved, and more complex interaction. So let's see, somewhere, it has staircase, so these kind of games, and as you can see on the bottom, there is actually React Native rendered. Right, so look at the performance. It's still 60 frames per second. And let me just get to the top. As you can see, we have here our robot working in the outer space. And it's working pretty good. Now I get back to my React Native app to a different view, it's a different scene within Unity, and it works like a charm. So let's see how we can actually achieve that.

So after the talk, you'll have a video of the same demo, so if you want to watch that, and in React Native, it's as simple as just use Unity View, a component that I've created, and we'll see how to create that. And I have on Unity Message callback, which basically sets the state within React, and you have a message posted from Unity within the view. Now, in order to post something to Unity, I have basically, let me just put the cursor here, or rather, do it that way, I have these function, postMessage, and I use the GameObject within Unity Editor, its name, and I use the method name of the script that is attached to that GameObject, and I pass the payload. This payload will be in Unity, basically, parsed back into a serialized object, and then I can do whatever I want with it. In my case, I will set the color and set the text. Now, in Unity, it will look like this. I have, like, our interop object on the left, the one that is marked there. And basically, that's, like, the name of the object I'm calling. Now, in order to enable this functionality fast in Unity, I need to create these two files. These are two native plugins. That's the option how you can bridge Unity into React Native.

You create these plugins. They have to be in assets, plugins, iOS folder. In this example, we will go through iOS example, but the same possible for Android.

And then, these native files are basically two files. There is the header file and a main file. And we'll get into a little bit of Objective-C, and a little bit of C-sharp, and a little bit of JavaScript code. So, it's going to be kind of three languages in the talk, so hopefully you're fine with that.

So, the first one is this header file, and it has two functions, basically. It has the register API for native calls, which will be called by Unity framework, and it will register our sendMessageToMobileApp function, and we will be able to call it from Unity. So, we basically use X and C, and we write our message here under sendMessageToMobileApp. We basically make it available for Unity. So, that's it. That's the whole thing we need to do from native plugins perspective in Unity.

Now, sending message to native in Unity will be done in the following way. In Unity, we will use the allele import internal, and we will specify the method to be called on the native side. In that case, Unity framework will search for the method name and will call it. And we'll attach that to a button and call the class NativeAPI, sendMessageToMobileApp, And that's what you've seen in the demo. Now, in Unity, you attach that to the component, and you specify the method button click press to be called, and that's how the interaction from Unity to React Native works on the Unity side of things.

Now, receiving message from React Native in Unity, basically as simple as just creating C sharp script with the method that you will be calling from React Native. You've seen, you just call the name of the object, the method, and it will be passed to Unity. Now, it's up to you what to do there. I just changed the color and the text. But basically, you can pass any data and do whatever you want with that.

So, in order to do this kind of integration, you need to do a few things. And now we will be going through the steps, so you will be able to do that yourself. So first of all, in Unity, you need to build your iOS project. That's the build settings window from Unity that every time you export to iOS, it's just a regular screen. You build as normal. Nothing different except having these two native plugins, two files for the native plugins. Now, I put in my example, I put it under Unity builds iOS folder in the React Native project.

Now then on React Native project, I will open an Xcode, my regular workspace React Native, under iOS folder, there is Xcode workspace that I will open. And then just add my Unity project, the build Unity project, into this workspace. So at this point we have two projects. Unity iPhone, which is the default name, and Unity Native, the React Native project.

Now then it's really important to actually add Unity framework to embedded frameworks under Xcode because we need to use the framework for certain interrupt. Another important step to actually remove linkage of, like, linking binary with libraries, you need to remove Unity framework from there, otherwise it will just break and you'll see like wild errors. And another helpful thing is to actually drag and drop the embedded frameworks before the compiled sources because what you want to do, you want to embed framework before actually compiling native files that we'll be writing in a bit.

You also need to change data membership. Unity iPhone project has a data folder which has all the assets and data for the Unity project. So we need to change membership to be Unity framework because when you add the project, it will be, by default assign membership to our React Native project, which is something we don't want. So we need to change that. We need to change build products path of Unity framework to point to our build directory because we want everything to be built into one iOS builds directory. And then we also need to find our like native plugins that were wrote. And we need to change their membership as well, first of all to Unity framework and then to be public because by default they're private. And then we need to build the Unity framework separately.

Now these steps absolutely must, you need to go through them. Everything that on this slide, you need to go through them every time you build your Unity project. And that's a little bit caveat because if you want to change things in Unity, you basically change things, you rebuild, you do the steps, and you get stuff in React Native.

So in order to integrate with React Native, we will start by regular integrating native modules, the same way we do with by creating React Native View Manager in our example. And we will use RCT export module, which is the way to export the module to the JavaScript side. And we will use RCT export view property, which gives us an ability to basically send events to register property on Unity message in our example and send events to JavaScript. We will create separate class for React Native Unity view. And then this class will have the one method that we just had like postMessage was the message, the method we will call it from JavaScript. We again will use RCT export method to export that to the JavaScript side. And we will use Unity postMessage bit on the top for actually doing the communication. And we also need to register supported events. Now there is a link on how to do that in official docs of React Native on how you actually send events to JavaScript. Another really important thing that we need to basically make sure we run on the main thread because otherwise there will be a separate thread and we don't want to deal with that. So we use this one method queue dispatch getMainQueue.

So we use this one method queue dispatch getMainQueue. Again that's in official docs, how we integrate stuff. In React Native Unity view, it's not the view manager, it's the view naming a little bit off. But we basically send message, we use this send message to go with name. Now this is the default name of Unity framework API to actually send a message to these game objects within Unity and call the method on it. And then we have these like wrappers for Unity framework to just make sure it loads, and it says the data bundle to be like this data folder, etc.

Now, getting back to JavaScript side. On JavaScript side, you remember this Unity view that we had, right? So that's how we write this Unity view. We get UI manager, native synthetic event, and we basically require native component and use our view manager as React Native Unity view, like so. Now, in post message, we will actually use UI manager, UI manager get view manager config. And we post the command to basically send that to Unity, so it will call stuff on the native side. And that's the way how we use that.

Now, let's get some really cool news. Now, that can be a little bit complex, right? To write all these, like, native code and stuff. So there is a library for that. It's called React Native Unity. Ta-da! And so no need to write native code, right? So that's a huge part of things. You just use React Native Unity. It has Unity view. It supports Android as well. So I just described how to use it with iOS. You still need to do all the setup and build steps. So it's really good now you know how it actually works under the hood.

So there are some helpful links that you can screenshot and get. There is a demo code in this repo. In order to learn Unity, you can go to learnunity.com. There are amazing paths to actually learn Unity development. There is a Unity assets store where you can purchase assets for a fraction of the cost. And if you have any questions for me after the talk, et cetera, so reach out on Twitter. That's my Twitter on the top, and you can ping me for consulting on my website.

And I also have these forums. I will be doing workshops and courses in upcoming years, so if you're interested in learning more about when it will be done, you can just fill it in. And I promise not to spam. I will just send you an email whenever any course or workshop will be ready.

And I think at this point the clicker stopped working. And the reason for that is I'm kind of done, and I think we have several minutes for questions here, and thanks a lot for listening. Good working! Yeah, thank you very much Vladimir. Just go ahead. Come come come come! We do have some questions, though. Are we running short on time? Almost, yeah.

So, there's a question. What would be a real-world use case for integrating Unity in React Native? I saw it coming! That's like, yeah. The real-world case, scenario nowadays, mostly it's AR. Because if you want to integrate AR into your React Native apps, right now it's either you use other frameworks such as Viro, for instance, which is a good one as well. But if you want more native capabilities and getting into all the capabilities ARKit has to offer over ARCore, Unity has a ARFoundation framework which enables you to leverage almost everything you can do with ARKit. And now you can bridge that into Unity. That's one of the use cases. There are quite a lot. I mean, you can do gamification, you can do like fancy ads within, I know we all hate ads, but I mean, sometimes it's driving business. I've heard that the Unity ads are pretty good. You can use ads, you can use some 3D views which are really performance oriented. For simple games, the ToggleView before mine was pretty good to just try things in like for 2D games and publish them as a web view. I really like Unity capabilities in general.

That's actually a good answer because you also covered another question which if AR is actually possible. There is another follow up question to the first one. What would be a real use case for Unity inside Reagnative? You showcase fireworks and things like that but what do you think in your perspective it is a real use case of Unity? Apart from AR, what do you think about UI libraries that already exist in Unity? Can we leverage those? Definitely sometimes and that's good you bring it up because Unity assets store has lots of amazing assets that you can purchase and some of those are really fancy UI libraries with bunch of effects and fireworks so if you want to have generally something like physics related and I think like particles are also like physics related like fireworks and stuff you can use these assets from Asset Store and embed them within React Native App and as you've seen you getting pretty good performance on that.

Great speaking of performance how performant is such UI in fast in-game interaction scenarios especially given that on every call the JSON serialization and deserialization is included. So basically you're not introducing another bridge you're basically in the same world as in general like React Native because basically what you're doing is serializing the same data and passing that through the bridge that you call in native methods that go into Unity so there's no like another layer here. So performance will be pretty much the same like talking about like passing through the bridge will pretty much the same as regular React Native apps. With that said if you want to make things more performant I would think of like serializing the data passing to the Unity and handing over the logic to Unity to do like the whole animations and stuff and then passing the data back.

So I would, for instance here I had like lots of renders and passing things through the Unity that's for the sake of the demo but I would advise against that because that's eventually lots of traffic through the bridge, you don't want that. Yeah, the thread will be bloated anyway so I think that's one drawback of React Native but I think that the new architecture is going to solve some parts of that as well.

Speaking about Native modules, have you ever thought about Expo Native modules, like to expose the Unity view using Expo Native modules? What do you mean, like exposing um.. Like Expo has created these Native modules... Ah, yeah, so basically with Expo, yeah, I've thought about that, like something I haven't tried and you need to experiment with that a bit. Just to bring in some context, Unity at some point they had like really like vendor-log sort of thing, they didn't have this feature, Unity as a library is fairly new one, relatively, as things go slower in game development world, it's like for couple of years, but still like from Unity perspective it's a relatively new one, so there is no much experimentation done on that regard, especially with React Native. But yeah, that's definitely something to look into and try it out.

And we do have one more question, why not use Unity and build iOS without additional wrappers, what is the profit itself? If just using Unity? Yeah, without going through React Native, what do you think is the benefit here? I think that you need to identify what's your app use case. So there's a regular statement from consultant, it depends, right. So usually, I would say if your app is really heavy oriented into fancy UI and rendering and physics and stuff like that, there's no reason to use React Native, but in lots of cases we see that clients will already exist in React Native apps, they want to introduce more gaming capabilities and then to some degree it limits their imagination, I would say. If they want to see something fancy, they cannot do that with the TypeScript and JavaScript, they need to do something more game-ish. In that case, Unity and React Native will be the good choice. If it's something more simple, then it's either React Native or Unity. But it's good to know that integration exists and it opens up a lot of opportunities for experimentation and actual projects.

Speaking about integration, have you ever thought about the size of the bundle after integrating Unity? Yeah, the bundle will be absolutely, I wouldn't say huge, it will be way higher because obviously with Unity you need to deal with all the 3D assets if you're dealing with the game. These demos, I think 150 megs bundle which is pretty huge, but if you think of game development in general, usually they have a huge bundle. So that's something to take into account, obviously. Okay, so it's basically something new that is going to enhance over time, but now we should also think about trees and nature, just saying, because 150 megabytes just for a Hello World or a 3D box is quite a lot, but definitely maybe it's going to improve. There is optimization, speaking of that, there is an option with Unity to use asset bundles, so basically you don't download all the assets, you get them through the network on the fly. There is capability to do that, and that's something, obviously, to take into account when you develop your app, especially with AR, for instance, because you have all these 3D models, which is really taking some space on the device.

Thank you. I think we're running out of time, so yeah, let's give a huge shout out to Vladimir. Thank you very much.