Extending Unity WebGL With Javascript

Rate this content
Bookmark

Unity can build games to run in a web browser using tools like Emscripten, Web Assembly, and WebGL. It provides integration with the browser, using browser APIs to simulate native APIs. Sometimes it is useful to interface with the browser in ways that Unity does not natively provide support for. In this talk, I will discuss how Unity builds games for the web, and how to extend Unity using Javascript to enable support for features not otherwise provided.

This talk has been presented at JS GameDev Summit 2022, check out the latest edition of this JavaScript Conference.

FAQ

Unity uses a tool called iotcp to compile .NET C Sharp code into C++. This tool handles stripping and massaging of .NET assemblies to reduce code size before generating C++ code, which is then compiled with Emscripten to produce WebAssembly.

Unity uses WebGL, a variant of OpenGL, for graphics when building web applications. It translates DirectX shading language used in Unity into GLSL to be compatible with WebGL through shader compilers.

JavaScript plugins for Unity are placed into the Assets Plugins folder. They include JSLib files, which define public APIs that can be called from C-sharp, and JSPri files, which contain supplementary JavaScript code. These plugins allow Unity to access additional browser features not available in the regular Unity API.

Unity can be extended with plugins to integrate WebXR, particularly for augmented reality. This integration involves creating a JSLib file to declare public APIs, which allows C-sharp to initialize WebXR and access its information. This approach is used to extend Unity's capabilities as it does not natively support WebXR.

Developing for the web involves dealing with frequent updates and changes in web technologies and browser behaviors, necessitating close collaboration with browser developers. Unity actively works to ensure compatibility and performance across different browsers and to leverage web-specific features like texture compression and advanced graphics options.

Shared data management involves allocating memory in C sharp on the Emscripten heap, which is also accessible from JavaScript. This allows data to be read and written directly from JavaScript, facilitating seamless interaction between Unity's C sharp environment and JavaScript code.

Unity has been enhancing support for mobile web exports by addressing specific mobile-related issues and integrating features like mobile keyboard support and texture compression formats suitable for mobile devices.

Unity has a public roadmap where users can vote on or suggest new features. This community-driven approach helps prioritize developments and ensures that Unity evolves in alignment with user needs and industry trends.

Brendan Duncan
Brendan Duncan
32 min
08 Apr, 2022

Comments

Sign in or register to post your comment.

Video Summary and Transcription

Unity targets over 25 platforms and technologies, including desktop, mobile, and virtual reality. They use Emscripten to compile the engine and game logic into WebAssembly for web development. Unity can be extended with plugins to access browser features like WebXR's augmented reality mode. The speaker demonstrates intercepting Unity's calls to the browser to modify its behavior. Unity is actively working on mobile support for web export and improving documentation for extending Unity with web plugins.

1. Building for Web and Extending Unity

Short description:

We target over 25 different platforms and technologies, including desktop, mobile, and virtual reality. When building for the Web, we use Emscripten to compile the engine and game logic into WebAssembly. Unity supports different graphics patterns and can be extended with plugins to access browser features. One example is integrating WebXR's augmented reality mode into Unity, which is not supported by Unity's built-in APIs.

One of the things that we do to support this idea is target over 25 different platforms and technologies from desktop, PC, Mac, Linux, to mobile, iPhone, Android, PlayStation, Xbox, Virtual Reality, Augmented Reality and one my favorites, the Web. When we build for the Web, we build like for any other platform where we compile the engine and the game logic together into a final executable. In this case, the final executable is WebAssembly and we use different tools to achieve this.

We use this Emscripten which is a C++ compiler that can generate WebAssembly in JavaScript. All the libraries for the engine and the built-in libraries are written in C++ and all the user code and public APIs are written in .NET C Sharp. We have to compile the .NET C Sharp code into C++ in order for it to be compiled with Emscripten. So we use another tool called iotcp to do this. It takes the .NET assemblies and does some stripping and massaging of those assemblies to reduce the code size and then generates C++ code from there. The C++ code can then be compiled with Emscripten to WebAssembly.

For the graphics side of things, Unity has different graphics patterns that it supports from DirectX, Vulkan, Metal, and OpenGL. On the web, graphics are defined by WebGL, which is a variant of OpenGL. So when we build for the web we tell Unity to use the OpenGL graphics device. And when it does this, Emscripten and the compilation processes will generate WebGL calls for all the OpenGL calls that Unity are making. And for shaders, they're typically written in a shading language, a DirectX shading language in Unity, but those aren't directly supported by WebGL so we have to convert those into GLSL to be used by WebGL. To do this, we have shader compilers that will translate HLSL into GLSL.

We can extend Unity with plugins that provide new APIs to Unity written either in C++ or in this case we'll use JavaScript. With the JavaScript plugins for WebGL, we can extend Unity to access browser features that aren't available in the regular Unity API. We can call these JavaScript functions directly from C-sharp and vice versa you can call C-sharp functions from JavaScript. When you define a JavaScript plugin, you put it into the Assets Plugins folder for WebGL and there's two different types of files from the JavaScript function. The main type of file is a JSLib file. This is where your public APIs from JavaScript will be and define the functions that will be callable from C-sharp. A JSPri file is just arbitrary JavaScript that you can include with your plugin. This will get compiled before the JSLib files so that it can provide JavaScript objects and functions that can be used and shared between your JSLib files. This is just a way to keep your projects clean so that your JSLib files can be your public APIs and you put all the rest of your code in JSPri. There's no requirement to do this. You could just use a single JSLib file. I like to keep things separate. The example of plugin I'll be talking about today is integrating WebXR's augmented reality mode into Unity. Unity currently does not support WebXR with its built-in APIs, but we can extend Unity using plugins to do this. This is not an official plugin.

2. Implementing WebXR and Sharing Data

Short description:

This is a minimal implementation of WebXR, demonstrating how to use it. The source code is available on GitHub. The JSLib file declares the public API for the plugin, allowing direct calls from C sharp. The merge into function exposes the declared functions to C++ or C sharp. The JSPRE file contains the main objects and methods for interacting with WebXR. You can also call C sharp from JavaScript using delegate types and callback functions. Data can be shared between C sharp and JavaScript by allocating memory on the end scripting heap.

This is for demo purposes only, and it's a very minimal implementation of WebXR. I'm not implementing all the fun features of WebXR. I'm just showing you how to do this.

All the source code for this project can be found on the GitHub project here, and you're welcome to use it for any purposes.

The JSLib file is where we're declaring the public API for our plugin. And here we'll provide functions that we can call from C sharp to initialize WebXR and get its information in its current state. These functions are callable directly from C sharp, and this is the public API that we're providing.

The merge into function here is part of the script, and what it does is it takes all the functions that we're declaring in the subject and exposing them to C++ or C sharp. To call JavaScript from C sharp, we declare the functions as external static functions in C sharp, and we also tag them to be encoded from the internal DLL in Unity. What this is doing is it's telling C sharp that these functions are not defined in place but they're coming from an external source, and because WebGL does not provide support for external DLLs, everything is bundled together and the internal DLL name defines that they're coming from the built-in DLL.

Here I also implement MTEW versions of the functions for non-WebGL platforms, because non-WebGL platforms do not support JavaScript. We want to call dummy versions of these functions to keep C sharp happy and keep the editor from complaining that you don't have implementations for these functions.

The JSPRE file is where I put the bulk of the code to keep the JSO file simple, and this is where I define all the main objects and methods for interacting with WebXR, and here you can put all the functions that can be called from your JSO and managing the state of your plugin. Again, there's no requirement to use a JSPRE file. I find it convenient.

Sometimes you want to call C sharp from JavaScript. In some cases, you have an asynchronous function in JavaScript that will be called sometime later and you want to call a C sharp function when that asynchronous process has finished. You can do this by declaring a delegate type in C sharp that is then toggleable from JavaScript. It can pass that function pointer of that delegate type to the JavaScript function, which it can hang on to as a pointer, and then it can call that C sharp function when it's done. And then we define the callback function in C sharp as a static function. JavaScript doesn't have any notion of C sharp objects, so we declare it as a static function, and from there you can use global variables or singleton access to whatever C sharp state you want. And then from the JavaScript side, we can use the inscription's dyne call function to call the C sharp callback. This can also be a C++ callback. So the vi argument for dyne call defines the return type as void and it takes a single integer argument and this can define the function declaration that you have. And the state change callback is the function pointer I passed in from C sharp, and these are the arguments I'm passing to the C sharp callback function which is an integer state. You can also share data between C sharp and JavaScript. If we allocate memory on C sharp side it's allocating on the end scripting heap and that end scripting heap is visible from JavaScript. We can read and write to that heap from JavaScript. Here I'm allocating 16 floats that are used to store the view matrix of WebXR so that I can update the camera's view matrix in Unity. I can pass that data to JavaScript as just a function data pointer and then now JavaScript will have access to that data.

QnA

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Optimizing HTML5 Games: 10 Years of Learnings
JS GameDev Summit 2022JS GameDev Summit 2022
33 min
Optimizing HTML5 Games: 10 Years of Learnings
Top Content
PlayCanvas is an open-source game engine used by game developers worldwide. Optimization is crucial for HTML5 games, focusing on load times and frame rate. Texture and mesh optimization can significantly reduce download sizes. GLTF and GLB formats offer smaller file sizes and faster parsing times. Compressing game resources and using efficient file formats can improve load times. Framerate optimization and resolution scaling are important for better performance. Managing draw calls and using batching techniques can optimize performance. Browser DevTools, such as Chrome and Firefox, are useful for debugging and profiling. Detecting device performance and optimizing based on specific devices can improve game performance. Apple is making progress with WebGPU implementation. HTML5 games can be shipped to the App Store using Cordova.
Building Fun Experiments with WebXR & Babylon.js
JS GameDev Summit 2022JS GameDev Summit 2022
33 min
Building Fun Experiments with WebXR & Babylon.js
Top Content
This Talk explores the use of Babylon.js and WebXR to create immersive VR and AR experiences on the web. It showcases various demos, including transforming a 2D game into a 3D and VR experience, VR music composition, AR demos, and exploring a virtual museum. The speaker emphasizes the potential of web development in the metaverse and mentions the use of WebXR in Microsoft products. The limitations of WebXR on Safari iOS are discussed, along with the simplicity and features of Babylon.js. Contact information is provided for further inquiries.
Making Awesome Games with LittleJS
JS GameDev Summit 2022JS GameDev Summit 2022
34 min
Making Awesome Games with LittleJS
Top Content
Little.js is a super lightweight and fast JavaScript game engine that has everything included to start making games right away. It has a tiny footprint and no dependencies, making it perfect for size-coding competitions like JS13K. Little.js is built with an object-oriented structure and comes with several classes. It provides a fast rendering system, a comprehensive audio system, and various starter projects for different game types. Little.js is designed to be simple and easy to understand, allowing you to look at and modify the code.
How Not to Build a Video Game
React Summit 2023React Summit 2023
32 min
How Not to Build a Video Game
Watch video: How Not to Build a Video Game
The Talk showcases the development of a video game called Athena Crisis using web technologies like JavaScript, React, and CSS. The game is built from scratch and includes features like multiple game states, AI opponents, and map editing. It demonstrates the benefits of using CSS for game development, such as instant load times and smooth transitions. The Talk also discusses optimizing performance, supporting dark mode, and publishing the game to other platforms.
Boost the Performance of Your WebGL Unity Games!
JS GameDev Summit 2023JS GameDev Summit 2023
7 min
Boost the Performance of Your WebGL Unity Games!
The Talk discusses ways to boost the performance of WebGL Unity games, including issues with bundle size, memory usage, and runtime performance. It suggests using Brotli for compression and non-exception support for better performance. Choosing the appropriate texture compression format and experimenting with separate builds can also help. The Talk also covers optimizing textures, models, audio, and assets by reducing build size, using compression, disabling unnecessary models, and optimizing audio quality. Unity's optimization tools and profilers are recommended for analyzing performance and memory issues.
Embracing WebGPU and WebXR With Three.js
JSNation 2024JSNation 2024
27 min
Embracing WebGPU and WebXR With Three.js
The 3JS project has evolved into a community-driven effort with numerous contributors over the past 14 years. It started with 3D engine work in Flash and transitioned to using SVGs for rendering in HTML5 before adopting WebGL. The project showcases various projects and frameworks, including a no-code tool powered by 3.js. The team is working on a new render using WebGPU and developing a new shader language called TSL. The hope is that WebGPU will eventually replace WebGL, offering better control and performance.

Workshops on related topic

Make a Game With PlayCanvas in 2 Hours
JSNation 2023JSNation 2023
116 min
Make a Game With PlayCanvas in 2 Hours
Top Content
Featured WorkshopFree
Steven Yau
Steven Yau
In this workshop, we’ll build a game using the PlayCanvas WebGL engine from start to finish. From development to publishing, we’ll cover the most crucial features such as scripting, UI creation and much more.
Table of the content:- Introduction- Intro to PlayCanvas- What we will be building- Adding a character model and animation- Making the character move with scripts- 'Fake' running- Adding obstacles- Detecting collisions- Adding a score counter- Game over and restarting- Wrap up!- Questions
Workshop levelFamiliarity with game engines and game development aspects is recommended, but not required.
How to make amazing generative art with simple JavaScript code
JS GameDev Summit 2022JS GameDev Summit 2022
165 min
How to make amazing generative art with simple JavaScript code
Top Content
WorkshopFree
Frank Force
Frank Force
Instead of manually drawing each image like traditional art, generative artists write programs that are capable of producing a variety of results. In this workshop you will learn how to create incredible generative art using only a web browser and text editor. Starting with basic concepts and building towards advanced theory, we will cover everything you need to know.
PlayCanvas End-to-End : the quick version
JS GameDev Summit 2022JS GameDev Summit 2022
121 min
PlayCanvas End-to-End : the quick version
Top Content
WorkshopFree
João Ruschel
João Ruschel
In this workshop, we’ll build a complete game using the PlayCanvas engine while learning the best practices for project management. From development to publishing, we’ll cover the most crucial features such as asset management, scripting, audio, debugging, and much more.
Introduction to WebXR with Babylon.js
JS GameDev Summit 2022JS GameDev Summit 2022
86 min
Introduction to WebXR with Babylon.js
Workshop
Gustavo Cordido
Gustavo Cordido
In this workshop, we'll introduce you to the core concepts of building Mixed Reality experiences with WebXR and Balon.js.
You'll learn the following:- How to add 3D mesh objects and buttons to a scene- How to use procedural textures- How to add actions to objects- How to take advantage of the default Cross Reality (XR) experience- How to add physics to a scene
For the first project in this workshop, you'll create an interactive Mixed Reality experience that'll display basketball player stats to fans and coaches. For the second project in this workshop, you'll create a voice activated WebXR app using Balon.js and Azure Speech-to-Text. You'll then deploy the web app using Static Website Hosting provided Azure Blob Storage.
Tiny Game Live Coding Workshop
JS GameDev Summit 2023JS GameDev Summit 2023
115 min
Tiny Game Live Coding Workshop
Workshop
Frank Force
Frank Force
Dive into the captivating world of micro-game development with Frank Force in this interactive live coding workshop. Tailored for both seasoned developers and curious newcomers, this session explores the unique challenges and joys of creating games and demos with extreme size constraints.