Unleashing Next-Gen 3D Web Experiences with Babylon.js 6.0

Hello everyone. My name is Ranaan and I'll be presenting Babylon.js, the open source 3D rendering engine.

First, a bit about me. I am a software developer at Microsoft. I am Babylon.js' Web tech lead and a core contributor since roughly 2014. I am a director responsible for Babylon's infrastructure, its build system, the WebEx our abstraction, and a few other features along the way.

But let's talk Babylon. Our goal at Babylon.js is to create the most powerful, beautiful, simple and open web rendering engine. We want to provide any developer, designer, and hobbyist all of the tools needed to create GPU-accelerated 3D experiences and games on the web, without the need to deeply understand 3D rendering while allowing experienced developers the tools to get the most out of these APIs.

So Babylon is a fully featured web rendering engine. It is written in Typescript, transpiled to JavaScript, and it is built on top of open web standards. Standards like WebGL and WebGPU for rendering, Web Audio and Video for media consumptions, and other standards like WebXR, WebRTC, WebAssembly, in general, every API that starts with web, we want to support it. Babylon is always backward-compatible, which is a guiding principle for the team when developing new features or modifying older ones. Our code, written in Babylon 3, will still work in Babylon 6, 7, or 10. The framework is roughly 10 years old and is being used in production websites throughout the web.

Apart from our core package, we offer different packages like loaders, serializers, materials, UI, and others, including all of our editors and tools. Everything is open-source and we publish them on both NPM and on our CDN. Out of the box, we have full support for WebGL 1 and 2, WebGPU. We have canvas accelerated UI, advanced material system, physics integration, WebXR abstraction, device input system, and many, many other features. You can visit our website and doc page to see a full list of the features.

Babylon has a set of powerful, powerful tools to allow you to make the best out of the engine. The first tool is the Playground. The Playground is a side by side code editor and lets you see your scene you're working right in your browser. It's based on Monaco, which is the editor used in, among other, VS Code. You can write code in either JavaScript or TypeScript, and you have full access to all of Babylon's features and published packages. The Playground allows you to save your code, share it with others, and even download your scene and host it yourself. The Playground is available at playground.babylonjs.com and is a perfect place to get you started with the engine and understand its capabilities. It includes many, many different examples.

The second tool is the Inspector. The Inspector is a debugging tool for your scene.

You can enable and disable features using a very simple UI. See or edit your scene graph, check your meshes, your materials, your textures in your scene and transform them to your needs. The Inspector can be enabled on any Babylon scene. You need just a few lines of code. And it is integrated in the Playground to simplify debugging your Playground scenes.

The third tool I want to talk about is the Node Material Editor. Node Materials Editor or NME allows you to create shader materials using a set of blocks available in our core library. Using a very intuitive UI, you can create materials, save them as JSON files or directly on our free snippet server and add them to your scene using a single line of code.

The last tool I want to show here is the UI Editor. This is the tool that you're seeing right now. The UI Editor allows you to create 2D-accelerated UI for your Babylon experience. Using all of the features of our powerful UI library, you can add, remove, and transform different types of UI elements, and just like the NME, save them as downloadable JSON or on our snippet server and then use them directly in your scene. Using that couldn't be simpler. Simply parse the JSON or provide the snippet ID to integrate the UI in your scene and hook up any callbacks you need by using Babylon's simple device input API. All of those tools, as you can see, are integrated in the Playground, so if you add a node material or a UI, you can edit them directly in the Playground.

I want to shortly mention Babylon Native. Babylon Native brings the power of Babylon Engine to Native platforms. As a developer, you can reuse the same code you use for web experiences in Native environments like Android, iOS, Windows and others. If you want to know more about Babylon Native and Babylon React Native, you can visit GitHub or our documentation page.

That was a general overview of BabylonJS. Now I want to dive into the new features added in our latest major release, BabylonJS 6.0, that was released last April. Babylon 6.0 was a major milestone for the team as it brought a lot of improvements and new APIs to our community of developers. The most remarkable feature we added in 6.0 was the integration of the Havok physics engine. Havok is a world-known physics engine used in many, many, many AAA games. Together with the Havok team we have worked to bring Havok to the web using WebAssembly. Up to 6.0 we had support for MOJS, Canon and OIMO physics engine using a single unified API. For Havok, we have also introduced a new version of our physics API that simplifies creating a highly performant physics experience on BabylonJS. The physics API 2.0 allows us to add new and exciting features to our physics capabilities like advanced physics ray-casting, better joint support, full mesh and convex hull support and a lot, lot more, while making the engine perform 20 times faster and more than the older API. We've also released a game called Playroom to show the new capabilities. You can visit the game at theplayroom.BabylonJS.com.

So let me jump to the playground and see some physics example. The first example I want to show is the stress test that we have implemented. You can see that when using MLJS, which we're right now seeing, with a large number of objects, MLJS is a bit choked and, on my machine, it works roughly in 6 frames per second. Now, let's see what happens when we switch to Havok. When we switch to Havok, you can see that the rendering stays at 60 frames per second and even provides a larger potential frames per second. This is the same scene using our old physics API with ML and the new physics API in Havok. Even when we're using a height map as a ground, Havok still maintains its high, high, high performance.

So let's open a second example that I want to show, which is our physics helper functions, which are a part of our core physics support. The helpers allow you to create special physics effects with ease. We have effects like vortex, updraft, gravitational field, and radial explosion. They can be triggered using a single function call on any physics-enabled scene. In this example, you can see exactly how to use this function. The code is available on the right. You can also enable the debug viewer, if you want to see exactly how the physics engine is working. You can see how performance would improve if you're using instances or an aggregate.

The last example I want to show is the constraints in Havok. You can use them directly using our physics API, connecting two or more physics bodies to one another. All of these examples are on our playground, which also includes many other demos. You can find a lot more on our documentation page.

Fluid rendering is a feature our community has asked for quite some time. In Babylon 6.0, we have added screen space fluid rendering. Enabling this feature requires just a single line of code. You can enable it on the scene, and afterward you can render our particle or GPU particle system using the fluid renderer. The feature is fully documented, including many, many different demos, and you can get started right away. You can see here, two demos that are available on our documentation page.

Until Babylon 6.0, we have offered screen space reflections, or SSR, using a post process. It was working well, but was not highly performant, and a little hard to fully configure. As part of making Babylon more powerful and simple to use, we have deprecated the post added a SSR rendering pipeline that simplifies adding screen space reflections to your scene. It supports both WebGL 2 and WebGPU, and can be enabled using just a few lines of code. On top of that, SSR has a debug mode that allows you to see how the SSR rendering pipeline is seeing the scene and help you understand how it works. Enabling this mode is as simple as turning on a flag in the rendering pipeline. This demo is also available on our playground and in our documentation page.

Babylon has added WebGPU support in Babylon 5.0 already, a support that was improved immensely in Babylon 6.0. It is fully backward compatible, always making sure that even if the WebGPU API itself changed and it does, Babylon still maintains the same API, and it helps you upgrade to a better, more performant rendering engine. We have also full support for compute shaders that allow you to fully utilize GPU in your scene. WebGPU now comes with different snapshot modes that can accelerate rendering your scene, depending on your use of the engine. WebGPU lives along WebGL 2, so you as a developer can decide if you utilize the stable WebGL 2 that is available all over, or the newer, more performant WebGPU. We make sure that every new feature added to Babylon is tested on both WebGL and WebGPU. All features are fully documented, and the WebGPU engine is also integrated in our playground if you want to experiment with that right now.

Another feature that we added as part of our scene optimization suite was the performance priority modes. Using this simple tool you can optimize the rendering process of your scene. The default mode, the one you see right now, is the one you get out of the box. It makes no assumptions regarding your scene and will always work correctly. The two other modes, intermediate and aggressive, will make certain assumptions regarding your scene and will enable or disable certain features that might not be needed. Depending on the features that you use or don't use, you can set the selected priority mode using a single function call to get a rendering boost for your Babylon experience.

In this example, rendering 2,500 spheres with 50 different materials, you can see how moving to the aggressive mode, the one that we're seeing right now, has improved rendering by more than 50%. An exciting and, in my opinion, underrated feature that we added in 6.0 is the accessible screen reader support. Simply put, it is a way to generate an HTML tree from your scene. The HTML tree can be read by a screen reader, making your scene accessible for people with disabilities. It fully supports our 2D UI as well. Every HTML element is fully configured using a JS object located on the nodes or UI elements in your scene, so you can decide how the screen reader understands your scene. It also supports keyboard events using our Action Manager. For example, you can use a tab to navigate the scene, and have the screen reader read what you're currently pointing at, and then select it with any key that you want, for example an enter or a space key.

Babylon has had mesh decal support for a long time now. Mesh decals add a mesh on top of another mesh to paint decals in the right place. However, adding them might hurt performance to your scene if you add too many of them.

To improve performance when using decals we have added texture decal support in 6.0. Using texture decals you're only adding a texture on top of the existing material to render the decals. The result is faster to generate and faster to render. This example you see is in our documentation page and is running in our Playground.

So babylon 6.0 was an exciting release for the team. On top of the features I presented we have fixed many different bugs and have added hundreds of improvements and additions to our existing API. If you want a full list of changes you can change our change log in the GitHub repository.

And now we are marching forward developing the upcoming major version 7.0. It has a growing list of new features. Let me share just a few of them with you right now.

The first feature is a whole new editor that was added to our arsenal. The Node Geometry Editor. Inspired by Blender's Geometry Node, the NGE allows you to procedurally generate meshes for your Babylon experience using a very simple UI by adding different types of blocks to a unified graph. Very similar to the way NME works for materials. Using a procedural building process instead of exporting a mesh from a 3d editor can improve download, render and startup time and in general improve your scene's performance. The initial version of NGE was already deployed. It is available on NPM, our CDN and also in our playground. If you want to visit the standalone version, the one that we saw right before you can go to NGE.BabylonJS.com and already start experimenting with it.

Another feature that was requested by our community for quite some time is what we like to call the Flow Graph. The Flow Graph is a way to programmatically introduce interactivity to your scene, kind of like an advanced action manager. Using predefined building blocks, you will be able to define triggers and actions in your scene using the same simple and known UI the node material and the node geometry editor are using. The Flow Graph will be fully compatible with GLTF's new interactivity extension, which is currently in draft mode. It is a work in progress, and if you want to follow it and see what we are doing, you can follow our GitHub repository and the issue existing there.

Another new feature coming to 7.0 is the experimental support for pure ES modules. Babylon is currently packaged in two different ways, UMD, so ES6 and UMD, and both of them have side effects. Having side effects means that tree shaking is not fully supported and requires a bit of work to achieve, and with UMD, tree shaking is not even possible. The idea behind our ESM support is to provide a function-based API with no side effects that will allow the developer to only use what they need for the experience.

This way, the developer is only using the minimum they need to render their experience, minimizing their package size and download size, and avoiding side effects. It will also compile with the latest JavaScript version to allow us, the Babylon developers and devs using Babylon to use modern JavaScript.

This is still a work in progress. We will first provide a revamped API to our base components like the engines and the scene. The current packages, the UMD and the ES6 packages will continue being published of course and they will not be converted to ESM.

What else is coming to Babylon 7.0? A lot. We have a long list of features we want to implement for 7 and we can't wait for the community to try them out. From new WebXR features to revamped audio engine, improved device input system, new physics capabilities and of course community driven features that are already developed like the USDZ serializer and the greased line material. Babylon 7.0 will be another exciting release for us and for our community as well.

Babylon 7.0 is coming next year. Roughly around April. Before I'm done, I want to share some links to get you started with BabylonJS. First, our mono repo on github holds all of our code and issues for 7.0 and future releases, including all of our tools. Our main site, babylonjs.com can give you a wonderful overview of the engine and its capabilities. If you have any questions about Babylon, I can only recommend you to join our forum, forum.babylonjs.com and be a part of our super friendly community. The team is there daily to answer your questions as well. And the best way to get started with Babylon is our documentation page that has many tutorials and code examples integrated in the documentation page itself. And with that, I want to thank you so much for listening. I hope you enjoyed this presentation as much as I enjoyed making it and giving it. If you have any questions whatsoever, stay afterwards for the Q and A or contact me or the rest of the team on either our forum or Twitter.