JS Nation

JSNation 2025

Joshua introduces Transformers JS and Hugging Face, emphasizing community collaboration and pre-trained models. Transformers JS evolution led to 1.4 million monthly users, supporting 155 architectures. The library's browser-based capabilities offer real-time processing, cost-efficiency, and scalability. Integration enhancements include native web GPU execution and React Native implementation. Web ML implementation focuses on Onyx Runtime for device execution and web GPU for resource optimization. Browser-based ML applications cover vision, speech recognition, and text-to-speech. Advanced implementations include multimodal applications and educational tools. Interactive AI demonstrations showcase semantic search and conversational AI scenarios. Model licensing transitions to ECMAScript for efficiency and model redownloading factors are discussed.

Learn about Transformers.js, an innovative JavaScript library for running state-of-the-art machine learning models 100% locally in your browser. With support for nearly 2 000 pretrained models, spanning a wide range of modalities and tasks, Transformers.js empowers developers to seamlessly integrate advanced ML capabilities into their web applications. We'll also explore how we leverage emerging web technologies like WebGPU and WebNN to create interactive, privacy-preserving, and scalable web experiences.

Transformers.js: State-of-the-Art Machine Learning for the Web

 Joshua Lochner

JSNation 2024

The 3JS project has evolved into a community-driven effort with numerous contributors over the past 14 years. It started with 3D engine work in Flash and transitioned to using SVGs for rendering in HTML5 before adopting WebGL. The project showcases various projects and frameworks, including a no-code tool powered by 3.js. The team is working on a new render using WebGPU and developing a new shader language called TSL. The hope is that WebGPU will eventually replace WebGL, offering better control and performance.

In the rapidly evolving landscape of web technologies, the introduction of WebGPU and WebXR represents a significant leap forward, promising to redefine the boundaries of what's possible in 3D web experiences. This talk dives into the heart of these new technologies, guided by the Three.js library. We begin by exploring WebGPU, a next-generation graphics API offering enhanced performance and efficiency for rendering 3D graphics directly in the browser. We'll demonstrate how Three.js is adapting to harness its full potential, unlocking unprecedented opportunities for developers to create visually stunning interactive experiences. Transitioning to the immersive realm, we delve into WebXR, a technology that opens the door for virtual reality and augmented reality experiences right from the web. We'll showcase how Three.js enables creators to build immersive experiences.

Embracing WebGPU and WebXR With Three.js

Mr.doob

Speaker introduces a side project involving JavaScript and iconic car recreation at JS Nation. Nico, a frontend developer, explains the concept of KIT from Knight Rider and its human-like features, emphasizing the use of machine learning in building it. Using Transformers.js involves defining tasks and models, with options like Whisper for transcribing speech to text and Kokoro.js for text to audio conversion. Adding intelligence through LLMs enhances the capabilities of the pipeline. Models small enough to run on device, allowing browser usage using LLM library. Creating engines and new conversations, streaming replies with added tokens for efficiency. Implementing Kit reasoning process for problem-solving and fun in learning and building using LLMs.

In the TV series Knight Rider, the concept of an intelligent car was pure sci-fi. Over 40 years later, we now have the tools to bring it to life - purely in the browser. This talk explores combining AI techniques like RAG and function calling with advanced browser APIs (e.g., WebBluetooth) to create a browser-based AI agent with personality, memory, and autonomy, powered by WebAssembly and WebGPU - fully offline and privacy-preserving!

Let’s Build K.I.T.T. With JavaScript

Nico Martin

The Talk delves into the integration of generative AI tools in web development, emphasizing the AI revolution's impact. It explores creating a browser-based translation application without backend servers, emphasizing speech-to-text and translation APIs. The discussion highlights browser-based speech capabilities, different voices, and the challenges of translation within the browser. The exploration of the prompt API, Gemini Nano, and specialized APIs showcases experimental features and language model capabilities. The advancements in browser-based AI, privacy-focused AI usage on Chrome, and the utilization of Langflow for server-side generative AI experimentation are also discussed. Progressive enhancement, mobile integration, real-time translation, privacy concerns, and model integration into browsers are key topics.

AI progress is running away on the server-side, where the GPUs live, but the web platform is never that far behind. In this talk we're going to explore the incredible applications you can build in the browser today and in the near future.You'll see how to build an offline-capable, speech to speech language translation app, a babelfish, using nothing but the web platform. Through this app we'll explore the different ways we can run AI models in the browser today and what may be built into the browser in the future. From low-level standards to high-level experiments, you'll learn how to work with AI on the web.

Practical Web AI: Built-In, Browser Based, Brilliant

Phil Nash

JS GameDev Summit

JS GameDev Summit 2023

This talk explores the differences between WebGL and WebGPU, with a focus on transitioning from WebGL to WebGPU. It discusses the initialization process and shader programs in both APIs, as well as the creation of pipelines in WebGPU. The comparison of uniforms highlights the use of uniform buffers for improved performance. The talk also covers the differences in conventions between WebGL and WebGPU, including textures, viewport and clip spaces. Lastly, it mentions the differences in depth range and projection matrix between the two APIs.

In this presentation, I'll explore the transition from WebGL to WebGPU, demonstrating how these changes affect game development. The talk will include practical examples and code snippets to illustrate key differences and their implications on performance and efficiency.

Migration from WebGL to WebGPU

Dmitrii Ivashchenko

JSNation 2023

With AI and web GPU, it's an exciting time to be a developer. The speaker's journey involves combining programming and design, leading to the creation of Pure Blue, a powerful programming environment. Adding AI to the mix, the speaker discusses the potential of AI in the creative process and its impact on app development. The talk explores the role of React components and WebGPU in enabling fine-grained editing and running AI models locally. The future of app development is discussed, emphasizing the need to stay ahead of the curve and leverage the power of JavaScript.

React + WebGPU + AI – What Could Go Wrong? 😳

Aria Minaei

JSNation 2022

Welcome to MakePad, a new way to build UI for web and native using WebAssembly and Rust. JavaScript is not suitable for complex applications like IDEs and design tools. Rust, a new programming language, was used to reimagine MakePad, resulting in a fast and efficient platform. MakePad offers live editing, high CPU performance, and the ability to load native instrument components. The future of MakePad includes an open-source release, a design tool, and support for importing 3D models.

In this talk I will show Makepad, a new UI stack that uses Rust, Wasm, and WebGL. Unlike other UI stacks, which use a hybrid approach, all rendering in Makepad takes place on the GPU. This allows for highly polished and visually impressive applications that have not been possible on the web so far. Because Makepad uses Rust, applications run both natively and on the Web via wasm. Makepad applications can be very small, on the order of just a few hundred kilobytes for wasm, to a few megabytes with native. Our goal is to develop Makepad into the UI stack of choice for lightweight and performant cross-platform applications. We intend to ship with our own design application and IDE.

Makepad - Leveraging Rust + Wasm + WebGL to Build Amazing Cross-platform Applications

Rik Arends

JS GameDev Summit 2022

Alex Saint-Louis, co-founder of Wunder Interactive, shares the mission of bringing Unreal Engine to the browser, enabling cross-platform 3D applications and games. They are working on a WebGPU back end for Unreal Engine to push the limits of 3D on the web. Wunder Interactive has improved compression, built their own asset file system, and offers powerful tools for game developers. They are utilizing modern web technologies like WebAssembly, WebGL, and WebGPU, and plan to support other engines like Unity and Godot. The team aims to transform the industry by bringing console-quality games to the browser and providing an alternative distribution path. They are excited to bring Unreal Engine 5 to the web with WebGPU support and are working on WebXR support for cross-platform 3D experiences, including VR.

Traditionally, browser games haven't been taken seriously. If you want to target the web, that traditionally has meant compromising on your vision as a game developer. Our team at Wonder Interactive is on a mission to change that, bringing one of the world's premiere native game engines to the browser - Unreal Engine. In our talk, we'll dive into our efforts porting the engine to the browser and carrying on the pioneering unfinished work started at Epic Games nearly a decade ago in collaboration with Mozilla. We'll dive into what this means for the future of games in the browser, and the open metaverse on the web.

Unreal Engine in WebAssembly/WebGPU

Alex St. Louis

Unity targets over 25 platforms and technologies, including desktop, mobile, and virtual reality. They use Emscripten to compile the engine and game logic into WebAssembly for web development. Unity can be extended with plugins to access browser features like WebXR's augmented reality mode. The speaker demonstrates intercepting Unity's calls to the browser to modify its behavior. Unity is actively working on mobile support for web export and improving documentation for extending Unity with web plugins.

Unity can build games to run in a web browser using tools like Emscripten, Web Assembly, and WebGL. It provides integration with the browser, using browser APIs to simulate native APIs. Sometimes it is useful to interface with the browser in ways that Unity does not natively provide support for. In this talk, I will discuss how Unity builds games for the web, and how to extend Unity using Javascript to enable support for features not otherwise provided. <a href="https://www.froala.com/wysiwyg-editor?pb=1"> </a>

Extending Unity WebGL With Javascript

Brendan Duncan

WebGPU is an upcoming web API that provides low-level access to the GPU, offering better performance and enabling new rendering techniques. It allows for the use of compute shaders, which provide more control over memory synchronization and threading, and can be used for general computation on a GPU. WebGPU opens up possibilities for unique effects in games and promises future support for ray tracing in browsers.

One of the major new features WebGPU introduces is compute shaders. I'll cover the basics of how they work and go through some applications that weren't possible before on the web. Examples include Unreal Engine 5's compute-based rendering techniques, and complex particle simulations. <a href="https://www.froala.com/wysiwyg-editor?pb=1"> </a>

#webgpu