Today’s generation of web AI is largely confined to server roundtrips: users type into a prompt box, wait seconds for a cloud API response, and pay per-token fees while streaming sensitive private data to remote servers.
But the next frontier of web development is rich, sensory, and on-device. Modern web applications can now perceive the user’s physical environment through the camera and microphone, process visual and spatial feeds at 60 frames per second, and fuse that perception with on-device language reasoning—all executing locally on the client's GPU with zero cloud infrastructure.
In this fast-paced 20-minute talk, we showcase what the future of web applications looks like with High-Performance, Multimodal Web AI, powered by Google's LiteRT (@litertjs/core and @litert-lm/core):
Perception at 60 FPS: How WebGPU compute pipelines allow web apps to run real-time monocular depth estimation, multi-class segmentation, and 4x neural super-resolution directly on live video and canvas feeds.
On-Device Cognitive Reasoning: How running local language models (like Gemma 4) in browser VRAM enables instant comprehension, summarization, and conversation without network latency.
The Multimodal Fusion Loop: How perception models pass structured visual and spatial features directly to on-device reasoning models, enabling web apps to understand what users see and point at in real time.
Actionable In-Browser Agents: How local models can invoke JavaScript functions and browser APIs to manipulate the DOM, edit media, or interact with local storage automatically.
Through live interactive demonstrations—including real-time spatial vision copilots and in-browser media editing studios—attendees will discover how to transition from building passive web pages to creating responsive, intelligent multimodal web experiences.
This talk has been presented at JSNation US 2026, check out the latest edition of this JavaScript Conference.
