Pushing the Limits of Video Encoding in Browsers With WebCodecs

Rate this content
Bookmark
Project website

High quality video encoding in browsers have traditionally been slow, low-quality and did not allow much customisation. This is because browsers never had a native way to encode videos leveraging hardware acceleration. In this talk, I’ll be going over the secrets of creating high-quality videos in-browsers efficiently with the power of WebCodecs and WebAssembly. From video containers to muxing, audio and beyond, this talk will give you everything you need to render your videos in browsers today!

This talk has been presented at JSNation 2023, check out the latest edition of this JavaScript Conference.

FAQ

Slantit is a web-based tool developed by Akash Hameedwasia that allows users to create catchy product videos quickly. It uses the same video encoding process discussed in Akash's talk.

FFmpeg is used as a muxer to combine the encoded video stream into a final video container format. It allows for the inclusion of audio and the creation of a complete video file ready for download.

The main topic of Akash Hameedwasia's talk at JS Nation 2023 is pushing the limits of video encoding with web codecs.

Akash Hameedwasia is a software engineer at Razorpay who loves building web products and contributing to open-source projects. He is known for projects like Blaze, Untabbed, Dyod, and Slanted.

The purpose of the video editing tool Akash was building was to create catchy product videos quickly on the browser, avoiding the need to download large desktop-based video editing software.

Some methods to render video frames on the web include CSS animations, modifying the DOM directly using JavaScript, using SVG, and using the HTML Canvas API.

The HTML Canvas API is a feature in HTML5 that allows for drawing and manipulating graphics on the web. Akash recommends it because it offers flexibility and various capabilities that other rendering methods do not.

The Capture Stream API is part of the MediaStream Recording API standard and allows for video capture from HTML5 canvas elements.

The limitations of the Capture Stream API for video encoding include no constant frame rate, no option to change the video file format (always produces WebM format), low reliability, and output quality that can vary depending on device performance.

WebCodecs is a set of APIs that provide low-level access to individual frames of a video stream, allowing for asynchronous and hardware-accelerated video encoding and decoding. It supports both video and audio and is crucial for high-quality, efficient video encoding.

Akash Hamirwasia
Akash Hamirwasia
25 min
05 Jun, 2023

Comments

Sign in or register to post your comment.

Video Summary and Transcription

This Talk explores the challenges and solutions in video encoding with web codecs. It discusses drawing and recording video on the web, capturing and encoding video frames, and introduces the WebCodecs API. The Talk also covers configuring the video encoder, understanding codecs and containers, and the video encoding process with muxing using ffmpeg. The speaker shares their experience in building a video editing tool on the browser and showcases Slantit, a tool for making product videos.

1. Introduction to Video Encoding with Web Codecs

Short description:

Welcome to my talk on pushing the limits of video encoding with web codecs. My name is Akash Hameedwasia. I write code at Razorpay. I also love building products on the web and contributing to open-source. You might know me from some of my projects mentioned here, Blaze, Untabbed, Dyod, or Slanted.

I hope you're all having a great time at JS Nation 2023. Welcome to my talk on pushing the limits of video encoding with web codecs. My name is Akash Hameedwasia. I write code at Razorpay. I also love building products on the web and contributing to open-source. You might know me from some of my projects mentioned here, Blaze, Untabbed, Dyod, or Slanted. If you don't know about these projects, I strongly suggest you check them out. You can also connect with me on my Twitter, GitHub, or also check out my past talks, past blogs, and also various projects on my personal website, AkashHameedwasia.com.

2. Building a Video Editing Tool on the Browser

Short description:

In this talk, I will share the learnings I had while building a video editing tool on the browser. I encountered challenges with video encoding, audio, muxing, and codecs, but eventually figured everything out.

So before we get started on our journey of understanding how to use web codecs, here's a little back story. A while back, I was building a video editing tool on the browser that would allow me to make really catchy product videos really quickly. There are various video editing tools out there, but they are desktop based, so you have to download them and they're quite large. And then you have to learn how to use them, and then they take a lot of time for exporting a video. I thought, why not, let's make a simple tool. Because I have a lot of video editing needs for all my projects. Let's build a simple video editor on the browser so that I can use them to export videos really Now, everything was going fine until I started writing code for the export process. I got stuck at a bunch of places. Video encoding, audio, browse, muxing, codecs, a lot of jargon, a lot of places to get stuck. But I was eventually able to figure everything out, and this is what I want to share with you in this talk, all the learnings I had of how to go about building something like this.