A Different Kind of Serverless: A Case Study for SQLite and Whisper.cpp

Rate this content
Bookmark

You can build applications with a local-first focus. In this case study, we dig into how you might use WASM to run Whisper.cpp for Speech-to-text and also use WASM to host a local SQLite database. The only reason we need a network connection at all for this application is to get the initial JS payload and to download the Whisper models from a remote store since they can get quite large. 

This talk has been presented at JSNation US 2024, check out the latest edition of this JavaScript Conference.

FAQ

Wasm, or WebAssembly, is an open standard designed to support any language on any operating system. It enables running compiled code from languages like Rust, C++, or Go in the browser, offering major portability.

Popular applications using WebAssembly include Figma, CapCut, 1Password, Google Earth, uBlock Origin, and Leachess. These applications leverage Wasm for improved performance and capabilities in the browser.

WebAssembly allows browser-based applications to run more performant code by compiling languages such as C++ or Rust to Wasm, providing real data structures and memory access, which can be faster and more efficient than JavaScript.

Limitations of using WebAssembly in the browser include a lack of direct DOM access, higher potential RAM usage, and the need to run in a WebWorker. Outside the browser, system access may be limited without a WASI runtime.

WASI, or the WebAssembly System Interface, is a standard designed to provide a POSIX-like feature set to WebAssembly, enabling functionalities like file I/O and networking within Wasm code.

Common languages used with WebAssembly include C, C++, Rust, and Go. Other languages with varying support include C#, Zig, Lua, and more.

WebAssembly can be run outside of the browser using tools like Xtism, WasmTime, Spin SDK, and Wasmer, which provide server-side runtimes and environments for executing Wasm code.

Security concerns with WebAssembly include its binary format, which can be obfuscated and harder to analyze than JavaScript, and shared array buffers, which require secure contexts to mitigate vulnerabilities like Spectre Meltdown.

Orderly is an application built for voice dictation of books and blog posts, using WebAssembly combined with technologies like SQLite in the browser and whisper.cpp for transcription, allowing it to run without a server.

Hosting considerations for WebAssembly applications include the need for proper header configuration to support shared array buffers and OPFS. Services like Render, Netlify, or VPS can be used for hosting models and applications.

Chris Griffing
Chris Griffing
25 min
21 Nov, 2024

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Today's Talk introduces WebAssembly (Wasm) and its versatility in supporting any language on any operating system. Wasm's history dates back to 2011 with Emscripten and NACL, and it has gained importance with the introduction of Wasi, the WebAssembly system interface. Wasm is supported by modern browsers and can be used with languages like C, C++, Rust, and Go. Popular applications like Figma, CapCut, and 1Password utilize Wasm for improved performance. Web Workers and shared array buffers eliminate the need for object serialization. The Talk also discusses the use of Keasley and Drizzle for interacting with SQL. Building and optimizing the application involves considerations of migrations, app readiness, suspense, optimistic UI, and debouncing inputs. Whisper, an open-source AI project, offers models for audio-to-text conversion and is implemented using whisper.cpp. Deployment options include Render, Netlify, Vercel, and Cloudflare, but using a cheap VPS provider with a file server can be a cost-effective alternative.
Video transcription and chapters available for users with access.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Utilising Rust from Vue with WebAssembly
Vue.js London Live 2021Vue.js London Live 2021
8 min
Utilising Rust from Vue with WebAssembly
Top Content
In this Talk, the speaker demonstrates how to use Rust with WebAssembly in a Vue.js project. They explain that WebAssembly is a binary format that allows for high-performance code and less memory usage in the browser. The speaker shows how to build a Rust example using the WasmPack tool and integrate it into a Vue template. They also demonstrate how to call Rust code from a Vue component and deploy the resulting package to npm for easy sharing and consumption.
A Framework for Managing Technical Debt
TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Top Content
Today's Talk discusses the importance of managing technical debt through refactoring practices, prioritization, and planning. Successful refactoring requires establishing guidelines, maintaining an inventory, and implementing a process. Celebrating success and ensuring resilience are key to building a strong refactoring culture. Visibility, support, and transparent communication are crucial for addressing technical debt effectively. The team's responsibilities, operating style, and availability should be transparent to product managers.
Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
Watch video: Debugging JS
Debugging JavaScript is a crucial skill that is often overlooked in the industry. It is important to understand the problem, reproduce the issue, and identify the root cause. Having a variety of debugging tools and techniques, such as console methods and graphical debuggers, is beneficial. Replay is a time-traveling debugger for JavaScript that allows users to record and inspect bugs. It works with Redux, plain React, and even minified code with the help of source maps.
Making JavaScript on WebAssembly Fast
JSNation Live 2021JSNation Live 2021
29 min
Making JavaScript on WebAssembly Fast
Top Content
WebAssembly enables optimizing JavaScript performance for different environments by deploying the JavaScript engine as a portable WebAssembly module. By making JavaScript on WebAssembly fast, instances can be created for each request, reducing latency and security risks. Initialization and runtime phases can be improved with tools like Wiser and snapshotting, resulting in faster startup times. Optimizing JavaScript performance in WebAssembly can be achieved through techniques like ahead-of-time compilation and inline caching. WebAssembly usage is growing outside the web, offering benefits like isolation and portability. Build sizes and snapshotting in WebAssembly depend on the application, and more information can be found on the Mozilla Hacks website and Bike Reliance site.
Building a Voice-Enabled AI Assistant With Javascript
JSNation 2023JSNation 2023
21 min
Building a Voice-Enabled AI Assistant With Javascript
Top Content
This Talk discusses building a voice-activated AI assistant using web APIs and JavaScript. It covers using the Web Speech API for speech recognition and the speech synthesis API for text to speech. The speaker demonstrates how to communicate with the Open AI API and handle the response. The Talk also explores enabling speech recognition and addressing the user. The speaker concludes by mentioning the possibility of creating a product out of the project and using Tauri for native desktop-like experiences.
A Practical Guide for Migrating to Server Components
React Advanced 2023React Advanced 2023
28 min
A Practical Guide for Migrating to Server Components
Top Content
Watch video: A Practical Guide for Migrating to Server Components
React query version five is live and we'll be discussing the migration process to server components using Next.js and React Query. The process involves planning, preparing, and setting up server components, migrating pages, adding layouts, and moving components to the server. We'll also explore the benefits of server components such as reducing JavaScript shipping, enabling powerful caching, and leveraging the features of the app router. Additionally, we'll cover topics like handling authentication, rendering in server components, and the impact on server load and costs.

Workshops on related topic

Build Modern Applications Using GraphQL and Javascript
Node Congress 2024Node Congress 2024
152 min
Build Modern Applications Using GraphQL and Javascript
Featured Workshop
Emanuel Scirlet
Miguel Henriques
2 authors
Come and learn how you can supercharge your modern and secure applications using GraphQL and Javascript. In this workshop we will build a GraphQL API and we will demonstrate the benefits of the query language for APIs and what use cases that are fit for it. Basic Javascript knowledge required.
How to Solve Real-World Problems with Remix
Remix Conf Europe 2022Remix Conf Europe 2022
195 min
How to Solve Real-World Problems with Remix
Featured Workshop
Michael Carter
Michael Carter
- Errors? How to render and log your server and client errorsa - When to return errors vs throwb - Setup logging service like Sentry, LogRocket, and Bugsnag- Forms? How to validate and handle multi-page formsa - Use zod to validate form data in your actionb - Step through multi-page forms without losing data- Stuck? How to patch bugs or missing features in Remix so you can move ona - Use patch-package to quickly fix your Remix installb - Show tool for managing multiple patches and cherry-pick open PRs- Users? How to handle multi-tenant apps with Prismaa - Determine tenant by host or by userb - Multiple database or single database/multiple schemasc - Ensures tenant data always separate from others
Relational Database Modeling for GraphQL
GraphQL Galaxy 2020GraphQL Galaxy 2020
106 min
Relational Database Modeling for GraphQL
Top Content
WorkshopFree
Adron Hall
Adron Hall
In this workshop we'll dig deeper into data modeling. We'll start with a discussion about various database types and how they map to GraphQL. Once that groundwork is laid out, the focus will shift to specific types of databases and how to build data models that work best for GraphQL within various scenarios.
Table of contentsPart 1 - Hour 1      a. Relational Database Data Modeling      b. Comparing Relational and NoSQL Databases      c. GraphQL with the Database in mindPart 2 - Hour 2      a. Designing Relational Data Models      b. Relationship, Building MultijoinsTables      c. GraphQL & Relational Data Modeling Query Complexities
Prerequisites      a. Data modeling tool. The trainer will be using dbdiagram      b. Postgres, albeit no need to install this locally, as I'll be using a Postgres Dicker image, from Docker Hub for all examples      c. Hasura
Building a Shopify App with React & Node
React Summit Remote Edition 2021React Summit Remote Edition 2021
87 min
Building a Shopify App with React & Node
Top Content
WorkshopFree
Jennifer Gray
Hanna Chen
2 authors
Shopify merchants have a diverse set of needs, and developers have a unique opportunity to meet those needs building apps. Building an app can be tough work but Shopify has created a set of tools and resources to help you build out a seamless app experience as quickly as possible. Get hands on experience building an embedded Shopify app using the Shopify App CLI, Polaris and Shopify App Bridge.We’ll show you how to create an app that accesses information from a development store and can run in your local environment.
Build a chat room with Appwrite and React
JSNation 2022JSNation 2022
41 min
Build a chat room with Appwrite and React
WorkshopFree
Wess Cope
Wess Cope
API's/Backends are difficult and we need websockets. You will be using VS Code as your editor, Parcel.js, Chakra-ui, React, React Icons, and Appwrite. By the end of this workshop, you will have the knowledge to build a real-time app using Appwrite and zero API development. Follow along and you'll have an awesome chat app to show off!
Hard GraphQL Problems at Shopify
GraphQL Galaxy 2021GraphQL Galaxy 2021
164 min
Hard GraphQL Problems at Shopify
WorkshopFree
Rebecca Friedman
Jonathan Baker
Alex Ackerman
Théo Ben Hassen
 Greg MacWilliam
5 authors
At Shopify scale, we solve some pretty hard problems. In this workshop, five different speakers will outline some of the challenges we’ve faced, and how we’ve overcome them.

Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.