Machine Learning in Node.js using Tensorflow.js

Rate this content
Bookmark

Write and deploy machine learning models easily in Nodejs using Tensorflow.js.

This talk has been presented at Node Congress 2021, check out the latest edition of this JavaScript Conference.

FAQ

TensorFlow.js is an open-source library that allows for easy integration of machine learning models directly in JavaScript. It enables the use of pre-trained models or the development of new models from scratch, without the need to learn another programming language like Python.

TensorFlow.js is versatile and can be run on various platforms such as browsers, mobile devices, and IoT devices like Raspberry Pi. This allows for the deployment of machine learning applications on these platforms without the need for plugins.

Tensor/Flow.js architecture includes three primary APIs: pre-trained models that are ready-to-use JavaScript classes, the Layers API for building and training models using high-level blocks similar to Keras, and the Ops or Core API for fine-tuned control of model architecture and mathematical operations.

Using TensorFlow.js on Node.js allows for the handling of larger models that require more computational power, leverages the performance boost from Node.js's just-in-time compiler, and integrates seamlessly with the extensive NPM ecosystem, enhancing overall performance and usability.

The TensorFlow.js GPU package accelerates tensor operations on the GPU using CUDA, significantly boosting performance compared to CPU-based executions. This is crucial for running large-scale models or models requiring intensive computational resources.

For Node.js, TensorFlow.js offers three main packages: TensorFlow CPU for CPU-accelerated operations, TensorFlow GPU for GPU-accelerated operations, and a vanilla package that runs operations in vanilla JavaScript on the CPU, suitable for environments without TensorFlow binaries.

TensorFlow.js can provide better performance for certain models, such as the BERT language model, where it has shown up to two times better performance on Node.js compared to Python-based implementations. This is due to optimized JavaScript execution and hardware acceleration.

When using Node.js bindings for TensorFlow.js in production, it's important to implement them synchronously to avoid blocking the main thread. Setting up a job queue or using worker threads can help manage operations efficiently in server environments.

Shivay Lamba
Shivay Lamba
8 min
24 Jun, 2021

Comments

Sign in or register to post your comment.
Video Summary and Transcription
The Talk introduces TensorFlow.js in Node.js for machine learning, highlighting its open-source nature and easy integration with JavaScript. It emphasizes the benefits of using Node.js, such as the ability to write machine learning models directly in JavaScript, access to the NPM ecosystem, and improved performance. The different packages available for utilizing TensorFlow.js in Node.js, including CPU, GPU, and vanilla packages, are discussed. The importance of setting up Node.js bindings to avoid blocking the main thread is mentioned, along with the availability of APIs like dfNode and TensorBoard.

1. Introduction to TensorFlow.js in Node.js

Short description:

Hello, everyone. The topic of my lightning talk is machine learning in Node.js using TensorFlow.js. TensorFlow.js is an open-source library that enables easy integration of machine learning models directly in JavaScript. It allows easy access to machine learning applications on different platforms without any need for plugins. The architecture of TensorFlow.js includes pre-trained models, a layers API for building and training models, and an ops or core API for fine-tuned control. Utilizing Node.js to run TensorFlow.js provides the ability to write machine learning models directly in JavaScript, run larger models, utilize the NPM ecosystem, and achieve better performance.

Hello, everyone. The topic of my lightning talk is machine learning in Node.js using TensorFlow.js.

Hi, I'm Shivailamba. I'm currently a TensorFlow.js SIG member and also a Google Codename Mentor at TensorFlow.

So, the first question that comes to everyone's mind is what exactly is TensorFlow.js? So, TensorFlow.js is an open-source library that enables easy integration of machine learning models directly in JavaScript. Hence, it reduces the need to learn a separate language like Python to host machine learning models. And it allows very easy integration of some kind of pre-trained models or writing your own models from scratch directly in JavaScript.

And as we know that JavaScript is a really versatile language, it can be run across different platforms like on the browser, on the mobile, or let's say on even IoT devices running Raspberry Pi. So, all of these different platforms now get easy access to machine learning based applications without any need of plugins, right? And machine learning that can be used in TensorFlow.js allows not just preexisting models, but it can also use a transfer learning to retrain some of the existing models based on your own data set. And you can actually also write machine learning models directly in JavaScript using JavaScript and without the need of any other language like Python.

Now this is the architecture of TensorFlow.js. There are three different APIs that are provided. So the top layer that you see are the TensorFlow.js pre trained models that are completely ready to use JavaScript classes. The next one is the layers API that enables to easily build and train models using high level building blocks similarly to how Keras has been built on top of TensorFlow. And finally we have the ops or the core API that helps to give you fine tuned control of model architecture or let's say for doing mathematical calculations like linear algebra. And this can be run either on the client side and on the server side. So on the client side we have basically the browser or let's say the mobile based applications that use the CPU or let's say the web accelerated graphics that is web GL or WebAssembly. And on the server side there is support for TensorFlow CPU that are run via the C bindings and we also have the support for TensorFlow GPU that help get acceleration with the help of CUDA that is also being run on Python based machine learning models.

And finally, you know, what are some of the main advantages of utilizing node.js to actually run TensorFlow.js. So the first one is, you know, we can write the machine learning models directly from scratch in JavaScript and we are only coding in one language. There's no need to use any other language to run and train and also test out these machine learning models. Now by using the TensorFlow.js on node.js we are also giving the ability to run much larger models that require a lot more power. For example, let's say if you're running some kind of neural networks that are having a lot of epochs and will require graphics based acceleration. So, with the help of the server hardware, like in Python, we can utilize the power of these servers to accelerate our larger models that cannot be done in the client side. And also we have the support for the NPM ecosystem because the NPM ecosystem is such a large like the node modules are so heavily used and are there. So, a wide variety of node models can be used directly with your TensorFlow.js based code as well to help assist with your TensorFlow.js application. And most importantly, we get much better performance because Node.js uses the just-in-time compiler and the models that are written in TensorFlow.js utilizing the Node.js can also get this performance boost. And that is quite evident with a few examples. For example, this chart shows you the comparison of TensorFlow running on Python-based models and also on the TensorFlow.js for a mobile net, which is a convolutional neural network model. And as you can see that the comparison for the time that it actually takes for the model to actually run is not a lot different.

2. Utilizing TensorFlow.js in Node.js

Short description:

TensorFlow.js provides better performance than Python versions in certain cases, such as BERT classification. TensorFlow.js in Node.js can be utilized through three different packages: TensorFlow CPU, TensorFlow GPU, and vanilla package. The CPU package accelerates mathematical computations, while the GPU package runs tensor operations on the GPU for even better performance. The vanilla package, which does not rely on TensorFlow, can be used on other devices that support Node.js. Node.js bindings for TensorFlow.js should be set up to avoid blocking the main thread. APIs, such as dfNode and TensorBoard, are available once the package is imported. Feel free to connect with me on social platforms for any queries regarding TensorFlow.js.

So that means that TensorFlow.js itself is quite optimized for running very industrial standard models as well, and also for newer models at the same time. And in some cases, the TensorFlow.js model actually provides a lot more better performance as compared to the Python versions. For example, BERT, which is a state-of-the-art language model for natural language processing, we can see that over here, there is actually a two times better performance boost by actually using Node.js as compared to a Python-based model that is running this BERT classification.

Now, coming on to the most important part, that is, you know, how can we start utilizing the TensorFlow.js in Node.js. So we get namely three different packages that you can install, like the npm packages. So the first one is the TensorFlow CPU. The TensorFlow CPU is, you know, whenever we are importing this package, the module that we get is basically accelerated by the TensorFlow C binary and it runs on the CPU. And the TensorFlow on the CPU uses the hardware acceleration to, let's say, accelerate any kind of mathematical computations, for example, linear algebra.

Now, the extension to that is the TensorFlow GPU package. Like the CPU package, this particular GPU package will be accelerated via the TensorFlow C binary. But it also runs the tensor operations on the GPU with the help of CUDA, that heavily accelerates your performance. And this binding is definitely the fastest as compared to any other package that we have. And now finally, we also have the vanilla package. This package is very similar to the one that we run on the browser. And in this package, the operations are run in vanilla JavaScript and on the CPU. Now, as compared to the other modules, that is the TensorFlow CPU and the TensorFlow GPU, this does not have support for the TensorFlow binary, which actually makes the overall package very smaller. And because of this, since it does not rely on TensorFlow, it can also be actually used on a lot of other devices that support Node.js.

Now, one of the important considerations that need to be made is that the Node.js bindings run on the backend for TensorFlow.js that implements them synchronously. That means that whenever we are running the Node.js bindings on a production application like a web server, we should actually set up a drop queue or a server or like worker threads so that it does not block your main thread. Now, there are also support for APIs because once we have imported the package, all of the normal TensorFlow.js symbols that we use can be appear once we have imported the specific module. Like, for example, one of them is like the dfNode that contains Node.js specific APIs and TensorBoard is actually a notable example of the Node.js specific APIs.

Now, this is a sample code that should give you an example where we have defined like a model and we are training it by utilizing the tf.node.tensorboard. That helps in your model training. With that, that finishes off my presentation and I hope that you have liked it and you can connect with me on these social platforms and to ask any questions regarding TensorFlow.js. Again, do follow any kind of TensorFlow.js models using the hashtag madewithTensorFlow.js on Twitter and on LinkedIn. And again, feel free to connect with me for any queries regarding TensorFlow.js. I hope you have liked it.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Node Congress 2022Node Congress 2022
26 min
It's a Jungle Out There: What's Really Going on Inside Your Node_Modules Folder
Top Content
The talk discusses the importance of supply chain security in the open source ecosystem, highlighting the risks of relying on open source code without proper code review. It explores the trend of supply chain attacks and the need for a new approach to detect and block malicious dependencies. The talk also introduces Socket, a tool that assesses the security of packages and provides automation and analysis to protect against malware and supply chain attacks. It emphasizes the need to prioritize security in software development and offers insights into potential solutions such as realms and Deno's command line flags.
Towards a Standard Library for JavaScript Runtimes
Node Congress 2022Node Congress 2022
34 min
Towards a Standard Library for JavaScript Runtimes
Top Content
There is a need for a standard library of APIs for JavaScript runtimes, as there are currently multiple ways to perform fundamental tasks like base64 encoding. JavaScript runtimes have historically lacked a standard library, causing friction and difficulty for developers. The idea of a small core has both benefits and drawbacks, with some runtimes abusing it to limit innovation. There is a misalignment between Node and web browsers in terms of functionality and API standards. The proposal is to involve browser developers in conversations about API standardization and to create a common standard library for JavaScript runtimes.
ESM Loaders: Enhancing Module Loading in Node.js
JSNation 2023JSNation 2023
22 min
ESM Loaders: Enhancing Module Loading in Node.js
ESM Loaders enhance module loading in Node.js by resolving URLs and reading files from the disk. Module loaders can override modules and change how they are found. Enhancing the loading phase involves loading directly from HTTP and loading TypeScript code without building it. The loader in the module URL handles URL resolution and uses fetch to fetch the source code. Loaders can be chained together to load from different sources, transform source code, and resolve URLs differently. The future of module loading enhancements is promising and simple to use.
Out of the Box Node.js Diagnostics
Node Congress 2022Node Congress 2022
34 min
Out of the Box Node.js Diagnostics
This talk covers various techniques for getting diagnostics information out of Node.js, including debugging with environment variables, handling warnings and deprecations, tracing uncaught exceptions and process exit, using the v8 inspector and dev tools, and generating diagnostic reports. The speaker also mentions areas for improvement in Node.js diagnostics and provides resources for learning and contributing. Additionally, the responsibilities of the Technical Steering Committee in the TS community are discussed.
TensorFlow.js 101: ML in the Browser and Beyond
ML conf EU 2020ML conf EU 2020
41 min
TensorFlow.js 101: ML in the Browser and Beyond
TensorFlow.js enables machine learning in the browser and beyond, with features like face mesh, body segmentation, and pose estimation. It offers JavaScript prototyping and transfer learning capabilities, as well as the ability to recognize custom objects using the Image Project feature. TensorFlow.js can be used with Cloud AutoML for training custom vision models and provides performance benefits in both JavaScript and Python development. It offers interactivity, reach, scale, and performance, and encourages community engagement and collaboration between the JavaScript and machine learning communities.
Node.js Compatibility in Deno
Node Congress 2022Node Congress 2022
34 min
Node.js Compatibility in Deno
Deno aims to provide Node.js compatibility to make migration smoother and easier. While Deno can run apps and libraries offered for Node.js, not all are supported yet. There are trade-offs to consider, such as incompatible APIs and a less ideal developer experience. Deno is working on improving compatibility and the transition process. Efforts include porting Node.js modules, exploring a superset approach, and transparent package installation from npm.

Workshops on related topic

Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
JSNation 2024JSNation 2024
108 min
Leveraging LLMs to Build Intuitive AI Experiences With JavaScript
Featured Workshop
Roy Derks
Shivay Lamba
2 authors
Today every developer is using LLMs in different forms and shapes, from ChatGPT to code assistants like GitHub CoPilot. Following this, lots of products have introduced embedded AI capabilities, and in this workshop we will make LLMs understandable for web developers. And we'll get into coding your own AI-driven application. No prior experience in working with LLMs or machine learning is needed. Instead, we'll use web technologies such as JavaScript, React which you already know and love while also learning about some new libraries like OpenAI, Transformers.js
Node.js Masterclass
Node Congress 2023Node Congress 2023
109 min
Node.js Masterclass
Top Content
Workshop
Matteo Collina
Matteo Collina
Have you ever struggled with designing and structuring your Node.js applications? Building applications that are well organised, testable and extendable is not always easy. It can often turn out to be a lot more complicated than you expect it to be. In this live event Matteo will show you how he builds Node.js applications from scratch. You’ll learn how he approaches application design, and the philosophies that he applies to create modular, maintainable and effective applications.

Level: intermediate
Build and Deploy a Backend With Fastify & Platformatic
JSNation 2023JSNation 2023
104 min
Build and Deploy a Backend With Fastify & Platformatic
WorkshopFree
Matteo Collina
Matteo Collina
Platformatic allows you to rapidly develop GraphQL and REST APIs with minimal effort. The best part is that it also allows you to unleash the full potential of Node.js and Fastify whenever you need to. You can fully customise a Platformatic application by writing your own additional features and plugins. In the workshop, we’ll cover both our Open Source modules and our Cloud offering:- Platformatic OSS (open-source software) — Tools and libraries for rapidly building robust applications with Node.js (https://oss.platformatic.dev/).- Platformatic Cloud (currently in beta) — Our hosting platform that includes features such as preview apps, built-in metrics and integration with your Git flow (https://platformatic.dev/). 
In this workshop you'll learn how to develop APIs with Fastify and deploy them to the Platformatic Cloud.
Building a Hyper Fast Web Server with Deno
JSNation Live 2021JSNation Live 2021
156 min
Building a Hyper Fast Web Server with Deno
WorkshopFree
Matt Landers
Will Johnston
2 authors
Deno 1.9 introduced a new web server API that takes advantage of Hyper, a fast and correct HTTP implementation for Rust. Using this API instead of the std/http implementation increases performance and provides support for HTTP2. In this workshop, learn how to create a web server utilizing Hyper under the hood and boost the performance for your web apps.
0 to Auth in an Hour Using NodeJS SDK
Node Congress 2023Node Congress 2023
63 min
0 to Auth in an Hour Using NodeJS SDK
WorkshopFree
Asaf Shen
Asaf Shen
Passwordless authentication may seem complex, but it is simple to add it to any app using the right tool.
We will enhance a full-stack JS application (Node.JS backend + React frontend) to authenticate users with OAuth (social login) and One Time Passwords (email), including:- User authentication - Managing user interactions, returning session / refresh JWTs- Session management and validation - Storing the session for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Table of contents- A quick intro to core authentication concepts- Coding- Why passwordless matters
Prerequisites- IDE for your choice- Node 18 or higher
GraphQL - From Zero to Hero in 3 hours
React Summit 2022React Summit 2022
164 min
GraphQL - From Zero to Hero in 3 hours
Workshop
Pawel Sawicki
Pawel Sawicki
How to build a fullstack GraphQL application (Postgres + NestJs + React) in the shortest time possible.
All beginnings are hard. Even harder than choosing the technology is often developing a suitable architecture. Especially when it comes to GraphQL.
In this workshop, you will get a variety of best practices that you would normally have to work through over a number of projects - all in just three hours.
If you've always wanted to participate in a hackathon to get something up and running in the shortest amount of time - then take an active part in this workshop, and participate in the thought processes of the trainer.