Machine Learning in Node.js using Tensorflow.js

Hello, everyone. The topic of my lightning talk is machine learning in Node.js using TensorFlow.js.

Hi, I'm Shivailamba. I'm currently a TensorFlow.js SIG member and also a Google Codename Mentor at TensorFlow.

So, the first question that comes to everyone's mind is what exactly is TensorFlow.js? So, TensorFlow.js is an open-source library that enables easy integration of machine learning models directly in JavaScript. Hence, it reduces the need to learn a separate language like Python to host machine learning models. And it allows very easy integration of some kind of pre-trained models or writing your own models from scratch directly in JavaScript.

And as we know that JavaScript is a really versatile language, it can be run across different platforms like on the browser, on the mobile, or let's say on even IoT devices running Raspberry Pi. So, all of these different platforms now get easy access to machine learning based applications without any need of plugins, right? And machine learning that can be used in TensorFlow.js allows not just preexisting models, but it can also use a transfer learning to retrain some of the existing models based on your own data set. And you can actually also write machine learning models directly in JavaScript using JavaScript and without the need of any other language like Python.

Now this is the architecture of TensorFlow.js. There are three different APIs that are provided. So the top layer that you see are the TensorFlow.js pre trained models that are completely ready to use JavaScript classes. The next one is the layers API that enables to easily build and train models using high level building blocks similarly to how Keras has been built on top of TensorFlow. And finally we have the ops or the core API that helps to give you fine tuned control of model architecture or let's say for doing mathematical calculations like linear algebra. And this can be run either on the client side and on the server side. So on the client side we have basically the browser or let's say the mobile based applications that use the CPU or let's say the web accelerated graphics that is web GL or WebAssembly. And on the server side there is support for TensorFlow CPU that are run via the C bindings and we also have the support for TensorFlow GPU that help get acceleration with the help of CUDA that is also being run on Python based machine learning models.

And finally, you know, what are some of the main advantages of utilizing node.js to actually run TensorFlow.js. So the first one is, you know, we can write the machine learning models directly from scratch in JavaScript and we are only coding in one language. There's no need to use any other language to run and train and also test out these machine learning models. Now by using the TensorFlow.js on node.js we are also giving the ability to run much larger models that require a lot more power. For example, let's say if you're running some kind of neural networks that are having a lot of epochs and will require graphics based acceleration. So, with the help of the server hardware, like in Python, we can utilize the power of these servers to accelerate our larger models that cannot be done in the client side. And also we have the support for the NPM ecosystem because the NPM ecosystem is such a large like the node modules are so heavily used and are there. So, a wide variety of node models can be used directly with your TensorFlow.js based code as well to help assist with your TensorFlow.js application. And most importantly, we get much better performance because Node.js uses the just-in-time compiler and the models that are written in TensorFlow.js utilizing the Node.js can also get this performance boost. And that is quite evident with a few examples. For example, this chart shows you the comparison of TensorFlow running on Python-based models and also on the TensorFlow.js for a mobile net, which is a convolutional neural network model. And as you can see that the comparison for the time that it actually takes for the model to actually run is not a lot different.

So that means that TensorFlow.js itself is quite optimized for running very industrial standard models as well, and also for newer models at the same time. And in some cases, the TensorFlow.js model actually provides a lot more better performance as compared to the Python versions. For example, BERT, which is a state-of-the-art language model for natural language processing, we can see that over here, there is actually a two times better performance boost by actually using Node.js as compared to a Python-based model that is running this BERT classification.

Now, coming on to the most important part, that is, you know, how can we start utilizing the TensorFlow.js in Node.js. So we get namely three different packages that you can install, like the npm packages. So the first one is the TensorFlow CPU. The TensorFlow CPU is, you know, whenever we are importing this package, the module that we get is basically accelerated by the TensorFlow C binary and it runs on the CPU. And the TensorFlow on the CPU uses the hardware acceleration to, let's say, accelerate any kind of mathematical computations, for example, linear algebra.

Now, the extension to that is the TensorFlow GPU package. Like the CPU package, this particular GPU package will be accelerated via the TensorFlow C binary. But it also runs the tensor operations on the GPU with the help of CUDA, that heavily accelerates your performance. And this binding is definitely the fastest as compared to any other package that we have. And now finally, we also have the vanilla package. This package is very similar to the one that we run on the browser. And in this package, the operations are run in vanilla JavaScript and on the CPU. Now, as compared to the other modules, that is the TensorFlow CPU and the TensorFlow GPU, this does not have support for the TensorFlow binary, which actually makes the overall package very smaller. And because of this, since it does not rely on TensorFlow, it can also be actually used on a lot of other devices that support Node.js.

Now, one of the important considerations that need to be made is that the Node.js bindings run on the backend for TensorFlow.js that implements them synchronously. That means that whenever we are running the Node.js bindings on a production application like a web server, we should actually set up a drop queue or a server or like worker threads so that it does not block your main thread. Now, there are also support for APIs because once we have imported the package, all of the normal TensorFlow.js symbols that we use can be appear once we have imported the specific module. Like, for example, one of them is like the dfNode that contains Node.js specific APIs and TensorBoard is actually a notable example of the Node.js specific APIs.

Now, this is a sample code that should give you an example where we have defined like a model and we are training it by utilizing the tf.node.tensorboard. That helps in your model training. With that, that finishes off my presentation and I hope that you have liked it and you can connect with me on these social platforms and to ask any questions regarding TensorFlow.js. Again, do follow any kind of TensorFlow.js models using the hashtag madewithTensorFlow.js on Twitter and on LinkedIn. And again, feel free to connect with me for any queries regarding TensorFlow.js. I hope you have liked it.