GPU Accelerating Node.js Web Services and Visualization with RAPIDS

Hi, and welcome to GPU Accelerating Node.js Web Services and Visualization with Rapids. I'm Allan Ane-Mark, and I am the lead in the Rapids Viz team here at NVIDIA.

So, Rapids is an open-source GPU-accelerated data science platform, and then you can find more details at rapids.ai and nvidia.com, and Node Rapids, which is the project I'm going to be talking about, is an open-source modular library of Rapids-inclusive bindings in Node.js, as well as some other complementary methods for supporting high performance browser-like visualizations. It's currently in technical preview, but you can find more details about it at github.com slash rapids.ai slash Node.

And really, our main goal in this framework is creating something that can accelerate data science and visualization pipelines fully in JavaScript and TypeScript, which is something that is traditionally done mostly in, say, Python. And our second goal is bringing GPU acceleration to a wider variety of NodeJS and JS utilities, since we feel like the general community is sort of getting not as much access to these high-performance tools as we'd like.

So what do you get with Rapids, which is traditionally Python and C++? You get these data science libraries, such as DataFrame Operations in CUDF, you get CUML, which is a lot of GPU accelerated machine learning algorithms, Cougraph, Forgraph stuff, Spatial, Signal, all the like, and more being developed continuously, and these are continuously getting improved. The caveat being these are mainly around Linux-based systems, so if you want to use Windows with them, you can. It has to be through WSL 2, however.

So what kind of libraries in the Viz ecosystem traditionally happen in Python? So we have our own Cougraph filter, which is a notebook-based cross-filtering tool where you can create these dashboards very quickly in a few lines of Python code, and then very quickly interact with hundreds of millions of rows of data in a pretty customizable way. And we make extensive use of one of the other great Viz libraries out there called DataShader, which is great at server-side rendering hundreds of millions of points. All this is GPU-accelerated. And really part of this great ecosystem of Viz and Analytics tools, which sort of lie on the spectrum between your back-end C, C++, Python that sort of transitions into just front-side JS. And really when it comes to the Data Science and Compute, then mainly analytic stuff, more on the performance side, it all sort of starts with the Python C++ side. And it sort of is like then translated into JavaScript for like interface stuff. You have some that are a little bit more in an intermediary, but really it sort of starts there and then ends up in JavaScript, or just stays in JavaScript. What we're sort of proposing is the inverse. We're going to start with the JS libraries and then bring them back to this more high-performing back-end in Node.js. So give them access to CUDA, and CUDF, Cugraph, all those sort of things.

So our experience with this is sort of started a while ago when we were making lots of demos for RAPIDS, and in this case we were making a great mortgage visualization where you have DECGL and React, and it all was kind of a very fast nice interface. It fits all kinds of different screens and all that, but the backend was a mess. We had multiple languages, multiple servers. It sort of became unsustainable, and we just basically each gave up, said, oh, well, let's just do it in Python and Notebooks. But deep down we were really kind of sad because there's all these great JS Viz libraries and the kind of custom abilities you get using them that we sort of were lacking, and it's a shame because now you kind of have this two continental divides, right? You have Python and C++ and you have JavaScript and TypeScript and this chasm between them where you're sort of separating the capabilities between them. So on one side you get direct access to hardware, you know, most of the HPC, high-performance computing, data science and compute libraries are in this space. Not the best usability because it's like a high learning curve, but this is the place to go for that high-performance stuff. On the other side you kind of have the JavaScript-TypeScript where you have your nice browser environment which is great for shareability and accessibility and compatibility, in my opinion a little bit more refined visualization and interface libraries, but again you don't get that performance because you're sort of bounded by the browser sandbox. So it's kind of a shame because you have, you know, data scientists and engineers and folks in front of those and they're kind of all siloed in their side but they can mutually benefit from each other's tooling and experience. So, hence Node Rapids, where we're hoping to give the Node.js dev community the streamlined API to a high-performance data science platform, Rapids, without the need to learn a new language or environment. So you can then leverage Rapids and Node.js features, you can accelerate the catalog of great JS vis libraries already out there without major refactoring, you can learn locally or through cloud instances and it's sort of well suited for accelerated vis apps, Node service apps, and again you help enable these two communities to more closely work together and vice versa. So, that's sort of the high ideals and what's the actual meat and bones of this thing? Well, here it is, Node Rapids. It's a very modular kind of library. So, it's a buffet style. You kind of pick and choose what you need for your use case. It's kind of organized in these main categories. The main being the memory management that gives you access to CUDA, so GPU memory.

We also have a kind of really nice SQL engine that we bind to, which enables us to do multi-node and multi-GPU stuff when needed. Then there's a whole data science wing, all in Rapids, so you have your CUDF and CUGRAS stuff. And then this is the graphics column here, where we're sort of taking advantage of the fact that WebGL is a subset of OpenGL. So, really what we're doing with these bindings is, you know, you can take your WebGL code and you can then run it in OpenGL and get the benefit of that performance increase from OpenGL. We're also doing things with GLF bindings and, you know, node processes. But again, you can now use your LumaGL, DeckGL, SigmaJS, we're hoping to get two and three Jets and basically run them in OpenGL without much effort.

So, another component to this is since you're already on GPU, you get the benefit of GPU video encoding. So, by taking advantage of that and using WebRTC, you can do server-side rendering, stream that over to the browser and have the browser-side JS interact with it like a video tag and it's lightweight in that sense. It enables a lot more capabilities in that sense. Or you can just do stuff like, you know, interacting with all this stuff in a notebook.

So, what do we mean by kind of architecture? GPU hardware is a little bit different. You know, it's pretty straightforward when you have a single GPU and just doing client-side rendering. So, all the compute happens on the GPU, you send those computer values over, and the client JS renders it, those few values. You know, pretty straightforward and sort of where GPUs excel. It's excelled so well that you can have multiple users accessing the same kind of GPU data and it's fast enough to handle that. Or if you have particularly large data, NVIDIA has an NVIDIA NVLink. And so you can kind of link multiple GPUs together and kind of get expanded memory and compute power that way. Or you can go in a more kind of traditional cloud architecture, where you just sort of have multiple GPUs all separated, you know, you have lots of child processes running on each GPU, Load Balancer running across all of those. And so lots of, you know, instances of people accessing it. And basically whatever GPU is free at that moment is the one that is serving up the information to that user. So a pretty straightforward, you know, again, you still tend to need a lot of GPUs, but, you know, not something terribly unfamiliar.

So now, taking advantage of the server-side rendering and streaming component, if you have some pretty heavy workloads, then you can then separate out those tasks for each GPU. So one can do solely doing the compute stuff, and one can do the rendering and encoding part to the client-side video. And again, it can do this well enough that you can have multiple people accessing the same system and same data. The caveat here being that for consumer-grade GPUs, I think they're limited to three NV encoding streams at once, so keep that in mind. So something that is kind of interesting and new and a little bit more on the wishful thinking but possible side is this basically multi-GPU, server-side running, multi-user, massive system, where in this case it would be like a DGX, which has 8 GPUs and all and all that, and you can divvy up the compute and rendering between them, you have tons of GPU memory, and basically you have this mini supercomputer that you can then access and control with off-the-shelf JavaScript, so it's kind of wild. So, you know, you normally think you need some sort of HPC type software and all that and all the overhead that involves learning that stuff, but really there's nothing stopping you from leveraging a system this powerful with your run-of-the-mill JavaScript, which we think is pretty cool. But anyways, now for the good part. Examples and demos. So for this first one we're going to keep it simple.

We're basically just going to show off some basic ETL data processing stuff in a notebook. A notebook that is running Node.js and JS syntax, but we're kind of basically using it as a placeholder for very common Node services that could be used for like batch services and stuff which you'd need to do for you know parsing logs, or serving up some sort of you know subsets of data, things like that.

So in this case we're going to do it live. Right now I have a Docker container running in our Node Rapids instance, and a notebook. And so here I have a Jupyter notebook, a Jupyter lab. If you're not familiar, it's sort of like the data science sort of IDE. And you can see here all we're doing, like we did in any sort of Node app is require our Rapids AI QDF, for us we're going to be loading a 1.2 gigabyte sized dataset of US car accidents from Kaggle. And we're going to read this as a CSV into the GPU. And well, it was actually extra fast today. It only took under a second, which is kind of wild considering this is a 1.2 gig file. And so how big is this? It is 2.8 million rows with 47 columns. So for us, you know, not that big of a dataset, but for people typically used to doing data and like Node.js and stuff, it's pretty good. And how fast and responsive it is, is kind of impressive.

So we're going to just see, you know, what is the headers in this. It's sort of messy, and we're gonna go through this pretty quick, but basically get rid of some of the ones you don't need. See again, we're gonna then see what the columns are and say, all right, there's some temperature data in here. Let's parse this to see, you know, what the ranges are, sanity checks. And you say, all right, there's some really wonky values in there, as you always have with data stuff. Let's, you know, bracket it so it's sensible. So this whole operation of, again, 2 million rows, 2.8 million rows, took 20 milliseconds to kind of filter out. It's kind of wild. So now this is looking better. We have basically 2.7 million now. And we can start doing some other operations. In this case, we're gonna stringify it, and then with cuDF you get some regex operations. And so we're gonna have a pretty complicated regex here where we sort of mask out commonalities between these terms. And again, 2.8 million, 2.7 million rows, it took 113 milliseconds. So you can see how powerful and fast it is on a GPU. And basically at the end of this, you're saying that, yeah, you know, when there's clouds and rain, the accidents get more severe. But this is kind of just the first showcase of what you can do with GPUs and rapids.

And now you get access to it in Node.js, which we think is pretty neat, and, you know, the JS syntax, which is kind of cool. So we're gonna step it up a bit more. So the next demo is using Sigma JS, which if you're not familiar is a graph rendering library that's only client side. So it's pretty performant and pretty neat, they have a lot of functionality. But what we did is basically take out the memory load graph loading into system memory and load it into GPU memory, and then serve that up or stream it into the client size app.

So in this case, ignore the numbers. It's actually 1 million nodes with 200,000 edges, and you can see it zooms in and pans buttery smooth, even if it's in WebGL. So you get that great interactivity, and all that. But this is technically loaded onto GPU memory. So that means that you're not bottlenecked by web browser limitations. If without using a GPU, you basically are limited to 500,000 nodes, and then the tab craps out on you. So as you can see here, you know, it's using the GPU for the nodes and edges and, you know, straightforward, Sigma JS with very little changes. So we're hoping to do in the future is enable lots of Sigma JS users to kind of get that performance optimization and memory access without really needing to change much of how their code works.

For this next one, we're something more similar. We're going to do a surface side compute and a client-side rendering of a geospatial visualization. It's using a deck GL and Uber data set of 40 million rows. So, you know, not bad. Still not that big for us. And you can see, basically, it's all computed into source and destinations. So by clicking on each of these areas, you're kind of computing how many of those trips went where and what time. And you can see that clicking on each of these, it's essentially instantaneous, and you get the results from 40 million rows. It's so fast, you can basically just start scrubbing the values and getting that interaction back. So what this basically means is, you know, you can get away with having multiple instances accessing the same GPU and the same data set. And because this is a React app, the state is managed on the client side. So that means they all get their unique view of the data. But you're still querying that same GPU, and because it's so fast, it gets the values back, basically still in real time. And so again, like I said, really where the GPU excels is doing the server side compute, and the streaming over the values. So you can get away with quite a lot, even on a single GPU. So next one is a bit more complicated. You can see we're sort of building up modules as we go.

This was a video streamed server side rendered and compute of a graph visualization that's so basically then streamed to the client as a video. So you're seeing we're using LumaGL, JavaScript library to then convert it to OpenGL, encode that with NV encoding, stream it with Red Better DC.

We're going to use CUDF for the data frame, and Cugraph for the graph layout compute. So you can see we're sort of starting to wrap this in a little bit more of a GUI app. In this case we're just using a sample data set. We're selecting those edges, colors, and then we're going to render it. So you have this giant sort of hairball, but it is 1 million edges and 1 million nodes that you're able to basically filter and query real time. So you can see how quickly query it, clear the filter, you're back to a million nodes, million edges. You can basically tooltip and hover over each of these.

Now visually it doesn't really make that much sense, but this is more again for a kind of performance benchmark and how you can still get each individual node. And now we're going to run Fort Atlas 2 in real time. So Fort Atlas 2 is basically a layout algorithm, and so it's going to be computing the layout for every single node and every single edge in real time and all the forces are being calculated. And you can see this is doing it on again a million edges and a million nodes, and it's all streamed to video to the browser. You can see it moving through the iterations in real time, which is pretty wild. Again, this is just a single GPU and you're able to interact with this much data. And I kind of love looking at this, it's sort of like looking at the surface of the sun, which is pretty neat. And for the last one, basically very similar, except we're going to be using point cloud data instead. We don't need to do any like graph layout and stuff, but similar thing where we're doing video streams. So in this case, it's a point cloud of kind of a building scan, so you can pan and zoom and interact with it. But again, because GPUs are pretty powerful, I'm just going to duplicate this tab and have multiple instances of the same data set from the same GPU. But I'm, again, able to interact with it uniquely for instance, because of how you're handling the data. And so, again, this is a video stream, pretty cool. So you can use a single GPU for a lot.

That's a lot to talk about. We have even more demos and examples. This is the location of that. We have things where we're showing graph renderings with the GLFW. We have all the Deck GL demos and local rendered OpenGL. We have a really good multi-GPU SQL engine query demo. So we are querying all of English Wikipedia using multi-GPUs.

We have examples showing the Clustering UMAP algorithm and Spatial Quadrant algorithm. So they're pretty cool. I recommend you check them out. If you don't even install it, we have links to YouTube demos of the videos.

So what's next? Going forward, we kind of have three main guiding roadmaps ideas. We're going to continue doing these demos and working on doing external library binding. Again, looking forward to doing Three.js and some more work with Sigma.js. From that, we're going to start doing more specialized applications, specifically around like Cyber and Graph, Geo Spatial, Point Cloud, stuff like that. Maybe something that makes it a little bit more easy for turnkey usage. And then, hopefully, more broader community adoption, specifically in non-Viz use case. We're biased being the Viz team for Viz stuff, but we know that providing these CUDA bindings has an opportunity for a lot more functionality that we're just not thinking of because we're not aware of them. So we'd love to hear back from the community of novel ways to use this.

So yeah, RAPIDS is a pretty incredible framework. And we want to bring this capability to more developers and more applications, and we feel like the JavaScript and Node.js, the dev communities can take advantage of some of the learnings and performance from the data science community. So that's what we're trying to do is bridge the gap between the two, in this case with a lot of Viz use cases. But this is what we're working on next, a mainly better developer UX. So it's a bit of a bear to install right now, or if you're just not familiar with Docker images, it can be a little bit of a learning curve. So we're hoping to get this installable through npm. It's going to be a bit of a work for us, but we know that will make it a lot more approachable to JavaScript devs. Like I said, we're going to make a couple more visualization applications that are going to be a little bit more turnkey for more just general analysts. And hopefully we're gonna get full Windows WSL 2 support. So currently you can run Node Rapids on WSL 2 and Windows, but there's no OpenGL support from NVIDIA yet. They're working on it. So you can do the compute side stuff, but not the rendered side stuff. And then maybe native Windows support. This is a little bit of a long shot, but it would be pretty neat for us to be able to have this. So you don't need to use WSL 2 or anything. So with that if any of this sounds interesting or piqued your curiosity, come check it out. You know GPU accelerated data science services and visualization with Node.js and Rapids. We still kind of technical preview. We're working out the kinks and it's still pretty new. But we'd love to hear feedback about all kind of interesting use cases you might have for us. And we're very responsive to that stuff.