JavaScript Beats Cancer

Hi, my name is Karel Prystalski, and I will tell you more today about how to use JavaScript to beat skin cancer. My experience is about 15 years in machine learning. So my background is machine learning, it's computer science, I did a PhD degree in artificial intelligence, how to use it in medical imaging and dermatoscopy as well.

You can find some of my papers, research papers in this topic on Google Scholar, for example. So feel free here is one of the articles that I have published, actually it is around five years ago, about analysing of skin cancer on multispectral images. Actually, in that case I use Python, but because of the, well, became more and more popular in the recent years, and also the usage of JavaScript specifically for this topic, I decided to also, well, prepare a presentation and also a solution app for skin cancer analysis.

So my background is not only scientific, I also have founded in 2010 so 12 years ago, a company, a service company working for fortune 500 companies, building also data science, machine learning solutions. And yeah, before that I had I did also some some, you know, some other commercial work, for example, at IBM. So, as I said, I have 15 years of experience in machine learning and specifically in medical imaging, I mean, in applications in medical imaging.

So, how, why I why I decided to actually cover this topic and to build some solutions in this area? Well, as you can see, I don't I'm not really in the risk group when it comes to skin cancer because, you know, the biggest group of of the risk group is actually the blond people with blue eyes. So, this is the phototype number one with the highest risk of having skin cancer, especially if you're becoming kind of your skin doesn't doesn't isn't well it doesn't become brown when you're exposed to the sun but actually it's more going in the direction of red, and actually also, the risk of actually getting skin cancer is high in this group.

So, the darker the skin is and how it reacts to the sun, the lower the the probability is to get a skin cancer. So, there are six type phototypes of skin. I'm more or less in the third group because of my color, hair color, eye color, and so on. That's why the biggest problem actually, it is the biggest, the countries like Germany, Scandinavia, and the Nordic countries, the US, Australia, especially Australia, this is actually where this problem is even more and more important. In the meantime, I also have done some partnership with some dermatoscopy companies, I mean companies who actually develop the hardware. So yeah, as you can see here, here's one of the device. This is our dermatoscope here. That's something, that is a device that is actually used by the dermatologists. In this case, I have also used an iPhone here on the front because this is actually an extension. So it's not a typical dermatoscope, usually it doesn't come with an iPhone or any kind of mobile phone. It comes alone, it's a standard on the device. Some dermatologists use also this kind of extension case just to take the pictures in an easier way. And obviously it's quite small, so we can take it to your pocket and actually visit even your patient to take a look on the mole like this. So this is how actually my solution is used and it is combined with the special lenses, special light to get the best possible image of the skin mole. When comes to the data set because any kind of machine learning topic, model should be fed by some data. Now when I started my research I actually started with 50, 53 images or less. So as you can imagine, that's not a big enough data set to do any kind of research. So what I did is I met, I guess, almost every company in public or private that do anything with dermatology in the city where I live, in Krakow, in Poland. Most of them actually declined to collaborate and actually build some models.

It was in 2007, 2008. So the way how people thought about machine learning was totally different than actually compared to what's happening now. Actually, machine learning is AI became a buzzword and everyone wanted to do AI. In the past, I mean, 15 years ago when I said AI, most people said, oh, no, thank you. I'm not interested. Now it's totally opposite. I need to explain people why not to use machine learning rather than actually use machine learning. So it changed dramatically in the COVID pandemic, even increased the hype on AI.

So when I reached to the companies, I obtained a data set of around 5,000 images. Now you can easily download a data set of about 26,000 images of skin moles. It's available on the ISICarchive.com website, and you can use it for your research. So now it's even easier to develop algorithms to find different kind of skin illnesses, not only cancer, which is technically, I mean, it's not the most popular. That's good illness when it comes to skin.

So for all of you that want to use of my code samples that I have prepared for you, you can always download my Docker image that contains a JupyterLab, JupyterHub, together with some kernels for JavaScript as well, and also some libraries, JavaScript libraries. It's a bit old because I am doing it for many years already. So it might be that I will update it soon, but it's still working. So you can easily use it with the notebooks that I will show you next. So the architecture, how I started to actually use machine learning, well, how I combined machine learning with JavaScript, because of this device, I decided, obviously, to use one of the JavaScript solutions to build a mobile app because the mobile devices are changing every year.

As you can see, this is an iPhone 6S, so quite old, and probably I will need to change it also soon to a newer one. Still, it has a good, good, it's able, I'm able still to use this phone to make good quality pictures because the quality is here, not here, right? So it's not in the phone, but it's actually in the lens here. So that's why I decided to use JavaScript instead of building a native application.

And in the past, I had to use some different kind of solution, starting from Cordova PhoneGap, now I'm actually working on React Native. And for the machine learning part, I used TensorFlow.js, so you might say, okay, why TensorFlow.js, and not, I don't know, Keras, or actually Torch, for example. Well, there is one reason why I use TensorFlow.js, because it's the most robust library, when it comes to JavaScript, obviously. So why I use, or why I choose JavaScript in this case, that's not exactly that I said, okay, let's do everything in JavaScript, and that's not true.

I mean, the true is that actually the model is trained in Python, but it is used with JavaScript. So I use TensorFlow.js not really for the training, and to be honest, I don't know anyone who actually do that. I mean, maybe because I'm into the data science field for a long time, and I know many people in this area, and they actually do mostly in Python, or I don't know, maybe Scala, some of them, especially related to big data. So in this case, in this specific case, I use JavaScript exactly because of the possibility, actually, to use TensorFlow.js to load the model, use the model, retrain the model, and use it on our phone, mobile phone. So in production, it is used together with some Kubeflow and TensorFlow servers. I mean, there are two models, one that you see on the left. Actually, you can combine it together with the main app. And that's how I did it here on the phone. But actually, in many cases, it also caused that service that actually is trying to find similar models, similar visions. And actually it is a web app. So the app is actually connecting to the web app and then also use that for retraining as well.

How does it look like? Let me just shortly move to some examples. Here we go. Here we go. That's partially, I did it also in the past for another conference at Hockey for Nukraine. I will just use part of the part of the notebook. You can easily find it on my GitHub account on my company's GitHub account. And a few repositories that are about machine learning, AI and JavaScript. When you start to do any kind of research related to data, when you do it in Python, you might use probably the first library that you think about is Pandas. And there is a fork of this library in JavaScript. JavaScript is called Pandas.js, obviously. To some degree, it's very similar to Pandas, to the original one from Python. But actually, it's quite limited compared to the original Pandas.

Still, many, many features are not yet implemented. So to some degree, you can use it in JavaScript. But still, there are a lack of many, many advanced features from the original Pandas, especially the one that actually are already just statistics.

Another library that you can find, or you can use in JavaScript for data analysis or data manipulation, DataFrame.js, Reclaim, DataForge, and so on. So there are plenty of such libraries to use. This is just a few examples, how to work with series, how to use with DataFrames. That's something normal for people that work on data, probably for JavaScript engineers, not so typical, but still easy to manipulate, easy to export to JSON, for example.

And so this is just a notebook on how to use that, just to show a few examples. When it comes to the visualization, in my opinion, JavaScript do a better job than Python. Obviously, there are some libraries like MATLAB, Seaborn, and some other ones and in Python, when you compare it to the one that actually are available in JavaScript. I think here, JavaScript has a huge advantage because there are good libraries for visualization for printing the charts. But I think in many cases, are just doing better jobs than the Python ones. So, that's good. So one, one, let's say.

And then, when it comes to Scikit-Learn, the most popular library for machine learning for the shallow methods. So, tensorflow, I mean, that's for building neural networks, but actually Scikit-Learn is more about the shallow methods. It's actually most of the cases that you want to deal with when it comes to machine learning can be solved or should be solved using shallow methods. So, Scikit-Learn, that's the first library we will use and easy to use in Python. In JavaScript, you have JS.kit.learn and Scikit-Learn as a fork of Scikit-Learn, but well, not updated in the recent years. So, it looks like someone did that, started something, but well, then dropped the maintenance and they are gone. I mean, not really a good library here to use. There are plenty of libraries that actually implement some specific shallow method, like I don't know, SVM, KNN, and then so on. But actually, if you want to have it in one place like it is in Python, then you don't have such a library in JavaScript, unfortunately.

So, when it comes to TensorFlow, here's a very easy example of how actually you can build a linear regression. Here are some examples how to import, how to use, how to implement a linear regression. And here's an example, here's also an image from the documentation of TensorFlow.js. You have many kinds of APIs. The most popular is in, written in Python, but actually back is written in C++, and you have also some other forks like Java, Go, JavaScript, and so on. So JavaScript is not, I mean, it's not special here. It's just another fork to get to the core of TensorFlow, really.

Here again, example of how to build a loss function, combine it together, how to use it, import it and train it. When it comes to build, when it comes to neural networks, how we do it actually in Python, so mostly we do it with cameras. I don't have PyTorch. Actually, you can easily combine the network. I mean, build it from a blocks. I actually have layers. Connect the layers together and you can easily build a very huge network shortly. In JavaScript, you have Keras JS, but to be honest, it's far, far away from the one that is developed in Python, unfortunately, right? Okay. That's the one thing.

Let's just move back to the slides. When it comes to the skin cancer, as you can see here, you have three images and only two of them actually are cancers. The one on the left and one on the right. Here on the left, you can see some patterns, I mean, the dots here and the stripes. Some vessels here visible that all of that is not very symmetric. The borders are quite smooth really. Here on the right, you have this kind of a pattern that is called the blue wave pattern. You can see kind of a white-blue colors here. It means that actually, well, it's going deep. I mean, the cancer is actually going deeper into the skin and actually trying to get to the vessels. That's bad for the patient, but that's also a pattern that actually tell us, oh, that's really bad. Here in the middle, you have an example of a suspicious care role, but not confirmed to be cancer because all of them actually are pathology confirmed or not. I mean, confirmed that they are cancer or not. So how do medical doctors do that? They use some kind of a scoring methods like ABCD, seven-point checklist, seven-point score, hunter's call, three-point checklist, and so on, and so on. So they have, they use some panels, like asymmetry, border sharpness, number of callers. I mean, in ABCD you have six calls that actually, it really counts, and they check how many calls there are. When you combine it together, I mean, you write down one by one how many different patterns are available, you can count easily more than 30 patterns that actually the doctor, the medicals can find out on the image, or just, you know, using the loop. Directly. What you can also do, and what I did in my research, is that I used different kinds of wavelengths of light to get not only what you see with the visible light, but also what's actually deeper in the skin using, for example, the infrared light. Actually, I used four different wavelengths of light in total, but actually here you can see one with the infrared, UV, to get the, to get the melanin, right? The vessels more visible, and that's how you can actually do a better research, because we have more details, more information. Oh, what you also do, you can also do some image processing, like binarization in this case, right, to find out, oh, this here border is not smooth, right? You can, for example, use fractal methods to analyze that.

All right, so the demo, another one, because I would like to show you also how you can also use that, how you can, what kind of, what kind of methods you can use. Really, here's one of the examples you can find more actually using, going to my, going to my GitHub repository, here's a way how to extract the asymmetry, actually extract the region here, first of all, right? So we could take this part, this region. I use here the ISIC as you can see there and next, that's the next step to calculate, in this case i can see it's Python, because I have developed, develop the modern Python, that actually exported to JavaScript, to be used in JavaScript, and here you can use, divided the picture into some regions and calculate the symmetry of the opposite sides of the blocks. So this is how you count easy the asymmetry. And this is how you can do it in many panels, I mean do some basic image processing, it allows us to find out some of the patterns. Some of them actually are more difficult, you need to build some sophisticated models based on neural networks. Please feel free to get more about that. But what is important, because I mentioned about the neural networks and also the shallow models, so just to give you a better understanding, here you have, because we have the black box and white box battles, so the black box are neural networks, in most cases you can see here a very short network with three layers here. It's easily trained with an accuracy of 98%, very high. But if you actually draw the weights, you can see, I mean, printed weights of just one layer, it looks like that. So if you try to explain that, it's very hard, even not possible, to explain each of this number, what does it mean. So there are some explainability, explainable methods, to explain the numbers, in the weights, in the neural network, but it's more complex.