Bring AI-Based Search to Your Web App

So, thank you for this very nice introduction. And I'm super excited to introduce you to this topic that, let's face it, a year ago I didn't have any idea about. So I'm going to bring you on my one year journey with me. So let's get cracking.

So you heard the introduction, my name is Sebastian Vitales, I build cool stuff, and I want to talk to you about it. So my journey begins a few years ago, a long long time ago in a galaxy far away. I went to a conference and there was a very interesting talk that I was very excited about that promised that you don't need a PhD to understand machine learning. And obviously I was super excited about it because A, I didn't have any clue about ML but I really wanted to get into it, like, hey, there was a promise, like, maybe I didn't need to study for five years to do something around ML. The thing is that 10 minutes into that talk I was so confused I didn't even know what my name was. And immediately my assumption was like, yep, you need a PhD, I'm never touching ML again in my life, you know. That's it. Over. But then I was like, OK, I'm not going to give up.

The thing that changed and is happening lately, and I'm sure you're all experiencing that, is that everybody's talking about ChargPT, AI, ML, like, all those things that you're hearing. Like, I currently live in Denmark. I don't speak any Danish but if I'm in a café there's, like, people speaking randomly Danish and catching ChargPT, something, something, like, you keep catching it. I went to get a haircut, right, like, and my hairdresser got confused. It was like, OK, I'm going to ask ChargPT what kind of haircut will go with you. Even ChargPT can help with it so that's OK. And the thing is, like, what changed? Why is everyone now talking again about machine learning? Where is this buzz coming from? Why is everyone excited, not just even people in tech, but even, like, regular people that, you know, don't, you know, don't use computers for professional stuff, right? And what changed is actually that those machine learning tools became accessible, right? Like, suddenly, they're, like, at your fingertips. Suddenly, you can go, you know, to OpenAI and, like, create a login, and then you could start writing prompts and ask questions to the AI and this is mind-blowing. And there are so many different applications, all sort of, like, image generation, all sort of things happening. But I only have 20 minutes for the talk. And the organizers, like, asked me already five times. So, I'm going to finish on time. So, I'm going to narrow down and only talk about, like, a very specific thing of the machine learning, especially I have 20 minutes and I want to do some live coding as well. So, let's stick to that.

So the topic of the presentation was search or using machine learning, search. And let's face it, everything that we do on the internet begins with search, right? You want to listen to music? You search. You want to watch a movie? You're going shopping? You want to find some information? You go to Wikipedia, you always search.

And I mean, you probably think it's like, okay, what's the problem? We've been doing that for decades. Search works. Well, I beg to differ, right? Like, it kind of works, but it could be better. And let me give you an example. So, with traditional search, you may face some challenges. So, if you went and asked, like, a traditional search engine, like, why do airplanes fly? So, maybe you have a whole database of documents that explain it. You may get an answer like, why you should fly with expensive airplanes? And it's like, well, I mean, it's pretty good because it matches airplanes, it matches why do and fly and all this. Why is this guy complaining? It's like a perfect match. Well, I mean, in reality, we asked how planes fly, and we were told to fly with expensive airlines. Well, that solves it for me. Thank you. Well, the solution for me is kind of like looking for the question from the semantic point of view. What is the meaning of the question and what sort of answer can I find for you? And actually, if you put this question into Google Search, you'll get an answer like this. You go and we'll find the dynamics of fly from NASA. And then in there, the bit that helped us find the answer was like airplanes wings are shaped to make air move faster over the top of the wing, blah, blah, blah, blah, blah. Like we don't really have any of the keyword matches, but the meaning is there, right. And that's basically the power of semantic search. So by looking at those two examples and the kind of things that you can get between the two, I mean, the conclusion is pretty straightforward, right? Like we should be going and looking more at the semantic type of search and like using machine learning for it.

So how does this work? And a little heads up, you don't need to know it, but I think it's pretty cool. Like you'll find it interesting, and I have like a very nice explanation that you don't need a PhD to understand it. Maybe it's not four or five years old, like it's more of like a 10 years old explanation, so just bear with me.

So first of all, in the machine learning world, we have this thing called machine learning modes. There's this like black boxes that they are constructing in such way that if you pass in some information into it, it will spit out this thing which we call a vector embedding. A vector embedding is basically an array of numbers. It's just like a bunch of numbers, and in reality, it's like how computers look at data and you look like binary, and computers understand it, oh, this is a piece of information. This is basically what machine learning thinks. Like, if machine learning looks at the vector embedding, that's how it understands information and data.

So if we took our previous two examples that we had between the two results, and we put fly with expensive, as modern fluid paragraph into machine learning model, you will get a vector embedding. If you take another paragraph, and we put it again to machine learning model, we get another vector embedding. And what these things are, like you can imagine they are like coordinates in like a multi-dimensional space. Think of it like you see in this multi-dimensional space like dog and wolf that is right on the left-hand side. They're really close to each other, right? And a cat is not too far off, while the bananas and apples are in completely different place. And of course, our brains are trained to think in three dimensions. But those machine learning models can go into 300 dimensions, thousands, or 10,000 dimensions, so they can understand information in a very detailed way.

The cool thing is now that if I want to do a search, I could take our original query, pass it through the machine learning model, and I get a vector. With that vector, visually, to imagine it, we could say, like, hey, my information is somewhere here. So any objects that are close to that new vector position would indicate here are the articles where your answer might be. It's really cool, right? By looking there, we can then find, like, oh, yeah, the NASA article was just there, and then we can pass it back. It's all math. It's all just, like, grabbing and calculating those vectors, which is really, really cool. See? So I think even 10-year-olds and people that are not in ML can understand that you don't need a Ph.D. I think this is pretty straightforward.

So there's a bonus thing. Some of those machine learning models, and you probably saw there were some pictures, there was some text. Some of them can understand both text and images. They can also understand many other things. But in particular, there's one called clip, and if I pass in a paragraph that describes a maxi summer dress and a picture of the same dress separately, I would get two vectors which are very similar, which basically means that I could use it as well to search through images and text in however way we want to do. So I could have a database of pictures, and I could write in English kind of or any other language, so like, oh yeah, show me a picture with a house and a dog and a tornado in the background, right? And the vector would then help us find that picture later on.

Well, if we have it in the database. So yeah, we could do images and text in one go and plus many, many other things.

So this is a JavaScript event, and you're like probably beginning, like, OK, what does JavaScript got to do with it, right? So let me take you on that.

Now let's talk about the practice stuff. So if you have a JavaScript application, whether your search is done in the back end or the front end, usually you have some sort of JavaScript app, and you have data that you want to be able to search. Cool. That's the first ingredient we have.

Then what you need to do is pick up a machine learning model or a vectorizer, as we call them, and you don't necessarily have to create your own one. There's companies like Open AI. You probably know them because of ChargeAPT. There's Cohere, they have these multilingual models that understands over 100 different languages. We have Hug and Face. Hug and Face, they have 80,000 machine learning models. Recently, Google at Google I-O launched Palm 2, which also does this job. So you don't have to actually create those. You just basically find you need to find a model that works for your data, and you're golden. You're all good.

So having all this, to tie it all up together, and this is why I'm here. I'm wearing a t-shirt from Weaviate. We have an open source vector database. So it's basically a database that takes care of all of it for you, right? Like, you say, hey, I want to use this machine learning model. This is my data. Go. Right? And now I can write code in JavaScript or many other languages to write my queries or perform many other things.

So I want to do some cooking and show you a really cool demo. So, whoops. I was not going to do this. So the first demo I have is I have this database with 10 million objects in it. So these are paragraphs from Wikipedia. And to make it better, each million is a bundle of articles in different languages.

So I have some in English, I have some in French, in German, I have some in Hindi, in Arabic, I think there's Japanese as well. So we have like a pretty wide database. So what I can do with it is, I'm not going to be teaching you necessarily like, hey, this is React application, this is how you connect it to the UI, you know this stuff, right? But what you really need to know here is the following. So with WeaveIt, I already have the data in my database. So first thing that we need to do is connect to my instance. So this specific instance lives in the cloud. So we have this cloud services where in this case, I'm hosting this demo and in there I'm passing in the API key of my cloud instance. But additionally, I am using the Cohere API, like an inference service. So basically I'm telling like, hey, this is my API key. So if Cohere needs to charge me, they know how to find me, right? So I create this client and with this client, I can start writing codes and then writing my queries. So I'm not going to show you how to import the data in this case, I just want to focus on the querying part. So at the moment, we are using the builder pattern. So I'm grabbing a query from based on GraphQL. And then maybe we have with class. So my data is in collection called articles. Oh, I should actually probably show you this. So my collection articles is defined to use text of echo here. And then in this case, I want to use this multilingual model from 2022 December. And then also just so that you're aware, I have properties like text, title, URL, wiki ID, views, etc, etc. So now I'll be using those two to query through my data. So I want to query my articles and then we will query, what is it? Fields, and I want to get text, URL, and title. That's cool. And then maybe I want to just get first 10 objects. So if I just run this query, I could maybe search for how planes fly. At the moment, I'm not doing anything as such. I'm just saying, give me a bunch of random objects. OK, that's fine. But now I want to actually do a query. And this is the bit where I can do machine learning query with this one line. So I could say, with neartext, there's different operators as well.

And I can pass in concepts. And the concept will be the query that I get from the UI. So this is the query I'm getting passed in. And now, with this one line, I can get articles that are related to flying. I can open this, and then this one is, what, is that Italian? Yep. So there's the Italian page on Wikipedia. And I found that article there. This is pretty cool. And of course, we can put many other queries.

The other thing I can do is I can write it in other languages, right? Musiczne instrumenty. So this is in Polish, like musical instruments, right? And I'm searching across my database, which is super cool. And if I just want to filter and only show English response, so this is to show if you that we can also do traditional database kind of queries, so we can do scholar search. So I could do with, I need a dot. So with where. Actually I'm not going to type it manually, where language English. So in this case I'm just saying, okay, where language equals yen. So this time I can get English results only. Just pretty cool.

The other thing that I can do as like the next level, I have another collection where I needed to make some changes and I introduced a generative module model. So this one uses GPT from OpenAI. And I can actually do some extra bits and then do, like provide prompts to, to the database to, to perform some extra operations. So let's try that. So this time the collection is called Wikipedia, but it's basically the same data. I just didn't have the time to rebuild the other one. So we're going to skip this one for now and then maybe I can do with generate. And then in this case, we need, what do we need? Show prompt. And then I can say summarize the following in a tweet. And then I'm going to pass text, which is this property. And because I'm using OpenAI this time, I am going to add an API key. So I have it in my local configuration.

Let's not run on 10 because that's a bit time consuming. What will happen is we run the query, we get the results, and then we'll pass it on to OpenAI to vectorize. And then also I need to do a little bit of parsing. So generate single result. Nope, not this one. Demo. Return. Generate. It's just like a simple map. I'm just after this field and I will replace the text. So if I rerun this and run the search, that's going to take, usually takes 10, 15 seconds, but yeah.

It takes each of the results and then creates a summarizer. What's really cool, everything's super cool when I'm talking right now, I summarize the following as a haiku poem. I save this and then I can rerun this and this is mind blowing. So it takes a while. And we have haiku poems based on the article with Galswin. All right. The next demo I have for you is, you could also do image search. So I have this table, collection, whatever you want to call it where, well, this is going to be tiny but you should be able to see now. So I have like a folder with like a bunch of actors, right? Famous actors, famous celebrities and this time I'm using a clip model. So I could, for example, do this where I say find a celebrity that looks like Brad Pitt. And then, boom! We have Brad Pitt but I'm always going to return two objects so not all of them would be perfect. Which actor do I look like from the like a hundred that I have? I look like David Tennant and Stan Tucci. They're really going after my Baldy here. This is offensive. But hey, another one, like I have a picture of a meerkat. So well actually Ryan Gosling does look a bit like a meerkat so that works pretty nicely. And I did the same thing for the conference speakers here. There's a bit of a running joke when it comes to Jeff Cross and Mike Harrington. They usually get mixed up.

At one point they even swapped their profile pictures. If I show you again the picture of Jeff, you know, there's some resemblance between them too. And if you want to search for Mishko, there was a different picture of Mishko. We're still able to find it. And this is really, really powerful.

And this application I built in Angular. So just to show you the little bit on how this is done. So we have this image service. So basically the step is this time I'm using the multi, like the clip model for the vectorization or to understand our images. And I'm saying, like, hey, we'll have name URL and image. And then after, then I go and import the data. This is not super important. The image search is just this. Basically, when you give me a file to search on, I convert it to base 64. And then the query is just like the one you saw before, with the difference being that now I'm doing with near image. So I'm passing this new image as a base 64 representation, and boom, I can do the search. And you see, like, none of this code makes you, like, requires you to understand how machine learning works underneath. You just have a job to do.

And so if in your websites or in your backend systems, maybe, I don't know, you have 500 million PDFs that your company has been storing for the last 50 years, and then you could never search through them. Like, you can load them into Weave8 and then you run them through a machine learning model and then query them. Like, something that was inaccessible before, now you can run it. As a bonus, our website, we've added this recently. So if I do Command K, we actually use vector search in our website itself. How to delete an object, boom, here is an article in a documentation. So these are all really, really cool things. So to summarize, anyone can bring AI-based search to their web application. Basically, you need your JS application, WebSum data, pick a machine learning model, and a vector database. And with that, and then, what, like there was like 20 or 30 lines of code, you're golden, and then you can use machine learning search for your use. So yeah, that's me. This is our website.

This is our Twitter. So the first question we have, and so I'm not really familiar with this space, but they're asking what are the benefits or maybe the differences between a product called Pinecone? Oh, yeah. The Pinecone would be like our competitor. I think one of the biggest differences, like first of all, WeaveIt is an open source database and we offer you both that you can deploy into the cloud, but also the open source option, you can run them with Docker and with Kubernetes. But the other thing also is that our developer experience, I think is superior. So like the fact that like I was able to run a query, like five lines of code, that's not possible with any other vector database that's out there.

Right, that's a good point. And this next one is asking about the comparison algorithm, but I guess that has to do with how it's actually dealing with those vector embeddings. So they're asking how does that comparison algorithm work and how performant is it? Yeah, how it works is actually, I could give like a one hour talk about it, by the way, but basically the idea is that because the vectors are like very complex, they're long and you can't just sort them by one thing. So we just have a special indexing mechanism that allows you to search very fast. One of the first ones implemented is called HNSW. It's very technical, but the idea is we just have a specific index mechanism for it. How performant it is? We've tested Weaviate on nearly a billion objects, which topped out 960 million because we run out of data. Even with this 960 million we're running queries under 100 milliseconds, which is insane because of the vast amount of data. Yeah, it's crazy.

This next one asks can you customize the distance metric? So like cosine similarity or Manhattan, is that something you would do? Yup, so basically out of the box we support five or six different distance metrics, but also Weaviate A is very modular and open source. It's very easy to add new ones.

This next question is asking about Postgres. So they mentioned Postgres has vector embedding, so I guess why would you use something like this if something like Postgres already support something similar? That's the thing. I could technically take a Formula One car on the street and then try to drive around. And it's all about is the tool built for the job that I'm trying to do? And when it comes to vector embedding search, there's no comparison. I think you can do it in Postgres, but the moment you hit like half a million objects, you will really struggle. In our case, I'm boasting and we go in a billion, and we're running out of data. And I think that's the big thing.

This next one is how much time is needed to train the AI? For example, on the Wiki. And I guess to test my own understanding, is it basically the time it takes to generate the vector embeddings? And at that point, you're ready to go? Yeah, so actually, it's not so much as a training, because the machine learning model that we use with Cohere there is already trained. So, the only time that you have to invest is as you import the data. All that data has to be converted into those vector embeddings. I think I imported those 10 million objects, it took me a couple of hours to do it from my laptop on a crappy network in a local café. That's crazy. So, you know, it's doable, it's not too bad. But there's an investment of time upfront required.

This next one asks, so what if you want to index like PDF data? Is it a similar process? It's slightly different, but actually we are working on it. We published a blog post about it last week, Tuesday. So, go to our website and check out our blogs. But yeah, we're dealing with unstructured. I think the trick is basically have something that can read through those PDFs, and maybe construct paragraphs, and then pass them into Weave 8. Because one of the things that's worth understanding, if you pass in, like, let's say a whole book into a machinery model, it will speed up one vector. Like and then if you run a query, it's just like, yeah, that's an average of the whole Bible, the whole thing. So, what you really want is something that can find specific parts. So, that chunking is sometimes usually the process that takes a little bit of extra effort. But, yeah, that blog post actually we explained pretty well. So, it's more about like, can I read through it, and then chunk up my content in a nice way? Right.

This next one is, is AI search useful for search that is completely deterministic or is it overkill? I think it depends on the case-by-case.

This next one is, is AI search useful for search that is completely deterministic or is it overkill? I think it depends on the case-by-case. And actually to that point, that's why in Weaviate we offer like we have AI search but we also have hybrid search where you can use both like the traditional methods and AI search in one go. And we also support this algorithm called BM-25, so that's keyword based search, so you could also do that. So based on what you need, you could do either.

Yes, there's cases where it just doesn't make sense, right, where you have like you know there's this exact thing you're looking for, you don't need AI for it. So actually the search that I showed on the website when I was searching for how to delete an object, we use hybrid there because in some cases the ML model might not know about some function names that we use. And I think just looking at the developer experience, it's almost like with just a method call you decide, do I want to use AI or not? So that's pretty cool.

By the way, I built this demo, the image search demo in like two hours like three days ago. I was just like in the evening, I was like, oh yeah, I want to add another demo and it just took me two hours, and I built it from scratch including installing Angular on my laptop and all this. Nice. Would it be possible to train a model on, for example, hundreds of product manuals like phones or TVs and get accurate results, or is a larger amount needed? So I guess for anything you're training or creating the embeddings for, how much data do you need? Yeah, that would be a good question to somebody in my team. I mean like basically what I'm being told is like the more data you have, the better quality results you can get. So I think like what you see with ChargeGPT, they had enough data that they had to spend six months training it, so we're talking about crazy amounts. So it's all about like the more data you have, the better it is. But what's enough? That's kind of hard depending on how good the results you want. I don't have a good answer for it, I'm sorry. Okay, no worries. I don't think there is one. Yeah. I guess it just depends. You have to try it out and see. Yeah. The next one is, if you get vector data out of something like text data 002, is it going to output the same vectors as some other model? Are they interchangeable or? No. So each model has its own thing. So the way I see this, I use this as an example. Imagine you have somebody that works at the shopping mall and knows where all the stores are, all the shops are, and then you go, like, hey, you know, like, coffee. And then he goes, like, this place, this place, right? And now you take him to Amsterdam and put him in a different one, and you're like, coffee shop is like this way, this way. He'll be sending you wrong directions, right? So that's like, what are they trained on, right, that matters. Yeah.

Yeah. Makes sense. Have you tried any other embeddings as, like, GloVe, Word2Vec? Don't know what that is but maybe you do. And how much better is it than Transformers? Is it always worth it? Imagine the cost of learning. Yeah. That's not a good question for me, honestly. Like, yeah, this will be for, like, the data scientists on my team.

Okay. I can definitely say for sure that it's easy to test it out in Wavi8, so I'll just encourage swapping them out and trying out. Yeah. Yeah, and I think we sort of covered on this one earlier but does it take a lot of time to add vectors for big data? But I guess you mentioned you were able to train on your laptop, so... Yeah. I mean, so the Wikipedia demo that doesn't run on my laptop, that actually we have an instance in the cloud. Okay. The image search that I did, that was purely on my laptop.

Yeah. I think the biggest thing that you need when you think of vector database that we don't say, it's, like, it's very RAM hungry. Mm. And the reason that index wide runs super fast is because we store it in RAM. I see. Yeah. Okay. And I guess the last one to kind of end on, the morals of it all, so should we be concerned about sharing our app data with an AI tool like ChatGPT? Yeah, I mean, so there is the thing of, like... So one thing that I know if you send queries to ChatGPT, they do use the input to train their future, basically, requests. And especially if you, you know, using ChatGPT for free, it's kind of like, it's an exchange. You give them training data in exchange for what you do. And I think recently Apple forbade their employees to use ChatGPT for any content. So there might be something to it. But generally, there is no malicious intent as such. We're friends, we're actually people at OpenAI. But if you're really concerned and say, hey, only I and another person should ever see this piece of information, like, yeah, just don't put it on the internet, right? Awesome. So that's all we have for you. Thank you so much, Sebastian. Thank you.