Video Summary and Transcription
In this Talk, the speaker demonstrates how to create an AI chat bot that can answer questions based on information it was never trained on. They build a basic RAG pipeline in just five minutes using live coding. The speaker also shows how to create embeddings and a vector database, set up a vector search index and endpoint, and modify the chat route to enhance the chat bot's capabilities. The program is run and tested, and the Talk concludes with an invitation to join a workshop for more information.
1. Introduction
I'm going to speed run creating an AI chat bot that can answer questions based on information that the LLM was never trained on. We're going to create a basic RAG pipeline and build it in about five minutes, live coding.
I'm going to pack as much information into this talk as I can, and if I go too fast, I lose you, if you have any questions, come find me afterwards, and I'm happy to talk. So what I'm going to do is I'm going to speed run creating an AI chat bot, and not just any AI chat bot, a chat bot that can answer questions based on information that the LLM was never trained on. So we're going to create a basic RAG pipeline, retrieval augmented generation. So on top of that, I'm going to build this in about five minutes, and I am going to live code, so nothing can go wrong, right?
2. Creating Embeddings and Vector Database
I ran NPX create next app using the next lane chain example. Installed MongoDB, React markdown, and .env. Checked the app, added dark mode. Tested with a question about MongoDB. Created fake documentation for a JavaScript library. Transformed markdown files into vectors and saved them in a MongoDB vector database. Used vector search to enhance LLM capabilities. Set up file system promises, open AI embeddings, Mongo client, and MongoDB Atlas vector search. Created embeddings for each document and stored them in MongoDB. Fixed a typo.
So I did already run NPX create next app using the next lane chain example. I installed MongoDB, the lane chain MongoDB integration, React markdown for some styling and .env because we're going to use a node script in order to run our ingest.
So I also have an open AI API key and my MongoDB Atlas connection string in my environment variables. So let's go ahead and check out this app. So this is the example straight, without any alteration. Well, I added dark mode so I wouldn't blind everybody, so you're welcome for that. So let's just test to make sure it works, so let's say what is MongoDB and hopefully the Wi-Fi works, and okay, there we go. And open AI responds to us with a pretty good answer. So it's working out of the box. Great.
Let's check out the code. So I've got this fake documents directory here, and I use chat GPT to help me create some fake documentation for a fake JavaScript library called fancy widget.js. So we have read me, usage, license, installation, contributing, change log, API reference. We have all the documentation that you'd expect from a JavaScript library. So what we're going to do is we're going to take these markdown files and we're going to transform them into vectors, vector embeddings, and then we're going to save those in our vector database. We're going to use MongoDB for the vector database. And then we can use, during vector search, we can use this to augment the LLMs capabilities so they can answer questions based on this information.
All right, so let's go ahead and get started doing that. So in the root here, I'm going to create a new file. We're going to name it create embeddings.mjs, and then we are going to do some typing here. So we're going to import our file system promises from recursive character text splitter, and then import open AI embeddings from lang chain open AI, and then our Mongo client from MongoDB, and then our MongoDB Atlas vector search from lang chain, and then we'll set up our Mongo clients, getting our environment variable there for our connection string.
Our database name is going to be documents, collection name embeddings. We'll set up our collection, and then we'll get our documents directory, those fake documents, and then get the files for those, and then console log the file names, and then the file name, look through those file names, get each document. After we read each document, then we're going to console log that we're vectorizing the document, and then our splitter is going to use our recursive character text splitter from lang chain, and we'll chunk those into different pieces, and then output those and store those into MongoDB using MongoDB Atlas vector search. We'll create those embeddings, we'll tell it which collection, which index name, which text key, and which embedding key to use, and then console log that we're done, close the connection to MongoDB.
And there is a bit of a typo here. Of course that didn't happen in practice. And of course I wasn't typing because that was a VS code extension. This is supposed to be import recursive character text splitter. So let me grab that.
3. Creating Vector Search Index and Endpoint
In under 40 lines of code, we have created our ingest for our documents. Ran node create embeddings and successfully created embeddings and saved them in MongoDB. Checked MongoDB and found the documents with text and embeddings fields. Created a vector search index using the JSON editor and defined the index path, dimensions, and similarity. Created a new endpoint called vector search in our route.ts file.
So let me grab that. And we should be good, I hope. Live coding, nothing ever goes wrong, right? So in under 40 lines of code, we have created our ingest for our documents. So let's go ahead and run that. So open up the console, let's kill this console, and then let's run node create embeddings. And yeah. Lang chain import, yes. Thank you. Lang chain import. Anything else that I'm missing? Yeah. That. Thank you so much. Okay, let's go back into our console, and let's run node create embeddings, and there we go, now it's working. Thank you. Sorry, it's looping through each one. It's creating those embeddings and it's vectorising them and saving them in MongoDB. So let's go ahead and check that out. So right now there's nothing in MongoDB. Let's refresh it. And now we should have some documents in MongoDB, which we do. Amazing. So we have a text field which is the original text chunk, and then we have the embeddings field which is an array of numbers, those are the vectors, and we have some extra metadata as well. So the next thing we need to do in order to use this is create a search index, a vector search index. So let's go ahead and create a new index. We're going to use the JSON editor, and let's go ahead and define this index here. So we're going to say the type is vector, vector search. The path is going to be the embedding field in our documents. We define our dimensions, our similarity, and then let's select the embeddings collection, and then next, and create, and good. Okay. While it's doing that, it takes just a few seconds to do that, but while it's doing that, we need to go over to our API end points, and we're going to create a new end point called vector search, and then our route.ts file.
4. Setting Up Vector Search and Modifying Chat Route
We import OpenAI embeddings, the Mongo client, and MongoDB Atlas vector search. Set up the database and collection. Configure the database with the collection, index name, and embedding key. Create embeddings using MongoDB Atlas vector search. Retrieve the top five similar chunks from the vector store and return the results. Modify the existing chat route to send the user's message to the vector search API and receive additional context to pass to the LLM. Add the extra context and user's question to the message queue. Continue with chat GPT or open AI to GPT 3.5.
Okay. So we're going to do some more speed typing here, and this time, we're going to import our OpenAI embeddings from OpenAI, our Mongo client, and then the MongoDB Atlas vector search from lang chain. And then we'll create our post route here, define the database name, the collection name, and then set up the actual collection. And then after that, we will set up our database config, so we'll define our collection there, index name, our text key, and our embedding key, and then we'll set up our vector store using MongoDB Atlas vector search. We'll create embeddings there for the question or the prompt that the user is sending in, the question. Then we'll set up our retriever, which is our vector store. So we're going to retrieve the top five similar chunks from our vector store. We'll get those results, and then we'll return those results. So this endpoint is using vector search in MongoDB to return the most relevant results for this.
So last thing we're going to do is go to our existing chat route. So this is the route that came with this example project, and we're going to just change this a little bit. We're going to intercept the user's message and do something with it. So let's do some more typing here. So we're going to pop off the current message in the message queue, and we're going to send that to that vector search API route that we just created. That vector search API route is then going to return additional context that we can use to then send to the LLM. So we're going to create this template. We're going to say, you are a very enthusiastic, fancy widget representative who loves to help the people. Given the following sections from the fancy widget documentation, answer the questions using only that information. I'll put it in markdown format. If you are unsure and the answer is not explicitly written in the documentation, say, sorry, I don't know how to help with that. We're going to add that extra context that we got from vector search, and we're going to add the user's question back into here and then push that back onto the message queue. After that, everything will continue as is, going to chat GPT or to open AI to GPT 3.5 is what we're using here.
5. Running the Program and Conclusion
Run the program again and ask it questions. It will answer based on the information we provided. If the question is not within the given context, it will say it doesn't know. End of the talk. Scan the QR code for more info and join the workshop.
Okay. So now if we go back into the console, and let's run this again, let's go back and refresh this. Now if I ask it, what is fancy widget.js? It should... Oh, no. Live coding. Vector search. Come on. You all saw that way before. You didn't say anything. Come on. Yeah, I know. I know. Huh? Let's kill this and run it again. All right. What is... Let me just make sure it's running. Yes, okay. What is fancy widget.js? And it should answer us from our information. Yes, it did. It worked. Now, what if I say what is MongoDB? It should say I don't know how to help with that, because we didn't give it that context. So it's only answering questions about the information that we gave it. Okay. So that is the end of my talk, all the time that I have. Scan the QR code for more info. Attend my free workshop and come see us at the MongoDB booth.
Comments