Integrating LangChain with JavaScript for Web Developers

Next that we're talking about, next type of prompt engineering is basically RAG. We've been talking about RAG a lot, which is Retrieval Augmented Generation. The thing that happens in RAG is in few short, in zero short, in chain of thought, you're writing a big prompt, right? You're writing examples, you're writing step-by-step instructions, you're writing your original input query, and then you also have your chat history. You can imagine their prompt will be so huge. Which basically means that any time you call an LLM, the money that you have to pay for every call is going to increase. Also, LLMs have something called as a token limit also, which means that you cannot have like a hundred page prompt or you cannot have a thousand pages prompt, or let's say a one million word prompt or a ten million word prompt. You have to limit it at some point.

Now, let's say I want to upload a PDF and then I want to upload or I want to connect my AI LLM to my Slack channel and all the history of messages that are there in the Slack channel. So how can I do that? It's not possible because the history of messages is so huge. All of this will not fit in the context. So I will have to do something that instead of looking at the entire history of messages, there should be some way that whenever I ask a question, somebody should be able to only extract the relevant context from here, give it to the user and then the user should extract the relevant context from here, give it to LLM, and then the LLM will answer. So the context length is shorter. This is where RAG comes into picture. We will talk about RAG in more detail once we are implementing this. But this is like how the structure looks like. You have a CSV file you loaded, you break it into smaller chunks, you create embeddings and then you store it in a VectorDB. If somebody asks a question, you again create an embedding, you do a search in the VectorDB and then let's say I ask, when is the timeline for this JIRA ticket? So maybe there are like 4 Slack messages that match the criteria. So all those 4 messages will come and then those 4 messages will be sent to the LLM model which is GPT and GPT will see where is the answer in these 4 messages. So now you can imagine, instead of 1 billion Slack messages, I'm only passing 4 messages as context to my LLM and then the LLM is choosing 1 answer out of those 4 documents that have been selected. This is where RAG comes into picture. These are the different kinds of prompt engineering that we are going to cover today. Zero-shot, Few-shot, Chain of Thought and RAG.

Any questions anybody so far on this? Because after this we are going to go into coding. We are going to write some programs. We are going to play around with this. But any questions on this?

The question, where API should be stored? Is it in env.local? I think, Pratana, it can be env.local but I would say it can be inside.env file. So it can be inside.env. OK, let's go inside this file called 2.js inside folder called Using LLM with LangChain. And the branch that you have to shift to is lab1. Using LLM with LangChain.

OK, just for everybody's sake, the version that I'm on is this 18.17.0. If you have not done NPM install before, please do NPM install again. OK, going back to the repo, everybody is asking where to put the OpenAI key. This is where you have to put the OpenAI key. OpenAI-API-key. This is the, we can remove this not needed OpenAI key, inside.env file, which has to be in your root folder. OK. Now inside 2.js, what we are going to do is the very first exercise that we're doing is it's very simple.

Again, we're starting from the basics and then we'll move forward a little bit. The first exercise is how do you talk to chatgpt or any LLM via LangChain? So that is our first exercise today. So the very first thing that we will do is we have to import the chatgpt model, which is chat OpenAI from LangChain slash OpenAI. OK, this is where the package is for every model. So let's say tomorrow you want to do it for another model. You can say Anthroobic model or whatever model is here and you can load that. You want to load MixedTrail, you can load MixedTrail from LangChain at the rate of Anthroobic, so you can have something like this. We have all those packages here. OK. Till this point, everybody is clear, import chat OpenAI from this, then we need our.env package, which is basically where all our env files will be loaded and then we load all our environment variables here, everything that you see inside env file has to be loaded here. OK. Now what we're going to do is we are going to initialize our GPT object. So for that, what we are going to do is launch chat, new chat OpenAI. And it basically needs a key, which is API key. And that will be loaded from this environment variable that we defined here. So we go here and that's it. So till now, our model is created. So this we will say this is our model LLM. We have connected to our GPT models.

OK. Can everybody see your screen? I cannot. Geeta Saree is not able to see my screen. Can somebody confirm please if they're able to see my VS code? OK. Alex is able to see it. I can see I can see the right screen, OK. I think the rest of the people are able to see. Geeta, maybe try to, I don't know, just see if you're able to see it right now, or is it still a problem? OK, Sibra, Sadek, Lars, Kamal, everybody following so far. We have added the API key, we have connected to our GPT model here. And last but not the least, if we have to ask a question, what we are going to do is we are going to ask the question, which is, let's say, what is the capital of France or you can just say what is the capital of India or any country that you want to. And then we are going to invoke the model. Sorry, I have this, I have a co-pilot enabled, so it helps me overall type a lot of these things. But I'll still type all of this for you. We are going to do a wait. We are going to call this LLM. Every LLM is always invoked in chat in this thing. We are going to pass our question. And then I will see the response here in console.log response. OK, everybody with me so far on this. This is your very first exercise. How to connect to any LLM. You want to connect to Anthrophobic? You can do that too. You can connect. You want to connect to some other model. That's also possible. How to run this? I'll tell you. But just need confirmation if everybody else has also written it down. And please change your capital to something else and see how it works for you. Does anybody get wrong answers? Anybody gets easy answers? Incorrect answers. What are you getting? OK, another information here, you can change the model name also if you want to have, let's say GPT 3.5 turbo. You want to have GPT 4. Or do you want to have, I don't know, something else. I'll just keep it whatever default comes in. OK, now let's run this and see if the output comes properly. We'll go to the console. Or the name of the chapter is using LLM one two. Yes, basically, here, we run this and see what happens. I get some response here, which looks like this. I want all of you to focus here. The response is called as an AI message. So remember, I think this was the response. The question that I asked was what is the capital of India? It has given me the answer. The capital of India is New Delhi. But this is all this information. Obviously, you don't care and you don't want to see. So we will format or let's see. I think I would want all of you to try it out and just print the answer. I don't want to see. I want all of you to try it out. Just print the answer. I don't want to see all this message.

Just maybe print to the answer. OK, new chat, open AI. Think I'll open AI, not found. OK, let me help Daniel on that. Anybody else who was facing a similar problem? Works for me, works for Vance. OK, Alex. For Daniel, I think the only concern could be, let me see, WebStorm project, workshop modules, open AI chat models. I think you've not loaded a dotenv file. It's OK, Daniel. I think you have to just just check the code once here. If import dotenv from dotenv is there and you have done this config, it's an object. It's basically a function here. And the name of the key is this OpenAI underscore API key. Did everybody try the answer? I think to get the only answer, I think you will have to to get the only answer and not just all this content. You have to do response.content. And then we run this again here. And you see, we get the answer, the capital of India is New Delhi. So the very first exercise you've seen, you're able to connect to an LLM. Right now, we are connecting to ChargeGPT, which is GPT 3.5 turbo and all those models. Let's say you want to connect to some other model. We can do that, too. We'll connect to GPT 4. My key will work for all of you for GPT 4 also. So I think don't worry about it. Let's run this again. The capital of India is New Delhi. I think nothing has changed, but this is what it is. This is our first exercise. Now we move to the next exercise, which is basically, let me open that for all of you. The next chapter that we are going to do now. Let's go back to the presentation. And just coming back to the reference here. If anytime you are stuck and you want to see how to search for something, this is where the documentation is. Let's say you want to connect to a model called Cloud A3. How do you do that? Instead of using chat OpenAI, you will actually import from Lankchain Anthropopic and import this. And the rest of the things remain the same, exactly remain the same. You don't have to worry about anything. As an assignment, what I would want you to do is or try out in this example would be, I will create one file here. I would want you to try the same with Cloud A and see if it works. So try it out with Cloud A and see if you're able to work. As an example, like after this assignment is done, when you go home, try with some other models. You can try with Cloud A, you can try with group, actually not group. You can try with Mixtral. You can try with Lala Ma. So all these models, I would want you to try and see how the difference looks like, what model gives you a different response. And if they change the response, if it's something else.

Open AI keys, I've already shared in the chat about, I'll reshare it with all of you. But if you want to get a Cloud A key, Cloud A. anthropobic keys can be found from here, which is console.anthropopic.com. You can create your API keys there. And then, instead of using OpenAI key, you can use the anthropobic key. I would recommend that after this, the code is exactly the same. You have to and you can see how, which is a LAPTU structured output. I'm just committing this to my repo, so you won't have to worry about this. Okay. Now, let's switch to our LAPTU, which is structured output. Okay. Now, a lot of times this has happened, and a lot of people have asked me as well.

Open AI keys, I've already shared in the chat about, I'll reshare it with all of you. But if you want to get a Cloud A key, Cloud A. anthropobic keys can be found from here, which is console.anthropopic.com. You can create your API keys there. And then, instead of using OpenAI key, you can use the anthropobic key. I would recommend that after this, the code is exactly the same. You have to and you can see how, which is a LAPTU structured output. I'm just committing this to my repo, so you won't have to worry about this.

Okay. Now, let's switch to our LAPTU, which is structured output. Okay. Now, a lot of times this has happened, and a lot of people have asked me as well. That, let me turn this off, Cloud A. Let's come back here. A lot of people have asked me in the chat or when we are doing something that my output is always coming as, hey, the capital of India is New Delhi. But when I'm coding, when I'm programming, a lot of times I don't need this response. I need the response like a JSON structure. I need it like this. Let's say, for example, country, India and capital, New Delhi. I need it like this. A lot of people have asked this, how do I get the structured response from the LLM? The next assignment that we're doing right now, or the next thing that we're doing is, how do you get structured output from the LLM? Right now, you only get text output. The next assignment is, how do you get JSON output from the LLM? For that, we are going to switch to the repo and we are going to go to a branch called structured output. If you open 1.js, you will see that the initial setup is already done for you, which is dot envconfig is already loaded. Model is already there. And LLM is already created. I'll just call this LLM. Okay. Now, what I would want all of you to do is, I will write it down. But I would want you to try it out first. Try out and see if you can get a JSON output. And let me know how you did it. And then we can go in depth of it. Always think that you have to refer to the Langstron documentation to make anything work.

Okay, so try it out. Basically, what we want is, when I ask a question like, what is the capital of France? I want the output to be something like this, like a JSON which says country. Sorry, I want it to be like, country France and capital to be Paris. Okay, I want it like this. See if you can, see if you're able to do this. Okay, this is the JSON output that I need. The code is already structured for you here. You just have to figure out how to get this stuff set out. Okay, at any point, if you have any questions, please feel free to unmute yourself or write in the chat and I'll be happy to answer them. Anybody have any questions on this? Like you have to go inside a folder called a structured output inside that one.js file. And then you have to complete the code to get a JSON output here. I'll be waiting for like two to five minutes. Then we can continue. Was anybody able to, okay, read the documentation? And then I will quickly write the code also here. Okay, it will be something like this. For everybody who's following through, LLM is structured output. But if you see, this is actually asking for a schema. So we have to define a schema. For that, we will use a library called a Zod. So we'll say import Z from Zod. We will define the schema. We will say output schema is Z.object. And I say I need a country. So I'll say Z.string, describe the country. And then I'll say capital. Say it again, the capital of the country. And this is what my schema will look like. I'll pass it here. And then finally, what I'll do is I will do the same thing, which is ask my question, what is the capital of France? Wait for the response, which is LLM with JSON response. I'll take a look with the question. And then finally, we do a console.log response here. And then I will try to run this. Basically, structured output, 1.js. You see, I get the country as France and the capital as Paris. So this is what I was looking at. You can structure your prompt. You can structure.

You can define what kind of output you need from the LLM also. You can do an XML output also. You can do a JSON output also. You want to do something else with it. You can do a JSON output also. I'm just reiterating again. You can do a JSON output also. You can do a JSON output. And then finally, you can do a JSON output. That also, you can write your custom output parser also. You can say, I need the output format in a certain way. That can also be done. The simplest of the output is via JSON mode. This is what we have done with structured output. We gave it a JSON schema. And it came out in the JSON schema. You need to do a JSON response and invoke it with the question, and you get the answer.

Any questions on this so far, anybody? Can you show us, please, the schema again? Can you show us the schema? Yes. Ah, okay. Sod string. Ah, okay. Got it. Thank you. So now what I would want you to do is just to try it out. Again, a very simple exercise here that we have done this so far. I would want you to ask it a question like this. What is the capital of France? And also tell me the population. Okay. And I want the structured output to include the country capital and the population also. So please try that out once. Anybody stuck on this? Anybody who's not able to get the population out of it? Or is everybody was successful in getting the population also as an out? Norman, Sadik, Alex, Lars, Camel, anybody? Daniel, are you able to move forward? Are you stuck on that? Okay. We'll wait for two minutes for everybody to try out the different schema and play around with it and see they're getting the output or not. And then we'll move forward to the next chapter, which will be around prompt templates. Like here, I've been writing prompts like this. Where is my prompt? I'm writing prompt like this, what is the capital of France? And also tell me the population. But the prompt won't always look like this. I will not always ask the capital of France. This has to be an input variable. There has to be some chat history. There has to be some initial message that I have to give to Chatterjee and then ask the question. So now next chapter, we are going to talk about that, like how to have dynamic variables in the prompt, how to have some initial system messages, how to have some history in the messages so that it knows what we are talking about. Okay, another two minutes and then we go to the next chapter, everyone. Okay, Lars, Sibra, Sidhu, Alex also at work. So when you're saying it gives me different responses every time I read on the question. I think that's okay. I think that will happen because it should not give you like a country capital response different. I think that should always remain the same. Population-wise maybe it is, I don't know, hallucinating or doing some random calculations there, but that is possible. But country capital should always be correct in my understanding. Alex is saying, gave me the country population by default, change the Zord field description to specify I wanted the population of the capital and it works super. Okay, so I think now we will have understanding that the description of the Zord schema is also important for the LLM to understand what it needs to answer. I think that's a very good answer that Alex has mentioned that it gave default population and what he wanted the population of the capital. So just changing the description helped him, which is excellent. Eugene is asking how can we see what exact prompt was sent to the model? You can hear there's no prompt actually. The exact prompt, if you're asking, this is exactly what was sent, what is the capital of France and tell me the population. There's no magic happening here. Here there is no layer of abstraction. This is exactly the prompt that is passed. If you're asking about the prompt with the structured output and those things, I think for that, we'll have to integrate Langsmith, which can tell you the exact prompt, but I'll show you that later. Okay, we move to the next chapter now, but how Zordic is schema passed. I got it, I got your question Eugene later. How Zord schema was passed to it? I understand I can tell you what it is actually doing internally, which structured for you to actually see the prompt. I can tell you later, but what it is internally doing is that structured prompt is actually telling the LLM that, okay, give me an output and try to match the schema of this JSON. So it's basically just altering that prompt. So if you want to see that exact prompt, I'll tell you later how to see that. Okay, I hope I can share that later with you. Continuing next to prompt templates. Basically, when you don't want to do this capital of France all the time, you want to have those dynamic variables. So what we'll do is we'll stick to the same branch here, not move anywhere. And let me see where have we done this? Just one moment. Okay, we will go to our prompt. Another branch now, which is, let's already come back to the changes that we all have done. If you want to follow along to your depot as well, I'll come at my changes, push it to my lab example. And then we will switch to the third exercise, which is lab three prompt templates. Okay, everybody please switch to this branch, lab three prompt templates. I'll mention in the chat also. This is the third assignment that we are doing. Again, all of this are the basics of Linechain that we talked about. We understood how to connect to an LLM. We understood how to get structured output via a Zort schema. And now we are going to see how to update our prompt templates to be dynamic, to have variables in it, to have some messages streamed in it, to have system messages, AI messages and all those things. Okay. We'll go to this folder called as prompt template and we'll open 1.js. The basic setup is already done for you. Okay. Everybody switch to that branch. Anybody having any problems switching to that branch? We open 1.js and we start our first exercise.

In the first exercise, what we have to do is, very simple, what we have to do is, we have always been asking this question to everybody. Like, what is the capital of France? I don't want it like this. I want it to be like this. What is the capital of country? And this country has to be a dynamic variable. So this question is always there. Like, how do I do this? How do I make this dynamic?

So initial setup remains the same. What we are going to do is, we are going to first rename this to LLM and we are going to import this package called as import prompt template from Lanchain-core-prompts. Okay. And instead of making this question, what we are going to do is, we are going to create a new variable here called as from template. And we are going to say, from template, from template. So we are going to say, make it from a template. And we are going to pass it this string. What is the capital of France? Instead of saying capital of France, we will say capital of country. Okay. So this is how our prompt template will look like. From template, from template, what is the capital of country? And before we actually pass this to our LLM, we are going to invoke this prompt template with a variable, okay. Sorry, let me just turn on the lights here. So what we are going to do is, we are going to actually create the prompt and say, from template.invoke. And we are going to pass the country here, which is France. Or I can pass any country. Let's say I pass India here. And then I will see what the prompt will look like.

Now, what I'll do is, I'll just call my LLM and say await. LLM.invoke. So the await LLM.invoke with the prompt. And I'll see my response here. And I will run this one more time to see the output. And the output has come as the capital of India is Nudelny. So you see, very simple example, but you could see the fundamental that how do you replace, how do you dynamically inject variable into it? Okay. Sometimes you can have two variables or three variables. You can only invoke it by one also. You can do partial filling also. If it's just one variable and one invocation, maybe string interpolation is okay, good to have. But when you want to do partial, let's say you want to only fill the country and population is something that I will not fill right now. So I can do partial also. I can just fill the country and not the population. Okay? So that is also possible.

Now we'll go to the next part of it, the next prompt template, which is 2.js. We are going to do the same here. We have LLM, but now what I would want you to do is I'll show you what I'm saying. Let me open Chad GPT for all of you first."

So you see, very simple example, but you could see the fundamental that how do you replace, how do you dynamically inject variable into it? Okay.

Now, one of the questions could be that like this came to my mind also that why do I have to do all these things? Why can't I just like do like string interpolation, right? What is the capital of country? And maybe just take country as a variable somewhere on the top. So again, that is also possible. But what we are doing here is, this won't be possible for you to take input like these. These variables might come from anywhere else. So that is why invoking it like this makes more sense because this is not just one function. You can have prompt template dot partial also. You can fill in partial variables also. If it's just one variable and one invocation, maybe string interpolation is okay, good to have. But when you want to do partial, let's say you want to only fill the country and population is something that I will not fill right now. So I can do partial also. I can just fill the country and not the population. Okay? So that is also possible.

We'll see the prompt here. I'll remove all of this and see what happens. We saw partial prompt. We see input variables for country and population. This was the template, country population. And the only variable that was filled was country India. What is the capital of country and population? Country is filled, but the population is not filled. So if you try to run it now, it will give you an error. So we can do partial templates also. Any questions on this anybody? So far clear. Let me see if anybody has any questions by any chance on this. Okay, prompt templates, no question.

Okay, prompt templates, no question. Now we'll go to the next part of it, the next prompt template, which is 2.js. We are going to do the same here. We have LLM, but now what I would want you to do is I'll show you what I'm saying. Let me open Chad GPT for all of you first. So, do you see Chad GPT on my machine? Here, this is basically a place with all of looks like, but in the playground, you'll see that sometimes or anytime you're talking, it can have some initial instructions and it can have, then the questions. For example, the initial instruction could be, answer, answer all questions. Okay, or imagine you are a Pokemon character, answer all questions as that character. Okay, so I can set some initial prompt, initial system prompt, and then I can ask any questions. So now I will ask the question, I will say, hey, what is your name? You see, are you getting the point? It's saying, it's considering Pika-Pika, hello, I'm Pikachu, nice to meet you. So how do I do this? Like here, there is a UI I know I can set some system prompt, I can send some user message, or then there is an assistant. Another thing you should notice here is, the initial prompt was called system. The user message was called user, and the reply was called assistant. The same terminology applies to prompt templates and messages in Lankchain. There is a concept of system message, there is a concept of human message, and then there is a concept of AI message, which is what we are going to do right now. We are going to do the same thing in this prompt templates. I want all of you to try. First, give a system message, Pika, so in the first example, you saw we only did a prompt template, from template, what is the capital? But now I would want you to build it so that you have some system message and a human message. I'll give you some. Easy. And then, okay, and it will content, and then prompt would be something like this, like, you put again, import chat, prompt template from code prompts, and you'll say, basically. So this is one of the examples of how you can do it. Is the screen visible to everybody? Okay. And I'm hoping it's visible to everybody. Okay. So you can ask the content here, whatever, and then what we'll do is we'll do the same thing, chat from template. Instead of from template, we'll do messages here and pass in the messages. Rest I will want you to try it out. This I've completed for you, from template. Please try the system message and the human message. Human message can also be given like this, like this, okay? Please try it. I give you five minutes to play around with this and see what I'm talking about. Your system message should be something like this, which I tried. Imagine you're a Pokemon character and answer it like this. Oh, sorry, somebody is saying that the voice connection is lagging from time to time. Okay, so sorry, Daniel. I think there could be an internet issue. Please tell me if you want me to repeat something, any part that you guys were not clear I can repeat. Is it audible now? Is it better now? It's better, okay. Okay, thank you, Daniel. Thank you, Alex. Okay, I've pasted the system prompt for all of you. Imagine you're a Pokemon character, answer all questions as that character. Please modify the prompt template and the message and try it out if you get the LLM response as a Pokemon character or not. Please tell me if that works for you. Okay, I give you like maybe three, four minutes to try it out and then we continue. Sorry, is anybody stuck on anything, any questions that they want to ask on this, any part that is not clear so far? And please feel free to message if the pace is too fast or too slow for you. We can speed up, we can slow down as well. Yeah, I'll stop sharing for two minutes and then we'll continue while you guys are practicing. I think Alex has replied something. It works now. I guess it worked, I got this answer. I'm a Pokemon character so I do not know much about human cities and capitals. My focus is on battling other Pokemon and mastering my unique abilities to become stronger. Okay, nice, I think it still worked like it was. Able to imagine itself as a Pokemon character and then answer accordingly. That's good. How about the others? Did you get any interesting answers? Did you get anything that was different? I don't know. Okay, I'll just wait another minute and then we'll go to the next chapter. Okay. I think everybody else is still doing it. Good, so we'll move to the next section now. Let me reshare my screen. This is now moving towards a more RAC sort of setup where you will have a huge document file that you would want to be able to, basically, if you want to query your custom data sources, like you want to query a PDF file, maybe you're managing your finances in let's say some, in some Excel file and you want to load that document and query that document somehow. We are going to learn that in this next chapter. Please switch to the branch called as lab4loaddocs. Okay, if you haven't, please take a pull off all the branches again and you have to switch to this branch. Take a git pull again, please, and then switch to this branch. After you switch to this branch, we are going to go inside basically 1.js file inside loading docs folder, okay? Okay, so now you can see Okay, so now in this example, what I'm going to talk about is more about the RAC flow. In the RAC flow, in the diagram, if you see, we saw this that whenever there is a CSV file or a PDF file, there are multiple operations that you have to do.

One of the operation is you have to first load that file into the system. Into the system means you have to load it somehow and then give that entire file as a context to the LLM. How are you going to do that? It's a huge file. It's like 20 pages, 50 pages, 100 pages, you don't know that. So for you to solve that, the terminology is basically RAC, which is retrieval augmented generation.

So you take a CSV file or a PDF file, you load it basically via some document loader. Now, Langchain provides a lot of document loader. Like, for example, if I see here, PDF loader, okay? So it provides this loader, which is for loading PDF documents. Similarly, there will be a CSV loader. Similarly, there will be a Word loader, Excel loader. So it provides all those loaders for you. So first we will use a document loader to load a file. Then, because the file is huge, right? We cannot just use it altogether. So we have to break it down into smaller chunks. Why do we have to break it down into smaller chunks? Because let's say you have a 20 page PDF, when you ask a question to a 20 page PDF, you also refer to it as chunks, right? You have like first 10 pages, you just scroll through, then the next 20 pages you search, then the next 30 pages you search. So that is why we are trying to break it down into smaller chunks. For simplicity, let's say that every page is a chunk. Chunks can be bigger also. You can say 10 pages is one chunk, 100 pages is one chunk, 1,000 pages is one chunk, you can decide that. But to reduce the context length, we break it down into smaller chunks for easier understanding also. And then finally, what we are going to do in this exercise is we are going to create embeddings. I will explain what are embeddings and how to load a file, how to create chunks, all of that to you, okay? So first, if you see, again, going to the documentation, how do you load a document? They're saying that you need to install something called as PDF files. It's already installed in your repo, so you don't have to worry. You just have to import this loader, pdfloader. So we are going to copy it from here. And we are going to go here and we'll say import pdfloader from Lanchain, community, document loaders, fsmpdf. In your repos, I've already created a document here which is docs, budget, speech pdf. If you open it, you will see what this is about. This is basically the budget of India 2023-24. It is a huge PDF. If you see, it's basically 58 page PDF. You can have any PDF you want. If you have a budget of your country somewhere, if you have a personal PDF document that you're saving somewhere, use that document. Feel free to use any PDF document that you want to. Now for me personally, the problem was when the budget comes out, I don't want to read everything. I don't want to go through the entire document. I just want to ask chatGPT that, hey, take this budget and tell me what does the budget allocated for railways? What does the budget allocated for agriculture? What does the budget allocated for, let's say, education in India or higher education. What does the budget allocated for automobile industry? I cannot just read through the entire document. I just put it in chatGPT and ask it question. And this is what I want to show you how to do this with LLMs and Lanchain. So far everybody with me on this, right? No questions that I'm assuming. I think everybody is able to understand their switch to load docs as well. We are going to write the code for it now. We go to loading docs. We have already imported the PDF loader. We are going to initialize the loader here. We'll say new PDF loader, and we are going to pass it the path file, the name, the file of the path. Sorry, the path of the file is basically inside. Let me get the folder. Where is it? Okay, so basically inside this, I'll just copy paste this entire path from here. I will go to the PDF loader. I'll paste it here. And the name of my file is budget underscore speech.pdf. Okay, and then what I'll do is, I'll say, has the doc loaded? I'll say await loader.load. And then let me see if it's loaded or not. So I'll do a console.log here. Okay, I want everybody to also try this. Any PDF file that you have, please load it. Or you can just rely whatever I've put in here, which is budget speech. And I will try to run this and show you how the loaded document looks like. I had to do nothing. I had to not do a PDF parsing. I don't have to write my own PDF loader. I don't have to save it somewhere, whatever. Just line chain, loader, done. CSV loader, done. Excel file loader, done. You don't have to worry about anything. So I do this node. My repo is basically loading docs 1.js.

It is saying loading docs 1.js, cannot find the module. Let me see what have I named this. Loading docs 1.js. Why did it not find this? Somebody's dropped me a message. CD- oh, yes, sorry. Thank you so much. CD dot dot. Okay. So you can see that the entire file is loaded here and you can see how it looks also. Document, page content, changes in central exercise and so it's basically loaded the entire document for me. Even now, I can't do anything with this. It's just the document that is loaded for me.

Okay. But try to load your own documents and see what happens. If you are able to load your PDF files or your Excel files or your CSV files. Here you can always see there is a PDF loader. You can see there will be a CSV loader also. For that, you have to install this and then you have to import the CSV loader. Everything else remains the same. We have reached to this point. We have loaded the file. Next thing we have to do is split it into smaller chunks.

Okay, I'll wait for everybody to try to load their files and see if it works for them. Anybody facing any problem? Was everybody able to load a PDF file, guys? Everybody in the group, whoever is following through this workshop. Are you trying PDF files? Are you trying Excel files, CSV files? What files are you all trying? Okay, yes, I think yes. Loading was successful. Yes, got it now. Okay, perfect. Now we move to the next section, which is how do you chunk it, which is basically here for that. Again, we go to LimeChain documentation and chunking or breaking it down into smaller chunks is basically called as text splitter. Okay, so what we are going to use is something like this, which is a recursive character text splitter and it is going to split my entire PDF. So you can see one example here. It is recursive text splitter. What it does it is always splits by new line character or double this character single space or basically no space. So you see, when I do this splitter, it's actually creating high as a separate file, IM as a separate file and Harrison as a separate file. So what I am going to do right now is I am going to import this text splitter for splitting my document into multiple chunks as we discussed here.

So we will go into this folder which is chunking docs and go to one.js here. We'll do the same thing. We'll copy paste this whole logic of loading the doc here. And now to chunk it, we will import this from here. We will initialize this. We will say splitter. We will initialize this and say splitter. So it will be a new recursive characters splitter and we are going to give it a chunk size of let's say 1000 and a chunk overlap of 10. Okay, the reason we are doing this is chunk size of 1000 would mean that every chunk will have, basically if you have like a 30 page or a 50 page PDF, you want to create smaller chunks of sizes, let's say 1000 tokens or 2000 tokens. And the overlap means that two chunks can have the same content like let's say in the budget example, the railway budget in the second page and the first page can overlap with each other. So it can have some overlap. And finally, what I'm going to do is I'm going to create chunks, which is splitter, whatever I've created splitter.split documents and I will pass it this documents that I loaded. And finally, I'm going to see my chunks here. I'm not going to see all the chunks. I'll just say chunks.slice 0 comma 3 only show me three chunks. This will be a weight and we will run this file now and see what happens in the terminal. Chunking docs, one dot JS and you can see I've got three chunks here. You see my first chunk is the government of India. This is the content page, speech of nirmala minister, February. It's also showing me where is it coming from sources here and the line also so you can get where is the exact line it is coming from. Then the second page is this. Third chunk is amendments relating to indirect taxes. So these three basically, but it has loaded all the documents and it has chunked it into thousand tokens, chunks and a chunk overlap. Everybody, please try this with your chunks and see if you're able to break it down. Also, I think Alex, Lars, people who are with Eugene, everybody who's on the call, do you want me to speed up the pace or the pace is slow, fast, please let me know in the comments so that we can move accordingly so that you all can benefit from it. How's the pace, please let me know slow, fast, what do you want me to do? Take it slow or take it faster. I am fine with speeding it up. Okay, Eugene, thank you. How about others? How about Alex and everybody else? I think it's okay. Alex is saying he's okay with the pace that it's going so far. All right, pace is perfect. Okay, I'm okay with the pace. Okay, perfect.

Okay, perfect. I think that we'll stick to the space and we'll continue like this. I'll wait for the others to try to split their documents into smaller chunks and see if it works for them or not. Once you're done, please let me know in the comments.

Okay, we'll wait for everybody to be done. I'm okay with the pace, what should it look like? So basically, when you load all your documents, camel the chunks, you can do like this. You can just load three chunks and the chunks should look something like this. You should see a document object with page content and some metadata. And you should see like this page content. You should basically see three chunks when you're done. And it should have some content inside it. Okay, while you guys are trying, I'll just wait for two more minutes. Stop sharing. Oh, Alex has said working for me for both PDF and CSV, super. I think that's great.

So now you can, I don't know, if you have your own CSV files with financial data or your budgeting data, you can just upload it on your own LinkedIn and ask it to, I don't know, ask it to play around with it, ask some questions that you want. Okay, we'll move forward now. I think working for me, working for PDF, everybody, I think it's working for them. Next step, we will go to now, going back to the diagram here. Next step we are going to do is, after chunking, we are going to create embeddings. Now, embeddings is sometimes a complicated topic for everybody, like what does embedding usually mean? So I would want to break it down into simpler terms that whenever you have structured, how do I say it? Okay, so embeddings are basically, you want to represent your data in some way, like when I am, when usually we are dealing with data structures, if I have to represent a class person, how do I represent it? I basically create a blueprint class person. I say it has name, age, gender, height, weight, or whatever, I give it these characteristics, and then I save it in a database called as a MySQL database, which is usually a relational database, if my relations, if my structure is more particular, if I know the structures. If it does not have any structure, I usually use a NoSQL database, like MongoDB or something else, for example. But in the AI world, in the LLM world, this data is not structured data, right? It's unstructured data. It is natural language data. Anybody's asking anything, any questions is coming.

The PDF file is broken into smaller chunks. It's a huge file. So what you do is every chunk that is created or every context or every question that comes, you want to represent it via some identifier. That identifier usually is an embedding, and in very simple words, if you know something about vectors or anything about the XY coordinate, an embedding, in very simple terms, is basically XYZ coordinates. So if you have a plane with X, Y, and Z coordinates, every chunk of my PDF occupies a space in that coordinate. So if I want to find PDF number two, I will say that, give me a zero comma one comma two coordinate, and that PDF will come out. If you have seen Google Maps geolocation, right, it uses latitude and longitude. That are also not exactly embedding, but that are also coordinates in a way to find a particular place. But those are just XY coordinate line, latitude and longitude. Imagine you have a third coordinate also. You have a third dimension, which is the Z dimension, XYZ. And in AI terms, these dimensions are huge. It could be 2000 dimensions. It could be 3000 dimensions. So embeddings are basically X, Y, Z, Z1, Z2, Z3, Z4, Y1, Y2, so those many coordinates, but they identify a particular chunk or a particular data in the vector space. So this is what we are going to do. Out of all these chunks, we are going to create those XYZ coordinates. And I'll show you how that looks like.

Going back to the documentation, we are going to use embeddings of OpenAI. So we are going to go here, we are going to say text embedding models, and it says we have to import this OpenAI embeddings. So that's exactly what we are going to do. Let's go back to our repo. I've already imported Open and we have to go to this folder now, create embeddings 1.js. Everybody with me so far, create embeddings 1.js. We do OpenAI embeddings.envconfig remains the same. What we are going to do is, whatever documents we created, we are going to create an embedding of that. But first I will show you how embedding looks like, okay? So first I'm going to initialize this embedding. I'm going to say embedding is equal to new OpenAI embedding. I'm going to have an input here, which I will say, let's say what is capital of France, again, the same question. Let me make this capital. And then what I'm going to do is, I'm going to see the result of this embedding, how it looks like. I'm going to say, embedding.embedQuery. And I'm going to embed this query that I have here. And then I'm going to see what is this entire sentence will look like in a vector space of x, y, z, y1 coordinates. So then I'm going to say console.log result.

But first I will show you how embedding looks like, okay? So first I'm going to initialize this embedding. I'm going to say embedding is equal to new OpenAI embedding. I'm going to have an input here, which I will say, let's say what is capital of France, again, the same question. Let me make this capital. And then what I'm going to do is, I'm going to see the result of this embedding, how it looks like. I'm going to say, embedding.embedQuery. And I'm going to embed this query that I have here. And then I'm going to see what is this entire sentence will look like in a vector space of x, y, z, y1 coordinates. So then I'm going to say console.log result. And then I'm going to go to my terminal and try it out. Create embedding, 1.js. It's a promise pending. I always forget this, all of this is always an await statement.

Okay. Okay, I did some error here. Embedding.embedQuery. We run this. And you see, this is how it looks like. I was giving you a more simplistic example that it is only x, y, z coordinate, but you see so many coordinates here. One, four, three, six. I think it's around 2,048 or whatever coordinates are there. But this is the embedding or a vector representation of that message that we sent query. What is the capital of France? It is stored somewhere in a vector space in one coordinate with the message of what is the capital of France. Okay, so now you can imagine when you store these many contexts, like the capital of France is Paris. And the capital of India is New Delhi. So they are all coordinates in some vector space. And when you try to ask a question, whichever is nearer, whichever coordinate is nearer to your question. So let's say I say the capital of India is New Delhi. And then I ask a question, what is the capital of India? So this question will have its own embedding also, and that embedding will be near to the capital of India is New Delhi. And when I search for it, it will be the nearest vector to this embedding and this answer will come. So this is how embeddings work. They're near to each other. So when I embed milk, you will see milk, coffee, tea, sugar, all these words as embeddings will be very near to each other because they are near in the vector space.

Okay, so what we did here is we actually embedded a query, but we can embed the entire documents also. So now let's embed documents. Create embedding for documents. How are we going to do that? Result one, we are going to say await embedding.create, sorry, embedding.embedDocuments, and we are going to pass it all the documents that we have, let's say, for example, we are going to say the capital of France is Paris. The capital of India is New Delhi. And that we are going to see the embedding result of these. So you can embed a query, you can embed a document, which basically means any question that you ask to chatGPT, that is a query converted into a vector embedding. Then a search happens to all the vector embeddings of all the data that it has. The nearest comes out and that is given to you as an answer. So let's see the output here. What does the embedding look like now? You see the embedding is even bigger and for two documents, I can see the embedding.

Okay, now you have seen this. What I would want you to do is create a two.js file here, copy the same thing into this. Instead of embedding this hard-coded data and this hard-coded document, I want you to embed whatever you created in this loading docs here, whatever docs you loaded. So basically all this that you have already done, go to embedding, okay, whatever docs you have loaded, I would want you to load it here for me. Basically pass it here and try this if it works for you. Create embeddings for the PDF or the CSV file that you just loaded. Okay, please try it and I'll stop sharing for two minutes and you guys can try and tell me if it works for you. Okay, everyone. Anybody stuck on anything? Let's see the message working for me. Okay, it works for me. I'm okay with this. I'll try to get the output from splitter dot split documents and use it as input for embedding dot embed document, but it didn't work. Did I misunderstand the task? No, I think it's correct. I tried to get the output from splitter dot split documents and use it as an input for embedding dot embed document. That is great. Can you show me the example how you tried it, Alex? I got this error, t dot replace is not a function. Embed documents chunks, I'll get this error, same error. Okay, let me see this. What is this error, quickly for all of you. Okay, okay, don't worry. Just one moment. We'll figure this out. Just give me a minute, I'm checking. What is the problem here? It is an array, okay. Okay, you guys have already done the chunking also. Let me also see where the chunking has been done. Splitter dot create documents, put it inside this and pass the chunks to embedding. And then try to run this. Sorry, I'll share my screen in a minute. Just trying to run the same thing that you guys are doing. See what problem you're facing. Yes, I think Alex could be one of the potential reasons. But what we'll do is, I think we can figure this out. I think I'll also have to check what is the exact error that is happening. But let's move to the next part because embed query will automatically happen once we switch to the vector store part of things.

So let's move there and we can come back to this or I can help you later on this, why this is not working, okay. It's failing for me also, but everybody is safe. So you're not alone in this. It's failing for me as well. But we'll switch, we'll move to the next assignment and we'll try if that solves some of our problems. Okay, let me share my screen again with all of you. Okay, coming back to this, I would want you all to take a good pull again and go back to this branch called as Lab 5 Vector Stores. Okay, Alex has already figured out embedding chunks.map c.cpage content, super. So I think for everybody else, Alex has already shared the solution in the chat, I think that is super. And I'm very happy that I think the best part is that for me personally, it's a win that you're able to figure things out on your own, which is great. And this means that you have understood until this far you've understood how to do this, which is perfect. Thank you, Alex. I think this is great. For everybody else, please follow what Alex has written in the chat, I think it'll work for you. We'll move to the next branch, which is, and so sorry, we are trying, we are running a bit on time. So we'll have to a little bit speed, increase the pace, but don't worry, I'll still not be too quick on this. Let's switch to this branch, which is Lab 5 Vector Stores. Okay, and what we are going to do now here is you've seen the diagram we created an embedding, we understood how to create an embedding, but we have not yet understood how to store it in a VectorDB. And the next question that comes is if somebody asks me a question, a query, how will I create an embedding again? How will I get that embedding? How will I do a semantic search of it from the Vector database? This is where we are going next. And you have to go inside this repo, Lab 5 Vector Stores. And inside that, we are going to go in the folder called as Vector Stores here, 1.js. Okay, everybody so far with me on this, I'm assuming we are going to go to Vector Stores. And what we are going to do is everything is installed.

And what we are going to do is everything is installed. You don't have to worry, we are going to use a Vector Store. There are so many Vector Stores. I can show you here in the repo. Let's say what are the Vector Stores that are available? You have basically, let's go here and see what are the Vector Stores. You can see the Vector Stores that are available is there are Memory Vector Stores that is available. You have Analytics DB, you have Astra DB, you have Chroma, you have Clickhouse, you have Elasticsearch also, you have FICE. So because we are doing Node.js, we are going to use something called as the FICE Vector DB. I've already installed this in your repos. The only thing you have to do is import this from here. So we are going to import it from here. And now what we are going to do is we are going to create a Vector Store.

Okay, we are going to create a Vector Store. We are going to say Vector Store from text. It can be from documents also. We are going to create it from text for now. And we are going to pass it some details like how we did for that, the capital of France is Paris. The capital of India is New Delhi. And maybe the capital of Germany is Berlin. For example, these three texts I have passed. And now imagine that this basically would eventually become an embedded thing. So I will do this, FICE Store from text, I'm going to pass this text here. I'm going to give it some metadata here, which is basically I'll say that, okay, your ID is one. Next ID will be two or three or whatever. And then I'll say the ID is two. This is just a metadata to identify this document is what. So you saw when we chunked the documents, every document has an identifier. Like this is document number one. This is document number two. This is document number three. So what we are going to do is we are giving the same identifiers here as well. We're going to give them the same identifiers here. And what we are going to do is here in the FICE Store, you remember how we were creating embeddings via embedding.embeddocuments or embedquery. Here what we have to do is we just have to pass the OpenAI embedding to this. And FICE.fromText function will automatically do that. So if you go inside from text, you will see it will take up metadata. It will take embedding and it will automatically create embedding for you and save it in the FICE Store also. So that is why what we are going to do is we are going to go here and we are going to pass it like this and this will automatically save it for us. It will create a vector store from this metadata and new OpenAI embeddings, and this will work for us here. We are going to save it like this. This works for us. And now when I want to ask a query, what will be my query look like? My query will be, I'll say just trans capital. And it will automatically search for that response for me. So how it will do? I'll do vector store.similaritySearch. I will do await vector store.similaritySearch. How am I doing this? Yeah. Vector store.similaritySearch. And I will pass this query that give me whatever, how many documents come close to my question? So here you see only one document come close to it. So I'll say, okay, vector store.similaritySearch and then give me the response. So now we'll run this and see what answer we get. We'll say node vector store 1.js, we're getting some error that is saying similarity search is not a function. We'll see what is the problem. By store.fromText, we have to do an await here. And then we can maybe run it again and see. And you see I got all the documents basically. It was not like similaritySearch basically did that. Okay. Whatever is similar, give me that. So in terms of similarity, my question was capital of, I did not say anything. I said, France capital. If I run it again, you'll see I get all this here. Capital of France is Paris, capital Germany is Berlin, capital of India is New Delhi. But you have to understand that these are ordered in the terms of whatever is more prior. So if I say that I only need one answer, then it will give me the answer here. The capital of France is Paris. So if you don't specify how many documents, so vector similaritySearch is not doing anything. SimilaritySearch is saying that, okay, whoever is similar, get me that answer. So everybody was similar, like capital of France, capital of New Delhi, everyone was similar. So we got all four documents. Now, in terms of priority, the capital of France is Paris, so it came as first.

But this is also not a foolproof solution. You will get, similarly, when you ask a question, you will get hundred such documents, but you won't get thousand, you won't get 10,000. You will shorten your search to 10 documents, and then you will give it to an LLM and say that, okay, I have this four documents. Now tell me the answer from it. So vectorSearch.similaritySearch is doing that. Was everybody able to run this example? Any questions on this?

Very similar to this, what we are going to do is we are going to go to here. We are going to whatever PDF document that we created. We created an embedding. We are now going to save that in our vector store. Somebody is asking, can it return similarity score? Similarity scores, we can, sorry, am I ordering? One slide only. Okay, so I think to answer Eugene's question, that depends on what vector store we are using. For example, if you use a Pinecone sort of vector store, Pinecone as a vector store also will give you the similarity score. So you can see that the similarity score of the capital of France is Paris, will be higher as compared to the capital of India or the capital of Germany will be lower. So Pinecone, Chroma DB, all of these things will give you the similarity score. In FI store, maybe it's not giving the similarity score overall.

Now we saw that we just did some embedding and storing of data into vector DB or some random text. But really my problem was that whatever embedding I have created, that all of that has to store into the vector DB. So that is something that PDF to vector store here. And we are going to write the same logic. Document is already loaded. I will copy paste my chunking logic from my chunking docs here. I'll get my splitter docs here. I'll take that logic, I'll paste it here. And then what I'm going to do is I'm going to initialize my vector store. Save like before. Vector store, I'll actually just copy paste from there only. That vector store gave me the entire thing. I will copy it here. And now instead of this random text and from text, I'll say from documents. And instead of having this random text, this will be my chunks here. I don't have to specify the metadata because you already saw when we created the chunks, the metadata was already there. I just have to give the embedding because I'm using OpenAI. So now I'm not going to do embeddings.createEmbedding because this file store from documents will already do that for me. And now the only thing that I have to do is const query. I'll say, what is the capital of France? Here I'll say... I'll say, but my chunks, this I'll say, what is and then vector.similaritySearch. And my query is this, and then I'll console.log the response. Somebody has a question here, let me see. Connection is... Yeah, Lars, I saw the connection, there was some connection error. I think it's back now, but what I was trying to do here in the PDF example was now you saw in the previous example, I created an embedding of random text. Now I'm actually creating an embedding of all the PDF that I loaded. So every other logic is the same. Load the PDF, split the PDF, and now from file store instead of from text, I'm doing from documents, passing all my chunks, new OpenAI embeddings, and then asking the question, what is the budget allocated for railways? And then I'm doing vector.store.similaritySearch and see the response. So now if I run this again, node pdf2vector1.js, it's saying recursive because I've not imported that. Let me import recursive text editor from here. Sorry, I think the meeting time expired, so we have to rejoin. Everybody else can continue staying on the call, I think the call has started again. We are almost reaching the end of this workshop. After this, the only thing that will remain is we'll talk about how to directly run this without having to do all this vector store and all those things. Let me re-share my screen again with all of you. Okay, so I did run this node vector.store. You see it has given me some response, Railways 50, a capital overlay, one document, two document, three document, four document. So there are so many documents, that I don't know which one is the correct answer right now because similarity score could be different. So I just say give me the top document by passing one, again doing the same thing. Now I'll only get one document. I don't need multiple documents, just give me one document. Here Railways 2.40 lakhs. So I can see somewhere the answer has come, a capital outlay of 2.40 lakh crore has been provided for the railway. The highest ever outlay is about nine times. But here you can see that vector store is giving me the similarity based on what it is able to understand, but this might not be the perfect answer, which is why what we have to do is once I get all the documents, so vector store should give me all the documents which match the similarity. So it might give me four documents or five documents. Then I will take those documents and I will pass it to an LLM like chat GPT, GPT 4, GPT 3.5 and tell the LLM to give me the right answer because LLMs are smarter, they are the brains. What I am doing is I'm just reducing the context length to four or five documents and then telling LLM to give me the answer. So the similarity score is just one way of getting an answer, but it might not always be perfect because similarity score can be wrong also. It can give you like four or five documents also. It might not always give you one document. Here I'm only saying give me one document, but in an ideal scenario, if you see it actually gave me around four documents which match the railway budget because the budget allocated for railways might be present in four or five documents. You see here one document, two document, three document, four.

You see here one document, two document, three document, four. Now who will tell me the right answer? This is where an LLM comes into picture, which will give me the right answer.

Okay, before we move on to that section, I would want all of you to try this Vector Store thing and see if it works for you. Okay, I'm keeping the example in front of all of you what I have done. The only changes instead of from text, we are doing from documents, passing our chunks, new OpenAI embeddings, and then asking the question. And with similarity search, trying to find whichever is similar.

Lars, Daniel, Camille, Alex, okay. Alex is saying working fine here, okay. How about the others? Camille is also saying working fine for her. Oh, you guys have already tried the similarity search, that is great. Okay, Daniel also saying working fine for him.

Perfect, so the last section that we will go here will be on Retriever and LLM. You understood till this part that we could save it in the database. We could also do a semantic search retrieval from VectorDB. You saw we got multiple matches.

Now, the only thing that remains is whatever matches have come, they might not be the perfect answer and they might not be the answer that I expect. Like when I say what is the budget allocated for railways, I don't expect this gibberish three, four documents to come back to me. I want an answer that the budget allocated for railways is $2 million or $3 million or $4 million. I can't read through this, so that's where I will take all these documents.

I will pass it to an LLM model, which will format it to one answer that I expect and finally give it back to the user. This will be the final section of our workshop, which will complete the entire RAG app that we have been talking about using Lanchain. The final section of this would be to go to this repo, which is Retriever and LLM. The solution is already here, but what we will do is we will understand part of it and we'll try to write it on our own. So I'll create a 2.js file here and we will write all of this on our own. This will be the last section and after this we will end.

So I think everything else will remain the same. We will copy paste our example from here till this point, similarity search. We'll go here, we'll paste it here, but except this query and the response part, we will not do it here. We'll just keep it till the vector store that we have created and saved every answer. Please guys do it till this point and then we'll go to the last part. Let me see. So now what we're going to do is we have, just walking through, we have loaded the PDF. We chunked the PDF into smaller chunks. We created the embeddings and we stored it in the vector store. Now, how do we connect it to a LLM model like ChadGPT or something like this?

So first, let's create our LLM model here. What will be our LLM model? It will be OpenAI and we will basically pass it our API key, which is process.env. What was the OpenAI key? It is.open.env. And then, C. OpenAI. If. Now, if you get open links. In Powl LLM, the chain.

Create a chain and it will search all those 100 documents. A bit of internet trouble in the last moments that are happening. But we are almost, please stay with me. We are almost at the last stage. We are at the last assignment of the workshop. And with this, you will be with all of you. The one thing that we have to do now is we have created the LLM here. Now we have to somehow create a chain, okay? For that. And we will. We have to create the chain and we have to import our open AI team also here, which is open AI from Launch and Open AI. And then we are going to import our chain here, which is going to search through all the documents, 100 documents. And we'll get the 10 or four similar matching documents out of it.

So for that, it will ask create stuff document chain from Launch and Open chain, combined documents. So what we are going to do is we are going to create this chain. And here we are going to say create stuff document chain. And what it basically needs, it needs an LLM and a prompt because this was the chain, which will search through all 100 documents and then pass it to an LLM. Okay? So it needs an LLM. So we have created an LLM here, we'll pass it that. And it needs a prompt also. I have not created a prompt. So let me create a prompt here first. So my prompt will have something called as a system template. Or quickly let me just copy paste it from here. This prompt here, and I will paste it here. My prompt template will look like this, answer the user's question based on the below context. If the context does not have relevant information to the question, don't make something up and just say, I don't know. Context will be provided like this. And I have created my prompt template like this from messages system template. And the user input will come here. So I will ask the prompt template here to my document chain. This chain is basically what it will do is it will take an LLM, it will take the prompt, and this chain will actually take all the documents and search for the, like the smaller chunks out of it. It will not take all the hundred documents, it will only take 10 documents and pass it to the LLM to find the answer.

And finally, to get the answer, I will create another chain here, which will be the retrieval chain. And I will import it on top here. This will be import, create retrieval chain from here, and I will create a retrieval chain. And what this takes is, this basically takes a retriever, it takes the combined docs chain and it will take an input. And how do I create a retriever? My retriever is basically a vector store. So I'll say vector store as retriever. And I will pass the retriever here.

Basically, I will say, a retriever is already there and it needs basically combined docs chain. So combined docs chain is basically this document chain. And this is how I'll create my retriever. So I created a retriever here that whatever is stored in the vector database, who's going to retrieve it. So this retriever is going to retrieve all that is stored in my vector store here, whenever a question is asked, and then that will be passed to this combined docs chain, which will actually pass everything that is searched based on similarity score here in the context. And finally, my retrieval QA chain will actually call the LLM and get the answer.

So now I will ask my query here, and I will say, what is the budget allocated for railways? And this will be my query. I will wait for the response. I'll invoke this retrieval chain, retrievalchain.invoke. Invoke. And I will finally pass in the input message here, which I have to pass, because I've specified that user will come inside input. So I'll pass this query here, and then I will wait for the response. This will be await. This will also be await. And finally, I think we are done. And we'll run this. I know we are going a little fast on this, but I'll explain all of this to you. One moment, let's run this. Chat prompt template is not defined. Let's import that also. Import chat prompt template. Now it should be okay. Okay, we did not get any answer here. We got all the documents that have to come. Let's see, what are we doing wrong? Combined docs chain, input has to be passed. Let's see, what did we do wrong? We loaded the documents, we got the chunks, we created vector store, we created an LLM also here. Where are we going wrong? Maybe I have created a system template. Question answering prompt is also there. Document chain is also created with this prompt. Retrieval is created. Great retrieval chain is also created. And retrieval is also passed. Combined docs chain has this document chain. Okay, I think we missed an await here. Then if we run this, we got the answer here. A capital outlay of rupees 2.40 lakh crore has been provided for the railways. What we are going to do finally is that because we only care about the answer, we'll say response.answer. And finally, we'll see the answer here. The budget allocates a capital outlay of 2.40 lakh crore for the railway, which is the highest ever outlay and about nine times the outlay made in 2023 to 14.

Coming back here to explain what happened in this entire scenario. We all are aware we first loaded our PDF file. So we loaded our PDF file. We created chunks here. Okay. Then we created embedding and stored in vector DB. Then we, sorry, let me just put it on top. Then we created a retriever to fetch data from vector DB. Then we created LLM to answer all the questions. This is the prompt that we created. Finally, we created a chain to load all docs and pass to LLM. This is what we did. And here we created a chain to fetch the data from vector DB and then pass to doc chain, which will eventually pass it to the LLM. So this is basically fetching all the data from the vector DB. When I ask what is the budget allocated for railways, it will actually go to the retriever. The retrieval will go to the vector DB and fetch all the similarities that are matching the vector DB. And then it will give all of that to the document chain, which will put all of that in the context here and pass it to the LLM, and the LLM will then try to deduce an answer from it, which is what it did. And finally, we did invoke this chain. And we got the answer. Any questions on this anyone, I think? I'm getting part of random string before the action, this case in the current fiscal year question mark. Alexander, yes. Is system capital or 2.0 telecurrent divided by the radius of the current curve, which is the highest ever outlet, or nine times out limit. Yes, Alex, I think I am also getting that. I think that comes because the chunking might not have been done properly, or the overlap would have been there. We can increase the chunk size or the chunk overlap to see a better answer. Anybody else, any question? I think this is the last assignment, or this is the last part of the workshop. Okay, Eugene, thank you so much for joining. I hope it was helpful to all of you. If you learned something new, please share about it. If the entire workshop was helpful to you, then these are some of the resources that you can follow from where I have learned. And if you want to ask me any question, if you want to share anything with me, any learnings that you want to share, or you want to talk to me, you can always reach out to me at this Twitter handle, viveknayer09, and I'll be very happy to help.

I keep sharing more resources also about LaNche. And if you want to ask me any personal questions also later on about LaNche or anything, you can reach out to me on Twitter. And if you so far liked what I've been teaching, I would request if you want to basically follow my YouTube channel also, where I keep creating more contents like this. You can basically follow me, follow the channel for those content as well. This is the link to the channel for anybody who wants to, I don't know, subscribe to the channel or see more content around Lanchain around AI or anything.

But thank you so much for joining. I think this is the last section and I think all of you have been very helpful, very supportive, and I hope all of you would have learned. I can see Alex was trying a lot of things, Eric tried out a play down with a lot of things, Camille understood, and a lot of everybody else also, I'm hoping understood and learned something from it. And thank you so much for React Summit also for this workshop. And I hope we do more of these workshops in the future.

Thank you so much, everybody. Bye bye, everyone.