NodeJS & AI: Building Smarter Applications

Hi all, I'm excited to talk today about how we can combine Node.js and one part of artificial intelligence to build applications that aren't just functional, but smart. There has been an increasing importance of artificial intelligence in software, and I want to encourage you TypeScript and JavaScript developers to learn and apply these technologies.

My name is Aileen. I'm currently working as a software developer at Orama, and I've been doing software for 10 years now. I organize technology meetups at my city, and I'm really passionate about the intersection of AI and software.

But before we dive in, I want us to align with some concepts that we'll be talking about. And let's start with defining what is a model. And I want you to think of a model as a specialized program that has been trained on massive amounts of data to recognize patterns and make predictions or decisions. These models have been trained to do specific tasks, like understanding text, generating content or classifying images. And we're going to interact with them by providing these models an input. It would use all of these learned patterns and generate an output. This output could be an inference, a generation or a prediction.

Today we're going to be using large language models. And a large language model is a model that is good at understanding and generating human language. They're based on transformer architecture. There's a paper called Attention is All You Need, and I really recommend it to read it if you really want to understand how they work underneath it. They're trained on large amounts of data, and they work for many tasks. So they can do summarizations. They can do question answering or translation. Today we're going to talk about two types of models, which is the closed models. And these models are models like chat GPT, like Cloud, From Anthropic, like Gemini, and some of Mistral models. These models are accessed via an API, and they come with a cost. You cannot download it, so you basically rely on these providers to use these models. The other type of models are open source models. And these models, you can find them hosted in Hug and Face. You can download them and run them on your local machine using Olama. You could also download them and host them on any cloud infrastructure, and these models are models like Lama, like DeepSeq, or like Gemma from Google. But as you can see, these models are really powerful, and we can definitely make them more powerful. Let's remember that these models have been trained with huge amounts of data, and it's no question that these models are not good at math or very specific needs or niche of an industry. So that's when a new technique comes into play.

And that technique is RAC. RAC stands for Retrieve of Method Generation. And RAC definitely helps out on things like hallucination. While it does not eliminate hallucination, it could definitely reduce it. Because if a model doesn't know something, it might have made up an answer that's not accurate. So with RAC, if we provide that information, that model can answer correctly. Or we can enable private data use. So definitely, if they were not trained with data that you have available only for you, it would definitely help make your model more powerful.

But also, let's remember that maybe all of these models are not trained daily. So there's knowledge that they don't have, or they might be outdated. And there is a specific use case that I want to show you. So I asked Claude who are going to be the speakers of Node Congress 2025. And as you can see in the answer, it mentions that the knowledge of that model at this current date was cut off on October 2024. So it doesn't know who are going to be the speakers of this conference. And so for today's exercise, I'm going to go and walk you through how to create and enhance this model with RAC to actually answer this question. We're going to follow a couple of steps. And our first step is data. So there's many ways that you can encounter data for this improvement of our models with RAC. And that could be a PDF, that could be a JSON, a CSV file, a crawler, a DocuSign site, or a database. In this case, I'm using a crawler to go into all of the Node Congress web page. And I'm just adding some markdown files that I'll later have available for them to read. So this is an example of how the markdown looks based on the crawler that I did for this web page.

The next step is the split. So large documents can be overwhelming, and our AI model can make it hard for you to find relevant information across all of this data. So with splitting, we're going to cut it and maybe make it easier to look for something like in a book. So you can look for information in a chapter rather than reading the whole book. So split is a good technique that is used to improve retrieval performance. And also, it's very good at representing context. In here, I'm using a library called LimeChain, and I'm using the function recursive characters text splitter. This is also something that you will need to decide based on your use case.

You could divide by sentences, you could divide by paragraphs, you could divide by characters and how many sentences, how many paragraphs, and how many characters. There's some things that would be very specific of your use case that you will really need to try and test which one works for you. In this case, I'm using a chunk size and characters of 15,000 and a chunk overlap of 150.

Chunk overlaps are crucial and it basically ensures that we don't lose context at each chunk. So as we are going to chunk all of this information, it's important to have a small preview of what's next and also from the next chapter or if we're talking about an example of books. So overlaps are very important to have. So what we're doing is basically going through each of this information, characters, chunk them, and then now that we have this chunk of data, what's next? We're going to go through something called embedding.

So embedding is a technique that transform data, in this case text, to capture the semantic meaning of it. So each embedding is essentially a list of numbers, typically between 700 and 1,000, and it positions the text in a high dimensional space where similar meanings are closer together. As I mentioned, it's 700 and 1,000 dimensions. If we want to do it or see it in a very basic example, we can look at this image from SuperBase. So this is a two dimension and how it's going to be mapping things. So if I have the sentence, the cat chases the mouse, it's very similar to the next sentence. So they're going to be close together. But another sentence that has nothing to do with it, it would be far. And this is just in a two dimension. So we're going to rely on embedding models to transform these chunks of data and put it in a dimension space so that later we can retrieve that information based on how close and semantically these meanings are.

So as I mentioned, we're going to use an embedding model. In this case, I'm using OpenAI Ada. And I'm going to go from each of these chunks and make it an embedding. So the chunks that I did for 15,000, these are going to transform into an embedding. And it's basically vectors, a representational number of where they're put in a dimension. And now that I have this data, I'm going to need a place to find it later on. And this is where vector databases come into play. I will need to start these data in this special type of databases, vector databases. And it would look something like this. Remember, I have my 15,000 character content. So this is basically the content. And in the other side, I have all the vectors that have the meaning of that content. This is going to be used to later retrieve this information with something called similarity search.

Awesome. So we have our data. We have split it. And we have stored it. Now we need to retrieve that data. How are we going to find that information later? There are many types of retrievals that we can use. This can be keyword or term-based, which is basically looking for that exact query or word or sentence. There's also semantic, which is most likely looking to similar things based on the meaning. And there is hybrid. So basically, it's a combination of the two. This is also something that you will need to rely on based on your use case. On some places, it might be great to use one or the other. And this is something that you might want to explore based on what gives you the best results for your user, which is the goal.

In this case, we're going to use semantic. And so in order to retrieve, we're going to follow these steps. First, we will need the user query. In this case, we need that question. So the question is, who are the speakers? So that would be our query. In order to make a similarity search, we need to transform that query into an embedding 2. That's the only way we could actually find something similar to it. And this would find anything that's similar and get it back. So step by step. We have an endpoint API search. We're going to get the embedding of the query, which is anything that the user asks. And then we would find anything that matches that embedding in a similarity search. And finally, we're going to generate a response.

So let's go step by step. Again, I'm using the same model that I used before, in order that it can actually make sense. It's not a good idea to use different embeddings for different for storing and for querying. So I'm using the same model.

Now that I have my embedding, I would do a similarity search. This is going to go for a function called match conference docs in super base. With the embedding that I have transformed. And I'm going to get... I want to match threshold of 0.5, which is basically 50% similarity. Anything that's 50% above, I want it back. But only give me eight. If I have 10%, I only want eight back. And this is something that also, it's something that you can change. This is not the right answer for all of the use cases. This is the one that I use for mine. So once I have this, what's next?

I need to generate. Remember large language models are good at generating human language. So I can not just give back the results. I need to give a human response. So in this case, what I'm using is a Gemini model, and I'm going to give the context. The context is going to be everything that I retrieve from my vector database. And the query. So I'm saying to Gemini model, I need information about the following questions regarding node Congress, which is the query. And then here is the information that is relevant, which is the context. And this model will generate a human readable answer.

So let's see it in action. So this is the web page, and I'm asking who's speaking at node Congress. Great. So if you see, I'm getting the name, the companies, and even the title of the conference talk of these speakers. And this is something that we made more powerful. So we are relying on a model, a large language model, and we were able to answer this question because we provided this answer.

So if you see, I'm getting the name, the companies, and even the title of the conference talk of these speakers. And this is something that we made more powerful. So we are relying on a model, a large language model, and we were able to answer this question because we provided this answer. We could provide an answer to the user and we didn't hallucinate on creating fake information.

Great. So what's next? There are many, there's another way that we could definitely enhance these models. And this is by agents. But what is an agent? And an agent is a system that don't just respond to prompts. So it's not how we were doing it. As I mentioned, Cloud asking questions. They actually work to achieve those goals. So if a user have a question that agent is gonna fulfill that goal that was made to do. So think it's very different from what we did on Cloud asking. It is very similar as if you have used coding assistant or any planning for a trip or something like that. So it would give you information, but it would try to accomplish a task. And what makes them special is because they plan their approach, they use tools when needed, and they adapt when they hit a roadblock. So instead of returning an answer that they don't know, they would try to look and adapt and use something to fulfill the mission or the goal.

Planning. Agents break complex goals into logical steps. If you ask for a competitive analysis of five companies, an agent will plan how to research each of one, what aspects to compare and how to structure the final report. Reasoning. This is where a chain of thought comes into play. So agents think step by step through problems, just like we do into solving something that is complex step by step. Agents maintain context across interactions. So they remember what they've learned and build on previous work. This is very common on coding assistants. And remember, tools. Tools is one of the most important things, and this is crucial because they can call external functions like searching the web or querying a database or making calculations. So these basically extend a capability that goes beyond the training data of the model.

So how does it look like in code? So first off, we're going to define functions.

This is basically the tools that the agent can use to look for information. So we're going to do a trip, a planning trip agent. So I have these functions available. The LLM might not be trained with the whole list of hotels or events. So we're going to do something to search for this information. So I have these functions.

One, retrieve event information based on query, option filters, or finding hotel API. So these are two functions that the agent has available. We're going to need a mission. So here we define what is the mission of this agent. What is the goal? So you are an AI trip planning assistant. Help the user with their travel query. So this is the mission that the agent needs. It's very important to define that.

Next, it's the React pattern. Basically it's the reasoning. The model analyzes the user request through the mission. So I have a mission, but I also need the model to analyze it. We have the action. What is the next step? The model calls the appropriate functions if needed. And then execute the function to get that data. Once I have that, I do an observation, which is collect the data from the function calls. And then finally, synthesize into response with a context. So I'm going to rely on what are the steps that the agent needs to do. And then just put it all together to give a response.

Let's see how this works. This is a AI trip planner. And I'm finding hotels from New York for the next month. So this is information that I have, but it's also asking for more context. What are the days? What are the budget? Where are you going to be staying? What are the priorities? So this can definitely give a more specific answer based on the user needs.

Right now, it's already given information from hotels that it's using from the hotel API. And it's also asking for more information, so it can actually go and look for rooms available based on the guests that are going to be needed for it. So this is how it works.

We have these patterns and we have these needs in order to make an agent work and fulfill the mission that it has. There are many common use cases, as we mentioned previously. And this is a very powerful tool that would also enhance our LLMs.

I hope you find this talk amazing and you learned a lot. And please connect with me if you have any questions or you would like to further ask about it. These demos are going to be available with the code. And yeah. Thank you so much for joining.