OpenAI in React: Integrating GPT-4 with Your React Application

Artificial intelligence is just a fad, right? It's going to blow over like a blockchain. Well, actually I don't think so. In fact, AI is far from a fad. It's a revolutionary change. It's helping businesses solve real problems, and making employees and individuals more productive. So let's talk about why AI matters now more than ever, and how AI can take your React applications to the next level.

I'm Jesse Hall, a Senior Developer Advocate at MongoDB. You might also know me from my YouTube channel, CodeStacker. So throughout this talk, we're going to explore the demand for intelligent apps, practical use cases, limitations of LLMs, how to overcome these limitations, the tech stack that we're going to use to build a smart React app, and how to integrate GPT, make it smart, and optimize the user experience.

So if you're new to the AI space, maybe you don't know all of these terms and technologies that we're going to talk about, or maybe you're scared that you're going to miss out on what all the new kids on the block are talking about. But don't worry because we're going to define and demystify a lot of these concepts. And then we're going to go deeper and discuss some of the considerations that you need to make whenever you're building AI into your applications.

There is a huge demand for building intelligence into our applications in order to make these modern highly engaging applications, and to make differentiating experiences for each of our users. You could use it for fraud detection, chatbots, personalized recommendations, and beyond. Now, to compete and win, we need to make our applications smarter and surface insights faster. Smarter apps use AI-powered models to take action autonomously for the user, and the results are two-fold. First, your apps drive competitive advantage by deepening user engagement and satisfaction as they interact with your application. And secondly, your apps unlock higher efficiency and profitability by making intelligent decisions faster on fresher, more accurate data.

Almost every application going forward is going to use AI in some capacity. AI is going to wait for no one. So in order to stay competitive, we need to build intelligence into our applications in order to gain rich insights from your data. AI is being used to both power the user-facing aspect and the fresh data and insights that you get from these interactions is going to power a more efficient business decision model.

Now there are so many use cases, but here are just a few. Retail, healthcare, finance, manufacturing. Now, although these are very different use cases, they're all unified by their critical need to work with the freshest data in order to achieve their objectives in real time. They all consist of AI-powered apps that drive the user-facing experience. And predictive insights make use of fresh data and automation to drive more efficient business processes. But how did we get to this stage of AI? Well, in the early days of computing, applications primarily relied on analytics to make sense of the data. This involved analyzing large datasets and extracting insights that could inform business decisions. As computing power increased, it became easier to analyze larger datasets in less time.

Now, as computing power continued to increase, the focus shifted towards machine learning. Traditional batch machine learning involves training models on historic data and using them to make predictions or inferences about future events, about how your user might interact in the future. The more data over time that you feed your model, the better it gets. The more you can tune it and the more accurate the future predictions become. So as you can imagine, this is really powerful because if you can predict what's going to happen tomorrow you can make really great business decisions today.

So batch AI as the name implies is usually run offline and on a schedule. So it's analyzing historical data to make predictions about the future, but therein lies the problem with batch AI. It's working on historic data. It can't react to events that happen quickly in real time. Now although it's really great for industries such as finance and healthcare, we need data on things that are happening now. And so this is where real-time AI comes in. Real-time AI represents a significant step forward from traditional AI. This approach involves training models on live data and using them to make predictions or inferences in real time. This is particularly useful for fraud detection, for instance, where decisions need to be made quickly based on what's happening in real time. What good is fraud detection if the person defrauding you has already gotten away with it?

And then finally, that brings us to generative AI, which represents the cutting edge. This approach involves training models to generate new content. Now this could be images, text, music, video. It's not simply making predictions anymore. It's creating the future. Now, fun fact, the images here were all created using Dolly. So over the years, we've seen AI evolve from analytics to real-time machine learning and now to generative AI. These are not incremental changes. They're transformative. They shape how we interact with technology every single day.

So let's zoom in a bit. We have something called Generative Pretrained Transformers or GPT. These large language models perform a variety of tasks from natural language processing to content generation and even some elements of common sense reasoning. They are the brains that are making our applications smarter. But there is a catch. GPTs are incredible, but they aren't perfect.

One of their key limitations is their static knowledge base. They only know what they've been trained on. There are integrations with some models now that can search the internet for newer information. But how do we know that that information that they're finding on the internet is accurate? They can hallucinate, very confidently, I might add. So how can we minimize this? Now, they can't access or learn from real time proprietary data, your data. And that's a big limitation, don't you think? The need for real time, proprietary and domain-specific data is why we can't rely on the LLMs as they are. This is especially true in the business context where up-to-date information can be a game changer.

So what's the solution? How do we make these models adaptable, real-time, and more aligned with our specific needs? Well, this brings us to the focus of our talk today. It's not merely about leveraging the power of GPT in React. It's about taking your React applications to the next level by making them intelligent and context-aware. We're going to explore how to augment React apps with smarter capabilities using large language models and boost those capabilities even further with Retrieval Augmented Generation, or RAG. And so we're not just integrating AI into React, we're optimizing it to be as smart and context-aware as possible.

Now what's involved in Retrieval Augmented Generation? First up, vectors. What are vectors? These are the building blocks that allow us to represent complex, multidimensional data in a format that's easy to manipulate and understand. Now the simplest explanation is a vector is a numerical representation of data, an array of numbers. Now these numbers are coordinates in an n-dimensional space where n is the array length. So however many numbers we have in the array is how many dimensions we have.

Now you'll also hear vectors referred to as vector embeddings or just embeddings. So here's a real life example of vectors in use. When you go to a store and you ask a worker where to find something, many times they're going to say, go to aisle 30, bay 15. And so that is a two dimensional vector. And we also notice that stores that similar items will be placed near each other for ease of searching and finding. The light bulbs aren't just scattered all over the store. They're strategically placed to be found easily. And video games use 2D and 3D coordinates to know where objects are in the game's world. With these coordinates, we can compute the proximity between objects to detect collisions, for instance. And the same kind of math is used to compute the similarity between vectors during vector search. And now if you're a Stargate fan, the gate addresses are made of at least seven dimensions that are like vectors. And so to locate Stargates in other galaxies, you can add an eighth or ninth dimension, just like you would a phone number's area code and country code. So this shows how adding dimensions significantly increases the size of the virtual space in which our data is organized. And so again, what makes vectors so special? They enable semantic search. In simpler terms, they let us find information that is contextually relevant, not just a keyword search. And the data source is not just limited to text. It can also be images, video, or audio. These can all be converted to vectors.

So how do we go about creating these vectors? Well, this is done through an encoder. The encoder defines how the information is organized in the virtual space. And there are different types of encoders that can organize the vectors in different ways, depending on the use case. There are encoders for text, for audio, images, and so on. And many of the most popular encoders can be accessed through Hugging Face, OpenAI, and many others. So now let's tie all of this back to Retrieval Augmented Generation. RAG leverages vectors to pull in real-time, context-relevant data to augment the capabilities of an LLM.

Vector search capabilities can augment the performance and accuracy of GPT models by providing a memory or a ground truth to reduce hallucinations, provide up-to-date information, and allow access to private data. So first, we take our private data or custom data, whatever it may be, and generate our embeddings using an embedding model, and then store those embeddings in a vector database. Now again, this data could be documents from our site, could be blog articles, videos, images, PDFs, et cetera.

Now you don't have to use LangChain to facilitate all of this, but it's very helpful, and we're going to talk more about that later. And once we have our embeddings for our custom data, we can now accept user queries to find relevant information within our custom data. Now to do this, we send the user's natural language query to an LLM, which vectorizes the query, and then we use VectorSearch to find information that is closely related, semantically related, to the user's query, and then we return those results. And we can do anything that we want with these results. We could summarize the answer to their question based on the custom data. We could respond with links to specific documentation, pages, and so on.

So imagine your React app has an intelligent chatbot with RAG and Vector Embeddings, this chatbot could pull in real-time data, maybe the latest product inventory, and offer it during a customer service interaction. With RAG and Vector Embeddings, your React app isn't just smart, it's adaptable, real-time, and incredibly context-aware.

Now, talk is cheap. How do we actually do these things and what technologies can help us? Well, the first tool that we're going to use in our tech stack is Next.js, and we're going to, of course, use version 13.5 with the app router. Next.js and Versel just make building apps with AI technologies so easy. And then OpenAI has been spearheading advancements in language models with models like GPT 3.5 Turbo and GPT4. And while there are many other language models out there, today we're going to focus on OpenAI and we're going to use them for embedding and for generating responses. And then Lankchain is another crucial part of our tech stack. It helps us in data preprocessing, routing data to the proper storage, and making the AI part of our app more efficient. And then there's the Versel AI SDK. It's an open source library designed to build conversational streaming UIs. It abstracts so much boilerplate code that you would have had to write otherwise. And then last but not least, where are we going to store our vector embeddings? MongoDB. We're also going to leverage the MongoDB Atlas Vector Search. It's a game changer for AI applications, enabling us to provide a more contextual and meaningful user experience by storing our vector embeddings directly in our application database, instead of bolting on yet another external service. And it's not just Vector Search. MongoDB Atlas itself brings a new level of power to our generative AI capabilities. Each technology in the stack is chosen for a specific reason. And when combined, they enable us to build a smarter, more powerful React application.

So let's take a look at how to build a React application with these technologies. And this is what we're going to build, an AI-powered documentation site. Now, this site will not only answer questions, but also provide contextually relevant information, summarize answers, and provide links to relevant pages to dig deeper.

What will we need? An open AI API key, Node.js 18 Plus, and a MongoDB Atlas account. The first thing that we need to do is create embeddings for our custom data. Now, since this chatbot is going to reference our custom documentation, we'll assume it's written in markdown files. And so this embedding function is just a Node app, a single file with less than 60 lines of code. And we're going to run this when we build our next JS app. So we'll need to make sure that we have .env, LangChain, and MongoDB installed. And then we'll import everything that we need. We have the LangChain text splitter, the MongoDB Atlas Vector Store, open AI embeddings, MongoDB Client, and then .env.

Below that, we'll prepare our MongoDB connection using our connection string and get the collection that we want to use. And then we need to process our documentation files. We have to split them into chunks that the encoder can use. So we'll create a splitter using the recursive character text splitter from LangChain to split the Markdown files that we're fetching. Now notice that you can choose the chunk size and overlap, which is helpful to make sure that we have enough context in each chunk. And then we'll create our output, awaiting the splitter createDocuments function, passing it our documents. And lastly, we'll use LangChain to store these embeddings in our MongoDB collection. We're passing to MongoDB Atlas VectorSearch the output, creating a new AI Embeddings, and then giving it the metadata for the collection, index name, text key, and embedding key. These keys are the fields that contain the original text and the embedding vectors. And then we close the MongoDB connection.

And this is an example of what ends up in our database. We have our original text and Markdown format, our vector embeddings, and metadata that identifies the portion of the document that this text came from. One last thing that we have to do to prepare the data for searching is to set up a search index on our collection in MongoDB. You can do that through the Atlas UI using this JSON configuration. So here we're specifying the dimensions of the encoder model that we're using. In this case, it's 1536. And then you can also define the similarity and type to use.

Okay, so now the fun part, let's set up the Next.js app. Now, if you'd like, you can use the Next.js lang chain starter, which already has everything set up except for the MongoDB driver.

So you'll also need to npm install MongoDB. Next, add your OpenAI API key to the environment variable file. And then for good measure, you can test the app by running npm run dev. You should now be able to chat with OpenAI. But this is just a standard chat with the ordinary models that we all know and love. So now let's make our GPT model smarter by giving it our custom information.

Now, this is the default chat route provided with the Next.js lang chain template. And we can see that it utilizes the Vercel AI SDK along with lang chain, OpenAI chat models, and some lang chain. This is also utilizing the Vercel Edge runtime to lower latency and help with streaming our responses back to the user. Now, further down this route handler, we are creating an OpenAI chat. And here we can specify which model we'd like to use, GPT-4, GPT-3.5 Turbo, et cetera. We're also setting streaming to true because we want the response to start populating in the UI as fast as possible and stream to the user. This is a much better experience than just throwing up a loading spinner and having the user wait one to 30 seconds to have a response come back. And then it returns the streaming response. And so in this route is where we need to inject our own custom data.

To do that, we're going to use the LangChain MongoDB VectorSearch method. And in this route, we're going to connect to MongoDB just like we did before. And we're going to pass to this the question that the user asked. And then we'll use the MongoDB Atlas VectorSearch LangChain method to create vector embeddings for the user's question. We have to also create vector embeddings for the user interaction so that we can then compare it with the other vector embeddings that we have stored in MongoDB for our custom data. And then we'll tell it which collection index, text key, and embedding key to use. And then when we do the search, we're using something called maximal marginal reference to find the related documents. We can specify the number of results to fetch and how many of the top results to return. This allows us to refine how accurate we want to be. And finally, we close the connection and return the retriever output.

Back in our chat route, we're going to get the last message from the user and pass it to the new vector search route that we just created. Now we can include the results of that search and pass everything to the standard LLM. We'll include some context, the vector search results, and the user's question. So the prompt we're using here is, you are a very enthusiastic MongoDB representative who loves to help people. Given the following sections from the MongoDB documentation, answer the question using only that information, output it in markdown format.

If you are unsure and the answer is not explicitly written in the documentation, say, sorry, I don't know how to help with that. And finally, we replaced the last user's message with our updated message that includes our custom information and context. That then gets passed to the LLM and we receive our response back.

And these are the results. The question is, how do I get started with MongoDB and Node.js? It answers with a summary and links to specific pages in the documentation for further help.

Now, if you want to build something like this from start to finish, be sure to attend my workshop. You can get more details on how to sign up for that on the conference website.

And we covered a lot of ground today from understanding the role of AI in modern applications to its seamless integration with React using GPT models and RAG architecture. We explored the limitations of large language models, how you can overcome them using real-time and custom data. We looked at a great tech stack that seamlessly integrates AI into your application using Next.js, OpenAI, LaingChain, the Vercel, AI SDK, and MongoDB Vector Search. And these are not just buzzwords. These are tools that can revolutionize how we think about building smart applications.

Now, remember, the future is not just about smarter AI, but also about how well it's integrated into user-centric platforms, like your next React-based project. So be sure to give MongoDB Vector Search a try in your next AI application. Thank you, and I hope this was helpful.