Automated Customer Support Bots with LangGraph on AWS

Hey, everybody. So in this presentation, we are going to be talking about Langraff and how to build a customer support bot with Langraff and the concepts involved for why we're going to be using Langraff, what kind of capabilities Langraff provides for our agentic support bot so that we can accurately and effectively help customers with various different questions and insights. Various different questions and inquiries. And we will also talk about Amazon Bedrock as our base for our rag and knowledge base for the customer support bot to be grounded in its answers when it comes to product support. My name is Saurabh Dahal. I'm a developer advocate here at AWS, and I'll be taking you through this presentation in this talk.

So I just wanted to make sure we're aligned on some of the fundamental terms and concepts that we're going to need to know about so that we can understand why we're even using Langraff and what Langraff is. So a large language model, as you know, we've all been using large language models. It can make up things. So if a customer support bot is only attached to a large language model without things like RAG or Langraff, then it might just hallucinate. For example, I have this website here that has various different products, right? You have the Define Curl Cream. These are hair care products. There's prices listed here, ingredients, and how to use it, when it's best to use these particular hair care products.

Let's talk about LangChain now. So what is LangChain? LangChain is a Python library that allows us to make the LLMs, large language models, use tools that connect to data.

It doesn't really have memory either of the last message if we were to just attach this to a large language model. So RAG, that's one way to help make this better. RAG stands for Retrieval Augmented Generation. And what this allows us to do is it allows us to attach this source of knowledge, our company information in terms of the products, frequently asked questions, policies, if we have any special promotions that are currently limited time. Here in the United States right now, we have July 4th coming up. So many companies are doing special discounts for products for July 4th promotion. So maybe we have temporary promotions that are going on. Maybe we have certain refund policies going on. Maybe we have product recalls for certain products. We want the large language model to know about all that. And RAG is a way for us to allow our large language model to essentially have access to current information that is relevant that the large language model otherwise would not know about. Because the large language model, as you know, it just knows about information that is trained on. And maybe the training cutoff date for the large language model was several months ago, and it would have no idea about the latest information about our products in our store.

So we're going to talk about how that happens. But essentially, when I ask a large language model a question, instead of it just responding to its pre-trained data, we can use Python to program it. So that the large language model will be classifying our prompt based on what are we asking and what kind of tools does it need to use in order to answer our question properly? And what kind of data sources would this large language model can be attached to so that it can also get the latest information that is current related to maybe our products or our discounts or refund policies? In this app, we're going to be using Bedrock model. So LangChain is going to use Bedrock model. It's going to be connected to Bedrock to connect to certain models. And also, it's going to use the Amazon Knowledge Base Retriever to retrieve from our Bedrock Knowledge Base where we're going to have a bunch of company product specific data, frequently asked questions, refund policies, things like that.

LangRaph allows us to build agents as a graph. So think about like a flow chart. Maybe there's conditions, right? So if I ask it a certain question, it might classify it as, is this question related to products or is this question related to like order, specific order information? So if I ask it, hey, what's going on with the curl cream for their hair care products? How do I use it? It's going to classify that message as, is it related? Should the large language model look at the tool related to searching up information about the products in our RAG system on Bedrock? Or should it, is this question related to an order that we need to help process a return for a customer? Well, for that type of question about a curl hair care product, and how do I use this curl cream? It'll classify that as intelligently as it can and it'll classify it as a product related question. And it'll use the tools relevant to searching for products and reply accordingly. Or if I ask it, hey, I need to return this shampoo. Well, it'll classify that message as, well, it's not really related to searching for products. It's related to handling the return. So instead, it'll use a different tool related to return policy so that it can search up return policy related information and maybe even involve a human in the loop to approve the refund based on certain criteria if it's eligible for refund. So you can think of each of these boxes as a node, right, where it's going to, it has one job.

So instead, it'll use a different tool related to return policy so that it can search up return policy related information and maybe even involve a human in the loop to approve the refund based on certain criteria if it's eligible for refund. So you can think of each of these boxes as a node, right, where it's going to, it has one job. Like, for example, this node classify message is going to determine whether the prompt that it was sent was related to products inquiry or return inquiry. So edges, these are these arrows right here. Those are, you can think of these as like branches and it decides which node to go to next. You can think of it like conditional branches. And LandGraph also has something called state. So think of it like a memory or some type of shared notepad. So like as it's going through these nodes and edges, it keeps track of what information was collected, what information was spoken back and forth. Remember, LangChain is the foundation. But LandGraph is built on top of LangChain. So LangChain, think about it just one direction. But LandGraph. It's multiple different options and scenarios, multiple paths. Even it can even pause for a human in the loop intervention, which we're going to see in this demo. And it's going to be connected to Amazon Bedrock for the knowledge base. So and also the model. So we're going to be using Amazon Bedrock for the foundation models that we can connect to. Where Bedrock is actually hosting a cloud model that we're going to be using. And then the library of the knowledge basis, our product information. I'll show you an example shortly of that. All right. So you can think of the agent as a graph when it comes to LandGraph. Classify intent and it'll determine whether to. Which agent to route the query to. If it's a general question like, hello, right? It'll go to general returns question. Returns agent, product question, product advisor, agent. All right. So let's take a look at a forced knowledge based search. So.

There's no guessing allowed, right? So a large language model by default, it might just guess. It wouldn't know about my products. Honestly, Clod has not been trained on this app I created two days ago about random fake hair care products with random fake made up prices. It has no idea about this. So how would it know? It would know that because it's connected to a knowledge base.

So I'll show you an example and I'll also show you an example later of how I was able to put this into a knowledge base. So for now, I'll do a demo. Here's a chatbot. As you can see, there's a different products here. I can ask it a question like. So, yeah. For example, I can ask, is the curl cream safe for color treated hair? Now searching the product knowledge base. Perfect. So now it's answering based on information that's uploaded.

So, for example, how much is the divine curl cream cost and what ingredients does it have? So I can ask it a question very specific to a certain product. And it accurately is able to tell me it's twenty four dollars, right? This is twenty four dollars and it'll tell me the key ingredients like shea butter, flaxseed extract, glycerin. Let's double check here. Yes, she is butter, flaxseed extract, glycerin. So it knows about current information. Right. And if I ask it what is the price of this curl cream? It was able to answer that. But what about for wavy hair, can I use it for that? Right. In my next question, it remembers the thread of what we're actually talking about. So it knows we're talking about the curl cream.

Right. When I say, can I use it? It knows what it is. It knows I'm referring to the defined curl cream. So that's kind of how a chat GPT is created, right? When you talk to chat, GPT underneath the hood. It's not just the chat GPT model. There's some type of memory involved that allows it to understand the context of what you're talking about. And a tool like land graph helps provide that foundation.

So I'm pretty sure these chatbots that we're using right now use these types of tools attached to a model. So it knows I'm talking about the curl cream. Awesome. Go back to the slides here. So, yeah, that has something to do with the model. There's a visible tool call and also memory and rag involved. Right. So let me go back to here. For land graph, we're going to have this workflow. It's going to classify our intent for our prompt. Like, if I ask it a question about a product or if I return.

This is the flow that is going to go. If I ask it about a return, there will be a human approval, a human in the loop approval because refunds are monetary involvement. Right. So we want to make sure that LLMs are just one part of it. But we want an actual human to approve the refund or deny the refund at the end because we don't want large linguist models to be responsible for our business refunds. We might go broke. Who knows. Right.

So here's another graph that kind of shows. The flow. And fun fact, it's a graph, right? You can see the graph in this flow and we're using land graph to create this type of flow. That's probably why it's called land graph. All right. So it's grounded in. Knowledge from bedrock.

So how are we able to do that? Well, I will show you a quick little demo here. I'm using Kiro. Kiro is my agent ID. And here if I create a new session. Let's say I have all these knowledge based products right about products like these are just markdown files that talk about the product information.

And I've already uploaded this to Amazon Bedrock knowledge base. But let me let me also upload a new file. So summer sale 2026. So this is for the 4th of July promotion. I want the large language model to know that actually some of these products are now 50% off instead of $24 for that curl cream that we just talked about. It's $12 right. So I want some new pricing updates for up until July 4th. Midnight. Right. So I will actually use something called the agent toolkit.

Which is, I'll share how to set that up later in the demo, but it comes with the AWS MCP server and a bunch of agent skills that tell it how to properly use Amazon web services, tools and services so that the large language model properly actually uses these tools when I prompt it. So I will say I want to. Upload the. Summer sale 2026 dot MD. Doc to our Bedrock. Knowledge base use the. Agents AWS agent. To get to help you properly set that up. And my my laptop is already configured to AWS account so. It will figure out which AWS account I'm connected to and it'll actually look up. Current documentation. Using this MCP server. On Bedrock and everything else. That it needs whatever types of services that it needs to use on AWS, so it knows the latest information and then it'll use certain agent skills.

For example, I'll go here. So the Amazon the AWS agent toolkit also comes with agent skills that tell the agent how to use like the best practices for working with different services. For example, for Bedrock. It has all this information. That is very specific to how to properly use it so that LLM does not make mistakes in setting up our AWS knowledge base in Bedrock. So as you can see, it's making the tool call. And it's going to add this document to a knowledge base so that that's how I get it to add that document to knowledge base. Let's go back to the slides.

There's also another flow for intent routing, so I want to refund for my order, right? So let's take a look here. For this, it's going to verify my order if it's an actual order that exists in the system. And then it's going to see if that order is eligible for a refund. So if the order was made more than 30 days ago, it's not eligible for a refund based on the docs that I've provided it in the knowledge base. So it'll check the knowledge base. So let's take a look at that. There's a fake order called LLM-999 that should not be found.

There's a fake order called LLM-999 that should not be found. An order that's way too old called LLM-1002 is not eligible for a refund. And then a valid order for LLM-1001 that is eligible for refund. So let's ask it, I want a refund for my order. So I'll tell it, I want a refund. So the intent classifier is going to decide, hey, well, this is not related to a product. This is related to a refund. So it's going to ask me about the order number. My order number is LLM-10001. But if I make up a random order number, it's LLM blah, blah, blah. It'll look up that order for me. So the agent is actually going to know to use a little tool for looking up orders from a database. So LandGraph allows me to route that agent to use different tools. So in this case, based on the response that I gave it, it uses lookup order tool and it's not finding that in the database. So someone is not going to get a refund for this. But if I tell it actual valid order LLM-1001, it's LLM-1001. It will look that up. It'll use the tool. I think I need to give it all caps. Hold on. Perfect. So yeah, it's case sensitive. So maybe in my Python script I could have made it so that it's case insensitive. So now it's able to find the order. And in here, I actually have a mock order JSON file. So these are the ones. So for LLM-1001, that's Jordan Lee. And the item that he ordered was a weekly repair mask. And it was ordered within the last month. So it should be eligible for refund.

And while that's doing, as you can see, the AWS MCP server is running to upload this knowledge base. It's asking me for permission. I'll just say allow so that it can upload the document for the July 4th discount to my knowledge base. I want to show you that in action with the agent toolkit. And when it runs through an error, it'll actually automatically figure things out by itself.

Anyways, let's continue here. Now deliver, can you verify the name or email address? So that's some extra security features, right? So my name or email address, let me figure out what that is. I totally forgot. It's Jordan at example.com. I'll send that. If I give it the wrong email address, then it knows that it's a fraudulent refund request. Perfect. So it'll check a new tool for the refund policy. And it'll take a look at the documents related to our company's refund policy.

And it'll see, okay, is this item eligible? Yes, it is. Perfect. Now, if you'd like to proceed with the refund, please confirm and let me know briefly why you'd like to return. I just don't need it anymore. Now, notice the large language model isn't going to automatically just send me a refund. Instead, it's going to submit this refund to a manager, a human, for approval. So you can see this human in the loop here where a human, and I have it all in the same UI, but in a real world, you might have it email an actual manager or a different dashboard where you can have an approval or deny button here.

I just put it all in one thing for the demo. So I'll approve it as a human. So this is a human in the loop. That's another feature of LaneGraph. And refund is submitted. Perfect. So with that flow, this is what's happening, right? It's going to classify the intent. And in this case, we are working with the returns agent. Human in the loop is a feature where you would need LaneGraph for that rather than just LaneChain because LaneChain is a sequence of events. LaneGraph is many different conditions and also human in the loop where the agent can pause for a human to intervene before it continues working.

So after the human intervened and answer something, the LLM was able to add more information. That's the interrupt pattern in LaneGraph. That allows us to do that human in the loop where it can pause, wait for external feedback, such as a human in the loop, and then it can resume its workflow. And LaneGraph allows for that. And our RAG stack, our documents that we're uploading, let's take a look at that. How's that going? Allow. Yeah, right now, as you can see, it's making different API calls and it's working through uploading this to Bedrock. So right now. It's running some scripts and it knows how to navigate the different errors and policies for identity and access management on AWS because of the agent skills from the AWS Agent Toolkit. And it's uploading our 30-day, actually July 4th, discount promotion to the Knowledgebase. So AWS identity checks out with the current ARN. The Knowledgebase is active and configured with S3 vector storage and the data source is available pointing to the right S3 bucket.

Cool. So let's talk about how that works. So how does the Knowledgebase work? So in S3 is where I upload a bunch of documents. It's like a bucket where I can upload all of my product information, refund policy. And then in Bedrock, there's something called Titan embeddings, which basically turn these human readable documents into mathematical vectors. And then in S3, there's another bucket called the S3 vectors index where these mathematical representations of our product information represented as vectors, will put all of those vectors into this new S3 bucket. So you can think of S3 vectors index as basically a mathematical representation of all of our human readable words and files and documentation related to our products, refund policy, special promotions and discounts. They're represented in vectors and stored in this bucket so that when I use something called Bedrock Knowledgebase, Bedrock Knowledgebase is just something that allows me to query this vectors bucket.

And I can ask it questions like, what is the refund policy? And it'll actually connect. It'll be able to read these mathematical formulas and understand my question based on meaning. And it'll give me semantically matching answers instead of just looking up by raw keywords. And also there's IAM service role that allows all this to happen for access policy. And the AWS agent toolkit, what's the purpose of it? Well, AWS is always shipping out new updates and new best practices and new services. And even like in Bedrock, certain models are going to be deprecated. For example, Anthropic Cloud 3.5 is no longer going to be available. But instead, we're going to be using a different model. So it'll know which models are out of date. And it'll also know about how to handle certain errors and which order to do things like, should I put the documents in the S3 bucket first and then embed it with Titan? And then what's the workflow? It'll know about those best practices with the agent skills.

So how did we build this? Let's take a look at some code and I have this in my presentation guide here. We continue allowing it to do its thing here on the chat for it to upload the promotions discount document. So, yeah, in my backend, I'm connecting it to Amazon KnowledgeBases. So once I have uploaded the documents to a KnowledgeBase, there's something called a KnowledgeBase ID that Amazon would provide back to me that would link it to the proper place where this large language model can actually query that particular KnowledgeBase where my documents are stored. Now, I did not know too much Langraff about a month ago, right? I was able to build this out using QIRO by using something called a spec-driven development workflow. And let's pause real quick here. We can see that it's getting a pre-signed URL for the S3 bucket so it can upload documents. So it's doing its thing here. But anyways, this is some of the code.

Here's the tool example. So this is a function and it's labeled with a decorator called tool. And there's a doc string that tells it when to use this function. So any time it's related to a product question, it's going to use this particular function to retrieve from our KnowledgeBase. So that's how it works. So I want to talk about how I built this app. There's an agent skill that I use called Mastering Langraff. This has the latest best practices on using Langraff so that the agent doesn't hallucinate and doesn't create too many bugs when it's trying to build out my Langraff app. In this case, this here customer support bot. In QIRO, we can do something called spec-driven development. It's similar to plan mode.

When you're asking it to build an app, I will... First, it'll generate like a design document for how the app should work, what kind of features it should have. A requirements document talks about the features, user stories. And then it'll break down that into tasks. And it'll also use the appropriate agent skills.

So let's go back to the slides. So yeah, with QIRO, I was able to generate this app by first going through the spec-driven development workflow by generating requirements like user stories in the design. So even like code snippets or the architecture using it on AWS, Bedrock, things like that. And then it'll break that down into tasks for it to actually create code in Python to build out the app. And the mastering Langraff skill, it's an agent skill that I used with my AI coding assistant.

And it was able to catch certain bugs that otherwise a regular large language model, even Opus 4.8 probably would have missed. So yeah, that's how it was built. I have a video on how to use the agent toolkit to set up the AWS side of things with RAG that I can share with you. But if you wanted to connect with me on LinkedIn, please scan this QR code. I'm happy to answer any more questions you have. You can message me directly. Also, I have a full video on my YouTube on how to use the AWS agent toolkit, how to set it up.

Also a step-by-step blog on how to set it up including the relevant MCP server configuration and commands that you'll need to run in order to equip your coding assistant with the AWS agent toolkit so that you can just prompt it to set up your Bedrock setup for your Langraff app, and it'll just do it. And it'll do it very accurately with minimal issues. And then you can use your coding assistant to build your Langraff app and then connect it together. And that's how I was able to do this whole thing. Thank you for listening. I would appreciate any feedback you have by scanning this QR code.

This is one question in terms of if you got any value out of this presentation. I would love to hear it or any comments that you have. I would love to see your feedback. And I'll see you on the next one. Bye. And, yeah, please message me on LinkedIn if you have any other questions. Thank you. Thank you so much, Soroban, for your session. I enjoyed the most. I really like that you founded everything on open-source framework, Langraff. I'm a big fan of graphs myself for my previous employer. We and the team did a lot of experimentation and I'm truly a believer in graphs. So there are so many use cases where they just fit perfectly as the best possible solution. And I think you clearly demonstrated the value of that approach.

And we have a couple of questions so far. Chris is asking, in the product agent, what would be required to update the suggested prompts based on the overall conversation? Hi. Great question. I would make the app even more dynamic and more user-friendly because it feels very curated to the conversation where the suggested questions really reflect the conversation that's happening. So currently, right now, those little chips, those tabs of questions are hard-coded in HTML and never change because this is a small demo. But to make them follow and fly from the agent, it asks the model for two or three good follow-up questions based on what was just discussed. And in Langraff, there's an in-memory thread that we can continuously build on top of so that it knows about the context of the conversation.

So we can add that as part of the thread as well, like potential follow-up questions. So it allows the bot to actually answer those questions from the knowledge base as well if we make it so that the follow-up questions, the system prompt that we create for generating the follow-up questions are also grounded in what it knows in terms of the rag system that we have. So those follow-up questions can't be something super crazy either. It needs to be aligned to what we can expect the bot to know. And you can send those back to the browser using just tags as something like suggestions instead, instead of like chat response.

And the front-end swaps out those chips with new ones because of those tags. The nice part is the pieces are already in place. The app remembers each conversation and already streams type messages. So this is a focused add, not a rebuild. And the main trade-offs are a small bit of extra cost and delay per message because it has to generate the follow-up questions. So there are different models for different agents. So maybe for follow-up questions you don't need the most intelligent model ever because it already has the in-memory conversations.

So it can generate follow-up questions based off of that. Yeah, that's a super smart strategy to use different models for different tasks. They all serve different purposes. They all cost differently after all. So you also need to find good balance.

The key idea is that you can't rely on the bot's instructions alone to keep it safe because a clever message or even poisoned text in a knowledge based document can trick it into ignoring them. Concretely, the tools that touch sensitive data or money should check permissions themselves. As a rule of thumb assume the bot can be fooled and design it so that even if it is fooled it can't do any real damage.

Great question. The knowledge base needs continual updates to avoid stale or conflicting information. Setting up a system to trigger updates in the knowledge base when documents are updated in the workspace can help maintain accuracy. Sharing the demo code on GitHub for feedback and potential contributions can be beneficial for improvement and innovation.