Advanced Claude Code Techniques: Agentic Engineering With Context Driven Development

Welcome, everybody! I have two questions before I kick things off. I need some context engineering for myself here. So raise your hand if you have used an AI coding assistant before. Okay, most the room, that's good. Otherwise we'd have a problem. Alright, now raise your hand if you use one for most, if not all of your development now. Okay, that's also what I expected. Not a ton. Alright, good. So that's what I was hoping for. So welcome to your guide on context engineering.

And context engineering, it's a very, very broad topic. And so really, it's just developing strategies to give the context to your coding assistant about your project and what you want to build, and enough so that it has what it takes to get the job done. And since it is such a broad topic, I want to focus specifically on what you're looking at right here. That is using errors to our advantage, building an AI coding assistant that can self evolve. And what I want to give to you guys today, my goal is to give you a lot of things that you can take action on right away. And then also make it so that it's just really easy to grasp what I'm presenting here, but I still want it to be really powerful at the same time.

So I'm going to carry my mouse around to go between the slides here because I don't want to just stay at my computer the whole time. And before we get into the meat of things. There we go, I want to just introduce myself a little bit. So I'm an AI educator, consultant, and a content creator. And you'll primarily find me on my YouTube channel, I started a bit over a year ago, and I've grown it to 175,000 subscribers already, which is still unreal to me, like I'm the most surprised where it's gotten at this point. And then there and then also my private AI community, I focus on teaching how to build AI agents and leverage AI coding assistance in a way that actually scales.

And I've been an engineer pretty much my entire life, ever since I was eight years old, raise your hand again, if you've heard of the programming language scratch developed by MIT, probably not using it now for anything production grade, it's a block builder kind of programming language that I use as a kid to create video games. And that's where I started my engineering.

And I wish I could say that I've been building AI agents and using AI coding assistance for that long as well. But yeah, I couldn't really do that more than three years ago. But three years ago is when I started getting very heavily involved in generative AI, pretty much right when it exploded with the release of GPT 3.5. Turbo in November of 2022. And I have just been building everything with AI since then, all the opportunities that have come from my channel, people asking me to build anything from personal automations all the way to enterprise grade local AI deployments.

And it might be surprising to you what I'm about to say, especially given what I'm talking about here. But even last year, I was very hesitant using AI coding assistance, all the engineers I knew were starting to rely really heavily on Windsor of cursor, GitHub, copilot, all of these different tools, and I refuse to use them for a long time, I would use large language models to help me plan my development. I think everyone should absolutely be doing that no matter what.

But I always thought to myself, why would I trust the implementation to an AI coding assistant, I can do it better. And they're just so unpredictable. That's what I thought. Now, obviously, since I'm here, right now, my understanding has evolved, and so have my systems. And that's really what I want to get into today. The reason why I didn't trust AI coding assistance back in the day is because of the problem of one off fixes. And this is how most developers operate, they use an AI coding assistant to help them build out something. And then when they encounter an issue, because large language models, they're never going to be perfect, they just fix the bug themselves and they move on. But then the problem with that is they encounter the same thing over and over again.

And I'm going to be real, there are a lot of those things you'll run into over and over again, like your coding assistant is installing old versions of packages, it's completely over engineering your solution, your implementation is super tightly coupled, like all these things that I've seen time and time again, I'm not going to sugarcoat this talk here and say that AI coding assistants are magical, and you won't run into these things. But what I am telling you is there's a solution to all of these problems. And that's the philosophy that I want to focus on today. And that is that every error is an opportunity for you.

So it's a win-win. And what I mean by that is when you encounter an issue in your code, thanks to an AI coding assistant, like cloud code, and I'll actually get into a demo with cloud code later, instead of just fixing the bug, which obviously you have to do that too. It's also a time for you to reflect and find what I like to call system gaps, there's some process that you have created for working with your coding assistant. That's the system that I'm referring to. It's your strategies for researching and planning and implementing your validation process. There's something there that you can correct to avoid that problem that you just ran into.

Not to say that large language models won't just sometimes have that one-off hallucination or fluke you'll never see again. I mean, that's going to happen. But the productive mindset is to think like, okay, if the problem was with me and my system, what can I do to address this. And that's what gives us the power of having our coding system that can self-evolve. And I say self, because we can use the coding assistant to help us address these system gaps. And so, I keep saying system, system, system, it might be kind of vague in your mind what I mean by that. And so before I really get concrete with you on what it looks like to address these system gaps, I want to describe what a system is. And this really is the core of context engineering as well. And so, like I mentioned earlier, a system is really just the process that you have defined to work with your coding agent. And so, there's four main components that I have for this. First is your global rules.

These are the top-level constraints and patterns that you have for your code base. It applies to your entire code base. Like, these are the things you want your coding assistant to follow no matter what. Now, the thing with global rules is their context, it's context that's loaded into your agent's context window all of the time. So you want to keep it concise, otherwise you're just going to completely overwhelm the LLM. I mean, the classic mantra is just because the context can fit into the window for the LLM doesn't mean you should try. Because the more that you add in, the more you overwhelm it. It's called context rot, if you have heard of that phrase before. So you want to avoid that. And so that leads into the second component that we have here, which is on-demand context.

So, we're still high-level constraints and patterns that we want our assistant to follow, but it's for specific parts of our code base. For example, maybe we have a guide that outlines our patterns for implementing front-end components. Or we have a reference guide for how we build tools into our AI agents. That kind of thing that we definitely want loaded into the context of the LLM when we're doing that specific task type, but we're not referencing that all the time. Global rules are things like our logging patterns, testing patterns, architecture, the kind of thing that like literally no matter what you're doing, feature or bug fix, you want that loaded into context.

And then we also have third, our commands, which I really would call that the heart of the system. Because your commands are the processes that you've typed out and you now have as a reusable prompt, so that you have a predictable workflow that you can use time and time again. At least as predictable as LLMs can be, right? That's our goal, is to even though it's an undeterministic system, try to make it as reliable and predictable as we can. And then last but not least, we have our validation processes. What are the tools and strategies that we can give our AI coding assistant so that it can check its own work, find bugs through that, and then iterate. So we have the autonomous cycle of not just doing the implementation but actually correcting, hopefully, a lot of the issues that it encounters when it develops the code. And so that's a system at a high level.

And the main mental model that I always teach when building systems for AI coding, and this is where we'll start to get into what it looks like to evolve our system, is the PIV loop. So PIV is short for plan, implement, and validate. And as an engineer this kind of process you should be pretty familiar with even outside of using coding agents. But there's just some specifics we have to think about when we're working with a coding agent and how we give it the context that it needs, i.e. context engineering. And no matter the strategy that you make up yourself, or you find online, like maybe you've heard of the GitHub spec kit, or the BeMad method, or PRP, there's all these different strategies floating around. It all just boils down to this. You have to plan the implementation with your coding assistant, make sure you're actually on the same page for what you're building, then you give it the implementation, you delegate, hopefully, most of the coding to the assistant. And that's what I do nowadays. And then at the end you have the validation, where both you and the check the work, run the tests, manually validate things. And the important thing with the PIV loop is that it is not vibe coding. It's that kind of cringey phrase that you've probably heard thrown around time and time again, where you're just going with the flow and giving the entire implementation up to the coding assistant and accepting whatever comes out. And the reason the PIV loop is not vibe coding is because the implementation is the only thing that we're handing the reins over to the coding assistant. We're sandwiching it in between the planning and the validation, where we are heavily a part of the process. And I cannot stress enough how important it is to be a part of the process and not get lazy there, because otherwise it's not going to be reliable.

So digging into this a little bit more for our planning phase, we start with what I like to call vibe planning. This is the one place that I give you permission to just go with the flow. Because at first, it's just an exploratory conversation with the coding assistant. There's a new feature that you have in mind, you want to implement it. And your job here is just to get on the same page with the coding assistant for things like your architecture, if it's an existing code base, like what part of the code base do I have to touch to build this, that kind of thing. And then your conversation transitions from unstructured to very structured. So you have these key sections that you have in some kind of like PRD or document that you output at the end of planning, or it outlines the external documentation that you want to reference the parts of the code you have to touch, the task list for implementing that feature, your success criteria. If you're a product manager, you're going to be able to create this kind of structure plan very well, because that's basically what you're doing here, is you're a project manager for the coding assistant.

Going along the same lines there, we take that structure plan, and we give it now to delegate the coding to the coding assistant in the implementation phase. And so your process here, which by the way, this is all using commands to outline how we do these things, the process for implementation is the coding assistant will first read through the structure plan, grab the tasks out of that, and then knock them out one by one. So it starts by doing a bit of research for that task, implementing the code, doing some quick validation before then moving on to the next task. And it just does that systematically until it knocks out your entire plan. And by the way, when I say task list here, I'm not saying that you give it like an entire epic ngira, and it just knocks out like every single task, like no, your structure plan is for a single task in something like ngira. So you want to keep it really granular, but these are just like all the individual sub steps, like one task might be more something like, write this file or set up this configuration versus implementing an entire feature, right? So we're at the feature level here. And then once we are done with the implementation, we've done a little bit of validation at each step of the way, but we still want to do the larger validation process at the end.

And again, this is where both we have our us as the human and the AI coding assistant is responsible. So our coding assistant as a part of our command structure will tell it how to run the linting, the type checking, unit testing, integration testing, hopefully it built some more tests when it implemented this new feature as well, but it's doing the new tests and regression testing. And then it's our job to do the code review, and then also do the manual testing. So actually running the application.

And so I had to get to this point to explain what a system is and how I operate in a system. But now this is where we start to get into how we can evolve our system and find the system gaps, because in the validation phase, that is where both us and the AI coding assistant will find problems that we need to address. And it's not the case that you have to do what I'm about to cover here for every single bug you find. Typically it's the ones that you see emerge as a pattern, like, oh, this is the fourth time that it has done something like really stupid with my SQL queries.

Like, okay, now I should probably address the system gap here. And so you find the error, you analyze your system. So you're thinking at this point, like, is it my rules that need to be updated? Like, is there just a better testing pattern I need here? Or maybe it's my planning command, my process for how I research my implementation and using the AI coding assistant to help you figure out what this is as well. It's never up to you entirely.

It's never up to you entirely. And so then you update your system, you change the command, you change your on-demand context, whatever it might be, like maybe you update your guide to implementing front end components so that there's a better testing strategy. And then you can go so far as to completely retry the task if you want. And it's a bit time-consuming to do this. But for example, if you change your implementation or validation process, you might completely wipe this first attempt, go back to when you just have the structured plan, and then retry the entire implementation. And I'll do that quite a bit for those larger issues that I've seen emerge quite a few times, because it's important to really validate that your update to the system actually addressed the problem.

Typically, what I do in a PIV loop is I will do the planning, create the structure plan, and then make a commit and get so I have that safe state there. And then I go into the implementation. And if I really do want to fix something in my system and retry, I'll just scrap everything back to that commit with a structure plan and go forward again. And so what I've got for you right now is a live demo. I am that crazy person that is going to use a coding assistant in the middle of a presentation. It does not get more non-deterministic than this, but I also I worked really hard to make sure that it would be a smooth process. So here we go.

So I have this AI agent that I built using my favorite AI agent framework, Pydantic AI. It's just a simple, super simple agent I built for this demo here that helps with web research and managing my email. And so let's imagine that I went through a PIV loop here to add the web search tool to my agent. I'm using Brave for the tool here. And so like I said, we want to keep these feature implementations, these PIV loops, to be very granular, like I'm just adding a single tool to my agent. I'm not going to try to like build the agent infrastructure, connect the LLM, get the system prompt, get the web search tool, and all the gmail tools all in one PIV loop. Like that's not what I'm doing here, so keeping it really simple.

And so as a part of my system, one of the things I have is a validate command. So I'm here in Cloud Code now and I have all these different commands set up, which are called slash commands in Cloud Code. I've got quite a few commands. I'll talk more about some of the specific ones in a little bit, but the one that I'm going to use right now is my validate command.

What this is going to do, I'll show you the process really quickly. I have it just very simple right now for demonstration purposes. It's just running the unit test suite with 56 tests in total. While it does that in parallel, I'm going to do the manual testing for the agent. For demonstration purposes, I have a super simple command line interface here to talk to the agent.

So I can say, for example, what can you do for me? This is a good smoke test I like to do for my agents to make sure all the right tools are connected. I can search the web, summarize research findings, and create professional email drafts. Okay, cool. Now let's actually try something. I'm going to give it a request that will cause it to do a web search. And, oh, looks like we got an error. Run context object has no attribute brave API key. We have a problem in our code base despite passing all tests.

And, oh, looks like we got an error. Run context object has no attribute brave API key. Okay, well that's not looking good. So why is that? Well, well first of all, let's take a look at this. So the agent didn't actually fail. This is important because I want to cover the better validation in a little bit here. It just says that I'm currently unable to perform the web search. So we definitely have a problem in our code base.

This PIV loop was not 100% successful. But then going back to the logs here, so in parallel it finished the validate command, and it's just telling me that I got green check marks across the board. All tests are passing, our health assessment is good. But clearly it is not. We have a problem. Our code is just not working. And so this is where we get to identify our system gap, thinking about what part of our process here needs to be better. Skipping a hop later, maybe I did some planning and some troubleshooting with my coding assistant, I'm thinking, okay, I need to update my validate command.

Obviously unit tests are not enough here. And I know this is an overly simple example, but this is just generally what the process looks like. So I want to fix my validation, but I also want to use the AI coding assistant to help me think about how I would do so. I have a prompt that I already have crafted here that I'll paste in. Because I don't want to type this out from scratch with you guys right now. So I'm going to go ahead and send this in. Basically I'm just saying, like, I tested this manually. Here's the problem that I got. It's not able to get that brave API key because you're using the context from the tool call incorrectly. And so just help me think about the key word I like to use here is meta reasoning.

And so just help me think about the key word I like to use here is meta reasoning. That kind of keys into the LLM. Like I need to do some self-reflection here and think about what went wrong in the process. Like, why did my validation not actually catch this? Because our goal is to, if the AI coding assistant has to make a mistake, well it should at least catch it itself at the end of the PIV loop when it's doing its own validation. And so that's what I'm doing here.

And while this is going, I'll just go back to my slides here and talk a bit more specifically the kinds of things that we can update. And so the first thing is our global rules. And so all the things that I've already mentioned, but I just want to dive into this a little bit more when we think about addressing a system gap. And so for global rules we just have to think to ourselves, like, is this problem an overarching issue that's going to come up as a repeated pattern no matter what I'm doing? No matter what feature I'm building, what bug I'm fixing, is this going to come up? And if the answer is yes, then you should probably add something or update something in your rules.

And then the next spot is our on demand context. And so if this is an issue that's only gonna come up when I am building front-end components, or only coming up when I'm adding agents to my tools, just to keep using the same examples here, then I should update something in my on demand context. I don't need to bloat my global rules right now by adding that. It's just for a specific task type. And then last but not least is our commands. Updating something in our commands. And this really is the heart of our system, because the reusable prompts is what defines this workflow that we're going to iterate on and just keep using time and time again. And so the four core commands that I use, no matter what code base I'm working on, is priming, planning, implementing, and validating.

And so what I mean by priming is I will run this in a new conversation with a coding assistant when I'm working on an existing code base. And it's just a set of instructions for what to read to kind of quickly catch it up to speed on the code base. So you know, priming it for the next feature implementation. And so read the readme, make sure you read the main entry points of my application, that kind of thing.

The planning command is our process for how do we research this new feature that we're looking for. And this is also where I put in my structured plan template. So here are the sections I want you to fill out, like the success criteria and the task list. The implementation command is where we break down how to handle tasks. And then we have the validate command, that's the one that I'm running. That's, you'll see in a little bit the well, actually, we already ran that, so you already saw the output of that.

So when we're going to run again, but the validation is something that we have as a part of the structure plan. But it's nice to like take that out if we want to run our validation ad hoc as well, like we're doing in the live demo here. So those are the three things that we can improve. And right now I'm improving the validate command. So I'll go back here. And hopefully at this point, we've got the output from our coding assistant for what it recommends doing. All right. So let me go back up here. And okay, so I'm not going to read through this whole thing right now. But basically what it's recommending, and this is great, it's recommending running a test or creating a test runner CLI.

So creating a kind of CLI harness for it to actually leverage the agent, like send a request into the agent as a more end-to-end test, which is going to cover things that unit tests obviously cannot, because we're mocking things like the context with the brave API key, that's why the unit test can't find this when an end-to-end test can. And so I could just like tell it to go implement this right now, but obviously I don't have the time to do that.

And so I'm instead going to show you a demonstration based on an example that I generated ahead of time. So I have the CLI created here. I'm just gonna copy it from my example folder and put it in my root directory. And then I'm going to take my new validate command that I also had it evolve for me, that still has the unit testing. So I want to add on, like I'm not replacing the unit test, those are still great. But then I want to do the end-to-end testing as well, describing how I can use this CLI to test a couple of different queries. And this should hopefully surface the error for me.

So now I'm going to go back into Cloud Code, I'm going to clear out of this, and then start it up again. And then I'm going to run the validate command. And so this time it should find the issue and be able to address the issue, or maybe it'll just like tell me like you know here's what I found you want me to fix it. We'll see, it does it for my testing either one of the two just because my command is really brief right now, I'm not super specific. And so yeah, the last slide that I want to cover just while that is running here, you can see it's kind of an art form to switch between slides and waiting for a coding assistant to finish.

So yeah, last thing that I want to cover is this. This really is the true power of systematic agentic coding. Because first, we get to take advantage of every error in our system. So now if it works, that's great. If it doesn't work, well that's also great because now we finally have an opportunity to improve something. It's kind of scary to me when something seems to be working 100% all the time, I feel like there's actually a problem and I just don't know it.

And also, it's nice to think about the fact that like when we improve our system, it's also helping with all future projects. Yes, a lot of our rules and commands and on-demand context is specific to our current code base and you should evolve it to be very opinionated and specific, but we're still taking these learnings that we can now apply to future projects. Like how do we get the coding assistant to not write 1000 line the front end components, for example.

And then also our coding assistant just gets smarter over time. That's always a huge plus. And then last but not least, this helps us build trust. Coding assistants are never going to be perfect, but the more important skill is just knowing how far you can take it. And that's a muscle that you'll grow over time. You'll be confident in your system. Like you'll know where you're at with your system. You know how powerful coding assistants are in the current day.

And so then you with that, you can evaluate like with this new feature request or this bug I have to address. Like am I going to be able to knock this out in a single piv-loop, or is it something that I have to break down more. And you just grow that understanding over time of what you can do, and then as you evolve your system, you just stretch that more and more.

So it's never the expectation that you're going to be able to do everything in one go, but you know what you can. And so I'm going to, just to finish the demo here, I'm gonna go back to my agent and see what I've got. Okay, so it hasn't actually gotten to the point yet where it has given me the error, because it's still running the test. But yeah, we'll get to that in just like a second. Let me go ahead and fullscreen this.

So it ran the unit test, so it did that part of the validation. Now it's getting to the point where it is running the test harness. So it's using the CLI, and I told it to do two to three tests, and so it's just finishing the third one here, and then it'll report the error. And then it's either going to fix it, or just tell me what it found. So yeah, usually it doesn't take this long, but that's part of the non-determinism for using a coding assistant in a presentation.

Or my internet gave out, who knows, but I probably would have got an error if that's the case. So yeah, maybe I'll actually like flip it here and like end the presentation, and then show you guys the results at the end. So let me go back to my very last slide here. Oh, it logged me out as well. That's funny. All right, there we go. Cool, so yeah, very last slide here. I've got a couple action items for you guys.

The first one is just to apply the PIV loop, even if you've never done anything like this before. Just think about how can you plan, then implement, and then validate, and create your command library to help you do this. I'm not typing all my prompts manually for how I want to research and plan and things like that. I'm always doing it through commands, and then through this you're probably gonna hit a bug, but that's your first opportunity to address a system gap, so I encourage you to do that.

And also let your coding assistant help you. You don't have to do it yourself. You want to take advantage of the coding assistant as much as you can, to delegate as much coding as possible, and to help you think about how to fix these problems when they do come up. And so with that, I'll come back to the slide in a second. I hope that it's done now so I can at least show you that it found the bug. I don't need to like address it or anything right now. Okay, so it found a critical issue. Good, but tell me what it is. Okay, here we go. So it understands that there is an attribute access error.

So, okay, finally. All right, I'll end the demonstration here. I don't need to obviously like find the bug and retry, but this is it right here. The code is accessing the Brave API key incorrectly. It should be context.dependencies.braveapikey instead of not including the dependencies. And there we go. It actually went and made the fix as well, and now it's doing that iterating cycle that I was talking about where it is running the CLI test harness again to make sure now that the web search is successful.

Perfect. And I'll go back to the slide here to wrap things up. All right, so yeah, the last thing that I want to say is that if you're interested in diving more into this and you really want to learn how to build your own system for AI coding and create it in a way where it can self-evolve, I got a QR code right here. I'm working on a new agentic coding course in my Dynamis community, and it is just an insane amount of effort that I've been pouring into building this for you. So yeah, scan the QR code if you're interested. Hope to see you there. I had a blast with this, and thank you all for being here.

We have a couple of different questions up on the board, but one of them is you mentioned the PIV loop. Have you tried using test-driven development for it? Plan first, develop the validations, and then implement the solutions, or implement once the solutions are solid. Yeah, that's a really good question. So, I have not done test-driven development a ton in the past. I have a respect for it, just isn't my thing. I have tried it with a PIV loop before, and the PIV loop is flexible. It can work no matter how you want to work. So if you do test-driven development, that's just a part of your structure plan. Like, your first task in the task list for your structure plan is write the tests, and then go and create the code and test against that. And so you can definitely do that. It just flips the order of things during the implementation itself, and how you outline the implementation in the structure plan, but the exact same approach and the way that you evolve the system and everything is exactly the same.

Perfect, and then I think we have time for one more. Do you have your slash commands shared anywhere? Yes, I do. So, in the Dynamis agentic coding course that I mentioned, oh, it logged me out again. That's unfortunate. Well, yeah, in the QR code that I had there for the course, I share all of the commands. I have an entire command library, and then also, I do a lot with open source and Github, and so I do have a lot of the commands there as well. And so, if you go to my Github, there's one repository in particular called the context engineering intro, and that's where I have a lot of commands, as well as a more opinionated strategy for context engineering called the PRP framework that implements the PIV loop in a very intelligent, but more opinionated way. That's a another set of commands that I have available in that repository.

Awesome. Can you share Github name just for anybody who may have missed? Of course, yeah. It's coleam00. C-O-L-E-A-M-0-0. Perfect! Thank you so much. Let's give Cole another round of applause. Thank you, guys.