Spec Driven Development: The End of Vibe Coding

Welcome to my short talk about spec-driven development. We're at the end of wipe coding, or maybe better, wipe coding done right. First, let me introduce myself shortly. My name is Daniel Sogel. I'm from Germany. I'm working as a principal consultant for Thinktecture. My main focus normally is, of course, software engineering, but I'm also really interested in this kind of topic, how to make developers more productive, and of course, also about generative AI. I'm also a Microsoft MVP for developer and web technologies, because my main tech stack is web application programming and also GitHub Copilot, for example.

This is why I'm promoted as an MVP from Microsoft last year. Let's dive into this topic. I'd like to start first with the problematic part where AI agents nowadays are still struggling. First, maybe when a new LLM is released, you will find some benchmarks. LLM providers normally try to convince you that their LLM is really good when it comes to software development tasks. Of course, this is not a false thing. It's correct, but it depends on which benchmarks you are taking a look at. For example, there's this famous SWE benchmark, but there's also a pro version available or pro score. This compares LLMs to each other when it comes to more complex, more real-world tasks.

When you take a look into this benchmark scores, you can see that even new LLMs like, for example, SONET 4.5 are still struggling in solving those real-world tasks. Also, it depends, of course, on which numbers you are taking a look at. For example, the latest Stack Overflow survey from last year shows you that 66% of the developers need to spend more time fixing code when they use AI-assisted coding tools. Maybe on first glance, it looks like everything is right, but when you take a second look into the code generated by your favorite AI agent, you can see there are multiple mistakes and you need more time fixing them.

What's also dangerous and important to take a look at are security issues and security faults in AI-generated code. You can also see numbers like this, 2.7 times more security issues in AI-generated code. Of course, all those problems need to be solved. Hopefully, I can show you a solution to those problems using Spectre from development.

So, there are five core limitations identified, and all those limitations should also be solved by Spectre from development. First, there's context management. We only have small context windows. Of course, it depends on which LLMs you are using, but normally we have around 200,000 tokens of context window sizes. Of course, this context window is too small to fit all company knowledge and also not your full coding base into this context window. Then we have multi-file coordination. So, when you're working in complex projects, you not only have one or two or three classes that are tied together. Normally, you have more complex flows in your application, so your agent needs to update all those files and oversee all those dependency structures in your code base.

Then we have silent failures. More on that on the next slide. This is the most dangerous issue we have these days when it comes to AI-generated code. We have architecture decisions. This is also often a problem when you're working on an existing project where the architecture is already not good implemented. When it comes to AI-assisted coding, AI agents are also often struggling into following architecture decisions. When you use them in an existing project without a good architecture, of course, the results will be really, really bad. At the end, enterprise context is also a big problem because sometimes you have knowledge inside a company that is maybe not documented somewhere or not in your code base. So, how should the AI agent know that there are some rules in your company that the AI should follow when it implements code for you? How to solve those issues? This is when spec-driven development comes into place.

But before that, it's really important for me to talk about this and the dangerous patterns. Because when you use AI agents, and I'm using them every day, often you can see that they remove safety checks from your code. For example, you have a try-catch block and the agent thinks, okay I don't need this try-catch block because we have another safety checks in our code base. So, maybe it's not relevant. So, let's remove this or some locks are removed so you will never see that an error happened. We have also this problem of making you happy, I would say. So, AI agents try to make you happy by generating code that fits into your estimated you have into them. So, you give them a task and the agent solves it. But often, agents hallucinate and they add falsy or wrong data to the code. So, the code task looks like it's completed but only with the help of fake data. This also happens often when you, for example, write unit tests.

There's this quote. I think this is also a really good quote. This kind of silent failures is far worse than a crash because flawed outputs will often work undetected until they surface much later. So, maybe you know that in software development, as early as you catch some bug, it's cheaper, it's faster to solve this bug. And when you have a white coated code base and you try to solve this issue later, of course, this will take too much time. So, finally, we can talk about spectrum development. And first, it helps us to put the complete context up front. So, when we later use an AI agent to solve our tasks, we don't need to wait until the agent finds all the relevant context needed to solve this task because we have done this work up front. More to that, of course, later. We have structured reasoning, so the agent always knows what it should do step by step. We have this together with the validation checkpoints. This is also important. So, it's not just like, okay, the agent generates code, but there are also some mechanisms that help the agent to validate its results independently.

Then, we have this autonomous implementation. Of course, this is currently not working 100%, but this is maybe for future LLMs and tools coming in the next coming of months or weeks. The agent can completely solve all those tasks without our need to take a look into the code because we will combine spectrum development together with test-driven development, for example. So, we build really strong guardrails for our AI agents. I think this is really important.

I think this is really important. We should act with our AI agents like they are our pair programmers. So, this is why Microsoft calls it copilot and not like stupid search engines. Of course, they are really good when it comes into searching through our code base, but maybe when you already used such AI agents, you know often they lack context when it comes to complex code bases. So, we will combine spectrum development, the what and why, together with behavior-driven development, the system behavior, and test-driven development for code correctness. So, with these three categories of development flows, we should solve all those issues I showed you before.

There are two major frameworks or libraries you can use out there. Of course, there are multiple ones available, but my favorite ones are OpenSpec and SpecKit. OpenSpec is a bit more lightweight. It's made with TypeScript, and the workflow is really simple. There's only a proposal phase where you define your specification. Then, we have our apply phase. So, the AI agent is solving the specification or implementing the specification for you, and at the end, you can only archive the results. That's it. It's a really simple solution. On the other hand, you have SpecKit from GitHub. It's based on Python and a lot of shell scripts, and it's more comprehensive.

So, we have more tasks we can do with SpecKit. So, there are different so-called slash commands or custom slash commands you can run, and there are four phases. We have specify, plan, task, and implement, and I will later show you how all those phases are working together. And this compared to each other, there are different projects you can or should use those tools. OpenSpec, because it's more lightweight and more simple, is better for brownfield projects. So, you have an already existing code base you want to work on, and on this existing code base, you can use OpenSpec. But when you have a greenfield project, so if you have not touched any line of code yet in your new project, then SpecKit is the tool for you.

But you can combine it, of course. This is something I also like to do by myself. So, when I have a greenfield project, first I use SpecKit to specify everything more detailed, and when the foundation is done by SpecKit, then I can switch to OpenSpec to generate code or smaller code pieces. So, we can also see that, yeah, spec-driven development is already a thing in the industry. So, for example, there's an own IDE developed by AWS called Keyro. So, yeah, they launched this IDE, and yeah, over 250,000 developers are using Keyro after three months.

So, yeah, we can see a lot of developers are playing around with this technology or this technique to develop software with the help of AI. There are also some internal studies, for example, from Delta Air Lines. You can see that 94% of the developers using spec-driven development are really satisfied when it comes to this kind of workflow. And we can also see on GitHub, for example, that all those libraries are gaining a lot of attention. Also, here, SpecKit, as an example, after one week, we saw that over 16,000 stars on GitHub already reached. So, we can see a lot of developers are adopting this pattern really quick and really fast.

So, let's dive into the four phases, specify, plan, task, and implement. And before you even start writing one single line of specification – not code, but specification – what is really important is to define the so-called constitution. In this constitution, you define rules that the agent later should follow, no matter which task it's working on. So, for example, in this short demo, I want to build an authentication API with Express and Node as an example. So, it's really, really important that security is always first. So, for example, we should always validate our inputs, as an example. We should always log out errors. This is really important later. Also, I showed you this in the slides before.

This is really important later. Also, I showed you this in the slides before. I also like to combine spec-driven with behavior-driven and test-driven. So, when you use AI agents and tell them you have an existing function, as an example, and tell them, please write me unit tests, they will normally write evocreen tests. So, those tests are written to fit your existing code. So, those unit tests will never tell you that there is a bug inside your existing code. So, this is why test-driven development works really good when it comes to AI-generated code. So, in this constitution, I tell the agents, okay, when you write or implement later a task, you should write the test first and implement after that. So, you write a solid test, the agent generates code, the test will be executed, and then you can see, okay, is the test passing yes or no? And it's funny to see, because I'm using spec-driven a lot in my projects, you can see often that the implementation fails on the first try. So, it's a really good pattern. Use test-driven development in your constitution.

And, of course, it's really condensed down. There are more rules you can add to this file. It depends on your preferences. And, of course, you can always use AI to ask your favorite LLM to maybe improve your constitution. And after that, it's finally time to start with the first phase, the specification part. This is the only file you really write by hand, because it's the fundament of spec-driven development, because this is the name. This is where the name comes from, the specification. And the specification is about the what and the why, and not about the how. So, it's to define everything, what you like to achieve and how to achieve, but not with which kind of technology you like to achieve your goals. So, you don't write something about, I want to build a backend using NextJS, and the frontend should also use NextJS, for example. Now, you write what you like to achieve and why you like to achieve. And then, you can see here, some outputs are a bit flaky. You can see Copilot, in this case, created five user stories, but named them P1, P1, P2, P2, and P3. So, it's just a broken output sometimes, because this is how LLMs are working.

But in this case, Copilot created five user scenarios based on my specification on creating an authentication backend. And we can also see this specification is also broken down into functional requirements, entities, success criterias, and that's really important edge cases. You should also always think about edge cases, for example, when it comes to authentication, how should later the backend handle a token that is not valid anymore, or the refresh failed, something like that. It's really important to think about edge cases. Then, there's this planning phase.

And in the planning phase, you can get technical. And this is when software engineers can use their skills. Hopefully, you know how to create a good architecture, which libraries you like to use, and so on, and so on. And then, you tell your favorite LLM, again, of your choice, which technologies you like to use.

So, in this case, for example, I want to use JWT, I want to use SQLite, rate limiting, Express, Node 22, LTS, TypeScript or whatever, VTests. It does not matter. Everything technical is now defined in this phase. And it's also, yeah, really important to think about your tech stack here in this phase. And then, you can see here also some screenshots about the output.

You can see, first, there was research. The copilot or the LLM behind copilot fact-checked everything, if those libraries I like to use are compatible to each other, as an example. Then, a design and contracts were designed. You can see here is the OpenAPI YAML as an example to define how the API should look at the end. And there's the implementation plan. How we are just technical stuff together with the specifications should be implemented together later.

And then, only two phases left. We have the task phase, and this is really simple. Normally, you just run the task command in copilot or your favorite AI agent coding tool of your choice. And then, the LLM will split those technical tasks and your specification down into small tasks. And each task can be implemented and tested independently from each other. So, you can, if you like to, in this case, create 89 commits after the implementation. So, each task will be one commit, as an example. And each task can be tested independently. And we can see here, small task authentication back-end with JWT using Express. It's a Greenfield project, in this case. You can see, at the end, it will create 89 tasks. So, it's really, really detailed.

And the last phase is where the magic happens. You hand over this list of tasks and the constitution and the plan and the specification and everything else generated by the LLMs before to one or, if you like, to multiple AI agents, and they will now work on those tasks. And you can also see on this screenshot, because I used test-driven development, you can also see that each file or each functionality in this small demo was well tested. So, we have around 26 unit tests. As I described before in the constitution, test-driven first. So, yeah, the LLMs generated the tests before they actually started the technical implementation at the end. So, I can be sure that the features are working, but of course, you should later take a closer look by yourself into the code base.

So, where does SDD actually pay off? Because not everything is perfect, of course. So, where it excels in Greenfield projects, it's working really good. So, when you start from zero to one, you can use spec.hit from GitHub to create a really comprehensive and detailed description and what you like to achieve. For complex features, it's also working really good because you have the smart down files describing everything needed for the agent. So, you define those guardrails. The agent should hopefully not hallucinate at the end. You can also use it really good for multi-agent coordination. So, maybe you know this is also a thing that you not only have one agent locally running, but it can also hand over some tasks to background agents, for example. So, this also works really good. But SDD also struggles in some projects and conditions. For example, when it comes to small bug fixes, maybe spec.driven is a bit overkill.

For example, using spec.hit for one specification takes around 30 minutes, maybe longer, when you specify something really detailed. And when it comes to bug fixing, of course, maybe fixing the bug will take you five minutes. Why should you define multiple markdown files? Also, for rapid prototyping and maybe solo developers, when you like to achieve something really, really quick, then, of course, spec.driven will make you slower. Of course, maybe the results are better. But when it comes to prototyping, code quality, security is maybe not really important for you. You just want to try something out. Then, of course, spec.driven development is also a bit too much for your task.

So, what are the key takeaways to summarize everything up? So, AI agents are, of course, powerful, but they are limited when you take a look into the current numbers. Like they are creating false or false code, security issues, developers need to fix more bugs when they write code by themselves. Maybe they would be faster. So, they are powerful, but limited. And spec.driven development shifts from code as a single source of truth to intent as truth. So, our specifications define what is the real truth and not the code itself. Specifications become also more executable through AI with the help of your favorite AI agents or subagents.

But it's really important to choose the right tool. I only showed you two, OpenSpec and Speckit. There's also this IDE from AWS. Kiro, I mentioned. There are different libraries out there. So, feel free to try them out so they fit into your specific workflow. And we can also see, like for example, MCP servers or custom instructions. Spec.driven development will be a standard in the enterprise sector, because it's really important for your complex projects that you have a really strong specification. Everything is documented. You know why you achieved something and why you implemented something the way you implemented it. So, this will also be really important in the future.