Think Like a Tester: What to Look For in AI-Generated Code

♪ Developers write code to make things work. I write tests to prove they don't. We have a very complicated relationship.

Hi, everyone. I'm Wendy, QA automation engineer, professionally suspicious for over nine years. Today, I'm inviting you guys into my world. But first, let's meet John. John is a great developer, smart, fast, and now even faster with AI.

AI does all the heavy lifting. He does the steering. Best developer year of his life. What could go wrong? On Monday morning, John gets an epic. Add coupon code support to the checkout flow. There's a PRD. There's a Figma. Someone looked at it and was like, okay, two story points, two days, development and testing included.

He opens up his AI assistant, describes the feature, and the code practically writes itself. John says, this will be done by lunch. Ha, who needs two days? Spoiler alert, it wasn't done by lunch. Now, before we go any further, checkout is not just a feature. It is the reason the company exists. No checkout, no money, no company. I call these flows bread and butter. The things that keep the lights on. The things that you protect above everything else. John knows this. Everyone on the team knows this. Monday afternoon, PR submitted. Happy path works perfectly. Enter a valid coupon code.

Discount applies. Beautiful. Now it's my turn. What happens when the coupon is expired? Bug. The discount still applies. A real user just got money off an order they were not supposed to. John fixes it, resubmits. This went on and on, to say the least.

Two story points, no. More like five. And climbing. Sprint ends Thursday, it's Wednesday evening. You don't wanna be the team that broke the checkout. You do not wanna be the name on the epic that killed the sprint. I've seen it. Nobody plans for it.

It's always the features that seem the simplest. The two story points, the things, you know, that were gonna be done by lunch. This was not John's fault. He was just looking at the code through the only lens he had. The developer lens. The developer lens asks a question. Does this solve the problem? That is a great question.

It just shouldn't be the only one. John sees exactly what he built. But the same code looks very different through different lenses. Developers build towards a goal. Testers look at the same path and ask what crack is going to break first. When AI generates code, it builds towards a goal, confidently, without hesitation. It does not stop to ask, but what if? And on bread and butter, that confidence without caution is very dangerous.

Same code, two completely different perspectives. But here's the thing about the perspectives. They don't cancel each other out. They balance each other. Developers and testers are a partnership. We are two sides of the same thing. You see what works. I see what breaks. Together, we see everything. AI makes development faster, but AI does not know what AI does not know.

When it comes to quality, that's still my job and I wanna share it with you. So when your code lands on my desk, the conversation isn't about bugs. It's about making something good, even better. This is the partnership I want for us. So let me introduce to you the QAI. When John, what John needed wasn't better code. He needed the QA perspective, baked in, automatic there, waiting to be triggered. The QAI is a perspective developers do not naturally have.

The one that doesn't ask, the one that does not ask, does this work? But can this break in a way we cannot see or control? You already handle your assumptions, but QA assumptions are different. This comes from years of watching things break after they ship. Edge cases that usually only users hit. States nobody designed for, failures that only show up in production. This is what the QAI brings.

A completely different set of questions your code had never been asked. The QAI can be broken down into three sections. One for product or project managers, one for developers and one for testers to help them with their building process. Our focus today is to look into helping you as a developer. So let's get into it.

Phase A runs, code alignment. If what you built does not match what is agreed, nothing else matters. You fix that first. Once that's done, phase B takes over. It scans the code, each one looking for a completely different way things could go wrong. One command fires it all, one report comes back. Every finding in one place, organized, actionable, ready to be fixed before your PR even goes up. No switching between tools, no separate scans, no hunting through different outputs. You write the code, the QAI does the rest.

The process should always begin with the code alignment. It is the gate. If what was built does not match between what was agreed, everything else is irrelevant. You have your process to write your code, but did you actually build what you agreed upon? Not just the PRD, not just the tickets, not just the figment, any document, any thread, any Slack conversation where decisions were made are all different sources of truth. The code alignment takes and compares what was agreed against what was built and splits the binding into two groups. What you as a developer can fix right now and what needs products decision before any code is touched. If there's a contradiction, you stop, you resolve them. Then and only then do the other three skills fire.

Here's what it looks like in real life. Product forgot to spec what happens when someone applies 100% coupon code, a free order, $0 at checkout. John wrote the code, the coupon applies, the order goes through, $0 charge. But should it? Does the system still create the order? Does the inventory still get reserved? Does the fulfillment still ship? Does the payment processor know what to do with the $0? Nobody decided any of that. The PRD didn't say it. The tickets didn't say it. Code alignment caught it, flagged it, stopped, brought product in and got them to be involved before we could go any further. Once that was resolved, we moved on to the next.

Failures become dangerous when your code stays silent. Every place your code fails without telling anyone inside or out does not matter where it comes from. What matters is that nobody sees it. Empty cash blocks, swallowed errors, API timing out with no handler, coupon codes returning a 500 and your code just sits there. And if it's new code, treat every external call, every async operation, every data transformation as a potential hidden failure until proven otherwise. Be more aggressive, not less. A 503 from our coupon service. This happens rarely, maybe once every few hundred calls, the code had a catch block, it caught it, logged, showed nothing, just returned. The user saw a blank response, tried again, moved on, didn't purchase. No money, no alert, no ticket, no idea. We found it three months later in a session recording, three months of people quietly abandoned the checkout on bread and butter. The catch block was there, the failure was handled, it just wasn't visible. That's the difference between handled and hidden. Fast code needs a security lens.

AI builds fast and trusts everything, every input, every response, every assumption about who is on the other side of that question. This skill takes that same AI and teaches it to be suspicious. Two angles, two questions. First, the OWASP top 10. Every vulnerability on that list exists because code trusted something, it should have verified. We are running your code against all 10.

The trust gap. What your frontend assumes that backend validates was that backend actually validates. Those two things are rarely the same. We are specifically checking the gap between them. The gap is where your bread and butter breaks. The gap is where the coupon discount applied for free, and in this case, it has a literal price tag on it. AI does not think about any of these things by default. This skill does.

You cannot see accessibility failures unless you experience a product differently. Invisible to almost everyone. No screen reader. You will never know that the input has no label. You will never know that the button cannot be reached by your keyboard. You will never know. You will never know. Your user who relies on assistive technology, they know. An invisible code field on your bread and butter is not just a minor issue, it is a broken checkout for a whole group of your customers and it's been there since the first commit. Accessibility is not nice to have. It is a requirement and it's almost always the last thing anyone checks. This will help you find every missing label your code leaves behind. Four skills, one goal. They are separate skills because they are four completely different ways your code can fail. A missing requirement is not the same as a swallowed error. A security hole is not the same as a broken screen reader.

Each one of these has its own focus and questions for it to surface. If we were to combine this into one scan, it will miss a bunch of them. These four skills are a base. They will help you immediately but the real power is what you make of them.

Every team has a pattern. Things that show up often enough that you stop being surprised and stop being ready. Let me give you an example from my team. In our app, navigating directly to a specific URL breaks the page. Click a button to get there, it opens Fyde. I have seen this so many times that I have a test for it but by the time it gets to me, it's already a bug.

That pattern has gone into my skill. Now, the developers catch it before it even reaches me. That company specific section is empty and waiting, fill it with your team's nose. Without it, AI is just autocomplete. Fast, confident and completely indifferent. Right or wrong, it doesn't matter. Effectiveness does. Does it do what you agreed? Does it handle what it was not supposed to handle? Does it protect what it was supposed to protect?

John and I work together to make it as effective as possible for our code base. This is what the QAI allows you to do. Make it effective, make it yours.

Okay, take two. This time with the QAI. Same Monday, same Epic, same AI helping him build. This time after John writes his code, he uses the QAI. Code alignment runs, two gaps ready to fix, one contradiction.

The PRD and the Figma disagree. John does not guess. He sends it to product, gets an answer. Phase A is now clear, phase B fires. Hidden failure, security lens, accessibility, all scan the code. Findings come back, John fixes all of them. This is even before the PR is done. Spoiler alert, he goes to lunch. It's now my turn. One bug, not six, not eight, one. John fixes it, test run again, all green. Sprint ends Thursday, it's Wednesday afternoon, John is ahead. I join him for lunch. I wanna tell you what that feels like from my side.

When I pick up a PR that has been gone through with the QAI, I already know the obvious stuff has been handled. The expired coupons, the swallow errors, the unlabeled input checks. So instead of spending my time finding what should have been caught before it got to me, I get to do what I actually love. Find the edge cases that nobody thought of. The ones that take experience to see. The ones that only show up when three things go wrong at the same time. That's the version of the job that I love and I want to do it every day. And you could give that to me when you ship clean code.

And you could give that to me when you ship clean code. I know what some of you are thinking. That sounds like a lot more work. It's not more work. It's work at the right time. The bugs did not disappear, they were always there. In edge cases, nobody thought to name.

What changed is when they were found. Bugs found in development cost hours. Bugs found in QA cost days. Bugs found in production cost trust. Bugs found in production on bread and butter cost customers. It adds hours, but it saves days. We started with a day of building, four days of fighting bugs, waiting on QA, waiting on fixes, waiting on retesting.

And we ended with a day and a half of building, one round of QA, and done by Wednesday afternoon. Same developer, same QA, same AI, same epic, different results. Developers write code to make things work. I write tests to prove they don't. But today, I gave you something better than tests. I gave you the QA perspective before the code even gets to me. Four skills, two phases, one report.

AI makes you faster, the QAI makes the speed safe. John still uses AI, still fast, but now he shifts things that work, not just things that look like they work. The gap between it works and it works correctly is where the bugs live, where deadlines die, and where your user trust gets broken. On flows that matter most, you can't afford to find out the hard way. You never, now you have the eye to see it. Think like a tester. Thank you, I'm Wendy. Thanks for watching. You want to be in contact with me, please find me on LinkedIn.