Agentic CI/CD: from Pull Request to Production without the Paper Cuts

So welcome, everyone. Today, we'll be discussing about the next evolution in software delivery, which is basically moving our CI CD pipelines from simple autonomy to actually true autonomy. So today, like we'll explore about how to wire intelligent agentic workflows into the development cycle to create processes that are not not just faster, but also fundamentally smoother and more intelligent, which will eliminate the daily frictions that become that we have all come to accept as normal in our development.

So let's quickly talk about the paradox with so it is it is no longer a novel concept. We have been using CI CD for years and years now, and it is actually a bedrock of modern DevOps and high performance engineering teams. And it undeniably has accelerated software delivery, reduced downtime and also improved the quality of products through automation. And we have built these incredible pipelines that automatically build, test and also deploy our code.

However, there is a paradox here. Despite the high degree of automation, the day to day experience for developers is often far from frictionless. So we have successfully automated most of the discrete steps of the processes, which is like building, testing, the execution, deployment and things like that. But we haven't actually automated the intelligence aspect of it. The decision making part of it and the cognitive workload that actually connects these these steps and this gap creates a constant stream of minor but cumulative frustrations, which I call deaths by a thousand paper cuts, which is basically which drains the productivity and the morale of engineers. So let us define what are these paper cuts, because the title of my talk is about paper cards in these. So these are basically small, persistent points of friction that we all encounter daily as as engineers and developers.

First and foremost, slow and subjective reviews are a common issue in software development, leading to inconsistent feedback and delays in the review process. This phenomenon, known as reviewer roulette, highlights the dependency on individual reviewers rather than the quality of the code itself. Security bottlenecks arise from the integration of security tools, resulting in a backlog of alerts and additional work for developers, shifting their focus to patching vulnerabilities. Complex debugging complicates issue resolution post-failure, requiring extensive root cause analysis to identify the underlying problem, whether it's related to new code, testing, environment configurations, or dependencies. Inefficient testing practices often lead to excessive testing for minor changes, wasting computational resources and developer time.

Combining these challenges results in cognitive overload for developers, who are constantly switching between tasks like coding, review management, debugging, and security alert triaging. This continuous context switching is detrimental to productivity and contributes to developer burnout. To address these issues, a paradigm shift towards autonomy and intelligent agents in CI/CD workflows is necessary. By creating proactive, adaptive, and self-healing pipelines, developers can reduce the cognitive burden and enhance efficiency in software delivery.

The path forward involves enhancing CI/CD pipelines with intelligent agents to eliminate paper cuts and improve the overall development process. The goal is to create workflows that not only automate processes but also anticipate issues, make informed decisions, and resolve issues without human intervention. This shift towards self-healing systems aims to streamline development, enhance decision-making, and reduce the reliance on manual interventions, ultimately leading to a more efficient and productive software delivery cycle. The upcoming discussion will delve into the strategies for achieving this transformation over the next 15 to 20 minutes.

So this ambiguity, again, adds more to our time. And fourth is inefficient testing, which is like when we write code, we often have pipelines that take a brute force approach where we run an entire regression test suite for just a small minor change. So like just a one line edit, it shouldn't trigger hours and hours of testing. Yet it often does, which is a massive waste of computational resources and most importantly, developer time. Right.

To combine this, this all culminates into a cognitive overload. Developers are forced to constantly switch context from writing code, chasing down reviews to deciphering a failed build, triaging a security alert. And this constant switching is a well-known productivity killer and a major source of developer burnout. And this is something that we are trying to tackle with this new agentic CICD development.

So what is the path forward here? We basically require a paradigm shift. So we must move beyond simple automation and we must embrace autonomy. And I think my thesis here is that we can eliminate these paper cuts by adding these intelligent agents directly in our CICD workflow. So the goal here is not just to build a pipeline that is a sequence of automated steps, but also a pipeline that is proactive, adaptive and oftentimes also self-healing. A self-healing system is basically a system that anticipates problems, makes intelligent decisions and actively works to resolve the issues without human intervention.

A self-healing system is basically a system that anticipates problems, makes intelligent decisions and actively works to resolve the issues without human intervention. So over the next 15, 20 minutes, we'll also talk about how to achieve this. We will start by defining the agenda paradigm and then we'll walk through the anatomy of the agent power pipeline. An agentic workflow refers to an AI-driven system that can operate autonomously, adapt to changing conditions, and intelligently execute tasks without requiring constant human oversight. The core components of an agent give this advanced capability, including the ability to understand complex situations, plan goals, and maintain contextual memory.

The key differentiator of agentic workflows is the shift from reactive rule-based systems to proactive and adaptive systems. These systems can make real-time decisions to achieve human-defined goals. Continuous Agentic and Continuous Deployment (CCD) represents a deeper architecture change from traditional CICD. CCD transforms the rigid linear pipeline into a dynamic collaborative system akin to a high-performance pit crew in Formula 1, where specialists work together, communicate, and adapt to achieve goals in real-time. The AI pit crew, consisting of various agents like code reviewer, security analyst, test strategist, release coordinator, and orchestrator agents, collaborates to create a fully autonomous CACD workflow.

So this shift is not merely an incremental improvement, but it actually represents fundamental change where the decisions are made. Traditional automations are usually pre-programmed decisions made by humans. In the case of these agentic systems, they are capable of making their own decisions in real-time to achieve human-defined goals.

I'd like to introduce Continuous Agentic and Continuous Deployment (CCD), moving on from CICD to CCD. This transition involves a deeper architecture change, from a rigid linear pipeline to a dynamic collaborative system with a high-performance pit crew concept, akin to Formula 1. An assembly line in traditional automation is compared to a pit crew in agentic systems, emphasizing specialists' real-time communication and adaptation.

The evolution leads to an AI pit crew, comprising specialized agents like the code reviewer, security analyst, test strategist, release coordinator, and orchestrator agents. These agents collaborate autonomously to streamline the development process into a fully autonomous CACD workflow. The progression from traditional rule-based decision-making to autonomous agent-driven decision-making marks a significant advancement in CI/CD workflows, enabling proactive feedback and autonomous remediation for more efficient and secure development practices.

Imagine CSCD as an assembly line transforming into a dynamic collaborative system with a high-performance pit crew like Formula 1. An assembly line follows a predefined sequence where a failure halts the entire process, requiring human intervention. In contrast, a pit crew comprises specialists adapting in real-time, mirroring the agentic pipeline's self-optimizing system.

The AI pit crew consists of code reviewer, security analyst, test strategist, release coordinator, and orchestrator agents working collaboratively for an autonomous CACD workflow. This marks the evolution from traditional rule-based decisions to autonomous agent-driven decision-making in CI/CD workflows. The transition progresses from traditional CIC with AI tools to agent CICD workflows for autonomous decision-making.

The final orchestrator oversees all agents to ensure expected final delivery. The agent pipeline's anatomy illustrates the journey of a pull request through stages, each involving the AI pit crew's collaboration. Real-world tools drive the collaborative efforts of the AI pit crew at each stage, enhancing the CI/CD workflow with autonomous decision-making and proactive feedback.

They do as as what their name suggests. And all of these operate collaboratively in a single in a cohesive way to achieve this fully autonomous CACD workflow. So this is the evolution of these workflows, I'd say. This is a traditional CIC, CIC with AI tools. And then finally, CIC, as we define right now. We have these traditional rule-based decision-making in a YAML file in a regular flow. And we have reactive feedback, which provides us with like build status failures and things like that.

The middle column is what we are using right now. We have tools like GitHub Copilot, which provides suggestions, but you still have to have a human that must interpret the output and make the final decision. And this is still largely linear because the developer acts as a collaborator along with the AI. But we are thinking beyond that, which is our agent CICD or CICD workflow, which is the final part where we want to reach, where decision-making is autonomous and delegated to agents. And the feedback look is more proactive with agents not just suggesting, but also acting on the changes like auto refactoring code and security moving from simple alerting to automatic remediation. So this is what I'm talking about, the evolution here.

And then we also have the final orchestrator, which, you know, as the name suggests, is an orchestrator that overlooks all of these agents and makes sure that the final delivery is as expected. So this is the anatomy of an agent pipeline. This is how the journey of a pull request would look like. So this is how we are going to follow each and every step here. We're going to follow the journey of this pull request through this pipeline and see how our AI pit crew collaborates at each stage using real-world tools that exist. So we have these four stages that I've defined.

Now, let's see the first stage, which is the API programmer or code review. You might have used a lot of these tools. One of them is GitHub, GitHub Co-Pilot Code Review. The moment a developer opens a pull request, the first agent gets to work, which is the code reviewer agent embodied by tools like Co-Pilot Code Review. It directly addresses the slow subjective review paper cut where you don't have to wait for an engineer to review your code; the reviewer starts immediately as soon as you open up a request. It's not just a linter; it checks deeper things that a human would, understanding the code's intent, logical flaws, performance issues, and violations of complex terms or architectural patterns.

The second stage involves autonomous security analysis using tools like Semgraph Assistant. This complements CodeQL by introducing advanced agentic capabilities like intelligent triaging and organizational learning through 'memories.' The intelligent triaging filters out false positives based on the developer's view, ensuring time is spent on real issues. Memories allow developers to provide feedback to the agent, enabling it to learn and adapt based on preferences for future suggestions. This learning ability is a key characteristic of an advanced agent.

The third stage is the intelligent test strategist. After the code review and security analysis, the agent tackles inefficient testing by using predictive test selection. By analyzing code changes, understanding dependency graphs, and consulting historical test runs, the agent prioritizes relevant tests likely to catch regressions from specific changes. For instance, a gaming firm reduced testing time by 30% by intelligently skipping unnecessary tests after minor tweaks. The agent optimizes resource usage, saving compute time, and shortening feedback loops for developers. When orchestrated together, these specialized agents act as a cohesive team under the orchestrator agent in the final stage.

Firstly, it is intelligent triaging. It uses an AI model to actually analyze findings and predict which one of them are going to be false positives and automatically filtering them out based on the developer's view. This directly attacks the paper cut that we are talking about, which is alert for tea and noise, ensuring that developers actually spend time on real issues.

And second, which is perhaps the most impressive feature, which is it features organizational learning through a feature which it has, which is called memories. So when a developer triages a finding or customizes a suggested fix, they can provide feedback to the agent and the agent remembers this preference and applies it for future suggestions. For example, if your organization prefers a specific internal library for sanitizing inputs rather than something else, the agent, you can let the agent know that this is a preference and it will incorporate that into the knowledge and into its remediation advice going forward, which is a key characteristic of an advanced agent, right?

So the stage three, which is the intelligent test strategist, where once the code has been reviewed and it is now secure, there are no security flaws, which it moves to the first stage, which is the testing stage, which the agent tries to solve this inefficient testing paper cut. So instead of running every test in the suite, the agent uses predictive test selection. The agent will analyze the changes in the code that were created and it will understand the dependency graph of the code based and consult on historical test runs. And based on this context, it will select and prioritize the most relevant subset of tests that are most likely to catch a regression from the specific change.

And based on this context, it will select and prioritize the most relevant subset of tests that are most likely to catch a regression from the specific change. So basically, like to give you an example, a gaming firm actually implemented an agent that intelligently skips unnecessary tests after a small tweak. And this actually helped them reduce their overall testing time by 30 percent without any loss in effective test coverage. So this agent acts as a critical resource optimizer, which saves valuable compute time and also shrinks the feedback loop for the developers.

So all of these tools are specialized agents, but their true potential is only realized when they are orchestrated as a cohesive team, which brings us to our final stage, which is these orchestrating like building the team through an orchestrator agent. So these individual agents are powerful, but they don't collaborate together. So we change their actions through an agent orchestration framework, which is you can call it as a central nervous system of this whole setup. And there are multiple tools for that. So, for example, this is Landgraf, which is open source.

So Landgraf actually uses like a graph-based approach where it has a sequence of conditional flows and a conditional edge, which basically checks for outputs of the agents, like, for example, if it will start with the code review agent, it will check the output. If it found critical issues, it will go to the next one. And basically it acts as a flow. It acts in a singular flow. If all the checks pass eventually, then the workflow moves to a final human approval node where the entire process pauses, it posts a summary for the human and waits for the maintainer to give the final go signal. Only after that, the graph is proceeded to the end state where it will trigger the merge. This approach provides the control and reliability needed for production systems, addressing trust and safety concerns.

This approach provides the control and reliability needed for production systems, directly addressing trust and safety concerns inherent in autonomous systems, particularly through the final human approval node. Another perspective is the Autogen framework, contrasting Landgraf with a more dynamic, conversational approach where agents interact by discussing issues collaboratively. This method excels in complex, open-ended problems requiring creative brainstorming and multi-step diagnosis, fostering dynamic interactions among agents aiming to find root causes.

The traditional merge gates, such as test coverage and thresholds, are considered primitive as they simplify judgment into binary outcomes for all changes. To address this inefficiency and risk, an intelligent agent serves as the final quality gatekeeper, aggregating signals across the pipeline to provide a confidence score based on various factors like code complexity, test coverage, security, and performance impacts. This gatekeeper introduces a human-in-loop strategy, allowing high-confidence merges to proceed automatically, ambiguous ones for human approval, and blocking risky changes with detailed reports, thereby establishing a trustable system that incorporates human judgment.

Recapping the journey, the process began by identifying and addressing minor issues through an agentic approach, followed by implementing specialized agents like the AI pit crew. The orchestration of agents using frameworks like Landgraf and Autogen was discussed, leading to the design of an intelligent AI-powered gatekeeper to maintain a safe, trustworthy development pipeline under human oversight. The ultimate goal is to transform the development process into a seamless experience, free from frustrating obstacles, ensuring a state of super fluidity.

This is these all agents are working together and proposing solutions until the root causes is found, as you can see in the in the flow chart here. So that we have actually built a highly autonomous CI CD pipeline, but it naturally raises the question of trust, the traditional merge gates, which is test coverage and thresholds, link checks, vulnerability scans. These are vital, but these are primitive. They treat every change as the same, reducing complex judgment into binary. So this result is both inefficient and has risks of low impact changes like low impact changes can get blocked by unrelated noise, while critical changes can pass without sufficient scrutiny. So with that, we introduce a final quality gatekeeper, which is an intelligent agent that acts as the final arbiter of. Arbiter of of merge readiness. So it aggregates all of these signals across the entire pipeline, which is like code complexity, test coverage, Delta's security, fixed confidence, review sentiments, performance, impacts, things like that. And the agent doesn't just give you an answer, whether it's a pass or a fail, it gives you a most confident score, which is like a number. And it'll give you an explanation that. Low complexity, high test coverage, one they mediated medium vulnerability with like no performance aggressions. And then this course drives the human in loop strategy. A high confidence merges can proceed automatically. Ambiguous ones surface to humans for approval, and the risky ones are blocked with detailed reports. And this creates a trustable system that scales with human judgments. And it keeps engineers focused where they are more needed and elevates merge gates from static rules to dynamic intelligent assets.

So let's quickly recap our journey. We started by identifying these paper cuts and then we saw how an agentic approach can heal these cuts by providing instant intelligent reviews, providing automatic remediation of security problems and implementing efficient and adaptive test strategies. Then we saw our AI pit crew, which, which were the specialized agents. And then we also learned how to organize these agents using an orchestration framework like line graph and auto gen. And finally, we designed an intelligent AI powered gatekeeper to ensure that this pipeline remains safe, trustworthy, trustworthy, and also under human oversight, which is the final result here is a development process that feels like less like a series of frustrating hurdles, or I'd say paper cuts, and more of like a state of super fluidity. So this is a, this is a technological shift and which will inevitably transform our roles as developers. And the good news is that while it automates the most of the tedious frustrating parts, and we will spend less time fixing these vulnerabilities, deciphering build logs, build logs, or waiting for reviews, and which actually frees up to, or frees us up to focus on. Things that actually matter, which is high-level system design, tackling novel architectural challenges, and deeply understanding the business problems that we are trying to solve.

And of course, with this, the developer's role is going to evolve. From, we'll be moving from primary coders to becoming designers and orchestrators of intelligent AI systems. And I believe that the most invaluable engineering skill in this paradigm will be the ability to design, guide, and oversee the agentic systems. And as one expert might say, prompting is the new debugging. And it is not about replacement, but it is about elevation of roles as a developer. So what is the long-term vision here? So the next horizon is a pipeline that is just not populated by agents itself, but is an agent itself. So imagine a system that can monitor its own performance metrics, lead time for changes, change failure rate, mean time to recovery, and use all of this data to reconfigure its own workflows. So it might learn that certain class of changes rarely benefit from like a full security scan and a dynamic, and it will dynamically adjust its own rules. It will be able to experiment with different strategies to find the optimal balance of speed and coverage. And basically this will achieve the ultimate goal of pipeline becoming adaptive and a self-learning entity that is also self-healing entity, which is the ultimate goal of agentic CI CD, which is essentially a development ecosystem that will autonomously and continuously improve its own efficiency, reliability, quality, and allowing human engineers to focus on entirely new things and a new set of skill sets.