
Aleksandr Nikolenko
I'm an infrastructure engineer at Perplexity, specializing in secure sandboxing, runtime state management, and engineering automation for autonomous AI agents. I designed Perplexity's "Search as Code" sandboxed environments, which allow agents to safely write and execute complex data-retrieval pipelines. Most recently, I pioneered the automated verification and shadow-testing frameworks used to safely migrate core search pathways from legacy Python to high-performance Rust -successfully deleting 50,000 lines of code under live production traffic with zero regressions.
Perplexity, Armeniaaleksandr-nikolenko-4b272977
Delegating Production Migrations to AI Agents: Safe Verification and Shadow Testing Loops
AI Coding Summit NYC
Upcoming
Delegating Production Migrations to AI Agents: Safe Verification and Shadow Testing Loops

Rewriting legacy code is historically one of the most high-risk, labor-intensive bottlenecks in software engineering. But what if you could safely delegate an entire live-production migration to an AI agent?
In this session, Aleksandr Nikolenko walks through a real-world case study from Perplexity: migrating a critical search pathway from an old Python service to a new, high-performance Rust implementation in under a month. By leveraging autonomous agents, the team successfully deleted roughly 50,000 lines of legacy Python code with zero user-facing regressions.
This talk breaks down the strict engineering contract that made this autonomous delegation possible. You will explore the mechanics behind building ""shadow workspaces"" where AI-generated code can fail safely, and how to implement output verification loops that test against real, live-traffic samples rather than static data. Aleksandr will also reveal how this agentic loop successfully caught schema mismatches and ranking-order divergences that traditional unit tests and human code reviews completely missed.
What You Will Learn:
- Goal-Oriented Delegation: How to turn a massive codebase rewrite into a targeted agent goal with measurable pass conditions.
- Shadow Workspaces: The architecture required to let agents execute and test refactored code safely under production load.
- Live-Traffic Verification: Why continuous output verification against real user traffic is the key to trusting AI-generated code.
In this session, Aleksandr Nikolenko walks through a real-world case study from Perplexity: migrating a critical search pathway from an old Python service to a new, high-performance Rust implementation in under a month. By leveraging autonomous agents, the team successfully deleted roughly 50,000 lines of legacy Python code with zero user-facing regressions.
This talk breaks down the strict engineering contract that made this autonomous delegation possible. You will explore the mechanics behind building ""shadow workspaces"" where AI-generated code can fail safely, and how to implement output verification loops that test against real, live-traffic samples rather than static data. Aleksandr will also reveal how this agentic loop successfully caught schema mismatches and ranking-order divergences that traditional unit tests and human code reviews completely missed.
What You Will Learn:
- Goal-Oriented Delegation: How to turn a massive codebase rewrite into a targeted agent goal with measurable pass conditions.
- Shadow Workspaces: The architecture required to let agents execute and test refactored code safely under production load.
- Live-Traffic Verification: Why continuous output verification against real user traffic is the key to trusting AI-generated code.