Building for Agent Experience

Hi, everyone. I'm Shifra, and I'm a founding developer relations engineer at Render. That means I work at the intersection of engineering, marketing, and product. And in case you're not familiar, Render is the cloud for builders. Essentially, it is the fastest path to production for full-stack apps, websites, agents, and startups.

And I build up our developer relations org brick by brick the same year that developers conveniently stopped being human, or at least a large percentage of them did. And as you know, it's pretty much impossible to sell to developers. So instead, I build genuine relationships with them and I show them the real value of the product. But how do you relate to a user who's not even a person? That's what we're here to talk about.

Now, Render is growing pretty rapidly. In fact, the company will more than double in size by the end of this year. But there are blips. Our team once had an emergency meeting one day to investigate a precipitous drop in signups after a big spike. But even after we combed through all of the latest updates, we couldn't find any sort of outage or significant change that might have caused it.

The problem was we were looking in the wrong place entirely. When we first saw that sweet surge of signups, we attributed them to AI recommendations, specifically chat GPT suggesting our platform, since we're pretty well established in developer communities by now. And in this case, the call was definitely not coming from inside the house. So what actually caused the drop in signups? Well, OpenAI updated one of their models, and that new version just didn't show us as much favor as the previous version. In the current operating environment, AI answer engine optimization, also known as AEO, is the new SEO. It decides what gets seen by whom. And no one, not even the companies building it, can really stabilize it or guarantee a specific response.

And look, the old SEO had a lot of problems, but at least it was legible. At least the ranking signals were relatively public. Changes rolled out gradually and you could measure cause and effect. So you could actually kind of steer in a somewhat deterministic system.

Meanwhile, AI recommendation is worse because the algorithm, quote unquote, is an opaque black box that can change overnight with no notice. Attribution is nearly impossible to measure. And a single update to an upstream model can cut your traffic in half with zero diagnosable reasons. So you might be wondering, how does that affect the way you run your team? How does that affect the way you build your product?

Well, we see that everything has changed for day-to-day engineering, right. But the strategy side has not kept up. And from where we're standing at Render, this is where leaders are actually falling behind. So engineering orgs already have agents running on both sides, meaning that you use them to build. And in turn, they read your product and decide what to recommend to humans.

These are two emerging subcategories of developer experience or what we call DX, one that's human centered and one that's agent centered. The problem is that usually nobody owns that second one as a legitimate products area. And what you end up with is a lack of intentionality in both. So the question becomes, how do you build for this new symbiotic relationship in a way that's optimal for AEO? And there isn't a clear path forward here. Right.

So instead of trying to build one single solution, what you land on is what we call a portfolio of bets across product surfaces that might take over your annual roadmap if you're not careful. Now, we can start with the most obvious shift for engineers, coding agents. If developers are asking Codex, Cloud Code or Cursor or whatever comes next to deploy an app, the product has to be legible inside that workflow. Being searchable on the web is honestly so 2024. Right. So realistically, what companies like Render need to do is we need to be directly useable by the agent while the developer is working. That means having MCP servers, skills and CLI commands that agents can call independently and safely. It means having API examples that are unambiguous enough for a model to use and docs written for a reader that might not be human, but is acting on behalf of one. This is what we call interface design for agents.

Now, in addition to building out all of these tools, we also did a lot of unglamorous work on the Render docs and website, making everything available as clean markdown, adding LLMs.text and supporting markdown over HTTP content negotiation. All of this work falls under one incredibly important and underrated category, which is making sure that the same source of truth is usable by humans in a browser and by agents that want plain text. That work doesn't demo as well as a chat bot. But if agents are looking at your product, you need to decide the shape of what they read and make sure it just works. Then there are the agents we actually run ourselves, a docs AI answering user questions, an agent streamlining general messages in Discord, a diagnostic Chrome extension that our labs team is testing that actually reads your Render dashboard and proactively suggests fixes. All of them use Render's own surfaces to help a human, which means even within even within Render itself, our own product is being consumed by agents or as I like to say, they use us so that we can use them. Now, content is also a big part of this portfolio.

Our technical writer agent helps us draft and review articles on deployment, framework and comparison questions that agents and humans both ask. The workflow has a lot of agentic steps, but the most important part is that it stops with a pull request before anything gets published. Because if we're being honest, all of this work needs to have a human gate. The workflow stops at a PR, the diagnostics suggests instead of directly mutating and support agents triage to humans a lot of the time. Now, we mostly can't measure which bets are actually working yet. Did the MCP server change what agents recommend? Did the content workflow move any recommendations? So we steer by indirect signals and build the measurement tooling as we go. That's uncomfortable. But navigating ambiguity is an inherent part of our job that we can't offload. And it always will be. Anyone selling you certainty or full automation is selling you slop. The real work is slower. It costs human time. And most experiments won't survive the pilot phase. So at this point, you might be wondering, what exactly is that work, that real investment into this space? It's a fundamental rethinking of how we build and for whom.

But we can't run from that because we still own these outcomes. Right. What we need to do is face them head on by breaking them into the sum of their parts, delegating the necessary time and money and embracing the inevitable failure on this journey. So I want to leave you with the right question to ask on Monday. It's not. Are we doing AI? Because everyone will just say yes with no specifics. The question is, do we have a named function on each side that's about agents using us and us using agents? Maybe it's called something like agentic experience. That's how we think of it here at Render.

Then from there, do we have a budget? Do we have permission to be wrong? And do we have a measurement plan that it doesn't quite exist yet, but we're building it anyway? Do we have a way to really measure? If you can't answer yes to all four, that is the work ahead of you and your team. It's not about correctly predicting all the possible upcoming AI trends. It's about having a system that can self-correct instead of having literally nothing. And once you've planned out that work, you'll need to deploy those agents and experiments to production so your use cases can actually so your users rather can actually benefit from them.

Humans and agents are already using each other, not to mention all of your product surfaces. If nobody owns the agentic experience, that means both sides are being handled by accident. And if that's the case, you'll find out about it like we did by looking at a dashboard you can't explain. The move here is not to track token usage, not to give a trophy to the engineer that burns the most money on cloud code. It's not to generate a bunch of extra spaghetti code and exacerbate the burden of review. It's to learn about the productivity hacks that your engineers are already doing to burn down tech debt and double down on those to find out where agents can actually speed up your processes through team building efforts like trainings and internal hackathons.