Building the Next Generation of AI Developer Tools

Hello, everyone. My name is Krzysztof, and I will be talking today about developer tools, AI, and how we use both of those to create the next generation of developer tools.

Before going further, I need to tell you one thing. I'm not a data scientist. I'm not a machine learning specialist. I don't have a PhD in maths or anything like that. I'm just a software developer, as I imagine most of the people watching this talk today. And, yeah. So it will not be talking about some deep understanding of AI or anything like that. I just don't have it. However, I will focus today a lot on user experience and hopefully some lessons that we've learned and you all can also use while maybe building your AI applications.

Who am I? I used to be a software developer and I'm also a software engineer. I used to be an independent consultant for many, many years of my life. In my spare time as part of my open source night life, I was focusing a lot on developer experience and developer tools. And because of that, around four years ago, I joined GitHub to work on this very secret project back then called GitHub Copilot. At GitHub, I work at GitHub Next. GitHub Next is part of the GitHub that's responsible for investigating the future of software development. We are a group of developers, researchers, user experience specialists prototyping tools and technologies that push boundaries of what we've thought is possible. And we try to go where no one has gone before. Well, at least in the context of software developer tools. Our projects vary from creating familiar fonts designed specifically for developers, to creating a prototype of new extendable user interface for GitHub, to AI tools such as Copilot. They have one common feature, all those projects. They're always about developer experience, developer productivity, and developer happiness. And today, I'll talk about a couple of those tools, mostly focused on AI stuff because, well, that's the topic of the conference, right? And yes, to reiterate, this is the talk about what happens in the middle, how you design this applicative, how we build prototype. I will not be talking about models themselves. For sake of this presentation, we can treat models as the black box that does magic. I'm also not going to talk about building the fully-fledged product. Building those is difficult and it requires a lot of knowledge, not only about user experience or artificial intelligence but also about business strategy and so on, monitoring, observability and stuff. So I will not touch those parts of building real scalable products for millions of users. So let's start with talking a bit about history.

And the tools we all know and use and love. The first one that started it all, and I've mentioned it before, was GitHub Copilot. It was released in 2021 in June or July, I believe. And I've been working on this project since February, I believe, 2021. And I can't even describe to you all how big a shock it was, at least for me personally.

I remember that when I was first contacted by Uge Demur, who was leading Next back then, and he has shown me Copilot for the first time as he described how it works. I responded with, it cannot work. It's not possible. As I've mentioned, I've been working on a lot of developer tools in the past. And, you know, I've been focusing, I've spent years of my life building tools powered by compilers, statical analysis, especially for statically typed functional programming languages, where correctness and type system is everything. And then you come to me and you tell me that, oh, it's using some AI, whatever it means. And we send to this black box a string as an input, and we get back string as an output. And it's supposed to be magic and it works. My first reaction was that that just cannot be.

And yet, well, I joined the team. It was interesting. Interesting challenge. And over a couple of months with a very small team of, I think there was like six of us, six engineers, we moved from badly working internal prototype into technical preview that we presented to world. So let's talk a bit about it. Original Copilot has been designed in very, very extreme way. And I don't mean it in a bad way. We've done what most good projects do. Like what most books or literature about new projects, startups, or user experience design will tell you to do. Focus on one thing and solve it really well.

Copilot's original user interface, the inline suggestion, or coast text, how we called it internally, has been designed with such one thing in mind, keeping developers in the state of flow while they code. It's about reducing context switching. It's about spinning you up. It's about automating boring parts of your work. So you can focus more on interesting bits, building business domain, difficult algorithms, or whatever is the core value in your projects. We as developers are paid a lot of money for our ability to solve those difficult parts, not for typing some boring boilerplate code into editor.

Those assumptions resulted, or those assumptions have been resolved off. One of the, on the one hand, our belief of how AI coding tool should look like. We believe very much into helping users and speeding them up and we still do to this day. On the other hand, it was also related to the state of the quality of large language models back then. You need to remember that Copilot was probably the first mainstream, very popular tool that was using large language models. This was before chat GPT was a thing. And back then in early 2021, quality of those models was not what we expected. No one see today. So, so those assumptions about keeping you in flow and those limitations moved us towards particular vision of user experience. Suggestions are shown directly in editor. They are shown all the time as you type, you don't need to take any additional action to invoke them. Suggestions are based on implicit intent. You don't describe what you do, it's from what you're coding right now, the system will figure it out. You can very easily accept them with just a single keystroke. And similarly, then very easily reject them, again, with single keystroke or just by typing and going further. And suggestions are very clearly separated from the real code. This ghost text, gray text user interface is telling you you're sending a message that this is just a suggestion and it may be wrong.

And then a couple of years later, well, a year later, year and a half later, something else has happened. Compatible chat and similar chat tools exist mostly because of the one reason, the unimaginable success of chat GPT. Back when it was released in the late 2022, no one, not OpenAI, not Microsoft expected such a level of success. If you go to the first OpenAI announcement webpage, it's nothing special. It's just yet another model they built. They had so many of those announcements back in the days. It is announcement as a research preview and the page talks about some sibling models that were also interested. So nothing special. And yet we are in the world with hundreds of millions of users later and we clearly see how AI has became a mainstream topic.

Non-technical people who have no idea about software or artificial intelligence started using it every day. It caused crazy normalization of large language models as a technology. And also it created in society, in general expectation that artificial intelligence tool is chat interface. I will come back to this point a bit later in the presentation, but this has been a very impactful and what we've been doing. Just like I mentioned that the original copilot user experience, this inline suggestion was an extreme. So the grid chat interface is also an extreme. Just on the other side of the scale. It is designed to take your attention. It moves you away from what you are doing right now to a separate window or to separate panel in context of ID. It solves quality of the response issue completely different than inline suggestions. In inline suggestions when you see a wrong response, you just ignore it or you just keep coding. Here you iterate on it by asking follow-up questions of telling the model is wrong and it should try something else. And inline suggestions have been very implicit. Chat is very explicit.

You need to type into it what you want and what's your problem and kind of how we want to solve it. However, those things doesn't mean that the chat interface is bad interface. It's just good at different things. While inline suggestions are great at keeping user in flow, chat works great when flow doesn't exist at all. When you want to stop and to plan or want you to learn something new when you don't know what are the next steps. You can stop coding. You can iterate with chat on the solution and plan how to move forward.

A lot of my friends are using chat as a brainstorming tool. It's a great user experience for that. I also wanted to stress that I put it on the slides. Those like first generation and second generation terms. That doesn't really mean that one of those tools is better than another or that one replaces another. It just merely means that they were created at different time based on different assumptions. They can be used together. They just solve different use cases.

And... And that's kind of... We're moving slowly to the core of this presentation. As I've mentioned, those two user experiences are on the extreme sides of the same scale, the same axis. However, we still have this huge, huge space in the middle. And that brings us to the next generation of tools. The tools that we are creating or are getting created today. And the space in the middle, I personally believe, is all about structured exchange. So as a developer, I want... I know what I want to achieve. For example, implement new feature or fix a bug described in GitHub issue or maybe create new, small application from scratch. And I want to do in streamlined way where the processes... Where the processes I should follow to move towards my goal are modeled. I don't need open-endedness and option to go anywhere that chat gives me. But I also don't want this very extreme, streamlined, implicit AI suggestions that are given by the ghost text. And over the last 18 months or so, we've built two different tools that fits this space.

First one of those tools is Copilot Workspace. Copilot Workspace, if you haven't seen it yet, you should go to Google it and then try it. It's a complex multi-step system that takes developer from issue or ad hoc field description to the ready pull request generated by the AI. However, instead of trying one shot the problem in style of fully autonomous agents, you know, we've seen such demos online. What Copilot Workspace is trying to do is attempting to replace... It's attempting to create this rich workflow that enables user control, feedback and iteration at every step of the process.

Oh, I will just describe more about like what I mean. We start with brainstorming and suggestions on how given tasks can be approached. And then we move to high level specification. Then we generate technical, lower level plan, and only then we move to the code implementation. And after that, we can also run and validate the changes. Every step of this process is user-controlled. User can see what's going on and can make changes to proposed, for example, plan. Users can iterate together with AI and accept every proposal, every offering from it. At the same time, it's not just very loose and open-ended like chat. We still have this very well modeled workflow.

At its core, it's all about... It's very similar to normal application design and normal user experience design. We need to model user processes. That's what we do creating applications. Especially the more complex is our system, the less chance there is that we will achieve appropriate results, leaving artificial intelligence to its own devices. Those large language models are very powerful. And part of their magic is how general they are. They seem to be capable of performing any task, but users in our applications doesn't want to perform any tasks. They are interested in performing their particular task. So we need to understand how they move forward to achieve it. And then user experiences of our applications should model that and fulfill the role of moving people forward towards end goal. This time, just with help of AI.

The second product that we've created is GitHub Spark. It's a tool that takes you from idea described in single sentence to working micro application running in the browser. The tool was not designed to be replacement for developers or to build fully-fledged big applications. Instead, we try to lower the barrier of entry to just create small fun things you or your maybe non-developer friends always wanted to build, but they never had time or skills to. Stuff like an app for tracking a weekly karaoke night or maybe small allowance tracker for kids, things like that. The core principle of Spark is focusing on act of creation. We don't want users to focus on code. In fact, I believe that the code is, by default, it's hidden.

There is no code in this view by default that I'm showing to you. You need to press the button to see it. Instead, we give users tools to iterate. We put application preview as a main point. We give users ability to change application, both with AI, but also with tools like theming support or data browser.

What have we learned building those two projects? On the one hand, they're very different projects, but they have some common aspects and common themes that we've noticed while building those. First one is, well, the most important one. We always should design applications with user in mind, with human in mind. We need to think about enhancing user capabilities rather than replacing users. This AI tool should be directly integrated with inner development loop. So it shouldn't be some weird, separate chat window or anything like that. It should be living very close to the task that we are performing. We need to model the workflows and processes that users have. It's core idea of building any good applications. And we need to have a low cost in terms of how many actions you need to take of wrong suggestions. Those models are awesome. They're getting better and better, but they will always be sometimes wrong.

We need to design defensively for failure. So the wrong suggestions or any suggestion should be easy to ignore, easy to retry, easy to iterate or easy to modify. And those four principles, they can be implemented with different user experiences. I've previously discussed how both inline suggestions and chat user experience are very extreme approaches, very completely different approaches, but they both of them, they follow those principles. We also learned, and this is very important lesson, we learned that most tasks are ambiguous. It's very easy to create nice looking demo of autonomous AI agent that implements some kind of hello world feature without any human input. However, it turns out we've learned that from, especially from the Copilot workspace, the vast majority of real world software tasks are highly ambiguous.

However, it turns out we've learned that from, especially from the Copilot workspace, the vast majority of real world software tasks are highly ambiguous. They need extensive research. They need clarification, refinements, corrections, explorations. It applies all the way through the completion of the task from the very first point of the start to validating the changes.

Many tasks are actually the form of exploration. It's another thing we've noticed. And because of this, real world tasks need constant human guidance adjustments. And so on all the way through the process. We cannot leave the process of building software to the fully autonomous agents.

And that brings me to probably the core point of this presentation. We've coined internally this concept of co-agents. So agents are usually AI agents are usually implemented as a loop, which calls AI over and over again while performing some tasks, while getting its context, its internal knowledge updated. With co-agents, we want to make sure that the human is also part of this process and can take actions to steer agent behavior. Co-agent is controllable, guideable, cooperative, unfolding of the reasoning and decision points of the agents.

Probably last thing I've learned is that we should move past using chat for everything. ChatGPT created this expectation that I've mentioned. This expectation of chat is AI and AI is a chat. However, we can go past that. There are different approaches and we shouldn't be afraid to experiment either as users or as developers trying to build software.

Again, Copilot Workspace is a great example of the philosophy, but I don't really have more time to go into details of that.

Probably last thing I've learned is that we should move past using chat for everything. ChatGPT created this expectation that I've mentioned. This expectation of chat is AI and AI is a chat. However, we can go past that. There are different approaches and we shouldn't be afraid to experiment either as a users or as developers trying to build software.

Let's quickly peek into the future and those are just ideas, like those are... Don't quote me on that or don't tell anyone on Twitter that this is coming in next five months or anything like that. No, those are just random ideas that we had.

So one of the ideas is having this bidirectional editing experience. So you can imagine that you're in your IDE next to every function that you have in your IDE. On the right-hand side, you see natural language description of it and when you edit the code description updates. But also it would be really cool if we could update the description and then see the code changing. Idea.

Taking that further, maybe we can go from full-blown specification in natural language into app develop into fully-fledged application and then when we edit code of this application, we get back changes to the specification. Who knows? There's also the huge space currently that's not explored at all. As we know from years and years and decades of research about software engineering, software developers spend 80% of their time while not coding, but looking at requirements, trying to understand things, trying to understand codebase, and so on. Currently, AI tools are great at optimizing how productive you are in the context of code. And writing more code and more code. However, there is this huge space of whether we should that to explore research and explore understanding of the code bases.

And also we need to remember that we are still very early in the days of AI-powered developer tools. It's been four years since original copilot, slightly less than four years. And we still don't know exactly what is actually sticking on the market. Right now in 2025, there is a huge boom on AI agents, preferably fully autonomous AI agents. We have no idea if those are actually productive and making developers productive. So let's remember that there is still a huge, huge adventure in front of us.

And I'm a bit over time, but this is the slide that I always put on any presentation that I do about AI. When your boss, your CEO, your manager, whoever comes to you and asks you to build some AI-powered feature because our competition is doing AI features, it should be also your responsibility to think and to make a decision and to ask them about, should we do it? AI tools are very powerful, especially those autonomous AI agents are getting kind of like, some of those demos are ridiculous. But you should always, again, always think about human in mind. Do you need 100% accuracy in your tools? Then probably AI, any usage of AI is a terrible idea. Probably today there are people using chat GPT to doing their tax returns. This is probably crazy idea, like, don't do that. There are probably people that, some middle managers that try to convince someone to build some financial app, to add AI to some financial apps. Please don't do that, I wouldn't use your application. There have been this example of huge drama where someone who has built the webpage that helped you to connect online to mental health specialist. And they suddenly started to replace some of those mental health specialist with AI under the hood, not telling users, this is not ethical at all. You shouldn't do that. So when your boss comes to the room, ask you to do AI, because everyone is doing AI. Please, think very hard whether you should really do that.

And that's all. Thank you for watching and see you next time. Bye.