The Rise of the AI Engineer

All right. Thanks, everyone, for tuning in. We'll see you next time.

Okay, so we have 20 minutes to cover the rise of the AI engineer. It is a thesis I've been developing for the entirety of my career, and I'm going to be focusing on AI engineering basics. If you are new in some sense, if you are still here, you're at least somewhat interested in it, and I intend to do justice. I'm going to go very fast. This is the first time being at one of my talks. You know that I speak at the rate that I listen to podcasts on, which is at 2X and sometimes 2.5X. All my notes are on my website, as well as on the Latent space if you search rise of the AI engineer.

If you've been somewhat living under a rock, or you've been overwhelmed by too much info, these are the ones I want you to have in mind. AI is the Moore's Law of our time. These are all sort of progress metrics in the past ten to 20 years. What you have on the left is the image generation capabilities that we've had developed over the past eight to nine years. You know, in 2014, we used to sort of develop this as a grainy picture of a face, and now we complain if we can't place our dog in the middle of a generated image. In 2000, we barely solved handwriting recognition with the MNIST dataset, and now we can mostly are at superhuman levels in quite a few core human capabilities that we understand. But it's also caused a lot of AI madness. This is where I tend to distinguish myself from what some of these AI hypebeast people are. You don't find a lot of these at the top of the list of people that you might see online.

So a lot of people are sort of mentioning AI and try to have some AI strategy on their earnings calls, but there's very, very little usage, actually, and there's still a lot of stuff to be proven out. And in particular, one of the most impactful or one of the most hyped projects of the whole year is Auto GPT, which if you're on the stock market, you pretty much run into this over the years, but it's a pretty crazy case in Django, and it's pretty crazy to have that and then to have other people also commenting that no one's using this thing, which is really interesting. There's also sort of I call this sort of manic depression. In the same month, you have investors saying AI has peaked and then also AI is back, coming from San Francisco, it's very common to simultaneously have the feeling that this is what we're looking for, and that this is what we're looking for, and this is all short-term momentum against the backdrop that we are in a longer arc of history where we're just inevitably marching towards sort of a bootstrapping of digital intelligence that will surpass us within our lifetimes. And I would say it's very popular to blame sort of short-term impacts on AI, Chegg is one of them, but I think for the most part, it's not in sort of all the papers. I have a monthly recap on the In Space in my newsletter where I sort of recap the top things you should know as an AI engineer, but really it's just, follow the money. Like, that's, you know, it is the least worst form of figuring out what is valuable that we've developed in the time that humanity has been around. So who's doing a good job? Who's actually making money? Jasper's arguably not doing as well, but I think a lot of people are doing their own best. So there's a lot of people doing their best now, but they still went from zero to 80 million ARR in two years. Generative images, now reportedly at $100 million ARR with a team of 13 people. And chat bots, obviously, OpenAI has a billion dollars just on chat.GBT alone.

There's actually a team of 13 people on that team. And Danny Postma is a good example of that. So that's just some data points I want you to have in mind, that AI is real, AI has a lot of opportunities. There's a lot of hype, but there's also some real long-term progress here.

And real, like, the real central thesis that I want to deliver is that there's an incredible amount of progress and excitement coming from AI over the past few months. And how it's been moving over from React and JavaScript, just like myself. This is all summed up in this thesis, the rise of the AI engineer I wrote a few months ago. And the central thesis is that AI is shifting right.

This is an actual XKCD comic from 2013 where they talked about, like, oh, if I need to do bird recognition, I need five hundred percent, I need to do Bird recognition. I need to do Bird recognition. 17.7 percent zero shot performance up to state of the art. 78 percent on some metric. That is a very, very different state of AI as it is today. From, hey, you need to hire a research team to, hey, you need to prompt a little bit. That is a huge shift in the past ten years. That is a huge, huge shift in AI as it is today.

So, first answer is do all of the machine learning stuff. I have done the machine learning courses on Coursera, not helpful at all. The second answer is, I'm sorry, it looks like the screen is gone dark. The second answer is that you should do data engineering. The third answer is that you should do machine learning on Coursera, not helpful at all. After having done that, it is not really super helpful for me to understand all of the foundation model progress of today. Should I pause to get the screen back? I think the AVT is still figuring it out. I'm still trying to figure out what is happening here. It is still playing on the screen over here. I'm just going to try to keep this moving without screen assistance. I want you to imagine in your head a spectrum from left to right of the roles in an AI enabled organization. On the left you have the ML side, and on the right you have the ML side, and on the right side, you have the ML side. On the left, you have the ML side, and on the right, you have the ML side. Front-end and product engineers.

In the middle, you have the dot line, basically the API line. What is effectively happening is that the ML research org and the ML engineering org is being now implemented in the API and the full stack engineers are going to be working on the API. They're going to specialize into a dedicated role that I'm calling AI engineer. That's the mental image that I would show on the screen if we had time.

Are we fixing that? The projector lamp died. They're replacing it. I'm going to go ahead and put the computer in there and try to really give that AI engineer a good experience. We'll roll with it. A lot of times you have to do error correction in life. There's a lot of stochasticity in dealing with both data and sort of messy hardware issues.

So five main reasons for the rise of the AI engineer. The first one is the cost of engineering. And the second one is the cost of the product in its language. In terms of economics, there's simply not enough ML engineers in the world. The reason why it's about 50,000 ML engineers in the world and there's something like 50 million developers in the world. And the sheer demand for AI stuff means that there will not be a single ML engineer in the world just to fill the gap. There will be 100,000 ML engineers in the world and that's what you have. So like sheer demand supply-wise that there will be a secondary role to the ML engineer that will emerge that will do adjacent to slightly different things.

And the quote that I featured on my slides which you cannot see because the time is one from Andre Carpathi who was commenting on the rise of the AI engineer. He said this which I really love. He said if you have an AI engineer that can do all the things that you can do but that's a little bit hard to do because you're not ever training anything. And I think that's the main dividing line. Are you training models or not? Are you putting them to use? The role of the ML engineer is to help train models and serve them and the role of the AI engineer is once you serve them as an API, how do you take advantage of them and consume those APIs and deliver the last mile to the users. You have to have a high level experience in terms of skills that you can have and to have a high level of understanding of AI engineers and to have an understanding and high conviction that over the next ten years that ratio will flip, that there will be more AI engineers than ML engineers. It's also economics in terms of just underlying hardware. Hardware is hard. Things break all the time. We have to figure out how to protect our infrastructure and we have to protect the computing power for the computing power for the computing networking that's happening by all the big tech players. The term for this is the divide between the GPU rich and the GPU poor. Most of us are GPU poor.

In fact, I have a little hat back home that reflects this. People like inflection, pool site, contextual, and of course the sort of the kind of the sort of the sort of the sort of the sort of those situations. And so, most of these things are already spoken for for the next five years. So, if your plan is to build your own, you know, ML research team, build your own models internally within your company to take advantage of AI, it is not physically possible. But you can take advantage of the APIs that are being served and I think that's why sort of the relative role of the AI engineer will arise. So, that's why I wanted to sort of illustrate this chart.

A lot of the, so, I'm advising for a few start-ups and a lot of the start-ups are not traditionally AI start-ups, but they are sort of discussing AI. So, typically in like the slack channels they'll be like one, I have this like literal highlight of a channel I'm in that is like discuss AI, right? Like and people just kind of enthusiastically throw stuff on the side. My thesis is that those informal slack channels will become formal teams. Or, they'll become informal, but not formally, but I'm talking about like, like, you know, like people don't formally start teams on their own. This already happened in my previous company. So, this is like not even a prediction, it's just a description of what is happening live today. So, that's just, that also happened for me.

I run a discord for DevTools investors where AI and ML used to be one line and now it's pretty much half the channels that we discuss stuff in. It's very similar to, like, what you do on the weekends. You work from home and, like, you, you know, what you do on the nights and weekends becomes what you do in your day job. And then the next point, I think it's third or fourth at this point, it's also enabled by tech. Like, I'm not making pure economics and pure sort of contingent commentary on, like, why the AI engineer will rise. It's also enabled by fundamental advances in technology, like, AI and quantum engineering, you can read the design of the quantum computer engineering pieces, you can also read the segment anything paper that came out of meta, and finally, you can read any number of the quantum engineering pieces, mostly Kojima et al, which is the let's think step-by-step paper. These are all examples of in-context learning or zero gradient learning or zero-shot transfer, and this is where you can get the most out of the model trainer model. In other words, you don't have to be a model trainer to unlock capabilities. You can actually just prompt, or do, you know, other similar activities to prompting to unlock those capabilities. And I think that's very fundamentally different than the previous era of machine learning, where you had to specifically say, like, all right, I want to detect fraud, so I will train a machine learning model and serve it to detect fraud. Very, very domain specific application. So, you know, we will see what that looks like, what the application is going to look like, and then we will use it in a domain specific application by prompting it. So, there is a fundamental tech shift, and I'm not sure if I can illustrate it well without it, but I'm going to try to do that anyway.

Second last one, is a product shift. So I also think about things in terms of, like, being a PM, being a business person, a real business person, and then you serve into your products, and then maybe you improve things by two to 3% in terms of your recommendation systems, or your fraud detection, and whatever. And that's the traditional path of the data and ML engineer. What happens with LLM-enabled engineers, is you ship an MVP first. Then, if the MVP kind of succeeds, then you get a bunch of other people to serve with you. And, as a team, you can serve, but you can't compete. As an ML-enabled approach, you're now fire-ready-aimed, and that shifts a lot of the time to market by orders of magnitude. And I think that is if you're in the startups world, if you're into the agile versus waterfall approach at all, this should resonate with you in terms of the time to market arguments of the time that you're in the market. So, just to recap, ML was very Python, and now AI being available through APIs is available to JavaScript people, so everybody here. And so, like, different estimates occur, but, effectively, I think the JavaScript population is about the same size or maybe slightly bigger than Python.

So we've effectively doubled the tan of AI engineering. A couple, that's four, which is AI innovation, technology, social, sociology, which is the intrinsic desire of people to want to explore informally and then formerly. Three, which is tech, fundamental improvements in foundation models, zero gradients in in context learning. Four is products, shifting from ready in fire to fire ready aim. And five, which is language, just a demographic shift from JS people being able to take advantage of AI finally instead of just Python.

And so, I have this whole section on software 3.0, which is the future of AI. So, I'm not going to talk about this without slides, because I was trying to show you some code. But you can check out the thesis on the essay, if you just search the rise of the AI engineer, you should see it. But basically, the whole thesis is that AI with code is more powerful than AI with language. And I'll tell you a personal story of, like, why I got this idea. So, I was doing a lot of research on AI with language. And I was researching, like, social events. Some of them are tech. And I had a woman come to me and say, I'm all in on AI, I'm trying to learn everything, but I'm not technical. What should I do? And I gave her some tips on how to learn mid-journey, prompting. You can orchestrate your prompts, your chain of prompts, that's why lang chain is popular. And you can use your language models to generate code to do other things. And all of this is locked up behind a wall to non-technical people. So, answering Meddy's original question, when he was talking about, is AI going to take our jobs? I would actually posit that AI is going to be the job that takes away the other jobs. If you believe that humanity has no other desires and will buck the trend of continually being wrong about technology taking away jobs, then AI will be the last job to be automated. I have worked out that logic internally and I'm pretty comfortable with this philosophy.

Okay, so, I'm going to skip a lot of the charts that I show typically in this demo because there's nothing to see. I will just instead direct you to the AI Engineer website. If you go to AI.engineer, yes, engineer is a TLD, and yes, we bought it. It actually shows you a nice little survey of the central problems of the AI Engineer. I think every industry that is forming, just like data engineer, front-end engineer, DevOps engineer, I studied a lot of industrial shifts, and I think AI Engineer is one of the industrial shifts. Every industry has a shared set of central problems that everyone gets together to work to find. I have six of these that I've identified. I'm going to go through some of these, and then we will go for Q&A. I'm clearly not doing as good of a job.

The lamp doesn't like me. First is AI UX. This is where reactive front-end engineers do really, really well at. How do you translate the raw capabilities, the model parameters, into UX that disappears under the hood? I was not able to talk about this, but I do want to talk about what people are doing to try to make the world a better place, and many of you are familiar with Amelia Wattenberger, and she came to my conference to speak about this. A lot of people are basically trying to see beyond the chat box. Chat GBT was mostly opening very fond of saying that it was a UX innovation on top of GBT3. If that's true, then fine. If that's not true, then fine. But what I was trying to say is that the chat box is a very brutally empty state, and it doesn't provide any affordances for people to discover what that looks like. So Amelia Wattenberger in her talk, I highly recommend seeing that. It basically shows ways to toggle on and off different modes and ways to build affordances for you to enhance or utilize capabilities without actually really talking about the in-app as well. I would recommend that you do that. And one of the things that she's working on that's very interesting at ADET, everyone is talking about this concept of AI agents and how things, you can sort of, you can employ AI to do things for you autonomously. But a lot of AI agents today take the form of a floating textbox on your desktop as a Chrome extension within your browser. And I think that's also still not exactly what we're looking for. And that's also not the case, because you can always just punch it in there and then it automates the browser for you. That's the state of the art today in AI agents and that's not really inspiring or useful. But what she showed in her talk, which I highly recommend, which is kind of running everything in the background as though it was a table. And the agents sort of disappears, like the AI experience disappears. That's what innovation in AIUS could look like if you're using the API.

Here is a picture of another way of looking at the world. First is the world of AI. It's AI engineering tooling, so that's the light chains of the world. As well as the evals that you might want to see. Third is the productivity dev tools, here's where all the co-pilots and the sort of code assistance AI tooling is, because I think we need to be aware of the fact that the AI engineering tools are effectively a tool for progressing along, and you might need that for privacy, customization and other reasons.

Fifth is fine tuning and post training, and here's, you know, even more technically advanced open source hosting. And six is AI agents, all of which I cover within the sort of AI engineering curriculum. So I have a whole list of tools, if you want to talk about tool, we can talk about it in the Q&A, but I'm going to skip over all of it in this talk because I want to kind of wrap up. So, yeah, I would say that there's just a lot of content and a lot of detail. I would just say that if you understand the core five reasons why AI engineering is becoming a thing, you can decide for yourself whether or not you want to become an AI engineer, you want to hire AI engineers. You can decide if you like the scope of the AI engineer is something that you're interested in in building, and let's talk about that, you can decide for yourself. And if you want to join me in that, then please, the conference and workshops that I do is at AI.engineer. Thanks for that, and I'm happy to take questions.

Thanks a lot. First question, do you see any issues with equity in AI? Oh, yeah, tons. Yeah, yeah. I absolutely do not think that this is a mitigated good, I mean, I think that there is a lot of different areas of the world where you have different opportunities. In particular, I mean, language is an obvious one, race is an obvious one, and then just differences in sort of resource availability, like the rich definitely will keep getting richer. And so, like, yeah, I think that we have to have sort of the social mechanisms to adjust for that, but it's not really the domain of engineering. And it's not, like, one of the things that we have to build together, but we have to build together.

And next question, can you elaborate on what an AI engineer actually does? And maybe you can describe it with a day in the life. Yeah, I don't know, a day in the life is still very vague, because it depends. Like, many people don't have this title yet. Many people who do have this title differ in their central job description. But, like, as a person who's been working on marketing, engineering for a while, and you look at the jobs page or the team page, he has AI infra engineer, AI compiler engineer, AI product engineer. And so, like, there's all these sort of subdivisions within AI engineering already that are emerging that is really hard to describe which specific direction this thing is taking. I do try to describe the three sort of central types of AI engineering, which is AI product engineer, AI infra engineer, and then AI agents which is the nonhuman engineer. Which we have to accept that it's going to happen. People are working very, very hard at it. And there's some very encouraging early initial results.

So what is a day in the life like? You are either working on products, or you're working on infra. If you're working on products, you are translating sort of user needs into AI requirements and seeing what user needs you can solve with them. So both ways definitely work. And then on the infra side, if you're fine tuning or you're doing model serving, that's all closer to the ML engineering spectrum than sort of product side of things. But it's also a very important part of the job. And then finally if you're working on nonhuman agents, you are doing tests on cogeneration and maybe what we talked about with Amelia, doing like web automation as well. So like there's a lot of different elements. And what I said about the central problems of the AI engineer is effectively what that's going to be focused on. Like I don't focus on so much the day in the life so much as the problems that we share because the solutions will come and go but like the problems will always remain and go deeper.

It's just as wide as being a front-end engineer basically. So I think that's the thing. I don't think that's the way to do it. So how do you get that sort of base? It starts to give you an indication of the technologies that you work with. And that's the reason why subspecialties within software engineers arise. You specialize in different parts of the stack. You're familiar with a different set of tools.

Here if I say you're a front-end engineer, I probably know Reacts, probably no CSS, probably know about browser applications, I probably know about different types of environment. There's a lot of good stuff in there. When I say Evals, everyone wins this. When I say Cogen, everyone cheers. There's a common set of problems that are shared by these engineers, and I think that will be increasingly the focus of the engineering industry.

I'm not sure how to say it. Let me start with the dev. Their company is banning GitHub CoPilot. What can the developers do? Unionize. Unions are out of scope for me. Companies can ban CoPilot, but you can run local CoPilot. That's the name of these models. I can't remember what the other name of it is. I don't know how to pronounce it. The data never leaves your company. It's fine to not use CoPilot at work, but you should try to explore the productivity gains that are experienced by having a CoPilot-like experience, whether it's local CoPilot or any other similar tool.

I would say one hot take that I might offer is the fact that we still need Docs. You don't need Docs for any of the other things that are involved in your application. We still need someone to write the raw text and then to ingest it within LLM, but once it's in the LLM, you don't really need the docs, so you can just do it in context. Because what's better than going to react.dev is not going to react.dev and just having it inside of your code base. It's going to take a little bit of time to get to the docs, but it's not going to be a big deal. The exact reason we need docs, all the LLMs will be one to two years out of date for everything they generate, because that's the training data. Fortunately, for job security, React keeps changing APIs every two, three years.

You want to use that SQL in your components, it's not going to know a single thing about it. You need docs to update it and then you can do away with them.

I've also seen a lot of people who have used LLMs for docs, and I built a chat bot for docs and it was amazing. I could just say give me the code to generate the sticky note using curl and boom, super nice, great experience. It's really good at learning new APIs.

For me, I have never learned CSS animation, again, apologies to Rachel, she's really good at animations on the web. I just never had to learn it, because now I just type in the code, and it's really great to be able to do this with the web. If you really want to learn more, you can check out this chat bot. And if you want to continue the conversation with Sean, he's going to go underneath you, and you can join by clicking the link in the timeline below. For now, it's time for the biggest round of applause the U.S. React Summit has ever seen.