Panel Discussion: Redefining Engineering Careers in the AI Era

Hello, everyone. Thank you. I'm Nihan, engineering manager at Flow Health. I'm not sure if you're familiar with Flow. It's usually mostly a woman audience who is using the app. It's a woman's health app for tracking your period or pregnancy or trying to be more of a health and well-being app in general these days. So I'm an engineering manager there. I'm based in London and here to discuss more about AI.

Hey, everyone, my name is Gregor. I basically grew from engineer to CTO in my full-time career. I currently work as a fractional CTO, coach, and mentor and also advisor mostly to small to midsize companies. I talk a lot about career growth and also what to do as an engineer and also engineering leader in the AI era, especially. That was also one of the premises of my talk today.

Yeah, thanks for having me. And that's going to be a great discussion today. Hello, everyone. My name is Fabrice Bernhardt. I'm the co-founder of CIO, we're an international tech consultancy, and I'm also co-author of a book, The Lean Tech Manifesto, for those who are not there this morning. And yes, we've been heavily impacted by AI and we have 700 people. So the kind of discussions we're having today are very relevant to us.

Hi, my name is Lindsey Simon. I'm the VP of Engineering at Vercel. I live in San Francisco. That's where we're headquartered. And I'm excited to talk to you more today about AI. All right, so we're going to talk about the ways that engineering careers are changing. Let's start out kind of at the skill level. What skills do you see that are being obsoleted today in terms of AI and what skills are just as important as they ever were? And let's maybe start with you, Nihan. Yes, I think, first of all, like in every industry, skills which are like repetitive, like reading something, summarizing something, doing something in a loop, maybe a little bit of analyzing, but perhaps not interpreting. So AI can like summarize points or give you like quick numbers, quick data, and then maybe people still need to use their own judgment to make use of this data.

OK, now I got insight into numbers. How do I make use of it? And of course, also like people skills, relationships like coding can become easy now. But doing the right thing and understanding the users and what products should I build will stay with us for a while, I feel like.

Gregor, in your talk, you talked about how more and more engineers are becoming multipliers. What does that look like at the skill level? What skills do multipliers need? Yeah, I like to say that a lot of stuff hasn't actually changed to what it used to was before, considered to be a great engineer. Three important traits that I like to say are the first one is sense of ownership and responsibility that is even more important than it was before because engineers are owning projects end to end. The second one, I would say, is, you know, being a continuous learner because you need to be learning all the time as you use AI and a lot of other tools are happening. And the third one, I would say, you know, being a team player and making others around you better. That is actually the premise of being a multiplier. So those are the three core areas that I really believe are more important than ever. They were important before, but now it's even more important as a lot of engineers are becoming actually the tech leads.

Another area I would also mention is good judgment and taste. Because the thing is that you develop this with experience. You don't develop good judgment just by people skills. You need to actually know what good looks like and how to actually build the right way to develop that over time. But good judgment and taste is probably going to be more and more important, as a lot of stuff is AI-generated these days.

All right. And Lindsay and Fabrice, you're both leading larger organizations. What's changing in terms of the skills you're looking to bring into your orgs or how you're cultivating those? Yeah, it's interesting because we've always targeted people that had a growth mindset, were very curious and were very fast learners. And that principle is still very, very useful in the AI age. So I would say what has changed, I think as we grew, we started also hiring for experience. So somebody who probably would be less intense in their learning and curiosity, but had like experience in a specific language. Well, that, I think, is starting to go down again. The kind of skills that are just based on having practiced a lot, that is something that AI is making obsolete. And we're back to the basics, which is growth mindset and ability to learn super fast.

I didn't hear anybody say lead code. No, no. Yeah, for me, when I go out and look at people who want to come work at Vercell and people who are succeeding, it's interesting. I think perhaps it's a function of size. But I said before, I'm no longer hiring for people based on skills directly. I think skills and things you know how to do, it's a nice touch. But at a certain point, I want to see people who actually care about the mission of the company and the mission we're after as much as anything else, because I think you can develop fast learners and finding and hiring people who are good learners. If they already love the mission that you care about, the rest gets easier. You know, we don't just hire for skills.

I don't just need someone who wants a job, right? You want somebody who really cares about making the web better, making the web faster, free software and the principles behind it. So it's interesting hearing several of you have said, particular technical skills, the value is going down. And yet I also see all over the place it's very hard to be a junior engineer. It's very hard to get that first position. So what is the set of things that someone just getting started needs to be investing in to get that first junior engineering position? So we're pretty active right now and with internships, you know, it's essentially a way to spend three months. And I'm less interested in sort of what exactly someone accomplishes than how they build into the team, what they're able to demonstrate, like in terms of learning. And you get a lot of signal out of three months of time for someone doing that. So actually, I've been very excited about the new grad cohort now for two and a half years. I was historically a little more afraid of it. So I'm seeing a lot of people show up really capable at onboarding. And thanks LLMs, right? All right, hand the mic down. Yeah.

I think, yeah, it's getting harder to kickstart your career, but my advice would be to try to be more self-sufficient and not be like, oh, I just need to find a job, but try to do stuff on your own, maybe becoming more well-rounded. So these days, I think a developer can also have more design skills or product skills using AI, even sales and marketing, I think. So you can become like slowly more well-rounded, like a more one man band, and maybe that will get you kickstarted in a faster way.

Just to touch a little bit what Lindsay mentioned about lead code. I didn't, I wasn't a fan of lead code even before AI. So I wasn't even a fan of those kind of assessments because they don't actually target the right set of skills that you need to be doing on the road, right? It's a totally different way. Like I was basically solving more than 250 lead code problems and I did well in the interview, but as an actual job, there was none of it actually on the job needed. So now it's even more prominent these days. But yeah, lead codes, I mean, some companies still do lead code.

My recommendation is a bit different. I always enjoyed working with junior engineers who are very passionate, very energized, very motivated, driven to learn new things because that's something that can spiral down to a lot of other team members. Forward deployed engineers offer a good opportunity for junior engineers to get their feet wet in the industry. Freelance projects and building stuff can showcase credibility in the absence of experience. A great computer science degree is as relevant as ever, building problem-solving skills and learning deep systems like system architecture and algorithmic parallel systems is crucial. The Radical Academy training program aims to accelerate progression to tech lead architect level by compressing experience and focusing on both knowledge accumulation and rare edge cases.

That's a good opportunity for junior engineers to get their feet wet in the industry because it's a really booming, booming role these days. And you learn a lot about the users, the customers, the business, and also the same time engineering as well. It's kind of like a mash of marketing sales and also engineering. And a lot of junior engineers I know who do well, especially if they are decent communicators. I would say the third thing would be also freelance projects and building stuff because you don't have experience. You need to showcase your credibility in a certain way that's going to show that you can do the job that is there. So building project is going to be a lot more than just your degrees these days. So, yeah, that's my recommendation. You asked, you know, how do you prepare? I think a great computer science degree, a great university degree is as relevant than ever, if not more. You know, building your problem solving skills, if you're in the right environment and learning the deep, the deep systems, I don't know, system architecture, algorithmic parallel systems, I think will bring you the kind of mentor models that will allow you to better judge whether the AI is saying something completely bonkers or actually a decent thing. And then when we hire people, we are building a new training, a new training program that we call the Radical Academy because we do think that you need to get to the tech lead architect level so much faster than before. And the good news is where the good news is, is that there's two types of things that you need to compress experience. Part of it is the knowledge, like the quantity of knowledge you need to accumulate. And part of it is like the rare, the kind of rare cases, edge cases that are very useful. But, you know, you only see them once every six months. And that's why we're using AI to basically create whole simulators, including client simulators, you know, having whole conversations with difficult business stakeholders and the kind of discussions that you maybe have, you know, once, twice a year, and there you can like train yourself 10 times a day.

Let's talk a little bit more about seniors now, which I think is probably more of our audience. I've heard a wide range of how senior developer roles are shifting. In some large companies, they're ending up all focusing on architecture, enabling juniors to run and run and run and not getting any chance to breathe because the juniors can now code things so fast with the LLMs. In other situations, I see only seniors and they're able to delegate everything to the LLMs and get things to work. And I've heard more and more people saying, hey, I don't even know what my job is anymore. It doesn't look like it used to. So how are you seeing the role of a more senior engineer evolving right now in this LLM era? I mean, just to bounce back on what you said, I think we probably more in the first example you gave, which is focusing senior contributors on the harness, on the architecture, on the standards, making sure that all the negative externalities of using AI are controlled. And that's an intense job. And indeed, putting the builder role of translating features, translating product requirements into features to more junior people. I think senior engineering is about enabling others a little bit, so I would expect senior engineer to help with AI adoption, maybe help create a roadmap for how the company can create a shared knowledge base, how therefore like AI agents can be hooked into our internal systems, perhaps not only like documentation, but also code by enabling good cloud integration with our codebases, exposing our services. So somehow like make the company more efficient using AI and enable everyone else. What I can say, and I think it's a really important message, is that the more companies I speak, nobody really has things figured out. And they're all kind of like learning as they go, like even companies that you would think that have things figured out, like open AI and Tropic.

And they're all kind of like learning as they go, like even companies that you would think that have things figured out, like open AI and Tropic. The more people I talk to, nobody really has the best practice already. But what I do see, and I think it's really important to mention, is that the roles are getting closer together, especially staff engineer, engineering manager, architect, senior, also senior engineers are kind of like in that in that mesh as well. So it's expected to be more as a person who can take many different hats. If you need to be a manager, potentially you could become a tech lead manager inside a team. That's also a common pattern that I see. But yeah, the overall seniors are expected to be more like tech leads. That's something I mentioned a lot in my talk as well. A lot of focus is on good people skills, communication skills, being able to finish projects from start to finish and also delegate potentially to human engineers and also agents and so on.

One thing I want to mention is the pure roles like architects are going to get a lot more closer with staff engineers. And things like that. So, yeah, the expectation is going to be more system design is going to be the clear expectation. I think so for senior plus engineers. And yeah. And then also, you know, I like to say generalists, good generalists and specialists. You can also be an extreme specialist in a certain area. But that may be like a really, really small detailed framework or a certain part. But I think that that's more a minority. Majority are going to be more tech lead like roles. I'll just say I expect the senior engineers on my team to call bullshit. They should be able to detect it, smell it. They've seen it. That's what I'm looking for. We're all at the same level when it comes to like AI engineering. It hasn't been out long enough. You know, the senior engineers are now just we're all junior engineers again, which is fine. And you see this trend in Silicon Valley with companies changing everyone's job title to just member of the technical staff. It's not you're a PM. You're an EM. You're a junior developer.

You're a level four or whatever. It doesn't you know, that isn't the kind of environment we actually want to, you know, leverage our titles for power. It's like, no, I just expect senior engineers to be able to see it and be loud about it with conviction and back that conviction up with data evidence really well. That should be the skill they're carrying over from prior work, too.

So, another question around how all of this is changing is what it does to measuring performance and performance reviews. I think there are some famous examples out there of people saying, well, the use of AI tokens is going to count into your performance review. And surprise, surprise, that led to a lot of AI token use with perhaps not such good output. But realistically, what does good performance look like in this AI era? And how do you measure it? How does this changing things like performance reviews and promotion and that sort of thing?

And Lindsey, I'm going to go back to you because you're talking about a very even flat world. What does performance look like there? Ideally for the companies we work with, our performance should be a function of time spent to business impact. Historically, that's always been the function of a performance review. There's been a lot more nuance, vibes, charisma, and bias in the process historically. We should all look forward to a world where we actually get more data-driven there. We're still very nascent in terms of how we measure this, like, oh, we'll measure tokens. I mean, obviously, it's a very naive measurement with equivalent to lines of code, which we used to talk about.

But I will tell you, historically, you know, if you look at PR throughput over years, typically the top 20 percent is roughly, you know, contains the top 10 percent of engineers in the organization. I've seen it for 25 years. So, you know, I don't know about AI token usage as the singular measure, but we're going to get better at figuring out what gaming looks like and figuring out how to connect the dots between output and business impact. And I'm here for it. I'd like automated performance reviews. I think it's an interesting question because this era is also this era of a lot of bullshit around value. So you are saying we need to measure the value created by engineers. It was it's always been very hard. The definition of value is very messy. But when you live in a world where on top of that, nobody talks about value anymore, I think it makes it more messy. I agree with that. I do think that, you know, you can still evaluate skills and typically because AI has given us a kick in the butt to really strengthen our training program. We can also see how fast people go through the training program. So that's one data point. And then we've always relied on peer reviews. And I don't think AI changes that so much. But as long as the culture of the company is value driven, and I think we're you know, we're lucky to have that, we're lucky to have that and we're going to have to fight to keep that. We're fighting to keep that. Yeah. Yeah, one thing one idea came to my mind listening to you guys talking about how much time are you investing versus versus the value that you create is not that we have more end to end ownership of different engineers, especially this is what is common in companies like OpenAI. There can be easily easily mapped of, you know, how much value are your projects creating? You know, it's not only about the actual work that you're doing, but the outcomes of those projects, because a lot of people inside companies like OpenAI or similar, they're expected, a lot of engineers are expected to share ideas and own them from start to finish. Then it's very easily you can you can map that that effort from that individual to how much how successful has that project been and what has been the impact of that project? And that is like in terms of like users, user acquisitions, retention, business value in terms of like revenue created and things like that. That's what that's one idea. Another another angle would be obviously, token spend is a big topic these days. How much tokens are you spending versus value how much value are you creating? So you want low tokens spent and big value to create. But I wouldn't go that route. It's a more controversial thing to say. But, yeah, I think a lot of a lot of things are still the same as they were before. What entices a good engineer is, you know, how much are they helping others? Are they actually, you know, a team player? Are they making others around them better? Not just focusing on their own individual contribution. So those are still the same.

Just a bit of the process has been changed and the way we are working on different projects. There's a lot more projects simultaneously going on. But I think a lot of things haven't actually changed. Yeah, I also think it's been mentioned, but it's about impact and outcomes, not output. So I would see AI as a tool, but not as the end goal. So we shouldn't look at number of tokens used. But what does the engineer do using the AI? What is he or she producing? And hopefully right now more people can become 10x engineers, best version of themselves. So if you're a QA engineer, perhaps you could do more automation and you can get through your testing faster and that would impact your performance positively. Similarly, a developer can ship more features in a smaller amount of time. A product manager can perhaps prototype her ideas and demonstrate to the engineers what she would like them to build. Therefore everyone is more, I think enabled to produce better outcomes, which would shine in their performance reviews.

May we all get there and actually measure output. Slightly different question. Looking at managers now, I'm seeing and hearing more and more engineering managers back in the code doing coding. And before LLMs, the coding manager or tech lead manager, which I think Gregor, you mentioned that, that was common, but considered unsustainable, right? You could maybe do that up to a couple of reports, do that for a few years, but it was a recipe for burnout. Has that changed? Do LLMs make that the new norm? It's a hard question. I think that managers going, so first it's a good trend, at least temporarily, like managers going where the values created and trying these new things. I've done the same. I've gone back in the team to understand what AI was really changing for us and where the opportunities were. And really like, you know, yeah, see it with my own hands, like getting the hands dirty. So I think that's a good trend. I do think there's something interesting in the fact that in the management position, you're there to enable the team to improve the working conditions and the speed at which you could create tools or speed at which you can now influence some of the processes through AI probably means that, yes, even in quite senior positions, some of the things you would have delegated to someone, you can now implement them just by explaining what you need.

But it's also not the goal. The goal is to create the right working conditions where your team can thrive, where your team can learn and creating the system that will enable that. And that doesn't require full-time coding. That requires going ahead and implementing the tools that you need. That doesn't require full-time coding. That requires going and seeing what is really happening, but probably like a few hours a week and not like half time. And to see what's really happening, like, as always, I've always felt like as a manager, it's your duty to try to push code to see what your user developers are experiencing. If you don't know the pain of the CI pipeline, how can you really push for that work to get done? When people are like, oh, it's really hard to land code.

You're like, oh, no, it's not. But you don't know. So I think, you know, similarly, what's changing in terms of, oh, you know, my PR review came out really sloppy, you know, the review text itself was sloppy. I didn't think about that. You know, you really have to go through that experience to understand what the boundaries are. So I do think there's probably a call to managers to get back in it, like perhaps temporarily, but perhaps not.

I agree with what both of you said. The goal of engineering leaders hasn't actually changed. It's to do what's the best for the team, for your people, and for the organization. So that is the mindset that everyone should have. It's in your best intentions to understand what your people are doing. So in order to understand, you need to try it out yourself and see how things are going. When you feel the problems, when you feel the struggles, you're going to be much more in line to try to make things better.

A shift is noticeable in the industry where staff engineer roles, architect roles, and manager roles are getting closer together. One person could potentially take on another role if needed. This trend is leading towards more expectations from engineering managers. The role of engineering manager is evolving to be more hands-on, making contributions, understanding team struggles firsthand, and encouraging diverse contributions within teams.

They're getting closer together. And one person could potentially take on another role if it's needed. And that's where the trend is heading. And that's what I believe is going to be more expected from engineering managers. So, yeah, definitely. If you haven't already, I would highly, highly recommend to do so if if if time if time is if you can, of course, in your role. What I want to also say, like for directors and above, PVP of engineering. Yes, it's it's it is important that you that you that you have some touch with it. But the roles haven't changed as much as, for example, engineering manager roles that are a lot more expected to be hands on these days.

Yeah. Yeah, I also think the role of engineering manager is sliding a little bit to be more hands on, like my colleagues mentioned here. And it's a bit more about, I think, becoming more enabled and making a little bit more contributions and then seeing your team's struggles or what is most important for them from the first hand. I'm also seeing at work more and more colleagues, more and more engineering manager colleagues are making their first PRs and trying different directions like someone from a back end background is making a US PR or making a PR for tests, which is really encouraging to see. So, yeah, I would say it is becoming more maybe it will be like 10 percent or 15 percent of our time will go to contribution hands on. Yeah, it used to be hard for managers to dip in because it was contact switching, but we're all contact switching now.

We have so many agents to go between. We are just about out of time. Let's just do one quick run through. There's a lot of angst right now. There's a lot of like fear about what's going on in the industry and the pace of change. If you had one recommendation to folks who are thinking about how they're going to take their engineering career, whether they're staying in an IAC role or moving towards management or in management and looking forward, what would your 20 second recommendation be? And we can go whatever order we just walk on down the line. And real quick, just keep on learning and never have a fixed mindset. I think everything is changing rapidly and the only way to catch up is breathing and trying what's going on every single day.