From Business Buzzwords to Meaningful Change: Measuring and Improving Engineering Productivity

I'm Lena Reinhardt. I'm really happy to be here today and talk to you about business buzzwords like efficiency, effectiveness, productivity, and how to actually make them work as a tech lead with your teams. Because it's been a bit of a year, the first time I gave this talk was a year ago and things have changed in the industry over the last couple of years. We all know it, we've all been part of it. And efficiency specifically has been the big buzzword in many top tech companies. And I've spoken with dozens of companies and leaders in our industry over the last probably two to three years since things started going into the downturn that we're in right now as a tech industry. And like many of you, I've been trying to make sense of what's going on. And a lot of the things that have been done have been really shocking for a lot of people in our industries, including people in leadership positions. A lot of layouts were justified by efficiency. And as someone who's worked in finance before getting into tech, as someone who has a business background, I do understand that. And at the same time, I also think efficiency, effectiveness, productivity all have a really bad rep and for really good reasons.

And so I did briefly want to take you down a trip on memory lane and look at actually how the space of developer productivity has evolved over time. And of course, when talking about any trends, you have to show Google trends. That's the only thing to look at. This is a chart of how many times people were looking at developer productivity since 2004. And so I wanted to show you what happened over this time. This is a lot of illustrations for one chart, but a couple of really interesting things because developer productivity as a topic has been around so much longer than you actually may realize.

Not everyone here has been in the industry for 40 years. So for everyone who just joined in the last decade, for example, think of things like XP, extreme programming. That's been around since 1999 when first book was published and the author has been working on it much longer. The Adler Manifesto was published in 2001. Here, you can see a photo of one of the meetings of the people who were involved in that. The software craftsmanship manifesto that Fowler and others wrote was in 2009. DORA, the organization for DevOps research was started in 2015. The book Accelerate that is the basis for a lot of people thinking about developer productivity is from 2018. Then the space framework that kind of expanded it was published in 2021 during COVID. And then, of course, the tech downturn started around 2022.

I'm showing this to you to illustrate that developer productivity has been on people's minds for a really long time. And a lot of concepts that you've probably been working with for a long time as a developer like Agile or even XP, those concepts are about developer productivity at their core as well. And that also means this whole topic really matters. Leadership roles, no matter if you're a tech leader, a people leader, you have direct reports or don't. Leadership roles are business roles. They are about helping a business ultimately achieve its goals. In addition, you actually have the expertise to do this well. You probably work with a team, with a set of services in a specific domain. You know how to help the people there achieve the goals that they have and the goals that the business has by extension. And there is the disappointing, sad, annoying side of this. But the time that we're in as an industry right now also means this is also about becoming part of the spreadsheet that has the names of people on it that are going to be laid off. I wish that weren't the case. It's unfortunately the reality of things. And one thing to keep in mind as a lead is the more you're able to connect to the work that you're doing, the work that your teams are doing, to business value, to outcomes, the better your odds are at least going to be.

There aren't guarantees, but it's going to help you a lot in your daily work. And it's going to hopefully also help you with job security. Now that we've gotten the unfortunate part about this out of the way, I do briefly want to recap. Our industry has been changing, and this also means that our roles as leaders are changing. And I use the term leader quite a lot during this talk. And when I say leader, I mean anyone who influences, guides others, gives direction, provides facilitation skills. For example, all of those are leadership skills. Leadership can be at any level. It doesn't have to be about titles or roles. It's about the work that you're doing and the actions you're taking. And so all of these changes mean we really need to talk about engineering productivity, engineering efficiency, engineering effectiveness. And the first thing that we're going to do is tease apart all of those terms. And I've mentioned already that this is an uncomfortable topic. And it's probably not just an uncomfortable topic for you, but also for many of the engineers that you're working with. First of all, of course, a lot of bad things have been done in the name of efficiency. That is the thing that happened. There are also really big words. We will get to explanations just in very short time, but people don't really have a very coherent idea of what those things mean. Productivity can mean so many different things, and there's not a lot of clarity about them. Then in addition, in a lot of cultures, talking about money is pretty taboo, and it's actually kind of frowned upon. And at least for many other people, it's uncomfortable. And even at work, having conversations about money in the sense of what revenue is engineering work generating or helping to generate, for example, it's not just hard, but a lot of people find it deeply uncomfortable. And many leaders in tech weren't trained for this. Many leaders and managers never got formal business training, but they were promoted at some point. And these business topics are quite inaccessible, honestly. And so I still am here, of course, to help you get started, because I found that this is not as hard as it may look. And I'm here to give you tools to either later today or tomorrow get started on this together with your teams. This is me wearing a spreadsheets hat, because I have one. I'm going to move on and let's get started on actually helping you think about productivity for engineering. And with that, I'm going to introduce you first to what I call idea Impala.

Fun fact here is Impalas get just under five foot long, but they can leap 32 foot far. That's the equivalent of me making a 38 foot jump right now. That's, I think, about six or eight meters. I forgot the conversion. That is unfortunate. Someone please correct me on this.

So idea Impalas represent the leap. They stand for effectiveness. Effectiveness is one of the three big terms that I'm going to talk about. I'll share with you how they are defined and how they distinguish between each other and understanding those terms is going to be honestly hugely helpful to help you understand the productivity space, but also how to talk about it. So effectiveness is about doing the right thing. It's about achieving our goals. I always remember this because it has the word effect in it. As in the goal thing that you achieve an outcome and effectiveness means we do what's most important and we get the right outcome. So that means, for example, you're not just shipping something, but the outcome is that customers are actually adopting it. People are using the feature that you built. Effectiveness also, for example, as things around quality in it. So that's part one.

In contrast, there is efficiency. And with efficiency, I want to introduce you to calculator Capybara. The Capybara fun fact is that Capybaras live on land and in the water. They've partially webbed feet. So they're great swimmers in their natural habitat. They usually don't use calculators, but this one does. And this calculator Capybara stands for efficiency like a normal Capybara. It can navigate any terrain to figure out the best path forward. And it asks what actions are we taking and what's the right way of doing things. Efficiency is about limiting waste along the way. Waste can be things like money, time, technology. You know, huge infrastructure costs, for example, can be inefficient, but also mental focus like context switching, for example, can be really wasteful as well.

So and now here's the fun part, which is where we talk about productivity and productivity in engineering is about doing more with the same. So I'm not throwing terms at you that are often honestly used interchangeably, which is what makes this topic really difficult. The big difference between productivity and efficiency is basically how you calculate the outcome. So efficiency is about just doing the same things but using fewer resources, for example, shipping the same improvements, but doing so in less time or using this concept switching, avoiding waste productivity is using the same time or the same knowledge and the same context switching amount but doing more in this time. So productivity is doing more with the same. The basis doesn't change. Efficiency is doing the same with less. This is confusing.

I'm going to mostly use productivity as a term in the future of this talk. One reason for this is that productivity is honestly most widely used. It's also the term that most people have the best understanding of, and it's at least going to be a good starting point. In addition, if you have a set team, you're usually not able to hire people, for example, the main thing that you can do is improve productivity by doing more with the same, figuring out how can you work with the team that you have, with the people who are there, the knowledge they have, the skills, the time, the capacity. How can you work with them to do more with the same? For any leader in tech, productivity is the most common angle to look at. That also means you have these three terms that we've looked at. We have efficiency, productivity, and effectiveness. It's very confusing. You'd be correct. As I mentioned, a lot of people struggle with these terms and how to define them.

In short, I want to illustrate it a little bit. You help your company achieve its goals. Achieving those goals, here's the impala, means that you're effective. Every day, you make countless big and small decisions with your team to help you achieve those goals and achieving them in a better way than you did before. That's efficiency or productivity.

Of course, you can walk down this path, or you can just buy a helicopter and fly your entire team up to your goal on the mountaintop and then hope you don't run out of fuel along the way. That's not exactly efficient because it costs a lot of money. It costs a lot of fuel. You're wasting resources, but it can be productive because you achieve your goal in less time. As long as you achieve your goals, that can actually still be productive.

To recap, you have effectiveness, it's doing the right thing. It's about getting the right outcome and the impact you're looking for. There is efficiency, where you do things right. Efficiency is about doing things right and using fewer resources to get the output that you want. Productivity is about doing more with the same resources that you have. Now, we've got the terms.

What this means for you is, first of all, know the terms, know them roughly. I have at the bottom of the slide, at bit.ly slash eng-productivity, you can find a cheat sheet that has the terms for you, just so you have them ready if you need them, as well as a lot of the content that we'll have over the rest of this talk. You should know this roughly, so you can talk to people about it. The more important thing, almost, is listening for what leaders around you are saying. What terms are they using? Are they talking about productivity? Are they talking about efficiency? Are they talking about achieving goals? Are there synonyms for some of these terms? Listen on that and adjust your own language, as well as the things that you're focusing on with your teams accordingly. Validate your understanding. If you're not entirely sure what your company is trying to focus on, for example, efficiency is more important, or if it's more about productivity, validate it. Ask your boss, ask your product manager, ask the people you're working with.

Lastly, don't get too hung up on terms. As I mentioned, people aren't always using them in the way that they're technically defined, which happens. Instead, focus much more on what's measured and the behaviors that those metrics drive. We'll talk about how to do that in a second.

Instead, focus much more on what's measured and the behaviors that those metrics drive. We'll talk about how to do that in a second.

So, terms are set. What matters most to most businesses, that's my last pro tip for you on this, is outcome with limited resources, which means you achieve your goals, that's effectiveness. There's also things like quality in there, that customers aren't unhappy with the things that you're putting out, and with limited resources, as in with the resources that you have, and that goes to productivity. Limiting waste is a big principle, for example, in Agile, where things like WIP limits are coming from, or the idea of limiting context switching. Efficiency is important in some companies at some points. Don't neglect it entirely, but it's often not the big priority. So, companies optimize for a mix of those things. That's also important to know. Some companies, for example, over the last decade, spent a ton of money just on hiring and retaining talent. That wasn't necessarily efficient, but it was a really effective way to help them achieve their goals in a market that was really difficult. Now, those things have changed a bit. A lot of companies, and a lot of bosses, especially, are looking at productivity metrics. So, can we do more with what we have? That also means for you, as an engineering leader, it's your job to use resources smartly to help your business achieve its goals. And of course, resources are things like time, capacity, and knowledge. People aren't resources, but I'm sure you know that.

So, I promised you a long-winded way to get started, and this was it. Ultimately, as leaders, we facilitate productivity, and I'm here to show you over the next slides how to actually do that. One thing that's really important to me in all of this is to take teams along, because people, none of us can individually do all the things that our teams have set as goals. We need to take people along, and with a topic like productivity that so many people have made poor experiences with, they've worked at companies that just treated people poorly and called it productivity or efficiency. With a topic like this, it's even more important to take people along. And so, I'll show you a framework that you can use to do that. The steps in it are these five, where you know your organization, you define how you will know what success looks like, you know your position, and then you actually make it real. We'll walk through this quite quickly, because it's honestly pretty simple, and, like many leadership approaches you can see here, we start with understanding things, understanding what success will look like, and then actually taking action. This is just spelling this out a little bit more. So, engineering productivity is a team sport, that's our first rule. You can't do this alone, and productivity is in every decision that you make with your teams. So, you start by understanding your organization, what are your organization's goals, and also what is important to your boss. If you work in a company that's quite large, what your organization wants and the goals that it has, maybe not as important as actually just what your boss wants.

So, here are some questions you can ask to your boss, or you can also look at documents, for example, like roadmaps. Things like what are our values and constraints? What are we optimizing for as a company? How are we making and costing the business money, either as a team or as a domain, whatever organization or entity you're leading? In addition to speaking with your boss, you can also look at industry news for budget. If you don't have one, you can ask your finance or HR partner for an employee hour. That's a number that a lot of companies use in order to estimate how much things cost. It can be super helpful if you just need basically a placeholder and don't actually have the compensation of the people who are on your team. It's a good way to still help you make calculations.

The whole point of this exercise is essentially your alignment. You need to understand what your company wants, what it's going for, so then you can pass that forward to your team. Now, this step here is kind of a substep of each of the main steps that I've showed you. The whole idea here is that you validate everything you learn.

So, say you've looked at a roadmap, you've looked at organizational goals, you have a sense that currently your organization really cares about productivity. In addition to that, your organization also wants to make developers' work a bit more efficient. Like, for example, save some costs on infrastructure. You've seen those things. What you then do in this step is you summarize it and you share it with your boss and say, hey, I've been looking at our goals. I've been trying to understand a bit better our strategy so I can align my team.

Here are the main points that I've understood from this. Is this accurate? Does this align with your understanding? Is there any context that I'm missing or that I still need? This kind of validation is honestly a really useful trick in leadership in any case, because ultimately, you still need to own stuff. You need to be the person who tries and finds those things out. Not at every level is there going to be a boss who tells you those things proactively, if we're being honest. In addition, it helps you really make sure that you're on the same page as the people you're working with so you can then take appropriate action. I'm not going to show this slide more, but this should become something that you do on a regular basis. Now you know your organization's goals. You have a rough idea of where it's looking to go.

Next up is we define what success looks like. That's a step that's usually very easy to skip, but I think it's really foundational because metrics and metrics in any sense, either qualitative things or quantitative ones, metrics create incentives which create behaviors, which create culture. Any metrics that you use matter much more than you may think.

That's also why it doesn't matter if you end up using Dora metrics, the Space Framework, DevEx, Flow, or anything else that's out there. It doesn't matter whatever you use if these metrics don't measure what matters to your company and if you don't have your team and boss on board and you don't keep them in the loop on your decisions.

To help you get started, you will very likely need to make a bit of a case for ultimately what productivity means for your team. You can on this slide see a ginormous list of metrics. Those are for effectiveness. For example, goals are aligned. You have the quality that you're looking for in the delivery of things customers are adopting, the tools that you're using, or the features you're putting on. Here's a lot of efficiency and productivity metrics like deployment frequency, lead time to changes.

You may have seen a lot of those in other contexts already, those, for example, Dora metrics. It doesn't matter which metrics you pick. You pick two or three to get started. How to get started is really the most important part, because a lot of us tend to easily get hung up on basically precision in metrics, metrics that are really accurate. That usually doesn't matter. The main question you usually need to be able to answer at this step is basically, how well or how poorly are we actually doing? Getting that kind of visibility is a really important bit. That's why you also start with what you can easily know in impact. If you currently only have things like agile metrics available to you, like cycle time, for example, use that.

With any metric that you pick, it's also really important to avoid wrong incentives. I always like to ask, what if we took this to the extreme? What if four weeks on end, we only focused on cycle time? That might actually not be too bad. Depending on which area your team works in, that actually might have harmful consequences though. It's important to always keep that in mind. If need be, you could adjust with a balanced metric, so, for example, cycle time, and in addition, some sort of quality metric, if that's a concern you have. Always make clear if metrics are for or about the group.

I mentioned that it's important to take your team along, and this is where you're doing this. You can go to your team and tell them, hey, listen, I'm looking to understand a bit better how we can be more productive as a team. Here's what productivity for us as a team actually means, and here's how we're going to measure it. For example, I found that lead time and the quality of the work that we're putting out are really good measures to get us started. How does the team feel about that? Ask them.

They might have some concerns that you'll need to address in the beginning, and that's really an important part of this as well. I always like to tell people if the metrics that I'm using are for this group, as in for a team, for example, to learn together, grow together, or if the metrics are about a group. For example, if I'm reporting on a team, I'm making sure that my boss has visibility. A team should be able to use metrics as a way to learn, and they should always be aware of what metrics are used to report about them. Talk to your team. Keep them in the loop.

At this point, you have probably one or two, maybe three or four metrics. It's really important to keep in mind with metrics that there's signals and not targets. They're just helping you see how things are going. They're not the thing that you run towards. You have your organization's goals. You have a sense of how you will measure if you're doing well or not.

Next step will be to know your position. This is where you just do a short review of where your team is actually at. I usually recommend doing this at least twice a year. If you're in a very fast moving company, like a startup, it can also make sense to do this more often. At this point, you're, for example, looking at what's our budget, how are we making and spending money, where are the big gaps in our technology, where are some issues maybe in our delivery and our process? You should really also ask your teammates for input on this because they usually have things that really bug them, that make their job much harder than it should be, or that are just annoyances that could be resolved.

Look at where you can improve as a team and then prioritize. These improvement opportunities, you have some questions here that you can ask yourself or your teammates about this. Then you can cluster them into things where, for example, you're masting money, areas where you can be more productive, where you can do more with what you have, and near-term improvements, as well as things that are a bit longer term. The whole idea there is to really help you know your position.

Honestly, those three steps, if you can do those, and you can probably do them in a couple hours, that if you can do those, you will have a really good sense already of how your team is doing and where improvement areas are. That's when you start making it real. The main thing, honestly, is to continue talking about this topic. You've, at this point, begun having conversations about productivity with people, and you all have a sense of what it means, what it will look like for your team. You continue weaving this into the work that you're doing together.

In your stand-ups, in your reviews, in your planning meetings, talk about how you can do things better as a team collectively. Then, over time, you will actually build the productivity that your team needs, and that your company expects from it. You may have noticed that I've only showed you kind of three and a half to four steps to develop productivity. That's honestly because I just want you to get started. There is one more step. You can find it at the bit.ly slash n-dot productivity cheat sheet and the article that I've wrote about this. But thinking about this topic only, and talking about it with your teams regularly, is going to go a really long way in helping you and your teams be more productive. That's the most important thing I want you to remember. I really hope you can go from business buzzwords to things that are actually empowering for you and your team, and things that are going to help you take productivity from something that's tricky, difficult, hard to understand, to something that's actually useful for your team in helping them have conversations that they need to have, and ultimately help you to achieve the goals that you need. As mentioned, you can find all resources at the bit.ly link below. Thank you so much for having me. My name is Lina Reinhardt, and enjoy the heck out of the conference today.