Adapting to the Future of Work in Tech

The future is now. I'm going to talk about the AI-assisted programming paradigm shift. So we're all going to be running out of a job. I think you need to look for something else to do in your life. Thank you very much for attending my tech talk. No, I'm just kidding. Let's get some things out of the way.

My name is Sebastian Drede. I'm a senior software engineer at GitHub. I create technical content in my free time by working seven days a week. My partner is a Dutch psychologist, and no, she does not psychoanalyze me, as far as I know. This is a little bit about me, but I want you to retain what I'm going to say now in your minds throughout this entire talk. I'm going to ask you some questions and just think about them as we go through this presentation.

Do we discover the future? Do we create the future? Or do we recreate the past? For some people, this might be a very trivial question. What are you talking about? For the philosophers among us, this is a very, they've gone probably on a very deep dive right now, analyzing what's going on with this question and thinking about all the possible ways we could influence the future and the things that we build moving forward. So keep this in mind as we go through this.

I want us to understand the technology because understanding it allows us to effectively use it. A lot of people look at technology, and they start using it without really understanding it on a fundamental level, and that creates a whole new world of speculation about, you know, doomerism, about what's going on in the future, people being afraid. In order to not be afraid, you have to first understand. And in order for us to understand, I'm going to make my argument in three different pieces that I'm going to go through right now. And we are going to start with the past.

None of this would have been possible without the great work of Grady Bush. He is the creator and founder of the UML. I'm pretty sure you've used UML diagrams before. These are the brainchild of Grady. But also Grady has done a fantastic job in archiving and building whatever we're going to see now on the history of computing. So let us start our journey in the year 1842. Did you know that computers used to refer to actual human beings? Some of you might know. Some of you might not. But in that era of time, the term computation started with Annie Cannon and her group of Harvard computers.

And their job was to actually catalog the stars that we see in the night sky. Back then, computers used to refer to human beings doing some cataloging work, some form of computation, some form of calculation that is sometimes a bit tedious, sometimes a bit difficult for others to do. This is where computing has actually started back then.

Now we're not going to go through all of these different dates but I want to talk about some of these, some highlights, because I want to illustrate a point about the history of software engineering. Conrad Zuse created the language Planck Calcule, where in another universe, Germany might have been the leader in terms of computation and in terms of creating everything we see in the world. And all of the companies that Conrad has created have paved the way for a lot of the technology companies that are now in our world and we see them in today's world.

Who doesn't know John van Neumann? If you've taken any computer science or engineering course, you've probably seen the CPU architecture, which was the brainchild of this polymath and polyglot in every sense of the word. John van Neumann was a genius back in his time and his impact is still left until today. Roth Dietelbaum worked on the ENIAC computer and she co-developed the programming system for that machine. And the reason why I'm bringing up these people is that each one of them has introduced new terms and new words and new concepts into our vocabulary. They've introduced new ways of thinking about programming, about code, about how we interact with the machines.

Morris Wilkis introduced subroutines. Grace Hopper introduced COBOL. Many of you who work in the banking sector probably still touch COBOL to a certain extent. Now some of these developers are the most highly paid, but the idea of machine independent programming languages was pioneered by Grace Hopper. Before that, we had languages that were very specific to the hardware that they actually run on.

Now, the reason why I'm mentioning all of these things is because I want you to think about the levels of abstractions that we are creating from the initial points when we started introducing these machinations to do some work on our behalf. We have Margaret Hamilton, who coined the term software engineering for the first time and introduced it to our common vocabulary in this industry. Prior to that point, the concept of software engineering did not exist. Some people thought we were doing engineering, some people did not. Maybe things were a bit fuzzy. The reason why I'm talking about these things is I want you to reflect on that progression in the history and what it means for the progression we're going through in the world today.

Fred Brooks introduced the Mythical Man Month. Who did not read any of that book? If you're journeying into management, this is the first book that everybody tells you to read. Again, new vocabulary. We are now in the Netherlands. Edgar Dijkstra, who doesn't know him, but he introduced a lot of the fundamental concepts that we see in our programming languages today. Now, the reason I'm going really fast is because I have 20 minutes. This is a 60-minute talk, but just bear with me as we sort of speed-run through the history of computing.

Barbara Liskov. I love Barbara's work because all of the stuff we see in any computer science course today, when you first learn about abstractions, abstract data types, when you learn about the solid principle, who knows that the L in solid refers to Barbara's work? The Liskov Substitution Principle. This is Barbara's work. Again, new concepts introduced our vocabulary. Leslie Lamport worked on distributed systems. Anybody who's working in a SaaS company today building products that are highly distributed in nature, you are working or doing work based on Leslie's work. We have Brad Cox, Objective-C. We have Ed Yorton. He developed structured analysis techniques. We have the design patterns. Eric Gama, he was part of the Gang of Four, who introduced the design patterns. Who doesn't, you know, know about them, uses them in everyday work, in the libraries, in the, you know, programs we build every single day.

Mary Shaw advocated for recognizing software architecture. Again, new terminology added to our vocabulary. Software architecture did not exist prior to that point. Mary Shaw introduced us to this concept.

Jeff Dean. Everything you're seeing today in terms of big data, big tables, all of the amazing work that was done at Google, Jeff Dean had a role to play in it. Everything you're seeing in machine learning today, in addition to Yann LeCun's work and other people's influential work, Jeff had a role to play in it. Huge amounts of great publications coming out of this person.

Kent Beck, test-driven development, Linus Torvalds, besides Linux he also introduced another great thing which is Git, which led to, you know, the founding of companies like GitHub where I am thankful to be working today. We have also, you know, the term DevOps, Patrick Dubois, whatever.

This is a system. A system can be pretty much anything you build on any everyday-to-day basis. It could be a website. It could be a distributed system doing so much more than just a website. It could be an app. It could be a full entire banking system. It could be an HMS system, whatever. This system will be built or is built, and it has a lot of different forces acting on it. And remember the term forces.

Now what is modern software engineering? How do we write code today? What do we do as software engineers? A lot of people even complain and argue whether what we do is engineering or not. And this is the second argument I'm going to be making right now.

This is a system. A system can be pretty much anything you build on any everyday-to-day basis. It could be a website. It could be a distributed system doing so much more than just a website. It could be an app. It could be a full entire banking system. It could be an HMS system, whatever. This system will be built or is built, and it has a lot of different forces acting on it. And remember the term forces.

And I say forces because some of these are in our control. Some of these are not in our control. The context, the complexity, the compatibility of our system are some of these forces that are enacted on our systems. Cost, schedule, and resources. Who hasn't dealt with a resource shortage? Do we have enough developers to build our systems? Do we have enough developers that are capable of building our systems with the cost and the budget that we have defined for it? Does it even make sense to build these systems according to these three different factors? We have the development, deployment, and evolution work that we do and that we impact on our system. And this is a force in and of itself. And the reason I'm talking about this in terms of forces is because I want you to have a mental model of what software engineering is like. I want you to think about systems design and systems engineering in this particular way because I want to illustrate a point.

We have the legal and the ethical aspects. And sadly, we are way more concerned with the legal and less with the ethical. And this is something that as software engineers, I think we also need to change in how we approach building things. The ethical component is extremely important, specifically in this day and age where we're introducing stuff like AI to everything we do. Security, safety, reliability, performance, and functionality. We call them non-functional requirements. What a crazy term. But these are also some stuff that are fundamentally important to how we build and design our systems. So modern software engineering is this. It is these different forces that affect the way we build systems at different rates. At different capacities.

And our job as software engineers is to actually balance all of these forces so that our systems remain in a state of equilibrium. So that our systems stay stable. What we deal with in terms of operational overhead, in terms of fighting with the business, in terms of working with whatever resources we have, is simply trying to balance this complicated equation of all of these forces enacting, you know, trying to enact change and trying to destabilize our system. So our work as software engineers is to actually understand a lot of these things, coordinate with a lot of the people pushing these forces on our systems, and try to keep our systems in a stable state as much as possible.

Now obviously these systems will never be as stable as we would like them to be because if they are stable, that means we're not changing anything about them. That means we're not disrupting these systems in any way, shape, or form. So building systems is complicated. As much as many people would like us to think, we are not code monkeys. We are not just pushing out code and everything will work if we just write the code for it. That's not what we do. We are here to stabilize our systems.

Now the question is, how much of software engineering can LLMs do? You've played with LLMs, you've played with Chad GPT, you've played with Copilot. So how much of all of that stuff that we've seen can LLMs do? So this is the second question I'm going to be answering and we're going to be talking about. Autoregressive LLMs have problems. LLMs cannot be made factual. I don't care what anybody says, the current state, this architecture of large language models cannot be made factual. And toxicity can be filtered out but it cannot be entirely eliminated. They are not controllable. The probability that the produced tokens can diverge us to outside of the set of correct answers is quite high.

So how much of software engineering can LLMs do? A fraction of the development work. Barely. Right? This is where we are today. Even with the introduction of, you know, agents, even with the introduction of a lot of automation built on top of large language models, they cannot do much of the rest. So we still have a long way before large language models can replace software engineers. That's based on the argument that I'm presenting for you today. Take it or leave it. That's really up to you. Large language models are still very useful, though.

That's really up to you. Large language models are still very useful, though. We at GitHub have built a lot of great stuff on top of them, and we will continue building a lot of great stuff using them. They've proven to be great programming companions. They are quite helpful. I've been using Copilot even before it was released publicly, and right now, I cannot really imagine working without it. It's such a useful tool. Why would I let it go? It's like saying I want to write a program without a compiler, or I want to write a program without an IDE. Why would I do that?

They can remix different forms of art. I've been using yesterday the new version of ChatGPT to brainstorm some ideas for new videos I'm going to create. I'm not going to use whatever it outputs for my videos, but I'm going to use it to brainstorm. They can help you with your mental blocks, obviously, and they are very helpful for non-native English speakers or any other language, to be honest. Have you tried talking to ChatGPT in your own native language? It's actually quite interesting. Gergely Oraz says something very interesting. If you don't know Gergely, he has a great newsletter. Just go and subscribe. Very deep insights into a lot of the stuff that happens in tech. He says, got to give it to the GitHub team. While most AI developer tools are aiming for a workflow that takes a spec and much later generates code, with no developer input, GitHub created a developer-driven AI-assisted workflow. This is not by mistake. This is very intentional. Because we realised what LLMs can do and what they cannot do. This is the strategy that is guiding a little bit how we are working and interacting with these tools and paving the way for the future. These are some messages from our CEO and CEO along the same lines. LLMs are not AI, please. They are very useful, but they do not encapsulate the entire field. There is so much more in AI besides LLMs. The big question right now, which is probably why you all came here to hear me rant for 20 minutes, is how to prepare for the future. We talked about the past, we talked about the present, we talked about the limitations and capabilities of these tools. How can we prepare for a future with more proliferation of these tools and for these tools being more available in our future? Andrej Karpathy talks about Software 2.0.

If you did not read his blog post, this is a very seminal article about how this researcher sees the future of programming. He sees the future as a dataset defining a set of desirable behaviors, and then we build models using these datasets to solve problems for us. He sees our work as software engineers to be users and builders of models as opposed to writers of sequences of code with all of their intricate abstractions. The writing of code will be done by models and not necessarily by human beings. This is Andrej's perspective.

The reason why we have a 5-10 year horizon is because nobody can really predict what happens beyond this point. So let's be a little bit more practical and talk about what we can actually affect and control as opposed to speculating about what we cannot change. In my opinion, change will be gradual and expected. We're not going to wake up one day and have super artificial general intelligence dominating our life. I don't care what doomers say, I don't care what these people say. I appreciate the conversation that we need to have about the probabilities of this happening. This is very important still. We need to talk about regulation, we need to talk about compliance and laws. That's very important. But I don't expect this to pop up in the next couple of years. LLMs are not it. We need more object-driven architectures, and they will start to merge gradually with time. Human-level AI is beyond the time horizon that we're discussing today, but we will steadily make progress towards it. We need to stop the urge of competing with the machine. Often I hear people saying, can AI do things smarter than human beings? Yes, well, I mean, if you're going to compete with a calculator, calculators can do things better than you today, right? So why aren't we afraid of calculators? Same thing with large language models. In many ways and in many fields, these things, we are going to be doing things that we are not even capable of ourselves as human beings. We have very limited machination in our brains. So we just need to stop computing with the machine and understand it for what it is and use it for what it is. AI-assisted development will get better and better with time, obviously.

SAS is dead. Long live SAS. The reason why I say that is because we've done so much in this field right now. I feel like this is a solved problem. We need to start focusing on more difficult issues in the future. There's a lot of different methodologies to build distributed systems, to maintain them, to optimize them, to scale them.

We've been doing that for many, many decades now. It's about time we developed new problems to solve. You need to ramp up on the fundamentals and build depth in the hard sciences, again, my opinion, because if you want to be involved in the conversation that's going to be happening in the future, you need to be able to stay informed. And the only way you're going to be informed is if you understand what's happening under the hood.

So, if you're not being fed information by someone else, not listening to people like me, you need to develop your own perspective about the technology. And you are in a great position to do that, because you are technologists yourself, and none of this should be cryptic or completely mystical for you. Andrej Karpathy is a great teacher. He has a lot of fantastic YouTube videos where he demystifies the process of building your own large-language models within the comfort of your own home. I highly recommend going and watching his videos. I also recommend The Mathematics of Machine Learning as a book. These are none of my books. I'm just recommending stuff I personally like to consume. Feel free to consume it or not. These are the QR codes for it. Tiwadar had a lot of issues learning math when he was younger, so he created this book for the folks who also feel challenged or feel mathematics is a bit of a challenging area for them.

Less focus on syntactic sugar and more focus on what's happening under the hood, programming will go back to being a hobby for many of us. I cannot wait for that day to be honest. The spotlight will be shining on solving human problems. That's only going to get brighter, because it's not about the tech. It never was. Everything that we do is to solve human problems. We need to remember that. Now, in the ten-plus year horizon, honestly nobody can predict what's going to happen beyond the singularity. I hope that we are responsible about what we're going to be introducing to our lives in the future.

Finally, I want to leave you with these last few words. In many ways, we're going to be discovering the future, but right now, we have a responsibility to try and create a better one together. If you go back to the initial question that I asked you, do we create the future, do we create the past, or do we discover the future? In my perspective, we're going to influence the future, but we're also going to discover it, because we cannot really predict everything that we all do as everything that humanity will do. But what we can do right now is to be more responsible about what we bring to our future together. Thank you very much for listening. This is a QR code for all of the links.

Let's jump into the questions. Remember, you can ask questions in the Slido. We have a question which kind of quotes a very famous person in the world of AI, and that person said that coding and software development are no longer going to be needed in the short term. Are we safe? Are our jobs safe? What do you think? I love Jensen because his takes are always quite controversial, and he's quite an interesting person. I love his leather jackets as well, but that's besides the point. He always has these controversial takes for a reason. At the same time, he wants to market his own company. I feel it's a very bad take to say that people should not learn to code because we have created better tools that can help us become better engineers. In my opinion, everybody should be learning how to code, even if we have tools that can abstract that for us. Now what Jensen is mentioning or what he's referring to is that the average folks who are non-engineers, non-developers should not necessarily learn to code at that level, at the level that we do, and they should be able to greatly benefit from being able to build their own software. Why not? In my opinion, everybody should be able to code, whether using programming languages or whether using large language models and whatnot. I don't necessarily agree with nobody should learn how to code. You should learn it, but just choose the appropriate tooling and know we're safe for now.

Another thing that I also wanted to ask and highlight, but not me, the audience, which is for example, when social media came around, social media changed the way humans behave, and humans in some ways are kind of dependent on the social media. Please keep tweeting and using the hashtag C3Defest. But other than that, do you feel that human dependence on AI will make humanity dumber by day, and eventually AI would become a necessity for life in the future? I'm thinking kind of the Pixar movie Wardy, where they're completely dependent on the robots. Look, evidently if we're not aware of how we consume technology, we're going to become dependent on them. This is inevitable, right? But what we can do is we need to put in some guardrails for how we consume these tools. These are tools that, at the end of the day, no matter how much we can externalize our problems and say everything is the fault of the tools, right, if we say that, we're never going to make any progress whatsoever, because any tool you can pick up, you can do bad things with it. It's just about how we consume it, and I think we need to raise more awareness, we need to be a bit more vigilant, we need to gradually introduce these tools to our environments and be conscious of how we are introducing them and using them. And yeah, I don't think there's a quick shortcut for that, we just need to be aware.

Absolutely, and it kind of, this is more of a comment than a question, but it really highlights what you said as someone who says, as a junior dev, they wish Copilot wouldn't tell them the answer right away, but hint the answer. Like often a teacher will be like, you're close.

Look, evidently if we're not aware of how we consume technology, we're going to become dependent on them. This is inevitable, right? But what we can do is we need to put in some guardrails for how we consume these tools. These are tools that, at the end of the day, no matter how much we can externalize our problems and say everything is the fault of the tools, right, if we say that, we're never going to make any progress whatsoever, because any tool you can pick up, you can do bad things with it. It's just about how we consume it, and I think we need to raise more awareness, we need to be a bit more vigilant, we need to gradually introduce these tools to our environments and be conscious of how we are introducing them and using them. And yeah, I don't think there's a quick shortcut for that, we just need to be aware.

Absolutely, and it kind of, this is more of a comment than a question, but it really highlights what you said as someone who says, as a junior dev, they wish Copilot wouldn't tell them the answer right away, but hint the answer. Like often a teacher will be like, you're close. I mean, just turn Copilot off for a little bit, maybe? Yeah, that's a good exercise, but also, can you tell it not to give you the answer right away? That might be an approach, prompt it to do so. For sure. We are nearly out of time, but let's just do a few more questions. What is your take on retrieval-augmented generation? Yeah, I mean, it's proliferating quite hard. Every tool that is being right now that you want to customize and sort of make it relevant to a certain environment, you're going to need to introduce RAG to it. It's a very popular and effective mechanism to make LLMs more adaptive to the environment we work in. I think a lot of companies are using it right now to customize their products on top of the foundational models, so I'm not sure there's anything controversial about it. Nice, nice. All right, so I know that Dave, we've already disconnected your computer, but where can people find you if maybe they want to get the slides or just find out more? Right. So, if you Google my name, you're probably going to find some links. I'm everywhere, because I push myself everywhere, but also, if you go to glitch.stream and glitch without a T, I thought that was funny. That was a good way to design my brand. It obviously was a mistake. So glitch without a T.stream, you're going to find all the links for everything I have. Awesome, awesome. And what books are you currently reading? You recommended a lot of great ones. What books am I currently reading? I'm reading a book called Polymath by Ahmed Waqas, and this book is about a cataloging of all of the polymaths that are in our history and the people who have been experts in so many different disciplines, as opposed to the common idea today that we all need to be experts in a single domain. It's a very nice, interesting book about all of these great people and what they've done with the ability to extend themselves across different expertise, and I think it's an interesting read. Awesome. And the last one that I will take is not a question, but a comment that I totally, 100% agree with. Bassem, thank you so much for your talk. That's all we have to say. Can we give it up for him one more time? Thank you so much for having me. Thank you. I appreciate it.