Test Effective Development

Hello, everyone, and welcome to the Test Effective Development introduction talk! I'm Shai Reznick, and today I'm going to teach you a new way of thinking about test will change your testing life forever and will boost your productivity. So let's begin.

As I said, I'm Shai Reznick. I'm known as the Test Effective Guy and sometimes also known as the Angular Testing Guy. I'm also a Google Developer Expert and I founded HiRise.IO, which is an education and training company that teaches developers how to write more cost-effective tests in a fun and entertaining way. Also, my talks have been viewed by over 170,000 developers and a few random people. I'm guessing. Hey, WSI here.

Okay. So my main goal in life is to help busy companies become more cost effective via testing transformation. That means that we take your busy company with your schedule, not stop in development, and find a way to integrate testing into your day-to-day. If you're interested in that, let me know. Okay, so we could take your developers from sleeping like this into sleeping like this, where as I like to say, test right, sleep tight. Okay. And it's not only about sleeping tight at night, it's also about becoming more professional. So you could make the changes to your code or apply this new technique that you learn without the fear of introducing new bugs into production. And if you want to find out your biggest testing mistakes, I prepared a quiz, a few questions that you can answer, and we'll show you what your mistakes and how to fix them and we'll share with you some cool new free resources about testing. So check out in this link or in this QR code, it'll take you the same link and get to that quiz.

Okay. Now a disclaimer. So I'm known, if you search my name on YouTube, you might find crazy talks of mine, like, you know, sometimes it's a stage play and sometimes it's a game show or a rap song or a magic show about tests or one of my personal favorites. This is a talk I did in 2018, where I hooked a device that can read minds and I share with the audience what my thought process and all the shenanigans that go through my head while I'm test driving an app. So even if you're not an Angular developer you'll find value in this lecture, so check it out. But! So today I'm not going to do this crazy talk because I have a weird situation here. We have a problem with our Wi-Fi. We are sharing the Wi-Fi signal with all of the other neighbors here in the office building and there is a weird phenomenon today that, you know, people are going over other people's transmissions, so you might see glitches or you might see what other neighbors of mine here are watching. So I'm sorry about that. I apologize in advance for anything you might see. Okay? I didn't want to, that's why I didn't want to create like a crazy talk today because I don't know when these transmission interruptions will happen. So I'm just sticking to the old-fashioned educational talk and that's it. So this is the disclaimer. Sorry about that. And let's hope, let's hope we have a smooth process. Okay, cool. So now let's begin.

Okay. So back in 2009, I was a developer in my day job and I had a side project at night. So I worked from nine to five on the day job and until 2 a.m. On the startup. And that's where my days. And I was in this, what I call the circle of doom because I was rushing and trying to meet all the deadlines and moving forward fast, I produced poor code quality that produced unexpected bugs, that in turn produced more poor code quality because of the fear of the unexpected bugs. I got like so scared, okay. And, and I didn't want to change and improve the code and that led to frustration and trying to destroy the computer or I don't know, and that led even to a more scary place where I was in a feeling of a quicksand where every time I fixed the bugs, I introduced four more bugs into, into the app. And I, it felt like I'm drowning more and more into the quicksand. And I started having this dizziness and, and, uh, I couldn't breathe and this attacks and I didn't know what's wrong with me, so I went, I actually went to the hospital and got like checks when it became more severe. And they actually told me that I have nothing wrong with me physically. And they suggested that I might have panic attacks and I did. This is what I had panic attacks from not meeting the deadline. And, you know, I missing the deadlines and I have more bugs to fix and don't know, I have this uncertainty on when more bugs will pop up and I, it was truly a scary place in my life.

So then I decided that I need to work smarter, not harder, because I thought, okay, what do I really want in my professional life? I want to do meaningful work. I want to upgrade my skills and I want to live a balanced life. So I don't want to, you know, miss holidays or, you know, birthdays or family with friends, back then my girlfriend and what prevented me from doing all that were unexpected bugs. So I then decided I'm going to learn how to fix them and talk with friends and suggest us to write tests.

So I just learning about testing and I met this like weird terms, like unit and integration, and then to end, and I didn't know what to choose when. And uh, sorry, like I told you. Um, okay, let's, let's forget about what we just saw. Okay. So I had all these terms like unit integration didn't know what, you know, what to choose and what to write and how to write them. And somewhere I read that unit tests are for developers and integration and end to end are for QA. So I decided to start with unit tests. Uh, so then at the start, it was like very difficult to understand what, what am I doing? But after a while I became better at it and better. And I thought to myself, okay, I got this. But then, you know, I started having bugs like 100% coverage or 90% coverage, but still had bugs in production. And then I realized that unit tests are not enough. I also need integration tests.

So I tried, I really, really tried. So I tried to write integration tests and the start again, it was very difficult, but then in the end, it was still difficult because the integration tests are super difficult. It looked like easy in the start, but when you realize you have so much logic and moving pieces in your code and all of this, like, psychromatic complexity and other terms that makes you sound smarter than you are, but basically saying that you have lots of code path in your app, you know, decision points and stuff like that, that made integration tests much harder and slower to run and that led me to write only test for the happy path when stuff are successful and not to check error situations, stuff like that. And that led me to have less confidence in my integration tests because I realized that it's just like an illusion. They're not really give me full confidence in my code. And I always kept feeling that this blanket is too short, you know, every time I try to use a new technique, you know, integration or unit or something like that, I always like suffer from something else. So after a few years of testing, I became a testing consultant and help companies with their code. And I taught them the same traditional ways of like the testing pyramid and all that stuff. And I started to get frustrated because I didn't have the results that I wanted to have. And this is me frustrated. Okay. It came to the conclusion that I had enough. Okay? We need a new approach. We need a better way to do stuff.

Okay. So I started experimenting, you know, with, with strategies. And then I, I had a few epiphanies and I learned the new strategies that worked for me. And once I started to apply them into my client's code and to my own code, I finally got to the place where I can change my code with confidence and I'm not suffering from all the stuff that I suffered when I use the more traditional ways. And I want to share the epiphany that I had. Because again, like I told you, I want to do meaningful work and upgrade my skills and live a balanced life. And what I thought, the only thing that stops me from doing it were unexpected bugs, but in fact it weren't just unexpected bugs, it was also inefficient tests. So, so I just switching from time wasted on fixing unexpected bugs to time wasted on fixing inefficient tests. So then I thought about, okay, what can I do to handle unexpected bugs? I need to increase my testing confidence. And what can I do to fight inefficient tests? I need to increase my testing efficiency. I know kept then obvious, but it's important to know because now that we know these two factors, we can view them and view our tests from these lenses.

And if we think about like the strategies, tools and techniques that you are used to apply or use in our tests, okay. Now, when you think about confidence and efficiency and judge them by these terms, we can see that some strategies lead to more efficiency, but less confidence and some lead to more confidence and less efficiency, for example, end to end test, where you have lots of confidence because all the moving parts are integrated together and you test everything in one place, but there are the least efficient test out of them all because they run the slowest and they require more code and there fail more often and stuff like that.

So although you can never reach a 100% perfect point, okay, you always suffer from bugs. The point is to find a balance. Balance to the confidence and efficiency we have from a specific strategy term or two. So then I thought about, okay, what can define this balance? Okay. What can, uh, how can I use one single term to define this balance between the confidence and efficiency? And the only thing that came to mind was cost effective. Okay. How cost effective is this strategy? And then I thought it was too generic. So I kept thinking about, okay, cost effective, cost effective. Maybe I can call it task effective. And that's when I came up with test effective development. Okay. And the idea here is to find better terms, better strategies, and better tools to write more cost effective tasks. So there's lots of topics to cover. We test effective development, but today, because we only have limited time, we're only going to focus on two strategies.

Okay. The thing that I learned about being more test effective is if you want to be more effective, you need to be more specific. For example, if you go and want to do a surgery, you want to go to a surgeon who know how to use the right tool for the job at the right time, and not like a maniac with a chainsaw to just like know how to use this one tool. Okay. And the idea came from actually from Elon Musk and his first principle thinking method, where he applied it to building reusable rockets by going all the way to the first principles and trying to build back from that better solutions. So that's what I applied. And when I thought about unit integration, I said to myself, okay, let's try to find what are the first principles behind these terms and try to build them back up and maybe come up with new terms. Okay. So when you think about an app, let's say the part of the app, like classes or functions or something like that, and the connection between them, like they import each other, this is our app.

And the idea came from actually from Elon Musk and his first principle thinking method, where he applied it to building reusable rockets by going all the way to the first principles and trying to build back from that better solutions. So that's what I applied.

And when I thought about unit integration, I said to myself, okay, let's try to find what are the first principles behind these terms and try to build them back up and maybe come up with new terms. Okay.

So when you think about an app, let's say the part of the app, like classes or functions or something like that, and the connection between them, like they import each other, this is our app. Okay. And when you think about unit testing, the confusion there is that sometimes you have a unit that describe one class, for example, depending on which book you are reading, and sometimes it defines like two classes together that represent one idea. But then when you learn about integration tests, you learned that this, it's the same thing. Basically, no, it's two pieces together in integration. And you test them both. Uh, this is also called an integration test. And sometimes again, depending on which book you're reading, it refers to your code against external code that you don't own. So this is an integration test. So there's a lot of confusion. And then because of this confusion, I decided not to use these terms and to actually dig deeper. And when I dug deeper, I discovered new testing dimensions. We have several testing dimensions that we can analyze. The first one is boundaries, the second is action scope, and the third one is subject type, and we have actually more testing dimensions that I discovered over the years, but today I want to talk about only these first two. I have other lectures and courses and stuff like that where I talk about more dimensions and how to analyze them. But for today's two strategies, I want to talk about these two dimensions. Okay. Let's define them. Boundaries. Boundaries are easy. It just means isolated versus integrated. Most of you already refer to units and integration, and you actually refer to isolated or integrated. And we'll see, so isolated means that we only test one thing. It could be a class, a function or whatever, but it's only one thing. And the rest of the stuff that this thing imports, we fake. Okay.

This is an isolated test. In contrast, an integrated test is whenever we test more than one thing together. This is an integrated test. Sometimes it could be the entire thing, which is an end to end test. So this is the definition.

Action scope means single action or multi-action test. And what is it? So let's say we have a class for methods, okay? A single action test will test this one method. And a multi-action test will test several methods or functions or whatever two things together in one test. This is a multi-action test and we can have several combinations of the things like we can have single action, isolated test or a multi-action integrated test or a single action integrated test. You know, we can have multiple combinations of these dimensions, but choosing the right combination is the key here.

Okay. You have. More effective combinations for different situations. Okay. We gently lay our possum down. I don't know. That's. I'm speechless. Okay. Let's let's just continue. Okay. So let's judge the effectiveness of single versus multi-action test. In the confidence score, multi-action tests score higher score. It's relative score. It's not scientific. Just like relative to each other, because the more you test more stuff in one test, you get more confidence that you have less side effects. But in terms of efficiency, multi-action tests are much less effective than single action test. Why? Because of popularity or what I call the influencer curse. Okay. Let's see.

Okay. So let's say we have this class here and it has this method and let's say with several tests to test this method. Okay. Now, if this method that has a bug, now all these tests will fail, but now let's add another method to the test. So let's say we have all these tests that test this method, but they also run this method as well. Now, every time this will have a bug, all tests will fail. And every time this will have a bug the second method, all tests will fail.

Okay. So when you introduce more methods or more actions to your tests, you're increasing the probability of failing lots of tests together. And that's the influencer curse. The more followers your code has, like the more tests that follows your code, the less efficient your tests are. Okay. So the conclusion is to prefer single action tests. Okay. There are some times that you want to use multi-action tests and like in end to end tests where the amount of tests are smaller and I talk about these in my courses and other lectures, but for most tests, you want to prefer single action tests.

Okay. Now let's talk about the effectiveness of isolated versus integrated. Okay. I see a lot of talks about, no, write only integration tests and then the testing pyramid or write only unit tests are mostly units or mostly integration and all that stuff. And I follow these suggestions for a while. And it led to more confusion on my part, because I realized that what's important is the context of your app. And part of this context is the size size matters in testing.

Okay. So let's define size. On one end, we have the small apps, example for these can be like microservices or micro-front ends or open source libraries or, you know, proof of concepts, stuff like that. These are smaller apps. Okay. On the other end, we have medium to big apps, which most of the time examples of these are monoliths. Okay? If you, we don't break them into micro frontends, it's a big monolith with lots of moving parts compared to a monorepo or something like that. So these are big apps. Okay. These are the terms.

Okay. In terms of integrated versus isolated on the confidence score, we have a high confidence score for the integrated test and the lower confidence score for the isolated because we lose touch with reality when we fake stuff. But there is an asterisk there on the integrated part, because it means that we must cover everything in the integrated and not just the happy path. And most of the time it's not the situation. That's why it's a caveat. Okay. You know, if in an ideal world where we cover all the possible code path with integrated test, yeah, we have maximum confidence, but that's most of the time isn't the case on the efficiency score. We will look at integrated versus isolated. Integrated is much less efficient than isolated test. And there's a lot of reasons as to why that is. Okay. I suggest watching a great lecture about the scam of integrated tests or integrated tests are a scam by J B Rinesberger, where he covers that in depth. And I have other lectures about where I go deeper into why this is the case.

Okay. What's the scam of integrated tests, but there are much less efficient than isolated tests. And for that reason, what I came to realize is that for small to mid apps, the more test effective solution is to use a combination between single action and integrated tests, because that way we don't lose the confidence and we don't lose that much efficiency because we don't have a lot of tests in the first place, because this is a small size app, but for big monolith or mid to large scale apps, the more test effective solution there or strategy there is to use a combination of single action and isolated tests, because that we preserve a good balance between the confidence and efficiency.

I'm just going to ignore that. Okay. Okay. And because I didn't want to keep saying like single action integrated test or single action isolated tests and stuff like that, I want to find better terms that encapsulate, like what I mean. I want to introduce to you now new test types based on size. So type number one. Okay. For small apps, I want something that could cover all parts of the app. And then I thought, okay, what's a good metaphor for, let's say I had all these parts in one room. I want to shed a light on all of these together. And then I thought about like, Hey, I can use a flashlight flashlight, you know, with the ray of light can hit all the parts together. Okay. And that's when I thought about the name flashlight test, which are short distance, okay. Think about the small room single action integrated test, and these are our flashlight tests. And this is type number one.

Now the other new test I've thought about was for the big apps. Let's say I had like, it's a big stadium with lots of distant parts. Okay. Sometimes I want to reach this part or this part. What can I use to reach it? I cannot use flashlight because the ray of light can reach just a short distance. I need another device. And then thought about, I can use a pointer laser and that way I can reach other like more distant and just those parts because they are isolated. Okay. So then is when I thought about the name laser test. Okay. Which are long distance, think about a big stadium, single action, isolated test. Okay.

So again, to recap, we have flashlight tests for small apps and laser tests for big apps.

Okay. And you might ask yourself, okay, how do you know when to switch between two strategies? The rule of thumb for me is when the tests reach more than 500 lines of code, then I know that probably my integrated tests are becoming less efficient. And then I considered switching to a laser test strategy.

Okay. So that's our time for today. Let's summarize what we learned. We learn about test effective development. We learn about the different testing dimensions we have, and we learned about the laser test and flashlight test strategies.

Okay. If you want to learn about more test effective strategies, again, this is the quiz I prepared for you. This is the link. This is the QR code, check it out. And you get more free resources to learn further about this topic, about test effective development. And with that, I wanted to thank you all for participating in my first ever TED talk.

Okay. This is my first, and thank you very much. These are all the details. Remember check out the quiz and become more test effective developer.

Okay. Enjoy the rest of the event. Bye-bye.

Hi Shai, how are you? I was so excited for the talk and I gave my all and I lost my voice. So you can hear the, my sexy voice in the Q&A. It was great. It was super funny. I love the videos in the middle. So we had the pool before. So let's take a look at the results and in the results, I actually didn't knew the answer so my, my. Vote was on the hardware testing.

And afterwards I was reading about it. So what, what's your, your takeaway in regards to the results of the pool? So I'm seeing the results now and look like we have a tie between plumbing and hardware testing and the real answer is hand-teasing nobody picked hand-teasing it's like 0% or something like that, but no, actually, when you, you know, I always like thought it was, regarding to plumbing and then because, you know, they blow smoke through the pipes to see if you don't have any, like, you know, leaks but it actually comes also from hardware testing. Okay. So hardware testing, the test there was, if you plug it in and it doesn't blow up in smoke, it works like you press the smoke test. So it actually is a combination between. So the right answer is both A and B are correct. Exactly. Yeah. That's what I, what I researched afterwards. And the plumbing one came before, came first and then the hardware testing. So that's great.

We have a few questions on this court, so I'll read them to you. And the first one is from the S and the T-shirt in the the questions are, can we get a T-shirt with the test rights, sleep tight? I actually plan. Thank you very much. Yeah, I want to have one as well. I will. Yeah, I will. I'm playing on the, I have a design and I'm playing to issue like, I just needed to find the time to actually see where can I produce it internationally and stuff like that. But if you'd like to be on the list, so hit me up at shai.hi-res.io and I will put you in the notification list when it will go live. But yeah, I'm planning like a whole line of testing related funny T-shirts. You have created your own terms, so you can have lots of different shirts with different sayings. Awesome. Yep, laser tests and stuff like that, smoke and lasers and stuff like that.

So, we have another question here, which is, does the size of the app matter when choosing the test, a testing strategy? Uh, I, I like, I think it's a, I answered it in the, in the talk. In the end. But yeah, so, so yeah. This, this is the, the, the, the whole point that I tried to make was that when we read about the blog posts that get like, suggest that a specific strategy we, you know, we used to read it from like, okay, our point of view. And we might take like the wrong strategy to our app. If we don't apply the, or wait for a second, I think, what is the size? What is the size of my app? And yeah, does this strategy, like, does this technique really is the most cost effective technique? And that's why I said that if you judge it all, that's what I tried to do. If you judge everything and you have more, more aspects and more dimensions to judge. Like I said, like you have a subject and you have all these things that you need to consider. That's what I tried to do in my lab here in my office, by the way, this is my family. So what they tried to do is to figure out what is the most cost effective pair context. So for context of smaller apps is try to use a combination of single action, single actions anyway for your micro test. And do the flashlight test, which is the single action and integrated test. That's how you will get the most gains and most efficiency and most confidence. But for large apps, which most of us work in companies where we have still monolith and big monsters of apps, it's pretty hard to write integrated tests in such an environment in a very repetitive or a conventional way where everybody knows the borders, the right way. Where the tests will be very deterministic, right? Exactly. So that's why I recommend, okay, stick to isolated tests for that strategy and use the integrated tests for stuff like smoke tests, which I talk about on other lectures. Oh, we have a hand-teasing. Somebody answered hand-teasing, which is amazing.

Thank you. You won with the free t-shirt. Yeah. I like that you mentioned, and it's not just you, many of the speakers here in the conference mentioned that we should choose the right testing that are more cost-effective for our needs, for our context, and I think that's what makes more sense.

There is no one strategy that works for everyone. We need to understand exactly where we are in and where we want to be. Yeah. And that's something that I think we should talk about in the testing world, more like try to go to first principles and try to assess stuff, not by the terms that we're used to talk about, but for what is more correct to my situation. And let's define what is my situation. Am I working in a small app, in a big app? And then, okay, what are the different dimensions? What do I really need here? Do I really, like in Angular for example, in React it's pretty obvious to write component or DOM or test for the DOM. It's more like it's easier, it's faster. In Angular it's actually faster to write only a test for the classes and not for the DOM and again, leave the DOM tests for the reusable components. But that's not the common wisdom, let's say. The common thing you will read about but I personally don't care about what's common, I care about what works in real life situation. So that's why we need more critical thinking on these terms.

Just a final thought that I have as well is that the same project might start with a strategy and when it evolves it will have to change its strategy because the app grows and it becomes more complex so we have to have that in mind for sure. It was a great talk, Chi. I really appreciated it. Thank you very much for that. Thank you too.