Plants vs Thieves: Automated Tests in the World of Web Security

Hello everyone, and I'm so happy to see you here, or, okay, I cannot see you through the screen, but I know that you're there. I'm just happy to have you here in React Day Berlin. And actually this talk was one of the reasons why I got nostalgic when preparing it. And it's not, it's not only Christmas time, and of course this gives me nostalgic feelings as well.

It is the topic of my talk actually. Maybe a few of you guys already recognize the title. But I guess this is a theme which will keep you away from Christmas mood, at least for a little, okay? On the technical side, this talk is on security and it's on testing. So two topics I'm passionate about and I really love. And I hope we can get some wonderful defenses built up using test automation.

Okay, the few people who might have already got this nostalgia thingy, maybe you recognize the font in the title or the title as a whole, Plants vs. Beefs, because it's derived by Plants vs. Zombies. This is a game created by PopCup, company was called like that, in 2009, I think. Yes. And it was a wonderful game, cross-platform, actually the only game I played on my mobile phone because I'm a console and PC player, but that's not important right now.

It is a tower defense game where you try to basically defend some base against enemies, right? And in Plants vs. Zombies, as the name already implies, you will try to protect your house against zombies in a zombie apocalypse. I know, not that Christmas-ish, but a zombie apocalypse by using plants inside of your garden. Well, of course, this is a wonderful allegory, right? It's nostalgic, it's nerdy, but what does that have to do with web development?

Well, what if I told you it's a wonderful allegory on web security? I really think that because, let's see it like that, we have a zombie, which is a threat and attacker, a security risk and exploit anything which could be a danger to your system, right? And the plant here, it's the countermeasure, the mitigation strategy, a security fix, anything which could protect your application against attackers, against those security risk or exploits. With a more general lens here, you can see that the house is basically the protected area, the protected part in our application that might be data we want to protect because it's sensitive, it's important.

Or we want to fulfill laws, we need to protect our data or protect features because we want people to basically pay some money for applications, right? Or at least be registered users we know about. So the property here, the complete thing, including the house, including the garden, are our defenses against those tricks. So defenses against the zombies, right? And defenses against security threats. So if we consider our garden, our defenses, let's build our garden, right? Because our plants inside of this garden are our lines of defense. All the defenses we got under our belt. And my first thought would be like, okay, best practices. We are cool at building our applications in a secure way, right? Testing can be an idea too, in my mind, at least. Not only because I'm passionate about testing, because I do know that it might help us and be a line of defense as well. So can testing help us? And of course, the first thought inside of my mind, and I guess in your minds as well, is tooling, like Sonocube, like Snick, or all those things which test for security issues and notify you. But, of course, it would be the easy way.

It would just say like, okay, let's pay for some tools, install them, learn them, that's it. Well, what if I don't want to learn new tools or spend money on, again, new tools, if I don't want to be dependent on third party, basically, right? Or if I want to learn what those tools are already doing? Well, there's lots of reasons to go by testing as well. And even if it's just one line of defense from many.

But before that, real quick, let me do the important disclaimer or an honorable mention where you bought it. Learn your application. You're still the most important person in charge, right? It will pay onto your testing efforts, but generally on the quality of your application as well. So learn the vulnerabilities of your application so you know how to build your application in a secure way. And of course, know what to test, but we'll get to that a little later.

And be aware of one thing, tests are always the messenger, not more than that because they will just notify you on security issues, you still need to act on your own. So basically, be aware that you still play a role in that. You cannot automate everything, but listen to the warnings, listen to your messenger, listen to the tests.

Okay, so how can we learn our application and learn vulnerabilities and learn where to look to ride those tests? Well, there's a group of people or a project helping us a lot. It's called OVASP, which is a shorthand for Open Worldwide Application Security Project. And it's a project which goal is to raise security inside of the web. So they published a top 10 ranking of the most important security risks, and they raise it in a certain rhythm. I guess the last one was 2021. And as far as I know, right now in December, they're collecting the data set for a new ranking in 2025. So stay tuned, there might be some changes, but for now, the 2021 is the last ranking and it's the most recent one. So for that, we will take a look at the top three risks. So spoiler alert, we are all front-end developers, we are doing React, and even if you're doing other frameworks, you're still working in front-end.

So there's one point which we will explicitly take a look at because it's important for front-end devs. But we'll get to that. I don't want to rush through this talk, even though I'd be mindful of your time. Okay, so, where to look at? As said, the three most important risks by OWASP, which are the first one, the most important one, the broken access control one, which means that a user can act outside their permissions, being able to read or to change things they shouldn't be able to. The second one will be injection. No, I'm too fast, sorry. It's cryptographic failure. So basically, that cryptographic needs are not met. Stuff is not properly encrypted. Maybe data, maybe connection, you're not using HTTP, stuff like that.

So this is rank two. And finally, where I was a little too fast, and I guess you already guessed it that this is our focus point, it's injection. If an attacker supplies untrusted input to your program, so basically, attackers code will be compiled and interpreted and executed as if it was your own code, which is pretty bad, isn't it? So yeah, these are the top three. Injection is our main focus and yeah, let's take a look how test can help us in this regard. And you might have guessed it, but it's not too much of a problem if not because you're here for basically learning about it. We will try to fight security issues by writing our tests, writing proper tests, using test cases to cover some security issues. And this talk, I use end-to-end testing and Cypress for my examples, but I try to be as agnostic as possible. So if you're using Playwright, Test Buffet, or even be brave enough to use Selenium, sorry, no matter what test automation framework you use, in most cases, not all, but in most cases, you can do so in other frameworks as well. And yes, let's take a look how to design our test cases, especially for injection. Because it's a perfect example because yeah, it's easy to reproduce, right? As end-to-end tests are basically simulating real users, they can simulate a real attacker. And yeah, my thoughts about that would be like having a negative test where you basically try to inject script or some SQL at some input field, stuff like that, for a sample form, for example, and as next step, maybe even randomize the input fields you go for, right? Well, let's take a look if this is working out. UvaSp sort of, fortunately, will help us in this regard because they have a DemoChop. It's an online shop, which is insecure and a little sophisticated by design. So it is broken. It's not safe. Don't use it in production, but maybe run it in a Docker container and try out all the things they did wrong. They even have challenges to practice. So basically, hack the shop. And in this talk, actually, we will do two challenges as test cases. Yeah, the first one. Let's go for the test. So as you see, we will go for this letter of the menu, which is NAVBAR count. It should be visible, rendered so we can interact with it, and we click it to open the menu. Then, there will be a point for login, which I copied beforehand. Same as should be visible, so it's rendered, and I want to click it to get to the login form, right? Okay, let's see if the test is already doing it. Okay, there we go. The login form is there. Let's get the selector by copying it from the test runner still to be visible. And then, we want to type in all the things we need, like logging in. We're not using an email or something valid, but we will go for a small part of a query.

This query will be basically used instead of the problem, a problem, and it will get the first user of the database to login to. In this regard, the password will not matter because we will do it by SQL integration, exactly, as you might have guessed. Let's go and submit the form. To have a proper test, which should fail, I want to have an error message in there as well. Let's take a look. I want to click it, enable it, so we can log in.

I was a little fussed. Now, I want to just have a invalid login to have error message to test for invalid email or password. Let's see if the error would show up. It will not, but of course, let's go for a proper test. Let's try to not use any typos here. I have a typo in there. Let's go back and run the test. It should fail by now because as you see, we're logged in. We can see the basket of the user. We can see the account message. We can see the menu. We can see everything. Pretty bad, right?

If you want to catch injection by a test, you can just try to write a negative test, which is a test listening on error behavior and trying to basically see what happens if you do an extra injection inside of one input field. You could go for another way of testing as well when it comes to content security policies, which is pretty cool, like CFP testing using Cypress. This example is taken by Gleb Bamutov, which is a wonderful author writing a wonderful article. If you want to take a look at it, there's a QR code. You can check for CSP headers as well. Basically, the CSP, content security policy, will define trusted sources for content, which is allowed to load, so basically what's listed.

In order to test that, you can go for the Cypress config to enable it. Experimental CSP allow list would need to list the headers you want to take a look at. Then you can basically check it. Maybe in an API test, where you try to check for the headers that are there and they are valid, or you go for a normal test case like here, where you basically simulate a security attack by CSP validations, where you should see it blocked. If you want to have more details, because I want to be mindful of your time, try to take a look at Gleb's article. It's pretty cool.

Well, it would be really bad if I don't cover the rank one issue, right? Broken access control. Even though most of the things there will be done by the backend, like user roles. You can think about using tools like FGA or OpenFGA to have a RBAC, like basically resource control, access control, the solution in place. You can take some look at testing as well, because writing proper test case can help you here as well. Next is to mimic a user with insufficient permissions trying to do something or trying to read things, information they're not allowed to read. So, yes, let's take a look at that.

I'll get back to the ovas produce example. So, let's go take a look. There, we do the login based on the first test like we did before. I put it into a custom command to save us time. I visit the shop and I want to go for the administration panel, which we should be able to reach as a normal shop customer, right? So, let's get the card we have, which is basically the same one as in the login form as well. It should be visible so the page is loaded, we can interact. And then let's see if we are blocked. So, a 403 error shows up or any other error message. Let's take a look if this is actually happening. So, let's see when I'm testing it on my own. Yeah, 403, you're not allowed to access this page, so we can go by this error message. Okay, let's go. Let's take a look there. What happens if we are executing the test? So, we go for broken access control. We will log in and we will see the administration overview, right? This is pretty suboptimal, let's say like that. So, yeah, the test will fail, which is valid because we wanted to have a negative test. But as you see, we were able to access the administration page. We are not blocked by doing so.

The test is a messenger letting us see that there's a problem. Okay, some small words for cryptographic failures as well. And even though this is more a backend topic as well, test automation can help you with this one as well. So, it's inbuilt basically if you use Cypress, for example. As Cypress works from within the browser, they need to adhere to the rules of same origin policy, which will lead Cypress to error any time you attempt to navigate back to a HTTP site when you were using HTTPS before. So basically, Cypress tests will fail if you have an encryption failure on your page, which is pretty cool, right? So nothing we need to write as a test case or something like that.

But still, you might say, okay, tests have one major flaw, right? Well, how do we handle security issues we not know about right now? Obviously, CVE databases are there for that, but we cannot always take a look at them, right? So how can we handle such issues?

So, well, there are tools to help you. And in this regard, I will still do some tools as a Hanne will mention, because it would be difficult to do it without it. Because in the test, you will always test for the things you look for, which you as a developer write down inside for the test. So in this regard, take a look at those. But there will be some which are open source, and are at least community plans where you don't need to pay money for it. So the first one, which is my favorite one with a catch, is Cypress and Overstep. So it's a plugin for Cypress combining it with Overstep, and it's fairly old. So hopefully, myself or anyone in the crowd will have some time later to basically maybe upgrade it, because it helps a lot to have security testing inside of your applications. But for now, at least it's a great example on how to do it. You can even find example tests in this repo as well. So it doesn't hurt us to take a look, right?

And aside from that, Overstep has wonderful recommendations for tools, for example, code analysis, IADE helpers, all things of tooling, which are mostly open source, or at least having a community plan. So take a look at those, if you are interested to learn. Last but not least, I will introduce a small QA quality assurance lens to this question of testing security with like bare-method testing frameworks, basically how to incorporate the test security testing inside of my test plans, pipelines, or workflows. And first and foremost, I'm using end-to-end tests as an example here, because of the limited timeframe and it's basically the testing kind, which I have most expertise in. But you can use unit testing, integration testing, component testing as well. So you are not stick with end-to-end testing if you want to, especially when it comes to API test cases. It even might be better to do integration tests or unit tests because they are basically faster and more isolated. So please don't feel limited to each test, okay?

So when it comes to incorporating security testing into my pipeline, let's go like this. First, I will learn my app, as we spoke about before, and the vulnerabilities which are relevant for my application to know what to test. Then I come up with a test plan. So which layers do I want to include? Maybe unit integration, E2E, whatever. Maybe I'll do a small risk analysis to know, okay, which vulnerabilities are most important to cover first or cover more often. Which test cases I need to write for risk, and then writing my test cases, of course. Afterwards, I will execute them, maybe locally, maybe already inside of my CI and analyze the result to get some learnings to see if I need to buckle up for other tests or leave out some others, just analyze and learn from them. And then I think about include testing tools if I need to and to combine those. And last, I will repeat it. So repeat both rules, but also the test runs. So I have my pipeline scheduled for every night, when, for example, the time is not that much of an issue. And then basically, we run those tests on a certain rhythm, but also basically repeat my plan as well. So think about new test cases, execute them, learn from them.

Yeah, the way you used to write tests, right? So I do think that tests are a wonderful complement to keep your app secure. They are the perfect messenger if you're using proper test cases, maybe for injection cases, for broken access control as a negative test, for content security, policy headers, stuff like that. To come together with implementing best practices to the application, building it robust and securely.

I do think that the attacker cannot bypass our defenses such as this Muslim, we cannot bypass this walnut defense, right? A zombie cannot jump over it. So my wish for my applications and yours as well is that it looks like this. Many different defenses keeping all attackers, all issues, all vulnerabilities at bay. So your customers or your users feel secure when it comes to your application and trust you.

If these are like three or four parts I want you to remember from this talk, it's this. Automation is a wonderful complement to have wonderful messenger to let you know if you got security issues inside of your app. Simple steps in test automations are most useful and are basically low-hanging fruits when it comes to having test line of defenses, right? So write test cases, use your features of the test frameworks. These steps are simple to implement I think. Combine own test cases you write from your own mind basically. And maybe some tools for things you don't know to have wonderful defenses. And consider to use all testing types inside of your test plan. Not only in trend but also unit or integration or component so to speak. If we manage to do this inside of an application, I'm happy but who am I? My name is Ramona Schwering. I'm working as a developer advocate at Offseral and you can find all things about me inside of basically my blog. You can see it in this QR code. If you got any questions, I'd be online in chat or in Discord. And hopefully I can see you at another Good Nation Conference in person next time. So apart from that, see you soon. I hope you have a wonderful Christmas time. Bye!