Tiny Tests, Large Results

In today's tutorial, we're going to talk about automated atomic tests. Automated atomic tests are a wonderful way to update your UI tests and make them less brittle and drastically faster. We're going to see a few examples of those automated atomic tests, and we're going to even implement those automated atomic tests on two web applications.

One application will have an HTML web form and another will use a JSON web token for authentication. My name is Nikolay Advolatkin, Senior Solutions Architect at Sauce Labs and founder at ultimateQA.com. So, what are you waiting for? Let's go and check out how to implement automated atomic tests.

An automated atomic test is a test that tests only a single feature or a component. They typically have very few UI interactions and typically touch a maximum of two screens, especially with front-end UI automation. The reason why I say they typically touch a maximum of two screens is because these automated atomic tests will typically have a setup state where they need to set up the state of an application and then a post setup state where they need to perform a validation. Automated atomic tests have several good environments in that they are much faster and much more stable than your typical automated tests due to the fact that they decrease the amount of latency that we have to deal with and they decrease the amount of UI interactions that we have to perform.

Now I got a question for you. This test over here, if you look at it and analyze it, do you believe this test is atomic? At first glance, this test might actually look atomic, right? In that it's signing in, validating that we were able to sign in and then it adds an item to a cart and asserts that an item was added to a cart. It's close to being atomic, however it's not actually atomic. The reason why it's not is because it has to login and validate that we have successfully logged in. There is no reason why we can't test the UI login in a separate test and ensure that works and then afterwards, what this test really cares about, what it actually wants to test is this portion right here. And so we can save time and stability by performing these interactions without using the user interface.

Atomic tests can come in many forms. Some of them can be extremely simple. For example, when we used to write automated atomic tests that click links, we actually have a wasted action and a wasted validation in that we need to click a link and assert that the link goes to the right location. What we're actually interested in is the Ahrefs attribute of the link, which we could easily validate in this manner, making our test faster and less brittle.

Let's go ahead and take a look at our first web application, which is going to have an HTML web form, and we're going to be able to log in to this HTML web form. But instead of using the UI, which is not efficient and not atomic, we're going to be able to make a web request that's going to end dropping a cookie in our browser, and then we're going to be able to bypass the log in without interacting with the UI. If we come to this application, there are many tests that are here, but I'm going to focus on one at a time, just to give you a better understanding. The very easiest test to understand, which is the positive test case, is going to be this test case here called Redirects to Dashboard on Success.

This is the very positive test case where we visit the login page, and by the way, you can see our application on the right hand side, if you're not familiar with Cypress. It displays the test execution on the left, with the actual application on the right, and if you did want to explore the application in another browser, you definitely can. Here it is, and it performs all the same exact operations, but with Cypress, for example, it's very easy in that you can see what the test is doing and then see the actual application on the right hand side and even interact with it if you wanted. You can see that the very first step that this test does is it visits the forward slash login of our application that is on local host 7077, and after that, it performs the standard operations that you would expect in order to actually be able to log into the application, right? So it's going to get the username and type in Jane Lane. It's going to get the password field and type in password and then it's going to submit the form. Once the form is submitted, you can see we do a couple of assertions. First, that the dashboard page is there, right? So our URL includes dashboard forward slash dashboard. So that means that we were able to successfully log in. Second, we get the header and assert that it contains Jane Lane. So almost checking not only that we're logged in, but that contains the right user and we get the cookie that corresponds to our login and make sure that there is a cookie that exists. So take a look at the application tab here and in our cookies. Currently, there is nothing available here. I've cleared it. But if we rerun the test, and of course, as I mentioned, it does interact with the application actually logged in. You can see that now there's a Cypher session cookie that's dropped, hence why we are able to log in through the application. The challenge with this test is that, again, it's not atomic, right? In that it does interact with this login form, and then it logs in, and then it validates that we're logged in.

So that's great. But if you wanted to do any kind of actions beyond this, for the login, we'd be doing the same thing over and over every single time. And so it's repetitive and unnecessary, and maybe we can make that action more atomic. But let me actually show you the rest of the tests, so that you better get an understanding of what the application can do.

So I uncommented the only part of the test. And so now we have all of the tests that are available here. And these tests, of course, as good end-to-end tests or any kind of test should do, they kind of help us to understand the functionality of the application without actually manually stepping through it.

So the very first test here is called unauthorized. And so the expectation here is that you're redirected on the visit to forward slash dashboard. So if you're not logged in, you should be redirected. And so this is what this test does. It tries to visit dashboard, but then it gets redirected to you're not logged in and cannot access this page. Hence there's an assertion that gets the H3 and checks for that message. And it checks the URL includes unauthorized in it, right? So if you're not logged in, can't access the dashboard page. If we try to log in by doing a get on the dashboard page, we will also be unable to do that. And so we're expecting a 302 to come back to us as a result when we try to log in without a cookie. And we'll also expect the response to come back with unauthorized in the body.

If you try to interact with the actual form and you try to log into it with, for example, invalid user and the valid password, we expect there an error to be visible and the error to say that username and or password is incorrect. And on Success, you've already seen this. If we visit the dashboard, type in the valid user and the valid password, we get a success in that you're able to visit the dashboard by itself. This test is atomic and great and very useful to test a single time. Once you're able to log in with the web form through the UI, you know that it's going to work and it no longer needs to repeat it in the rest of the tests.

And so how do we make it atomic? In this case, for this web application with an HTML web form, the way that it works is you submit a request. How would an atomic log in actually look under the hood? Well, as you can see from the documentation of this test here, we can simply perform a web request that will allow us to log in. The web request is simply going to be a post to the log in URL, passing in the valid username and password. And then that will automatically set the session cookie and so now after this point, we can perform any operations that we want that are behind the private application interface. So, let me show you what exactly that looks like. And so, here you can see doing the post to the log in, right? And then showing you that the cookie does exist. And in fact, we can even simply do, just take a look at this test so that it's focused. Go back here to the application, get rid of this and then if we rerun it, we're going to be able to see that there is a cookie.

And nothing happened in our application, right, but if for example now, we come and try to access the dashboard because we are logged in and we have a cookie in our browser, we're able to access it without actually logging in. This time instead of our application being an HTML web form, it's actually going to use JSON web tokens. JSON web tokens are a very common industry standard method of basically authenticating into applications and allowing access to certain parts of the application based on this web token. If you want to learn more about it, you can go to this website here JWT.IO, but basically the way it works is you submit a request with your username and access key or username and password and then based on that, the application will verify whether those exist in the database. You get back a token that looks like this and now this token is what you can use for authentication, very common in many applications in today's day and age. So extremely useful to be able to do this in our automated test. Here is the repo that we're going to be working out of. As before it's very similar. This folder that I'm in is part of a much larger folder. I can show you exactly here. So you can see the folder I'm in is logging-in JWT. If I navigate up, you can see I'm in the larger Cypress examples repo. And then as I said, we're working out of logging in JSON web token folder. The application we're going to be working on is a Vue application, and you see the instructions on how to start the app here, which I did over here. Actually, I simply ran NPM run start to start the application. And so that's running in here. And then in another terminal window, I ran NPM run Cypress open, and that opened up my Cypress browser. And so now, let's go take a look at the tests to see exactly what they're doing and how to do it through the UI, and then how to do it without a UI through an atomic manner.

The way this app is going to work is it's going to send the login credentials, and then those login credentials are going to be stored in local storage in an item named user. You can see from this screenshot here, there's our token. And then any requests that we make afterwards that are behind this authorized information, that's going to require the authorization header with a bear token in order to be able to access the resources. And so if we, for example, take a look at the network tab here under the users, you'll see that there is an authorization and a bear token with the JSON Web Token, and that's how users can get access to different parts of the application.

Once Cypress is up and running, we'll be able to see these two specs here, right? One is using UI and then the other one is not. And so if we pull up this using UI spec, you'll see all of the tests that are performed, right? Using UI, it'll be something very standard. You've seen before, which is basically getting the username, typing in that username, typing in the password into the password field, and then clicking the login button and you can see what happens before and after. Afterwards, we are basically checking here. You can see there's a web request happening of a Pulse 200 and then we're doing a number of operations such as we're making sure that we are at the correct URL, expecting this header to be visible, and then we're even expecting that there is an item with a username first name to a last name and then it has this token that is a JSON web token and we're expecting it to be a string. And then there's a bunch more assertions on different parts of the web request. There's even more assertions like this checking that there's a logout link and then clicking the logout link and logging out and asserting that we're on the login page. So if you try to access the localhost 4000 slash users, this URL, you will get unauthorized. Right? So you get this error message. If we come here and try to access this application invalid token because we can't get there. And so we're supposed to get back a 401. Right? So if we come here and inspect and come here to the network, and then we do Yelp XHR and the users, we will get here a 401. You can see the users document returned was 401 because you're unauthorized. And so this basically obviously shows a positive test case and a negative test case both through the UI. Oh, and here is actually one with the UI that will try to log in with an invalid username. Right? So typing in username, typing in a wrong password, and then clicking to log in. And then of course, it's going to get that username or password is incorrect. And hence it's doing that assertion to make sure that it exists. So that's our UI test. Wonderful. Again, fast on local host will be much more cumbersome when we actually deploy this to an environment that have to travel over the network to interact with this and more flaky as well as a result. So, hence we can bypass all this using web requests as we did with the previous web application. So if we wanted to make the same test atomic, we have these specs here and I'm going to run them. Unfortunately, they're not as useful to see through the Cypress UI because they do make a bunch of web requests. And so the information behind them is not super useful, but you can see how we can make an authenticated request, make sure that we're logged in, and show a loaded user. So instead we can actually look at the code to understand how to make this possible with JWT. So if we look at this spec.cy.js, we can see that the very first step that we do is actually hit an endpoint to authenticate a user with a username and password, and then store the returning body inside of the user. Then, in order to be able to add the item to our local storage, we need to set that item called user and we stringify that user, hence enabling that Bare Token. And so, once that's set, we can try to hit this endpoint with this Bare Token and we are able to access this user, without which, right now, we would not be able to access it. But with it, we are. And then, of course, as a result, we can check for different kinds of conditions, to be or not to be visible. And that's it. We've created an automated atomic test using a JSON Web Token login mechanism. Now, there are actually lots of different types of login mechanisms and certain ones may be relevant to your application, others may not. There are many more examples here, as you'll see in this repo. You'll see all of these login examples right here, from Basic Auth, all the way to Single Sign-on, so depending on your application you'll use the relevant login mechanism. Also, as a bonus, I've linked a bunch of bonus resources to how to do automated atomic testing in the ReadMe, it's at the bottom of the ReadMe, so definitely be sure to check that out. Thank you so much for joining me in today's tutorial about automated atomic tests, and if you want to learn more about me or catch up with me, you can find me at ultimateQA.com or my YouTube channel or any of these other social medias down below. Thanks again so much for your time and I'll see you next time.