Who Guards the Guards? – Finding Bugs in Your Tests

Hello everyone, my name is Tibor Blanesy and I work on static analysis as a sonar source. Welcome to this talk which is called Who Guards the Guards? And I would like to show you three common pitfalls of JavaScript unit testing.

So the first issue I would like to show you is the following one. So imagine you have the following test and you write your expectations there. Do you see the issue? Whatever you do inside the expectation, this test actually never fails. Because what is missing is the assertion after the expect. So to have any meaningful testing, it has to look like this. You might be surprised how common this issue is. I will just show you two examples. So the first one is in Kibana project and the second one is in Angular.

The next issue I would like to show you might be a bit cosmetic. However it often helps to have the clear test to better understand the real issue. So in the following test, when you write your assertion, you always need to provide the actual value which is the result of the test and your expectation. And the order in which these values are provided matters. So to the expect you should provide the result of the computation and when you invoke some assertion like equal in this case, it should be the expected value. If you swap it around, you will receive such a message as here where expected is two and it's supposed to equal to three which is wrong and it might be confusing. Correctly it should look like this. You should have the addition in the expect call and it should equal to what your expectation is, in this case two, which gives better message where three is expected to equal the two. So the result is what we computed is three and our expectation was two. And here is the actual correct test where you need to correct your computation to have the test to be green. So this is cosmetic but when you are debugging a more complex issue it might be really helpful to have a good error message. So here is the real life project having this problem. You see that in the expectation the test and data is swapped.

The third issue I would like to show you in this lightning talk is the following one. Imagine you have a test and this test is going to be always green. Why? Because there is actually no assertion in it. So you are not testing anything. So to fix this test you should actually assert something. So for example, you want to assert if this string has the value of string. And here is the real life example from ESLink project where you might have this problem.

This often happens when you are dealing with exceptions. So the following test will fail if the exception is thrown as the comment suggests, however the message is not going to be very nice. Better way to do it is to wrap your call in the expectation and assert that either the exception is thrown or in this case it would be to assert that no exception is thrown. So this is the documentation from CHI assertion framework on how to do that.

So you might wonder how you can avoid having such issues in your code and I will show you a small demo how you can do that with SonarLint or SonarCloud or SonarQube. So first I will start with SonarLint. SonarLint is an extension which you can install in Visual Studio Code. So here is the extension, it's already installed and here I have the three examples I was showing you. So the first one here is the incomplete assertion where the expectation contains the comparison between some function and the result value and actually to have this test to make it more meaningful what you should do is to remove the expectation for here and add the assertion like this. And the issues here disappears. The second problem I was showing where you can hover here to over the squiggly line to see the description. So it says that the two arguments in the assertion should be swap and the correct order is the actual value first and then your expectation. So when I exchange the arguments the issue disappears. And the last problem I was throwing was the problem with the missing assertion. So you see here when I hover over the squiggly line I see that at least one assertion should be in this test. So let's add an assertion here. So let's say we expect the string to be empty like this and the issue disappears. So this is how you can quickly fix it in your IDE and your editor will notify you when you have this problem. And also if you want to integrate this sort of static analysis testing for your unit test in your continuous integration pipeline, you can use sonar cube which you install on-premise or if you prefer the cloud solution, there is sonar cloud. So they will both look the same. Here I have the sonar cube and here is the the same file which was analyzed in the continuous integration pipeline. So here you see again the squiggly line where the issues are located.

This is my lightning talk.