Play it Right with CodeceptJS: An Introduction to Supercharged End-to-End Testing

Hello. I hope you enjoyed the conference, and I hope you learned a lot today. Lots of new tools, new approaches. You met wonderful people, and I'd like to jump on the train and introduce you to a new, very cool tool, Concept.js.

So I would call it supercharged end-to-end testing. Well, because I can, and because I really do enjoy testing with Concept.js. So my name is Michael, and I will introduce you into this tool and show you the problem it actually solves. So I am from Kiev, Ukraine. I am actually a web developer, but I also work as a test automation engineers to practice, to see how people, what problems people do have and going with automated testing. And that's how I take this experience to make it an open-source testing frameworks like, surprise, surprise, Concept.js.

So yes, I am lead developer of Concept.js as well, but also I like do testing with Concept.js. Well, I think it's fair enough to say this. I started my journey to open source testing frameworks with my previous project, Conception, which is very popular in PHP community. And Concept.js takes this experience and brings this to a new level and gets into JavaScript ecosystem. Nowadays, I'm working for my own company and we are building a test management system for automated test, Testomato. Testomato is focused on JavaScript tests, so if you have a total mess with your tests, Testomato can organize it for you. That's the idea.

So I would start my talk with the question about do you remember that old good days when everything was pretty predictable, everything was stable, and I'm not talking about the things like that you could travel to all the places you want, you could talk to people you want to. I'm talking about just engineering stuff that previously we had only one tool. It was called Selenium, and we used it to test everything that is related to browser. If you need Selenium, you would probably choose Java. If you need, maybe you will need Python or Ruby, but you will choose Selenium. It's a certain decision. There was absolutely no alternatives on these days.

But today we live in the era of uncertainty. The JavaScript's found, and it created a wonderful ecosystem of different tools. And today choosing a right tool for the job is a tricky question, because we have Cypress, we have WebDriver, Playwright, Cucumber, Jest, Mocha. And well, how much of things you need to learn today to choose the right tool for the job. We definitely live in the era of uncertainty, and much more stress brings the questions I'm trying to list when I'm asking a client for new test automation project. because we can start writing tests, but then client came out and say, hey, I've learned about the BDD yesterday. It's so awesome. I want to write this test myself in Gherkin language. Can our tool write tests in Gherkin? Hmm, good question. Especially if we already have our tests not in Gherkin, but in JavaScript, can we easily introduce BDD level on top of it? Oh, the client came. I need Internet Explorer. You will hate me for this, but we need Internet Explorer because I have lots of enterprise clients. Can your tool easily switch to Enterprise Explorer on the fly when all the tests, especially with BDD on all the stuff we're already written? That's a good question. Hey, client comes again. I need Safari support. Well, Safari is fair enough. Safari is modern browser. Everyone uses it. And if our tool can do Safari or not, this also can be tricky for some kind of tools. But we need to be prepared for all those changes all this time. Can we adjust our tests to be executed on all those different environments? And especially for mobile browsers because the world goes mobile, and test it in mobile is also a hard requirement that client may not be fully aware of at the beginning of the journey but will be asking when you have some of tests and they will ask you, execute this test on mobile and this test. So mobile browser can be a new requirement as well. And against era of uncertainty, we don't know what a new tool will come up this year because every year, some good test automation tool emerges.

The previous year we received Playwright from Microsoft. Previously we got Puppeteer, there was TestCafe, Cypress. So lots of new tools are coming out here and there and maybe 2021 brings us something new. Will we be able to switch to it? If it's so new so-called Cypress killer, Playwright killer, absolutely anything killer, maybe we should try it as well and without rewriting tests. Is it ever possible? So this question we can address to concept.js.

Concept.js is BDD style testing framework. BDD style means that we write our tests in human-like language and we try to avoid some complex JavaScript interaction and keeping the code as readable as possible. And it's solely focused on browser testing that makes it different from Mocha or Jest but it have API for all browser engines.

So unlike Cypress, concept.js doesn't try tests on its own. It delegates the running to Playwright, to WebDriver, to Protractor, to Puppeteer, to Testcafe. So concept.js can easily switch execution between all those engines. And yes, it comes with Cucumber BDD support in case your client really needs it. So here is the test in concept.js. As you can see, you can read it. Yeah, it's very straightforward. You don't need to look into documentation. You don't need to think of what is happening on the screen. You can already understand everything that is happening here. So concept.js supports clicking by button names, by button tags, by CSS, XPAS, everything. So you can make your test as readable as possible when writing tests with concept.js.

So concept.js is pretty easy to install. It's just one command. And by default, it comes with PlayWrite because I think PlayWrite is really awesome nowadays. And I would choose PlayWrite for my next test because it's multi-browser, multiplatform, and I would recommend it to everyone. But concept.js can be installed without PlayWrite. It's just an opinionated decision which you can easily switch. So concept.js comes with a demo project, but let's start with a new empty project. You will need to initialize your tests and select browser engine. Select browser, answer this boring question, and create a first test. So when we did this, we get into the first test created.

The test should contain only these two steps, open a page and launch the pulse. Now, I'd like you to introduce another cool feature of concept.js is pulse. Pulse replaces all your tests that need to go after they open the page. So pulse is awesome.

Let me show you. So I'm executing this test. It's running in my browser. It opens Airbnb website with the playwright. And after the page is opened, the execution is paused, so now I can open my dev tools and think of what should I do next. You see this is my terminal. You see this pulse message here. So when I'm not sure what should I test, what should I start with, I'm just starting with a pulse. The page is opened as the dev tools are opened. I can navigate through the page and pick different elements, pick different commands, try them, and this is how it goes.

For instance, let me show you how we deal with calendars here. I have my dev tools opened on the bottom and terminal on the right. I write my commands into the terminal and I check if I can find elements by the locators I just picked up. For instance, I found that I can select date with Test ID Locator and I try it. Yeah, it worked and the date was selected. Then I press key Enter and I see that browser acts correspondingly. So I continue to do my actions by writing commands, selecting elements, and this is like I am writing my tests in the real time.

So when we finish picking up comments and we finished our tests, we go back to the editor and our comments were saved into CLI history file. What I do, I copy my commands from this file, only successful comments were saved. I remove the pause and voila, here is my test. It's included all the comments I had. So I was executing this browser once and I didn't do any blind guesses, blind searches. I executed the test and the test is here. By the way, I could write my comments and I have very nice auto completion support. So everyone can start concept GS because the comments are very easy to catch up. For instance, this is the test we finished with.

But as you see, it's very easy to reach so junior engineers can read it and they don't really care of how it is executed. They don't need to learn APIs of different frameworks, only these commons, which are very well documented and auto-completed.

How hard is to switch browser engine? Basically like this, we go to config and we change from player to web driver and we can expect that the test will be executed similarly, not the same, where because you see the web driver will work a bit differently than play, right? Maybe some points you will see crashes, but the idea is that the commons are the same, so all the difference can be easily fixed with maybe add some weighting, change some locator. So for most of the tests, the transition will work smoothly for 95% of test code. You will require no change.

So this solves the problem I mentioned. So if you need to transform your test to play, right, you can easily transform. If you need web driver, you can just change config. Maybe you will need to change to test coffee for internet explorer support, this also works. And yeah, this is really powerful.

Concept.js also comes with built in UI and graphical interactive test runners that can show step-by-step report of all executed tests. So you can use it as a reporting tool. You can see how the execution goes. And it has also some features that Terminal has, but some more advanced as well. For instance, you can keep one browser without relaunching the test and switch easily between headless and window mode.

Also in Concept.js, we have built in reporting support. So one of the best supported tool here is Concept.UI for sure, because Concept.UIs are done to Concept.js. But the screenshot on the right is not from Concept.UI. Maybe you already guessed what it is. It's allure reporting. So Concept.js has built in support for allure reports. In case you need better reports to show to your clients or just to see what was happened on CI in production so you could track the changes on your tests. We also do support the more enterprising technologies like Report Portal. We have official Concept.js agent in Report Portal organization. So you can have Report Portal reports with Concept.js and Testomatio, the tool I'm working on is also tries to bring better reporting to your end-to-end tests. So reports are pretty important in nowadays testing because everything can be shown from a console and for sure your managers, your clients won't look to console to see what tests are passed, what tests not, for what reasons. They want a clean, nice graphical UI to understand this. This is why Concept.js provides so many option for reports because depending on your budget, depending on your conditions, you can choose whatever tool you want. By the way, all those three tools are absolutely open source. You can start working with them and with Concept.js today.

So let me sum up some important features of Concept.js. Not mention features of Concept.js because you already heard of switching between engines, interactive pause, easy to learn syntax. What else do we have? And yes, I was already showing this a bit. Maybe you have already seen it.

So we support parallel execution by workers. So very modern, very elegant API. You can write your own custom runners so you could parallelize your tests however you want. For instance, group run tests in two groups, in two browsers from Firefox or make four groups at Safari. So you can split tests however you want into groups, bring as many browsers you need and have parallel executions and have single report after this. That's really cool.

Also built in, and it's a simply working solution for failed steps. So automatically tries for steps. So if some element is not found on page on first try, you won't even notice it. Concept.js will retries this and wait for it to appear without even notifying you. So it works for any command, click, see, so whatever it is. So it just works, just don't touch it and you're fine.

Concept.js takes the traditional concept of page objects and it's built into the framework so you can easily refactor your code to make it easy to be used in other tests. So page objects, step object, data objects are all built in. And as I mentioned previously, we have Cucumber BDD support built on top of Concept.js. So this is really awesome because you can combine true BDD with specifications in JavaScript like Concept.js tests. This is important because not all tests should be written in the truly BDD manner because not all tests are features. So because you need to have regression tests for box. So my point is that you should combine BDD approach with classical approach and only Concept.js testing framework allows you to do this.

So another cool feature I really like is if you have API and you could communicate with your application via that API, for instance, you could create a fixtures through this API, delete fixtures. And it is much easier to do it through the REST API interface instead of web interface. So you can easily combine them together. And that's my final points. Concept.js is a testing framework and what's different from other, it focuses on end-to-end testing without executing your end-to-end test by itself. So it tries to take the best user experience into itself to make testing as easy as possible. So Concept.js is absolutely free, open source.

You can contribute on it. There are already big and small companies using Concept.js and joining it. So join our team, join our community, try Concept.js on your own to check it and it works really great with Playwright. Thanks for watching, my name is Michael and see you soon.

Yeah, I thought that Cypress will take even greater numbers here. I'm pretty impressed that someone is using Concept.js. What I wanted to show with this polls that there are different engines and for different purposes, you can choose and the tools that fits right for you. So I think the Cypress is very popular because it's very developer friendly and it took on board lots of developers who didn't like testing, who didn't want to do this and browser testing seemed to them complicated because it was always a product of Selenium to deal with. And the part is WebDriver is more for QA engineers who work on end-to-end testing because they need more tool browser control. So I think it's like two audiences here and yeah, they're quite different but we need to engage all of them to collaborate and make better tests.

That makes a lot of sense. We've also started getting questions for your talk. So I've got a few questions for you, if that's okay. I'm gonna start with them and I'm not gonna mention the names of the people who have asked the questions because I'm gonna butcher the name so there's no point. But one of the first questions was, does CodeSapJS work in the multi-browser mode? For example, a pair of browsers are interacting with each other.

Sure, sure. We had this feature implemented like year ago, no two years ago. So yeah, it's quite some time. So it was working, it was better tested and yes you can use it. Or two different browsers if you want it.

Okay, okay. So that's really good. Somebody made the comment about CodeSap having different trade-offs like Cypress does. Cypress for example, says it doesn't have much tab support and you can't use it to drive two browsers at the same time. How's CodeSap fared compared to those?

CodeSap doesn't solve these problems. It delegates it to the engines that already solve this problem. So this is why I prefer tools like Playwright or Puppeteer which have full browser control, and this is why I dislike Cypress for most of my testing work. But yeah, this is user preference and whatever better fits to your job. So if you need to test single-page application with the single components, Cypress is really fine. But if you need full browser control and more and more features, CodeSap js is better because you can choose the engine while you write your tests.

So the code base is growing, but you're still able to choose a better engine if you need it. That makes a lot of sense and I like that, especially since people were really skewing towards one of the engines out there.

How does CodeSap deal with things that take a while to appear in the page? For example, a single-page app? Sorry, how does it, what? How does CodeSap deal with things that take a while to appear? So they're slow to load on the page because it's a big page, like a single-page application where everything kinda has to load before you can interact with it.

Yeah, so the problem with single-page application, we don't know is it loaded or not. So it never says us anything about it. So there is no state of this application. So we have like different strategies on this. The default strategy is outer, trying each step. So each step in the test will be automatically retried on failure. We have explicit waits. So you can wait for anything you want there. You can add manual retries. And if you have very, very bad applications that not very flaky, you can retry the full scenario. This is how it also works. I think that's all strategies, but maybe there are a few more of them. But this is that I use regularly. And that works.

People want to know more about the restrictions of using CodeSapp.js. Are there any restrictions like file uploading or working with iframe? What are some of the limitations of the framework? That's quite incorrect questions because I really feel it like Cypress auditory and you can't do it because CodeSapp.js is not driving the browser. It's delegated to Playwright and Playwright has no restrictions. So the only restriction you need to understand that CodeSapp.js can't do everything that Playwright can. So you need to write some Playwright code or Puppeteer code or web driver code if you need some very special APIs to be used. But yeah, you can do everything that you can do with Playwright, Puppeteer, web driver, et cetera, with CodeSapp.js.

Okay, that makes a lot of sense. That makes a lot of sense. Someone wants to know if you can generate CodeSapp.js test by recording interactions in a browser? I'm very skeptical about this idea. So I tried to use it, and I switched to this interactive pause mode. And I found this much more effective because I'm not just recording my actions, I'm testing it in real time, which I can't do with a recorder. So this is much more flexible and effective as for me as a tester. That's really good.

There's a lot of questions coming in from Michael. We won't get to all of them. We'll take one more. So when CodeSapp does the auto-reply feature, is that recording only on success on the first try, or for example, on the third try? The person asking the question has a concern about retries that can mask intermittent bugs. So how does the auto-retry record happen? I think it's quite a technical question. I'm not sure I will fit in time to answer it, but I would like to talk about it after my talking in the room.

Oh, perfect. In that case, that's a perfect like for me to remind everybody that they can join you in your speakers room on spatial.chat. The link to join is on the live stream timeline, and please join Michael for more questions. We won't get time to go into any more. Thank you, Michael, for a talk and for joining me for the Q&A. It was really good. Thank you for inviting me. Thank you for opportunity to be here with you.