A11y Beyond the Theory: Integrating Accessibility Testing Into Your Workflow

Hello, everyone. My name is Ntandala Kengose. I am a software developer, among many other people. I am giving this talk from the beautiful city of Johannesburg, where I was born, raised, and live till today.

Now, if you have no idea where Johannesburg is, it is a big city, not the capital, but definitely the economic hub of the beautiful country known as South Africa. And if you're not entirely sure where South Africa is, well, it's simple. It's in the to look at it. We are at the southern tip of beautiful continent of Africa. Now, this should be showing you that when they said that some of the most difficult things in computer science is naming things, they definitely didn't think about South Africans, because clearly we're really good at this.

As I said, I am a software engineer, and I work for a company called PbD. And again, because we're good at naming things, PbD stands for the founders. Now, PbD started about thirty nine years ago, right here in South Africa by three engineers. And we've now grown to be a global bespoke software development firm with probably about one thousand two hundred professionals spread across seven cities all over the world. We deliver bespoke software solutions into a number of sectors. But my job is slightly different from everyone else at PbD. That's because I work in a unit we call ATC. In paper we're responsible for a number of things, such as what we call specialized consulting. We are responsible for training PbD's 1000 plus employees. We're responsible for also doing community engagements and a whole bunch of other things that are quite interesting. I love community. And to fuel this passion, I also am involved with number of meetups around Johannesburg.

None of this matters. The most important thing for you to know about me is that my Twitter handle, X, is at unlikely underscore. Please feel free to follow me, let me know what you think of this talk. Let me dive straight into it. I turned 32 days before the recording of this video. Now, with this new age, I've realized that I need to start looking after myself a bit better. I need to get back into fitness and try and look after my health and what I eat. So a friend of mine strongly suggested that I try the gym. Now, there's a couple of problems with the gym.

That's because it can be quite intimidating when you walk in and you see all of the heavy equipment and some of the things that people are doing can just be daunting. So I decided to go back to things that I actually enjoy, which is sports, two sports in particular, rather, namely football and soccer. And no, I don't mean it like this, I mean it more like this because I seem to be really, really prone to injuries.

I play two sports, one where the players are known for faking their injuries, and the other way the riders are really known for trying to avoid their injuries and getting back onto their horse. So take a wild guess as to which one of these two sports ended up or resulted in me having this knee brace on my knee for over three months. It's soccer. I had a minor incident, and I ended up damaging my knee.

Now, something interesting started happening after this injury. I started being very, very grumpy. My friends said it was just old age creeping in, but I realized that something else was happening. I was starting to experience the world like never before. I was experiencing it as someone who couldn't walk as long as I normally do, and I realized that I was actually experiencing a temporary disability and that this was making inaccessible designs around me much clearer. And all of those things were now starting to affect and frustrate me.

And the first thing that really, really, really made me angry was the shopping mall. I live about a block away from one, and I've always thought that it had parking all around it and was a great experience to go to. I hated it because when people think accessibility, all they think about is ramps. And I found these ramps everywhere, and I don't know if you've ever tried, but walking with crutches on a ramp is really, really difficult. And I struggled a lot. I was so furious and I did what every normal person would do, try and figure out whose responsibility is it. And now that I was faced with my own physical limitations and trying to figure out who to blame, it dawned on me. I started reflecting about whose responsibility it is to ensure that the solutions we build are as accessible to as many people as possible.

And to answer this, I think we just need a quick, common working definition of what we mean by accessibility. Well, firstly, A11Y is a numeronym where the 11 stands for, it represents the 11 letters in the word accessibility. And the major dictionaries really gravitate towards this general definition, where they define accessibility to be the quality of being easily reached, entered or used by people who have a disability. If we bring our focus much closer to home, we see that web accessibility means that websites, tools and technologies are designed and developed so that people with disabilities can use them. More specifically, that they follow the PAW principles, which means that people can perceive them, understand, navigate and interact with the web that we build.

So to simply answer the question, the answer is that it's everyone's responsibility, everyone that is involved in the software development life cycle. We have long agreed that it works on my machine is simply not enough. So I've always wondered why we're so comfortable with shipping solutions that work for some people and not necessarily others. And every time I've been engaged in conversations around accessibility, it's often treated as a nice to have, but what we don't realize is that it actually affects the bottom line.

Inaccessibility costs businesses significant amounts of money. And I mean, we're also seeing it becoming a legal requirement all over the world. So the W3C has put together something they called the Web Content Accessibility Guidelines or WCAG for short. Now this is a set of technical guidelines that are aimed at making web content more accessible to people with disabilities. It effectively has 13 guidelines, as of 2.1. And the guidelines fall under four core principles, meaning that they are aimed at making sure that things are perceivable, operable, understandable, and of course robust.

Having guidelines and principles is one thing, we also need to be able to measure the success. We need to be able to describe just how well we conform to these guidelines. And for this, we've got really, really simple three levels of conformance. Namely, A, AA, and AAA, which range from simple and very common things such as missing alternative text, keyboard accessibility, all the way to the other end of the complex spectrum, where those issues are more concerned about the content and comprehension of our users. The interesting thing is that if you take a look at some of the most common challenges, they are often the easiest to fix. So things like the lack of alternative text, inaccessible forms and things like poor color contrast can actually be looked at quite programmatically. Even the lack of keyboard accessibility is still one of the most common accessibility issues that are found on the web. And, I mean, we've long seen that keyboard shortcuts or the use of those shortcuts can improve productivity quite significantly.

I recently had a conversation with one of my senior colleagues and he told me about how we once had a client and part of our service offering was that we guaranteed this public service client that we would be able to have someone test the solution to make sure that it's accessible. He told me a story about how because they had this one person, a person who was blind, use the system. A lot of the time when it failed, the client would actually be quite upset about the amount of time that we were adding to the process. And I realized something crucial about what he said.

For such a massive software solution, they had a single person and just said to them, tell us if you can use this. They often released a lot of these solutions without passing quality assurance measures because they were chasing deadlines. I then realized that we actually need to equip people that are involved with the tools that are there to put the tools that can educate them rather of what needs to be done and how to go about implementing these things. But where do we even start with this? I'm a practical person so I don't believe in simply sending people off to a training session or a workshop. I think we need to do more and it needs to be in practice. And for this, practicality is key.

So I started digging around for a couple of solutions, and effectively, this mission was to find accessibility testing tools for everyone that I work with. I started with some of my favorite people as a developer. QA engineers, test analysts. They come in many forms, shape, and names, but we all know that they are a developer's worst nightmare. Now, when I talk about QA engineers, I'm talking about manual testers, for instance. What I found is that there's something called the Web Accessibility Evaluation Tool List. I absolutely love this website, because on the left hand pane it also gives you an ability to filter as to what type of tools there are based on either the guidelines that you want to follow, the legal requirement, or the type of accessibility concern that you're trying to address. Now this tools list has more than 100 tools, because we are still in the web and we get a tool every minute, but the filters are really awesome.

So effectively, the rest of this talk is going to be me telling you about a list of some of my favourite accessibility testing tools. But to do this, we need a website. Now I thought of using the React US website, but then I realised that I really want them to invite me back to this conference next time. So I found this test website, it's called the Medical Mysteries Club. It's built by the Chrome DevTools team and they built it with a disclaimer saying that this page was deliberately made inaccessible for the purposes of the demo, which is quite cool. You can check up the site, there's a code pen for it and here I'm running it in debug mode. Speaking of the Chrome team, my first tool is Lighthouse. Lighthouse is a really, really great tool and it comes shipped with the Chrome browser. And this tool is quite common, you simply need to do a click analyse test in your dev tools and it should be able to give you a good breakdown of some of the accessibility issues. Now, Lighthouse is often used for other things as well such as SEO and performance, and so I won't dive into it a lot because I understand that a lot of people would have seen it. I'll show you similar tools instead. The first one is called Wave, which stands for the Web Accessibility Evaluation Tool.

Now, this is a browser extension and once you've added it, it actually annotates the screen that you have in front of you, so it will put all of these tiny icons to actually show you what these issues are. What I like about this is that when you go into detail about what actually is wrong in there, it's able to show you things like such as the low color contrast and even help you figure out what color is best at that point.

Another tool that's quite similar to Lighthouse is called XDevTools. Now, XDevTools also is added in your Developer Tools as a tab. You also just click Analyze, and it will give you a breakdown of all of the issues that are there. One thing to note though is that this is paid, and by the time I started building up this talk, I had already run out of my trial license. The important thing for me here is that I wanted something that I can use for everyone that I work with.

I looked around and I found something for the project managers. When you're managing a project, you're not necessarily interested in the very minute details, but rather overviews of things that are happening. For this, I found one of my favorite tools called Pally, the automated accessibility testing file. Now, Pally is an open-source tool that helps developers and testers identify these accessibility issues on web pages and applications. It can be found on npm, of course. When we take a look at this, we see that it can run in the command line or can be used in your own solution in a node application. What that looks like is that you would simply import it, and you would provide it with the URL. It would go and crawl that web page and return those results to you. You can then do as you wish.

Here, if we go and replace our testing website with the CodePen for the Medical Mysteries Club, we see that if we go and print out those results, this is what it start looking like. What I like about this is that it tells you which principle or which guidelines you are violating. It's able to show you a human-readable message. It gives you the context of where the issue actually came from, and even the HTML selector. Now, this is really interesting. I realized that I'm onto something here, but I'm a web developer. As soon as I start thinking that I'm building something very useful, I know that I need to stop because chances are someone else has probably built something similar to me. For this, I found Peli Dashboard. Now this Peli Dashboard, you can configure the URLs that you want to go and take a look at, so that you can see your transitions over time. This is absolutely great if you are making an already existing application more accessible because you can track your progress and the progress of your team over time. But of course, if the only time you identify accessibility issues is in production, then it's really, really too late. You generally want to intercept these problems at the developer level. You want to guide your developers to solve these problems while they are building. But how do you do this? Well, we automate this.

Automates the testing portion of this. Of course, developers are new to testing. We all write code that we've tested thoroughly. As a developer myself, it's not that I don't trust engineers that work on my project. But rather because I'm an engineer, I know that sometimes you just happen to forget to build and run your tests. I've learned that the best way to get developers to conform to these standards is to simply block pull requests if the right thing isn't in there.

Effectively, we need to automate this accessibility testing and then block the pulled requests if they fail. If your work cycle looks something like this, where you make some code changes, you push them to some server and you create a request to merge changes into your main branch and then you merge them. But this would mean that we would have to insert one more step, to simply check if the component you've added or the code you've added is accessible. What that looks like is quite similar to the manual portion is that we would need to start up a server, launch a browser, go to localhost. I have no idea why that's a chicken in there. We'd have to wait for it to load, and then once it's loaded, we then have to run our Pelly tool.

To do this, because now this needs to happen in the command line, I found Pelly CI, which is an accessibility test runner that's built with Pelly under the hood. But this one is particularly focused on running in continuous integration environment. This is particularly important because we don't just want to get results of what's broken. We also want to break the build itself so that we don't allow it to be merged. Putting this into action, I chose to go with GitHub Actions, because that is probably one of the most commonly used CI-CD pipelines. GitHub Action automates tasks in the development workflow, which means that we can customize how we do this thing. Everything that I've just described, we have to put into our CI pipeline configuration.

But now if we talk about testing, a typical testing setup would look like this. You would need a test runner. Examples of this are Mocha and Jasmine. You need an assertion library which defines your testing logic. This says I expect A to come out as B under these conditions. Then because we're talking about things that actually need to be rendered, we also need a browser. Now, one of my favorite tools because I use it on all sorts of projects is Jest. Jest sits right across those two. It's both a test runner and an assertion library. They describe it as being a delightful JavaScript testing framework that focuses on simplicity. What that looks like is that if you have a function called sum, you would simply write a test that says, we expect the sum of these things to be two or to be this result.

If it fails, as you see here, I said I expect the sum of one and two to be six, which we know it's incorrect. This is what the error would look like and it tells you that one of your test fails, which means that you can block the build using this.

I then realize that we can actually use gist with something called gistx. X is another core rules engine that helps you figure out accessibility issues. Once we start using it like this, we're able to replace the function with a render of a simple component that we want to test because we cannot always be testing entire pages. Sometimes we want to test a small component of what it is that we are building.

We see that once we pass it in there and we ask the gistx runner to test it for us, it's able to tell us which rules we've violated. But if you take a look at this one, it says that all page content should be contained by landmarks. This is great and true for full pages, but not necessarily always the case in smaller components that'll always exist in bigger pages. Which means that we need a way to configure these things, which is really, really great. You'll see that it also tells you what to fix and it gives you a link to Deque University's guidelines as to how this actually works. To configure it, it's nice and simple. We can enable the different rules for isolated components or for our different test cases because we know that we can never just have one solution for everyone. But again, this is a guideline, or this is a guard that we put when developers are building. What we really want to do is get even closer to the developer and help guide them and coach them while they're actually working.

For this, there's the X-Accessibility Ginter. It provides tooltips on why issues are being marked, why they are considered to be issues and even how to go and fix them. To go one step further at annoying my team, I also came across Husky, which is a beautiful GitHub that allows you to run these tests on commit. Not only are we stopping developers from merging, but if I really want to annoy the team, I'll configure this to stop their commits so that they don't even commit anything until they make sure that it's accessible. What it looks like is that once you try to commit, it will tell you that there's a problem here and this is why things are failing.