AI and Accessibility: We Got a Lot to Talk About

Let's start diving into the presentation because we only have 20 minutes to talk about everything. As Daphne said, I'm Kat Johnson and today I'm going to be talking to you about AI and accessibility and there's a lot to talk about and we only have so much time.

So one thing I want to give a disclaimer of in this presentation is that I am not an AI expert. I am an accessibility expert. Most of my career has been working at Microsoft and now I work at Khan Academy where I specialize in making accessible web components and widgets that work with screen readers and other assistive tools.

So you're probably wondering why I'm talking about AI. Well, it's mostly because everyone asks me all the time. All the time people are asking me about AI and whether it's going to be this revolutionary tool that's going to fix all of our problems with accessibility. And really this presentation is my way of trying to answer that question, get it recorded, so I don't have to keep answering it all the time.

So today I really want to talk about all the ins and outs of AI as it relates to accessibility. I want to talk about what AI can do for accessibility. What are the limitations of AI? And what are the pros, cons, and everything else in between? So with that, let's start diving in and looking at the nuances of AI and accessibility.

But first, let's talk about what is AI? Because grounding ourselves in what is AI will help us most in answering this question. So what is AI? Well, this quote I found online, and I really this is my favorite quote, hence why it's in this presentation. AI is simply a simulation of human intelligence by a machine. This we see all over the place within our society from really interesting little AI chat bots when you're trying to update your credit card details to the fantasy of Terminator and Skynet taking over the world.

So that's a broad range of AI. So let's break it down to some actual terms that we can use to define what AI is. And really there's three types of AI that I'm going to touch upon. There's actually a lot more terms around AI. Obviously an AI expert could probably go into more detail. But for this presentation, I want to point out three main types. Narrow AI, general AI, and super intelligent AI.

Now this presentation could easily dive into each one of these subtype areas. But that would eat up all of our time. For this presentation, we're really just going to focus on narrow AI and what it is exactly. So narrow AI is really a type of AI that performs specific tasks and doesn't learn anything beyond that. One thing to note is that most of the AI that we're seeing out in the industry right now that we're all really excited about is just narrow AI. And you see narrow AI in so many different applications from self-driving cars to face recognition software to even those cool chess-playing robots that beat grandmasters at chess. But all these are narrow AI in application.

Now some of you are probably curious about generative AI, LLMs and those large language models, how do they work and what type of AI are they? Well generative AI is really just a subset, an extension of narrow AI with the purpose of generating content based on large amounts of data that it's been trained on. So it's still narrow AI. It's meant to do a very targeted set of things, but just reproduce it by generating content.

And there's a lot of content out in the market that's doing that today, live-chat GPT, Copilot, applying it in different ways, Midjourney for generating images, and Synthesia. I'm glad I said that correctly. But Synthesia generates videos. I haven't tested it out, but it was really interesting, so I included it in this presentation. So that's AI, and this presentation is going to be focusing on narrow AI.

Let's pivot slightly and talk about accessibility. And what is accessibility just to define it for us? Well accessibility simply means the ability to access. It typically focuses on enabling access to societal and community services and tools for people with disabilities. You can think about accessibility in terms of the closed captioning you see on your Netflix shows. Or the powered accessibility doors when you get into this conference center. And a lot of developers, especially us web developers, when we're working with and trying to make our code accessible, we're usually focusing on ensuring that our code can be picked up by different accessible tools such as click... Come on. There. Oops. It's glitching out. One second. Boop! All right. Braille readers. Yes. One of the tools I have to work on at work is ensuring that my website will work with Braille readers and screen readers such as NVIDIA, JAWS, VoiceOver, and so many more. But these are some good grounding things to think about when we're talking about accessibility.

Now let's move on to the exciting part of the presentation. Talking about all the achievements in accessibility with the help of AI. One big thing I want to make a note of is that AI has made writing so much easier for so many people. Chat GPT, a tool to help us write essays, write letters and emails. I don't know about you, but I detest writing emails. And so Chat GPT has been very helpful for me.

But it's also been very helpful for a wide range of people who don't enjoy writing or have disabilities and limitations around writing content. Meetings are also so much more accessible. I don't know if you've noticed them, but some meetings will now have this meeting summary option, where I'll summarize the meeting so people don't have to take their own notes and those are provided for users.

There's also been some really interesting advancements in text-to-speech applications like Speechify that is really helpful for people with dyslexia. All that has been helpful with AI. There's also been some really cool advancements with voice recognition, improvements to captioning and translation tools. So now our captioning and translation options are much more efficient and much more exact. There's also improvements to voice command applications, like voice access and voice control that users who don't use mouse or keyboard can use their voice commands to interact with their device. And now it's so much more efficient.

There's also been some interesting things with image recognition and the ability to recognize images at much more efficiency and much more accuracy. And there's been some interesting advancements in applications for visually impaired users that they use to navigate the world around them. Now I'm going to play for you a video from Be My Eyes that will walk through a really interesting demo of this. Do you guys got the video up? I'm Andy Lane from Be My Eyes and I'm about to take a journey that may change lives around the world. Vision is such a dominant sense for most of us. Our eyes feed us an endless stream of information. But it is somewhat different for me. I'm blind, so I've had to learn new skills and use workarounds to live my life without sight.

I still think visually, but I have to build the images in my mind through other means. Today though, I'm experiencing something new. I'm using OpenAI's newest model and voice first interface to imagine the world through a conversation with an AI.

Nice sunglasses. They definitely add a cool factor.

Oh, thank you. What is in front of me at the moment? You're looking at a beautiful sunny day by the River Thames in London. Right in front of you is the Tower Bridge, one of the most famous bridges in the world with its majestic twin towers and blue suspension cables.

That's perfect. Can you tell me if the bridge is up or down at the moment? The bridge is down at the moment. As the AI is speaking, my mind is building a visual image. I'm now getting a real appreciation of the majesty and scale of some of London's sights as they're described to me.

Tell me what you can see. I can see the top of Big Ben peeking through the trees. It's got that classic neo-gothic design with the recognisable clock faces on each of its four sides. It's such an iconic landmark, isn't it? I can visualise the iconic sights that so many visitors come to London to see.

Is the King in residence at the moment? Yes. It looks like the King is in residence right now. The Royal Standard flag is flying above Buckingham Palace, which is the signal that the monarch is present. How exciting to be there while the King is home.

Try and tell me exactly what they're doing right now, please. Right now, the ducks are gently gliding across the water. They're moving in a fairly relaxed manner, not in a hurry. Occasionally, one of them will dip its head under the water, probably looking for food, and then pop back up. I even know when a taxi is coming with its orange light on. I think I'll hail it to get home. Yes, I spotted one just now. It's heading away on the left side of the road. Get ready to wave it down. Great job hailing that taxi.

It looks like you're all set to go. That's a good dog right there, leading the way into the taxi. Safe travels.

Blind people around the world have rich and fulfilled lives, but it can be so much more difficult without sight. But my experience today has given me an overwhelming feeling of power and joy, and I'm so excited that other blind people around the world are soon going to have access to OpenAI's newest model through Be My Eyes.

I think it's a bit like a driverless car. It feels like it's a lot to take in, but then the instant that you do take it in, it just becomes completely normal. And that normality is, like, so full of hope and optimism. Yeah. Yeah. Makes sense. I love that video. Went too far. And beyond that, there are so many more interesting applications that are happening out there. There's improvements to screen reader interactions, there's also some really interesting ways that Copilot can give accessible code suggestions for developers, and so much more.

This presentation could have easily become just a giant hype video of all the interesting things happening in AI. But like any tool in an application, there are some definite concerns with AI that a lot of experts in this field are obviously quite concerned about.

The first thing I want to point out with the use of AI is that AI does not replace knowledge experts. It's really important that if we're using AI, that we don't use that as a blanket reason to not hire experienced experts like our designers and content writers. Our designers and content writers are important professionals who know how to write accurate alt text for images and give accurate information on how to build our websites and use the correct text to both inform our users. And while AI may guess at the structure of your website to be best for your users, it really comes close or replaces the need for our designers and content writers to build out those experiences.

In addition, engineers, us, we still need to know how to properly construct our websites. It's still important for engineers to learn how to write compliant and accurate HTML code so it works with screeners and other assistive tools. And while AI might be able to give a good suggestion about how to write a button that will be accessible for all applications, it's rarely helpful when you're trying to create custom widgets like a graphing calculator. That's something my team has been working on.

And even if you hired all the correct professionals to be participants and to work and build this product, there are still challenges with AI. There's a lot of implemented and bias within the system that AI is built. Remember what we were talking about earlier. The AI that we're seeing most out in the industry right now is narrow AI, as in, it's trained on a very set of narrow tasks and it can reproduce things that are very wide and very interesting, like large language models, but it's still building off of data that we're feeding the AI engine. And as a result, it tends to amplify a lot of biases that exist within our society, such as race, gender, and that includes disabilities.

In addition to that, if an AI is able to understand who their user is based on their race, gender, or disability, AI tends to treat that user differently and not always for the better. There's also huge errors within AI. While AI, we're trying to build it to be much more efficient, more accurate, right now in the industry, we're still trying to move in this direction of getting AI to understand how confident it is in its output.

The example earlier about Be My Eyes, I don't know how the application works, but there's a very different level of confidence you need the AI engine to have between writing alt text for an image on your website versus describing the street and whether you can cross it. Both of those options have very high different levels of risk, and if you're going to be building an application that's important for a user and has safety concerns, you need to ensure that AI is able to give accurate results.

And on top of all that, a lot of users can be overly trusting of AI results, even when the information is inaccurate or very harmful to users. So if AI isn't this perfect blanket answer, if we can't just hand over AI the keys to make everything accessible, what is the thing that we need to do here? Well, here are some things you can do as an engineer. First, you can clean up your HTML. Most of the issues that are on the web right now are a result of improper HTML using the wrong tags, wrong attributes, and if you're able to go through and clean up and write well-structured code, you're going to fix a lot of the problems that hurt our users.

Also, get an accessibility browser tool. I use Axe Core Browser Extension. I also use Accessibility Insights for the Web, which is a Microsoft product. I've even tried Lighthouse a little bit. Using these tools will allow you to scan websites that you're navigating and see what are the issues, especially when you're developing. And please, if there's one big takeaway, please add automated testing into your development pipeline. It's a really easy thing to catch accessibility bugs before you ship them out to production and cause a lot of those issues that you see all over your website.

There's so many applications you can plug into your pipeline. I included a few here like Axe Core, Lighthouse CLI, Selenium from Deque, but I've also heard even at this conference, people using Plywright and the accessibility extensions they have there. I encourage you to check it out. Now say you're not an engineer and you're here at this conference. Welcome. If you're an organization, I highly encourage you to understand your compliance level and identify what you need to understand what your compliance level is.

And then after that, dig into all the issues that your customers are seeing and experiencing on your website. And if you have some money within your budget to hire accessibility experts, please do hire accessibility experts. Hire them in engineering, project management, design, user research, and testing. These are all areas that if you have an accessibility expert on staff, they can give you the best advice on how to make really clean and compliant code and a really great experience for your customers. In addition, listen to your users. If you have the ability to gather feedback from your customers, do that. Or if best case scenario, run user studies for them to navigate through your site using whatever tools they need and get honest feedback on how accessible your site is, what's interesting, what's clear, and what's confusing and not working.

And overall, my main point in this presentation is just to use AI strategically.

I'm not saying you shouldn't use AI. As you can see, there's a lot of really cool advancements and ways that AI is very helpful. But overall, AI really is just a tool. So use it. Use it to help generate alt text for low level images. Have your designers and content writers use it to make them more efficient at their job. But just be aware of its limitations and what it's intended to do and what it can't do. And thoroughly vet those results.

Overall, AI really isn't the magic answer. But the real answer is that AI is a tool, and we just need to use... Smartly use it within our applications without letting it run amok.

Here are my resources for this talk. I encourage you to read these articles. They are very insightful. I couldn't include all of their information. And if you want to reach out to me, here's the QR code for my slides. My name is Kat Johnson, and thank you so much for listening to my talk.

So this one is a really good question. Do you feel that AI generated code has a risk of degrading accessibility and web size over time because people are copying and pasting code with little review? That is a large concern of mine. It's a big reason why I created this presentation. Because AI generated code, sometimes it's very helpful. And sometimes it can perpetuate misunderstandings on how to structure code. I'm not too concerned with, say, help generate a button. Because usually it's pretty much there. But my concern is the proliferation of copy pasting code that isn't vetted and isn't the correct way. And it's just what that AI engine has been trained on. And based on all the data I've gathered is how to write a button. But did they really gather that button empirically on the proper syntax? Or is it just what most developers use? And that's where the fear is.

So you're saying there's a risk. And people can use that as a starting point. But use it with caution. Right. Next question. Will web agents like Claude work better with already accessible websites? Ooh, wait. What? This is the one that I highlighted. Claude. I'm not familiar with Claude. Maybe that's something I might need to check in on that.

Okay. On to the next one from Ben. What are your favorite resources for engineers looking to go down an accessibility rabbit hole? Oh, my gosh. I collect so many links. So if you ever want to reach out to me and be like, I want your resources, I will give you a data dump of all the websites I track. But I really love using WCAG accessibility guidelines. Some of them are kind of wordy. Some of them are pretty good. But they have a documentation on general patterns around the web.

And while their documentation isn't perfect, they give really good examples on how to make clickable content, how to make different tab navigation. A lot of suggestions on how to write semantically accurate widgets. I am a big fan of going on WCAG website. Sometimes just reading the definition, I don't know what it should sound like in screen reader. So I go on WCAG and have their example and use the screen reader on them and compare it to what it sounds like. I love using those. Yes, highly recommend. Okay. So we are big fans of WCAG. Okay.

So do you feel how do you recommend that someone can become an advocate for AI and make sure that stakeholders don't try to replace an expert with AI? That's a hard question. We are trying to ask you hard questions. Yes. Let me think. Can you ask that question again? Yeah. So how do you recommend, so I'm making you do like a neck workout, how do you recommend how somebody can become an advocate for AI and make sure that stakeholders don't try to replace an expert with AI? I think, you know, one thing that we do inside of Khan Academy is that we have a ethics and AI sort of sub-team group. It's not like a formal team that's hired by the company. It's more of a ragtag team of people who are interested in AI and also advocating for the ethics within our organization. So I think creating those smaller groups within your company to band together and advocate for certain things is sort of the best way that I know how right now. But if you're also building in the space of AI and working on these applications, advocating and pushing for advice from experts as you're building stuff and acknowledging that, hey, I don't know how to build this the right way that's going to be accessible. Can we consult with someone who is an expert or can we get some funding to hire consultants to advise on this area? I think that's a really good point.

Is the group of accessibility experts in your company, do they do that full-time or is it kind of their side project? We have a mix. We have a few people on staff that have titles as, like, senior software engineer of accessibility or front end architect with accessibility. But in combination with those experts that we've hired on staff full-time, we also coordinate with Level Access, a consulting firm that evaluates our site and gives us feedback on how to make our site more accessible. So if you don't have bandwidth to, like, hire people full-time, even just working with some consulting companies can be really helpful. Yeah. Totally. I think one thing that from my experience working in smaller companies is that when you don't have the resources to hire someone full-time, you end up having a group of people who volunteer their, like, side project time for it. And so a lot of context switching, and how do you go about doing that? It seems like that group of people ends up just taking, like, time to learn accessibility and a lot of times they end up doing, like, more work compared to other people. Right.

That's always hard. Yeah. I'm always one to advocate for your time as a developer. Like, accessibility is super important, and I know I could easily spend all of my time working on accessibility problems, but also the larger issues within the website are not going to be solved by one person within the company. So advocating for a change while also protecting your own time is paramount. It's not an easy choice. It's hard for me to say no. Yeah. So protect accessibility and protect yourself. Yes. Gotcha.

Okay. Next question. This is specific to your company. How do you collect data on the impact of accessibility improvements at Khan Academy? Ooh, great question. We have a... I think we just hired them, or we've had them around for a while, but we're trying to get in the flow of testing out our accessibility on our customers, and so we're currently doing a lot of work to make our math and science widgets more accessible. That's actually where I was hired into Khan Academy, was part of that effort. And so we've made some big lifts and big improvements, and we're going to be running some user studies with those customers and gathering feedback on whether the graph is more intuitive or what things would be best to make it a really crisp experience. But I'm not a user tester, so I don't know how they find customers to talk to. That's not my area of expertise. I'm usually the person that comes and says, here's the thing that we created. Go test. Yeah. So you cover the engineering portion, but there are also people that cover more human testing. Right. Exactly.

So you cover the engineering portion, but there are also people that cover more human testing. Right. Exactly.

That's good. Okay. Okay. Do you think that find two LLMs for accessibility could be available soon? What was it? Oh, LLMs. Yeah. The top one. Yeah. Potentially. It's definitely an area that I need to play around with. I don't usually use the copilot code generation functionality yet, mostly because sometimes I don't find it helpful. I feel like it just makes the code worse, personally. But I'm sure they'll, as time goes on, they'll get more and more accurate. And they can be helpful in small instances. Again, like, oh, how can I make this button widget more accessible? It can probably rewrite it to include some alt attributes or ARIA labeling. That will be really helpful. Yeah. So it will only get better with time. It's just awareness that it's not perfect yet. So don't expect the results to be the way to do stuff. So it will be helpful, but not, period.

Okay. Yeah. We have time for just one more question. This one's from Ben. Thank you, Ben, for submitting the question. Are there accessibility concerns specific to working on a large educational platform? Oh, yes. There are.

There's a lot of accessibility concerns. One thing that really drew me to Khan Academy was its mission around creating global education for everyone. And I feel like we need to add in accessible education for everyone. So Khan Academy has been around for a long time, and a lot of people have found a lot of great use of it. And I'm part of a big effort within the company to help get the accessibility a lot better. It's a little wonky right now, but it's okay, because we're working on it. But we're doing a massive migration to make things a lot more accessible and inclusive for education and for people all over the world. And we're doing some really exciting work with schools.

So yeah, accessibility within the education and ed tech is so paramount. You don't always get all the resources to make it accessible, but when you do, trying to be strategic with your time to fix the biggest issues affecting customers. And then after that, incremental gains to make your experience better and better for customers. So try to cover the biggest ground and then find two after that. Exactly.

Nice. Well, I want to thank you for your time today. It was very lovely talking to you and love the outfit of Lemon. So if you have more questions for Cat, make sure to find her outside or at the Q&A room. I'm sure there were so many more questions we didn't get to. So everyone give a round of applause for Cat. Thank you. Thank you.