Usability Testing Without a UX Specialist

Ask nearly anyone about the process of developing software, and somewhere in the answer they’ll (hopefully) mention the users. User research, user testing, user feedback – the end user is at the heart of everything that we build. However, for many companies, making conversations with real users actually happen is a real challenge – especially if you don't have a UX specialist on your team! If this is all sounding familiar to you, then I have a recommendation: take it into your own hands. In this session, we'll talk through setting up a basic user testing program and growing it, so you – the developer – can feel empowered to start usability testing for your own product!

This talk has been presented at React Advanced 2023, check out the latest edition of this React Conference.

Watch video on a separate page

FAQ

Katherine Grayson Nanz is a Developer Advocate at Progress Software with a background in both development and design. She has experience in various roles that require a broad skill set.

The main problem faced by the startup was a poor user experience due to a lack of design input. This impeded the growth of the customer base because users struggled with the complex and unintuitive user interface.

Usability testing is important because it helps identify user pain points, misunderstandings, and places where developer assumptions diverge from user behavior. It provides valuable insights into how users interact with an application, which can lead to better user experience and more effective software.

The two main types of usability tests are: 1. Observing a user move through an established flow unguided, and 2. Walking the user through a brand new flow with some guidance. Each method has its own benefits for understanding user behavior and gathering feedback.

Test subjects can be found through various methods such as reaching out to sales and support teams for established users, posting open calls on social media, adding banners or modals to your website, and offering incentives like free trials, discounted rates, or company swag. For new users, consider friends and family or offering small gift cards or meals as incentives.

Before running a usability test, you should identify what you want to test, choose specific tasks or flows, gather test subjects, and decide on the logistics such as whether the test will be in-person or remote. Prepare any necessary equipment for recording and outline a script to guide the test session.

To conduct a usability test, introduce yourself and explain the goal of the test. Reassure the user that there are no wrong answers and encourage them to think out loud while completing tasks. Ask open-ended questions to guide them if they get stuck and take notes or record the session for later analysis.

After conducting a usability test, review and analyze the data by grouping it by task or question, looking for patterns, and collecting hard data like time to complete tasks. Summarize the most impactful findings into a one-page document for quick review and share it with your team.

Challenges in usability testing include finding test subjects, organizing logistics, and bias in results. You can overcome these by leveraging internal resources, offering incentives, considering both in-person and remote testing options, and ensuring a diverse group of users.

Developers should be involved in usability testing because it provides them with a unique perspective on how users interact with their applications. It helps them resist making assumptions, check internal biases, and ultimately build better, more user-friendly software.

Kathryn Grayson Nanz
Kathryn Grayson Nanz
28 min
23 Oct, 2023

Comments

Sign in or register to post your comment.
Video Summary and Transcription
Usability testing is effective for uncovering user pain points and desire paths, as well as revealing loopholes, shortcuts, and hacks. Finding diverse users for testing can be challenging, but reaching out to sales and support teams and offering incentives can help. The logistics of usability testing include having multiple people to run tests, disclosing recording methods, and considering in-person or remote testing. During the tests, it's important to encourage participants to think out loud, ask open-ended questions, and gather feedback for improvement. Collecting and summarizing usability test results involves analyzing raw data, gathering hard data, and avoiding biases.

1. Introduction to Usability Testing

Short description:

I'm Katherine Grayson Nanz, a Developer Advocate at Progress Software. In a previous job, I was one of two designers on a team. The app was built by developers without much design input, resulting in a poor user experience. We weren't talking to our users and were making educated guesses about their needs. Despite limited resources, we combined experience with research to implement usability testing successfully. Making conversations with real users happen can be challenging due to organizational and resource constraints.

Hi there. I'm Katherine Grayson Nanz, a Developer Advocate at Progress Software. As a developer with a design background, I've often been in jobs and situations where I get to do a little bit of everything. To the point where just saying I wear a lot of hats can feel like a bit of an understatement. But honestly, I really enjoy that type of work.

In one such previous job, I was one of two designers on the team. We had one full time designer, and then myself, splitting my time between design and development. The company was a startup. And they had a great app idea that they had built, and proven, and gained a group of loyal customers with. However, the app was built entirely by developers without much design input at all. And the poor user experience was starting to become an impediment to growing the customer base.

As new tasks were getting assigned and discussed, we quickly discovered an issue. We weren't talking to our users at all. There was a hyper focus on adding new features and growing the functionality of the application. But no actual data suggesting that users wanted these features. Meanwhile, users that we had were struggling to use the existing features due to the complex and unintuitive user interface. In meetings, our discussions often included phrases like we think, assuming that, and hopefully. We were making educated guesses about our users, but we didn't actually know for sure.

We needed to close the loop, but we didn't have any UX specialists and we were working with a startup budget. I had a little experience from helping run usability tests at a past job, but those were larger established programs where I wasn't in a leadership position. And yet, as it turns out, expertise is relative. And relative to the rest of my team at the time, I was the one who knew the most about what a usability testing program looked like. And if this was something that we felt strongly about as a team, and it was, we decided that was just going to have to be enough. We would combine experience with research to figure it out as we went and find a way to make it work. And guess what, we did.

The idea of usability testing is one that most folks will support and agree with. However, for many teams and companies, making conversations with real users actually happen can be very challenging. And when this happens, we often think it must be due to a lack of understanding about the importance of user testing and that the problem we need to solve to start user testing is getting buy-in from other people. While this can occasionally be the case, I've often found that there already is a strong understanding of how valuable the feedback would be. The struggle, as it was in my previous company, was more with organization and resources.

2. Usability Testing Basics

Short description:

Usability testing is most effective when focused on testing a specific flow, task, or feature. Choose a task or chain of related tasks that will guide the user through the part of the application you want to test. The flow should have a clear starting and ending point. You want a task that a user can complete in one sitting, which generally means about 20 to 30 minutes. It can be challenging to find users, but asking for 30 minutes of their time is easier than asking for two hours. There are two main types of tests: observing the user unguided or walking the user through a semi-guided flow.

It's much more common for me to hear things like, yes, we usability testing, but we just don't have any budget for it. People want the chance to sit down with users, but they're not sure how to make it happen. And because it's not an absolute requirement in order to ship, it gets continually bumped in favor of more urgent tasks. This usually happens in situations where there hasn't really been a strong history of design process and where there are very few or no UX professionals on the team who can really take ownership of the task.

So, if this is all sounding familiar to you, then I have a recommendation. Do what we did and do it yourself. And yes, in an ideal world, this would be the responsibility of a UX designer or researcher. But for many teams like the one I was on and maybe like the one that you're on, for various reasons, that situation just can't be a reality right now. So, in this talk, I don't want to discuss the ideal big budget perfect world scenario. Because there are lots of resources that already exist for that. Instead, I want to talk about how we can still make usability testing possible for startups, small companies, or personal side projects that might not have lots of money or people for this work. Think of it more like DIY usability testing.

No matter what your situation is, I'm here to assure you, running basic usability tests is something you are completely capable of and it's going to come with the bonus of making you a better more empathetic developer as well. The first major hurdle that we'll need to address before we can consider anything else is just figuring out what we want to test. And no, the whole app is not an option. Usability testing is going to be most effective when it's focused on testing a specific flow, task, or feature. Take a moment to consider what questions you want answered about your app. Have you noticed more support tickets coming in? Is there maybe a place where you're not getting the interactions you anticipated? Have you been receiving negative feedback? Maybe you're getting ready to launch something new. All of these are great jumping off points. You want to choose a specific task or a chain of related tasks that will user through the part of the application that you want to test. The flow should have a clear starting and ending point. The ending points don't necessarily need to be exactly the same for every user, but you should still clearly know when a user has satisfied the task requirement. Some examples of flows that you could test are onboarding, searching and saving, exporting an asset, updating the user profile, et cetera. You really want a task that a user can complete in one sitting, which generally means about 20 to 30 minutes. You can potentially test multiple flows, but you'll still want to keep the entire testing session at no more than about an hour long. The longer the time commitment, the bigger the ask that you're asking your users to give you, right? That's a lot of time that you're asking of them. It can be challenging to find users to begin with, but it's a lot easier to ask someone for 30 minutes of their time than for two hours. There are two main types of tests that you can run. In one option, you'll observe as the user moves through an established flow, completely unguided. The second option is to walk the user through a brand new flow, kind of semi guided.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

Network Requests with Cypress
TestJS Summit 2021TestJS Summit 2021
33 min
Network Requests with Cypress
Top Content
Cecilia Martinez, a technical account manager at Cypress, discusses network requests in Cypress and demonstrates commands like cydot request and SCI.INTERCEPT. She also explains dynamic matching and aliasing, network stubbing, and the pros and cons of using real server responses versus stubbing. The talk covers logging request responses, testing front-end and backend API, handling list length and DOM traversal, lazy loading, and provides resources for beginners to learn Cypress.
Testing Pyramid Makes Little Sense, What We Can Use Instead
TestJS Summit 2021TestJS Summit 2021
38 min
Testing Pyramid Makes Little Sense, What We Can Use Instead
Top Content
Featured Video
Gleb Bahmutov
Roman Sandler
2 authors
The testing pyramid - the canonical shape of tests that defined what types of tests we need to write to make sure the app works - is ... obsolete. In this presentation, Roman Sandler and Gleb Bahmutov argue what the testing shape works better for today's web applications.
Full-Circle Testing With Cypress
TestJS Summit 2022TestJS Summit 2022
27 min
Full-Circle Testing With Cypress
Top Content
Cypress is a powerful tool for end-to-end testing and API testing. It provides instant feedback on test errors and allows tests to be run inside the browser. Cypress enables testing at both the application and network layers, making it easier to reach different edge cases. With features like AppActions and component testing, Cypress allows for comprehensive testing of individual components and the entire application. Join the workshops to learn more about full circle testing with Cypress.
Test Effective Development
TestJS Summit 2021TestJS Summit 2021
31 min
Test Effective Development
Top Content
This Talk introduces Test Effective Development, a new approach to testing that aims to make companies more cost-effective. The speaker shares their personal journey of improving code quality and reducing bugs through smarter testing strategies. They discuss the importance of finding a balance between testing confidence and efficiency and introduce the concepts of isolated and integrated testing. The speaker also suggests different testing strategies based on the size of the application and emphasizes the need to choose cost-effective testing approaches based on the specific project requirements.
Playwright Test Runner
TestJS Summit 2021TestJS Summit 2021
25 min
Playwright Test Runner
Top Content
The Playwright Test Runner is a cross-browser web testing framework that allows you to write tests using just a few lines of code. It supports features like parallel test execution, device emulation, and different reporters for customized output. Code-Gen is a new feature that generates code to interact with web pages. Playwright Tracing provides a powerful tool for debugging and analyzing test actions, with the ability to explore trace files using TraceViewer. Overall, Playwright Test offers installation, test authoring, debugging, and post-mortem debugging capabilities.
Everyone Can Easily Write Tests
TestJS Summit 2023TestJS Summit 2023
21 min
Everyone Can Easily Write Tests
Playwright is a reliable end-to-end testing tool for modern web apps that provides one API, full isolation, fast execution, and supports multiple languages. It offers features like auto-weighting, retrying assertions, seamless testing of iframes and shadow DOM, test isolation, parallelism, and scalability. Playwright provides tools like VS Code extension, UiMode, and Trace Viewer for writing, debugging, and running tests. Effective tests prioritize user-facing attributes, use playwright locators and assertions, and avoid testing third-party dependencies. Playwright simplifies testing by generating tests, providing code generation and UI mode, and allows for easy running and debugging of tests. It helps in fixing failed tests and analyzing DOM changes, fixing locator mismatches, and scaling tests. Playwright is open source, free, and continuously growing.

Workshops on related topic

Designing Effective Tests With React Testing Library
React Summit 2023React Summit 2023
151 min
Designing Effective Tests With React Testing Library
Top Content
Featured Workshop
Josh Justice
Josh Justice
React Testing Library is a great framework for React component tests because there are a lot of questions it answers for you, so you don’t need to worry about those questions. But that doesn’t mean testing is easy. There are still a lot of questions you have to figure out for yourself: How many component tests should you write vs end-to-end tests or lower-level unit tests? How can you test a certain line of code that is tricky to test? And what in the world are you supposed to do about that persistent act() warning?
In this three-hour workshop we’ll introduce React Testing Library along with a mental model for how to think about designing your component tests. This mental model will help you see how to test each bit of logic, whether or not to mock dependencies, and will help improve the design of your components. You’ll walk away with the tools, techniques, and principles you need to implement low-cost, high-value component tests.
Table of contents- The different kinds of React application tests, and where component tests fit in- A mental model for thinking about the inputs and outputs of the components you test- Options for selecting DOM elements to verify and interact with them- The value of mocks and why they shouldn’t be avoided- The challenges with asynchrony in RTL tests and how to handle them
Prerequisites- Familiarity with building applications with React- Basic experience writing automated tests with Jest or another unit testing framework- You do not need any experience with React Testing Library- Machine setup: Node LTS, Yarn
How to Start With Cypress
TestJS Summit 2022TestJS Summit 2022
146 min
How to Start With Cypress
Featured WorkshopFree
Filip Hric
Filip Hric
The web has evolved. Finally, testing has also. Cypress is a modern testing tool that answers the testing needs of modern web applications. It has been gaining a lot of traction in the last couple of years, gaining worldwide popularity. If you have been waiting to learn Cypress, wait no more! Filip Hric will guide you through the first steps on how to start using Cypress and set up a project on your own. The good news is, learning Cypress is incredibly easy. You'll write your first test in no time, and then you'll discover how to write a full end-to-end test for a modern web application. You'll learn the core concepts like retry-ability. Discover how to work and interact with your application and learn how to combine API and UI tests. Throughout this whole workshop, we will write code and do practical exercises. You will leave with a hands-on experience that you can translate to your own project.
Detox 101: How to write stable end-to-end tests for your React Native application
React Summit 2022React Summit 2022
117 min
Detox 101: How to write stable end-to-end tests for your React Native application
Top Content
WorkshopFree
Yevheniia Hlovatska
Yevheniia Hlovatska
Compared to unit testing, end-to-end testing aims to interact with your application just like a real user. And as we all know it can be pretty challenging. Especially when we talk about Mobile applications.
Tests rely on many conditions and are considered to be slow and flaky. On the other hand - end-to-end tests can give the greatest confidence that your app is working. And if done right - can become an amazing tool for boosting developer velocity.
Detox is a gray-box end-to-end testing framework for mobile apps. Developed by Wix to solve the problem of slowness and flakiness and used by React Native itself as its E2E testing tool.
Join me on this workshop to learn how to make your mobile end-to-end tests with Detox rock.
Prerequisites- iOS/Android: MacOS Catalina or newer- Android only: Linux- Install before the workshop
API Testing with Postman Workshop
TestJS Summit 2023TestJS Summit 2023
48 min
API Testing with Postman Workshop
Top Content
WorkshopFree
Pooja Mistry
Pooja Mistry
In the ever-evolving landscape of software development, ensuring the reliability and functionality of APIs has become paramount. "API Testing with Postman" is a comprehensive workshop designed to equip participants with the knowledge and skills needed to excel in API testing using Postman, a powerful tool widely adopted by professionals in the field. This workshop delves into the fundamentals of API testing, progresses to advanced testing techniques, and explores automation, performance testing, and multi-protocol support, providing attendees with a holistic understanding of API testing with Postman.
1. Welcome to Postman- Explaining the Postman User Interface (UI)2. Workspace and Collections Collaboration- Understanding Workspaces and their role in collaboration- Exploring the concept of Collections for organizing and executing API requests3. Introduction to API Testing- Covering the basics of API testing and its significance4. Variable Management- Managing environment, global, and collection variables- Utilizing scripting snippets for dynamic data5. Building Testing Workflows- Creating effective testing workflows for comprehensive testing- Utilizing the Collection Runner for test execution- Introduction to Postbot for automated testing6. Advanced Testing- Contract Testing for ensuring API contracts- Using Mock Servers for effective testing- Maximizing productivity with Collection/Workspace templates- Integration Testing and Regression Testing strategies7. Automation with Postman- Leveraging the Postman CLI for automation- Scheduled Runs for regular testing- Integrating Postman into CI/CD pipelines8. Performance Testing- Demonstrating performance testing capabilities (showing the desktop client)- Synchronizing tests with VS Code for streamlined development9. Exploring Advanced Features - Working with Multiple Protocols: GraphQL, gRPC, and more
Join us for this workshop to unlock the full potential of Postman for API testing, streamline your testing processes, and enhance the quality and reliability of your software. Whether you're a beginner or an experienced tester, this workshop will equip you with the skills needed to excel in API testing with Postman.
Monitoring 101 for React Developers
React Summit US 2023React Summit US 2023
107 min
Monitoring 101 for React Developers
Top Content
WorkshopFree
Lazar Nikolov
Sarah Guthals
2 authors
If finding errors in your frontend project is like searching for a needle in a code haystack, then Sentry error monitoring can be your metal detector. Learn the basics of error monitoring with Sentry. Whether you are running a React, Angular, Vue, or just “vanilla” JavaScript, see how Sentry can help you find the who, what, when and where behind errors in your frontend project. 
Workshop level: Intermediate
Testing Web Applications Using Cypress
TestJS Summit - January, 2021TestJS Summit - January, 2021
173 min
Testing Web Applications Using Cypress
Top Content
WorkshopFree
Gleb Bahmutov
Gleb Bahmutov
This workshop will teach you the basics of writing useful end-to-end tests using Cypress Test Runner.
We will cover writing tests, covering every application feature, structuring tests, intercepting network requests, and setting up the backend data.
Anyone who knows JavaScript programming language and has NPM installed would be able to follow along.