What is accessibility testing? Key concepts explained

Software companies invest a lot of time and effort into making their software usable for their intended audiences. But a great many development teams miss a substantial audience that they don’t invest in.

These are users who experience challenges interacting with computers through traditional means.

They might experience vision problems or have trouble with their hearing, and they might not be able to use a mouse or keyboard in the traditional sense.

The US government publishes Section 508 standards that outline requirements websites need to meet for the “statutory obligations to ensure equal access for individuals with disabilities, including federal employees and members of the public.”

Ensuring that websites meet the need for Section 508 is called accessibility engineering. Accessibility testing refers to the tests that teams use to validate compliance with accessibility requirements.

What is accessibility testing?

Accessibility is defined as ensuring that users with disabilities are able to access software services, but accessibility engineering is often underfunded for many teams.

For teams who do invest resources into accessibility, continued compliance with standards requires just as much ongoing investment as ensuring stability in every other behavior of your application.

Just as teams test for software bug regressions, teams need to test for regressions in their accessibility work.

This testing often includes steps like validating that your website operates correctly with screen readers, that your text and backgrounds are sufficiently high contrast, and that users can navigate your application without a mouse.

The process of accessibility testing is actively testing your software to ensure that it meets the standards set by accessibility laws and guidelines. Property accessibility testing is performed on a regular basis, ensuring that your site stays usable for all of your customers.

Accessibility guidelines and standards

However, website accessibility standards are a bit of a patchwork mess. The requirements for how your applications need to be accessible to users depend on where those users are when they access your app, and how your app works.

In the United States, accessibility is governed by Section 508, as well as the Americans with Disabilities Act Title III. When your user is in Canada, the relevant legislation is the AODA and the ACA. In the EU, the relevant law is the EAA.

The requirements for how your applications need to be accessible to users depend on where those users are when they access your app, and how your app works.

What are the types of accessibility testing?

There are three kinds of traditional web accessibility testing. There’s also a fourth that’s starting to build momentum, too:

1. Manual accessibility testing

Manual accessibility testing simply involves checking your product against the web accessibility checklist. In this method, the evaluated content gets accessed through assistive technologies, such as a screen reader.

To evaluate the website, you’ll consider factors like the content, size, code, markup languages, development tools, and environment. Thus, testing web accessibility manually can be time-consuming.

2. Automated accessibility testing

Software that assesses code for WCAG conformance makes web accessibility testing significantly faster.

However, there’s no fully automatic technique for determining if a website is accessible; developers and designers might still need to adjust according to the website accessibility scan.

3. Hybrid accessibility testing

Hybrid accessibility testing is always the ideal method for ensuring a website is accessible. Primarily, you can only use some features, such as page-scrolling and navigation, with the keyboard, and cannot completely test them using the software.

4. LLM-based accessibility testing

In recent years, research teams have started experimenting with using LLMs to help automate the process of identifying and fixing accessibility issues.

This is a promising new frontier in accessibility testing. While the experiments don’t cover the entirety of any accessibility guideline, they are a useful addition to your hybrid testing approach.

The best way to approach accessibility testing is a multi-pronged approach that allows each type of testing to do the things it does best.

How do the types of accessibility testing work best?

For starters, the different types of accessibility testing work best when you let each of them play to their strengths.

Manual accessibility testing is great at discovering flaws in your user experience.

Automated accessibility testing is great at detecting obvious misses, like no alternate text on an image.

Conversely, if you flip the responsibilities, automated testing is bad at discovering when your user experience doesn’t support accessibility tools. Manual testing is much too slow and tedious to discover simple misses like lacking alternate text.

Lena Persson, writing on behalf of the accessibility platform she co-founded, lays it out nicely:

“Automated scanners are useful. They can quickly detect common technical issues like:

• missing alternative text on images
• low color contrast
• form inputs without labels
• missing ARIA attributes

But these tools only catch a portion of accessibility problems. Many issues depend on context and human judgment. For example:

• Does the alt text describe the image in a meaningful way?
• Does the heading structure make sense when navigating with a screen reader?
• Are error messages clear and helpful?
• Can users complete tasks easily using only a keyboard?

These are things automated tools cannot reliably determine. Automated testing should be seen as a first step, not the final answer.”

Building a multi-pronged accessibility testing strategy

The best way to approach accessibility testing is a multi-pronged approach that allows each type of testing to do the things it does best.

So, use manual testing to collect feedback from users who leverage accessibility tools. In the best cases, you can do this research by working directly with your actual customers who are using accessibility tools.

Then, after you identify and fix outstanding issues, write automated tests or leverage tools like linters to ensure that you don’t reproduce these issues. Run those automated tests on every build, and fail if accessibility goes backward.

Finally, if it’s in your team’s wheelhouse, pick up an LLM-based accessibility tool to do broad-based detection of accessibility issues across many individual pages if your website is very large.

Web content accessibility guide

Arguably the best set of standards for web accessibility come from the W3C’s Web Content Accessibility Guide (WCAG). The WCAG aren’t legal requirements, which complicates the landscape even further, while the WCAG is daunting and comprehensive, it can be summarized into four main principles that guide how companies should think about their web content:

1. Perceivable: Using one or more of their senses, users need to be able to perceive and understand your website in some significant way.
2. Operable: Users need to be able to navigate through your website and UI elements, such as clicking a button with a mouse, a voice command, or other method.
3. Understandable: Your website content should be readable, understandable, and digestible to readers.
4. Robust: Your website’s content should be developed keeping in mind how it will work across different types of browsers, both now and in the future.

Each guideline in the WCAG entails one or more of these principles. Keeping these principles in mind when developing applications will assist you in meeting accessibility guidelines.

By regularly testing your applications for how well they meet those guidelines, you’ll make your path to legal compliance as short as possible.

Arguably the best set of standards for web accessibility come from the W3C’s Web Content Accessibility Guide (WCAG).

How accessibility testing works

Fundamentally, accessibility testing doesn’t work any differently from any other kind of software testing. To design an effective accessibility test, you must:

1. Establish the accessibility requirements for your software.
2. Define a clear test plan to evaluate your adherence to requirements.
3. Identify which testing requirements you can automatically test.
4. Write automated tests for tests that qualify.
5. Establish test routines for tests that you can only meet manually.
6. Regularly execute both manual and automated tests.
7. Document failed accessibility requirements as software defects.
8. Fix documented defects and design tests to ensure that you don’t suffer regressions.

Outside help

One approach that many companies take for accessibility testing is enlisting outside help. The simplest form of help you can get is doing an automated scan of your website with a tool like an accessibility checker.

These tools will provide a quick overview of how well a particular URL meets accessibility guidelines and call out specific rules that the site fails to meet.

If you’re just starting out with accessibility testing, these external tools are a great way to kick-start your initiative. Even better, you can use these results to start writing automated tests.

If you have an extensive web presence, you might also benefit from enlisting more targeted outside help.

For many companies, this comes in the form of hiring accessibility consultants. These consultants have more experience working with accessibility challenges, which will likely help your team avoid common pitfalls with accessibility initiatives.

A site containing video and audio content should provide an option for alternative text, such as captions and subtitles.

How to test for accessibility

To test the accessibility of your product, you can check it against existing web accessibility standards.

You can do this by determining whether the site as a whole or, more commonly, a subset of pages complies with W3C international accessibility standards, as defined by Web Content Accessibility Guidelines (WCAG).

There are various ways to test accessibility depending on a person’s disability. To test the accessibility of your product, you can check it against existing web accessibility standards:

1. Vision disability

Many eye problems can affect a user’s ability to see things as they are.

A person with blurry or poor vision or another type of vision disability might require larger text to perceive content on a website, whereas a person with color blindness might be affected by your website’s color contrast, whether high or low.

Therefore, it’s essential to build websites so that people with any vision impairment can easily access them.

2. Physical disability

Assume a user who happens to be an amputee (lost a hand) is trying to fill out a form on your website.

Due to this physical challenge, this individual might be unable to use the mouse or keyboard to navigate the site. However, if the platform supports dictation, the user can use the speech recognition option to dictate text.

3. Hearing impairment

A site containing video and audio content should provide an option for alternative text, such as captions and subtitles.

To illustrate, imagine an individual who is deaf or hard of hearing tries to access a video-streaming site. If this site supports ALT text, users can efficiently utilize that site regardless of their disability.

With the text present, a person who is deaf can read and understand the content of the videos.

Challenges of accessibility testing

The biggest challenge in accessibility testing is doing it. Accessibility is a large problem, even for teams with modest software footprints. Many companies aren’t willing to invest the time and resources into accessibility.

The changes needed for accessibility improvements aren’t obvious to stakeholders.

People who use accessibility tools comprise a small portion of your user base, so investing in accessibility doesn’t seem to provide much return on investment. Many companies choose to risk the possibility of fines for legal non-compliance instead of investing up front.

The best practice for accessibility testing is simply to do it, a first step that many teams do not take. If your team is currently thinking about investing in accessibility, the first step is just to do it.

Best practices for accessibility testing

There are additional best practices you’ll want to adopt once you get started. Here are some suggestions:

1. Automate everything you can. Tests you don’t run don’t do any good.
2. Regularly reevaluate your tests to determine if they’re still covering everything.
3. Update your event monitoring systems to detect when a user utilizing assistance technology experiences a failure.
4. Update your framework configurations so that shipping accessible code is the path of least resistance for your developers.

Accessibility testing checklist

1. Include appropriate images

Images have a greater impact than words. Therefore, consider supplementing text with relevant images. This will help individuals with literacy challenges to perceive the website’s content described through pictures.

2. Add subtitles and captions to videos

If your website renders video content, you should offer options for subtitles and captions—in multiple languages—so that users with a hearing impairment can comprehend what is being said.

Furthermore, even those who view the video content in a public setting without headphones can read the subtitles and understand the content without disturbing others.

3. Utilize keyboard-friendly navigation

Design and develop your software and hardware to support keyboard navigation without depending on a mouse or touchpad.

4. Take advantage of inclusive color schemes

Your website should have colors that are sufficiently contrasted so visually impaired users can perceive your website, products, or services.

5. Resize text to be viewable

Text on your website should be large enough for all users to read. Also, consider designing your website so individuals with low vision can zoom in and out of the text.

6. Use operable interactive elements

Consistent navigation throughout the website pages will come in handy for users with cognitive disabilities. They’ll be able to utilize assistive devices in accessing your website. Therefore, opt for elements such as drop-down menus and clickable images.

7. Avoid marquee text

Keep the website or software content as simple as possible. Anyone who has epilepsy or other related diseases may experience seizures when exposed to flashing lights or other elements that are both flickering and bright.

Improvise with headlines and descriptions. Use header tags (H1, H2, H3) to separate material and make it easier for people to read and take in information. Screen readers use heading tags (<h1>, <h2>, etc.) to navigate and prioritize the content.

Often, the most instructive type of accessibility testing is real, hands-on usage flows with real users who rely on accessibility tools.

Examples of accessibility testing

Accessibility testing takes many forms. Often, the most instructive type of accessibility testing is real, hands-on usage flows with real users who rely on accessibility tools.

That style of testing is heavily manual and carries a substantial cost. But you’ll rapidly learn things that automated testing and traditional QA flows won’t tell you.

This kind of testing involves exactly what it sounds like: you identify some users (either through recruitment within your application, or by contracting with a company that brokers connections like these).

Then, you need to provide a set of objectives for those users to accomplish, like purchasing a particular product or learning about a particular offering from your company. Usually, you’ll do something like this on a video call, or with screen recording software.

From there, you want your team to take notes about what the users find easy or difficult to accomplish. Then, at the end of the process, if possible, you want to conduct a full user interview and ask about their impressions of your site usability.

Automated and emerging approaches to accessibility testing

At the opposite end of the spectrum, you have fully automated accessibility testing. This is by far the cheapest and quickest option.

It’ll discover mostly surface-level issues with your site, but if your site has these surface-level issues, it’s still quite valuable.

Performing one of these tests usually looks like engaging with one of the tools available at w3.org and then running the test against your site. The tool will report back any issues it discovers, and then it’s up to you to fix them.

Hybrid testing lives in between these two types of testing, and it’s self-explanatory. A hybrid test is not a specific test but rather a combination of both automated and manual testing, at varying levels of intensity.

Finally, there’s an emerging type of testing that requires explanation. LLM-based testing is structured like a manual test, but it operates like an automated test.

In an LLM-based test, you provide an LLM, like Codex or Claude Code, a website to interact with and a set of goals to accomplish.

Then you ask the LLM to perform the tasks using a limited set of tools or browser APIs to see how easily someone can interact with your system while missing particular capabilities.

LLM-based testing is still an emergent technique. If you’re interested in trying it out, be ready for heavy experimentation as you figure out what works and what doesn’t.

Importance of accessibility testing

There are a high variety of factors that determine just how much importance accessibility testing carries within your organization.

At the most foundational level, accessibility testing is important because it’s the right thing to do. You have users from all walks of life and many parts of the world. The last thing you want to do is exclude someone because you didn’t spend time investing in testing.

Beyond the moral argument, there are solid business arguments for accessibility testing, too. As we’ve noted elsewhere, there are legal requirements for accessibility testing in certain jurisdictions.

Not meeting those requirements and finding yourself on the business end of a lawsuit is a terrible place for a business to be. That kind of attention brings negative press headlines, costs substantial sums to defend, and can lead to penalties in the five digits per offence.

Once we accept the value of accessibility, whether that’s because of the positive argument or simply because you’re trying to avoid negative consequences, the value of accessibility testing clears right up.

If you’re like most businesses, you’re not building your software and then just letting it sit. You’re probably constantly adding features and updating the content of your site.

Every time you push an update, you need testing to ensure that your accessibility meets expectations. Every time you add an image or redesign a menu, you’re opening the door for accessibility to get worse.

Clearly, that’s not what you want when you push an update. The only way to prevent those changes from degrading your accessibility is to test your software for compliance to accessibility guidelines.

The testing process, in all four forms—automated, manual, hybrid, and LLM-based—is critical to ensuring that your site meets the highest standards for your customers.

Even if the monetary benefit of the work isn’t obvious, every improvement in accessibility is an improvement in your software.

Every accessibility improvement is worth it

Accessibility improvements require real investments of engineering time and effort. Sometimes, the gains from these investments might not be clear to people watching the bottom line. For those people, there are opportunities to be had.

Asad Khan, writing for the Forbes Technology Council, has this to say about the benefits of accessibility testing: “Apart from enhanced UX, accessibility testing can be a key to achieving brand credibility and market research. According to a Forrester report (via the Bureau of Internet Accessibility), accessibility and improvements yield $100 for every $1 spent.” 

Even if the monetary benefit of the work isn’t obvious, every improvement in accessibility is an improvement in your software. Improving one aspect of your accessibility story is still a worthwhile investment.

And once you make one of those improvements, that’s where you should invest in accessibility testing.

Those tests ensure that you don’t experience regressions that render your investment moot. Even more, those tests drive compliance with legal requirements, which will help your company avoid fines.