Ability Bazaar guide
Automated vs. manual accessibility testing: what does each find?
The short answer
Automated accessibility tools are good at repeatable checks that can be decided from code, such as some missing accessible names, contrast failures and invalid relationships. They can scan many pages consistently and help teams catch regressions.
Manual evaluation is needed when the answer depends on meaning, sequence or successful task completion. It includes keyboard review, content judgement and assistive-technology testing. There is no authoritative universal percentage of issues that automation detects; coverage varies by tool, page and how 'issue' is counted.
What can automated tools do well?
- Check large sets of pages quickly and repeat the same rules consistently.
- Identify some missing labels, names, language declarations and structural relationships.
- Calculate many text and interface-component contrast failures.
- Expose the affected page, element and rule so developers can reproduce the finding.
- Run in development and continuous integration to catch known regressions early.
What still requires human judgement?
A tool may confirm that an image has alt text but cannot reliably decide whether the text communicates the image's purpose in context. It may find headings yet miss that they create a confusing document outline, or find a form label while a user still cannot recover from an error.
People test focus order, keyboard traps, status messages, reading sequence, instructions, language clarity, complex widgets and complete workflows. Users with disabilities can identify practical barriers that conformance evaluation alone misses, although W3C cautions that one user cannot represent every disability or technique.
Why do percentage claims vary so much?
Published detection percentages often measure different things: WCAG criteria that are fully automatable, unique rule violations, pages with at least one finding, or all individual defects in a sample. Tools also implement different rules and return results that still need review.
A more useful vendor question is: which rules, pages and workflows are automated, which are reviewed manually, which assistive technologies are used, and who resolves false positives and ambiguous results?
What does a sensible combined process look like?
- Inventory page types and critical workflows before choosing a sample.
- Run automation broadly to identify repeatable patterns and regression risks.
- Manually review representative pages against all applicable success criteria.
- Complete every essential workflow with keyboard-only interaction.
- Test relevant combinations of browser and assistive technology.
- Involve disabled users, then distinguish usability findings from formal conformance results.
- Fix root causes, re-test and add automated regression checks where possible.
Frequently asked questions
- Can an automated tool prove that a website is accessible?
No. Automated tools can find useful patterns, but a clean result does not establish accessibility or conformance.
- Is there a reliable percentage of accessibility issues automation finds?
No universal percentage applies. Results vary by tool, content, sampling method and the definition of an issue.
- Why involve users with disabilities?
They can reveal practical barriers and usability issues that standards-based checks alone may miss; their evaluation should complement, not replace, conformance testing.
Sources
- W3C WAI, Web Accessibility Evaluation Tools List
- ADA.gov, Guidance on Web Accessibility and the ADA
- W3C WAI, Selecting Web Accessibility Evaluation Tools
- W3C WAI, Involving Users in Evaluating Web Accessibility
Sources were checked on August 21, 2026. Regulations, standards and product documentation can change; follow the linked source for the latest version.