Website accessibility testing: automated vs. manual
· 5 min read
Automated checks find technical errors such as missing alt text, insufficient contrast or unlabelled form fields quickly and reliably. Whether a website can actually be used with a keyboard and a screen reader, however, only people can judge. The best strategy is a combination: automated, regular and across the whole site; manual, targeted at the most important flows.
How automated checks work
An automated test loads a page in a browser and applies rules to the code. A widely used open testing library is axe-core; version 4.11 ships with more than 100 rules, about two thirds of them related to WCAG criteria at levels A and AA. Typical rules are:
- Does every image have an
altattribute? - Does every form field have an associated label?
- Does the text have enough contrast with its background?
- Does every button have an accessible name?
- Is the page language specified?
- Are the ARIA attributes valid?
Rules like these can be checked unambiguously: the attribute is there or it is not, the contrast is above 4.5:1 or below it.
What automated checks do well
- Breadth. Hundreds of pages in a short time, without fatigue.
- Repeatability. The same check every week, with comparable results.
- Early detection. New errors after an update show up immediately.
- Cost. Much cheaper per page than any manual check.
- Evidence. A history shows that you work on accessibility continuously.
Where automated checks reach their limits
Automated tests cover only part of the possible barriers; studies differ on how large that part is (GOV.UK test, Deque study). Typical gaps:
| Question | Why automation cannot answer it |
|---|---|
| Does the alt text describe the image meaningfully? | “IMG_4711.jpg” is formally an alt text – but useless. |
| Is the tab order logical? | Automation sees the code, not the path through the page. |
| Is the focus always visible throughout the flow? | Depends on states, animations, overlays. |
| Can the checkout be completed entirely by keyboard? | Requires real interaction over several steps. |
| Are error messages understandable? | A question of language, not syntax. |
| Does the screen reader announce “Added to cart”? | Requires a test with assistive technology. |
| Does the cookie banner cover the focused button? | Depends on layout and screen size. |
On top of that, there are false alarms in edge cases, such as text over images or gradients, where contrast is hard to calculate.
So the rule is: an automated check is not a full audit. Anyone who promises otherwise promises too much.
What a manual check looks like
A manual check follows the WCAG success criteria and tests with real assistive technology:
- Keyboard: operate the page using only Tab, Shift+Tab, Enter, Space, the arrow keys and Escape.
- Focus: is it always clear where you are? Is the focus not hidden by sticky headers or banners?
- Zoom and reflow: 200% magnification, narrow screen width, no horizontal scrolling.
- Screen reader: go through the main flow with NVDA on Windows or VoiceOver on Mac and iPhone.
- Content: assess alt texts, link texts, headings and error messages for their content.
- Multimedia: captions, audio description, controls for videos and sliders.
A suitable method for a structured check is the W3C evaluation methodology WCAG-EM. A full audit maps all findings to the success criteria and, depending on scope, takes days, not minutes.
An example from everyday shop life
Take the “Voucher code” field in the checkout. The automated check reports: the input field has no associated label, only placeholder text. That is a clear, technically detectable error – quickly found, quickly fixed. But whether the “Invalid code” error message after submitting is also announced by a screen reader, and whether the focus then lands in a sensible place, only a manual test shows. The two checks complement each other: automation finds the error on every page, a person checks the flow.
The comparison at a glance
| Automated | Manual | |
|---|---|---|
| Speed | minutes for many pages | hours to days |
| Coverage of criteria | part of the criteria | all criteria |
| Repetition | as often as you like, e.g. weekly | occasional |
| Assesses quality of content | no | yes |
| Tests real flows | limited | yes |
| Cost per page | low | higher |
The sensible combination for online shops
- Automated and regular. Check all important page types weekly. That keeps the technically detectable errors under control, and new errors show up immediately.
- Manual for the core flows. Search, product selection, cart, checkout, registration – after every major update and at least once a year.
- Manual for new components. A new filter, a new checkout plugin or a new cookie banner deserves a quick keyboard and screen reader test before going live.
- Include feedback. Customers’ reports about barriers are worth more than any rule.
A list of the criteria, noting what can and cannot be detected automatically, is in our WCAG 2.2 checklist.
Why overlays are not a third option
Sometimes a widget is offered as an “automatic solution”. But an overlay neither checks nor fixes anything in the code; it puts a layer on top. In 2025, the US Federal Trade Commission (FTC) obtained a payment of one million US dollars from a well-known provider and a ban on certain advertising claims. More in our article Overlays are not a solution.
What this means for your obligations
For shops covered by Germany’s BFSG: the requirements concern actual accessibility, not a particular testing method. A regular automated check is a good, verifiable building block. For the accessibility statement under Section 14 BFSG you also need a description of your service – no scanner can provide that. The framework is explained in the BFSG guide.
How we check
We check automatically against WCAG 2.2 AA with axe-core in a real browser, explain every result with a rule card and show the trend from week to week. Every report states: automated check, not a full audit, not legal advice. How it works in detail is described in How it works. Try it with one page of your shop.