The power of Accessibility Testing: Unlocking deep insights and boosting test coverage

The human world is talking about empowerment and the technology world is talking about accessibility. 
Organizations are showcasing their constant commitment towards developing a technology world that empowers everyone, including people with disabilities. Understanding user requirements and implementing them to make user friendly products is not just helping these organizations to expand globally with a larger customer base but also driving further innovation to satisfy the end users. 

The technology trend is shifting too from a concentrated market to a fragmented market which is highly competitive in nature. This is leading to constant innovation in the field of digital world where Web and Mobile technology is evolving each passing day, leading to a huge demand for Accessibility Testing. At the same time, the usage of mobile applications and web-based services has increased multifold over last few years and more people are consuming content online.

As per WHO, approx. 1.3 billion people (that is around 16% of world’s population) live with some form of disability and this number is expected to touch 2 billion by 2050. This means, we have this huge set of potential users who are unable to use digital content due to accessibility barriers.

A recent study done by Straits says the global market for accessibility tests will cross an estimated $606.46 million by 2027 and is growing with a CAGR of 4.1%.

Let’s look at some of the most prominent success stories and new design inventions which were made possible due to accessibility testing.

• Apple’s ITunes U educational program: California State University was not able to use Apple’s iTunes U educational program because the application was not fully accessible. This was later made accessible by Apple for both Mac and Windows platforms.
• Bank of America has become a pioneer in providing accessible banking solutions like talking ATMs and ADA-compliant banking services through its iPhone mobile app, online banking chat, etc. This helped them reduce their customer complaints related to accessibility and increased online customer engagement.
• Accessibility led to the invention of VoiceOver feature on iPhone, the world’s first gesture-based screen reader.
• Legal & General group (L&G) increased website traffic by 50% and doubled their conversion rate within the first three months, once they fixed accessibility issues. In terms of ROI, the investment they made in their site’s accessibility resulted in 100% return on investment within one year.
• Another major design product innovation considering accessibility issues was “PriestmanGoode’s Scooter for Life”.
• Search traffic increased to 6.86% for weekly broadcasts at National Public Radio stations in the US as they opted to provide transcripts for the audio content.

It is also estimated that organizations which are not taking the Accessibility path are losing $6.9 billion a year to competitors whose sites are accessible.

This is why it has become imperative to make digital platforms accessible to all, by create universal and inclusively-designed products. This creates the need for integrating “Accessibility” in our product development cycle by using standard guidelines, followed by a verification & validation of software content as a part of the normal QA process.

So, what is accessibility testing?

“Accessibility Testing is a type of software testing approach where QA engineers follow certain guidelines to verify and validate digital web content to make it accessible for all kind of users, including disabled users. It is a subset of Usability Testing.”

Through this blog, we intend to highlight 4 key points that must be considered when incorporating accessibility testing during the QA process. We will then also evaluate ways to maximize accessibility test coverage through this journey. 
                                              

1.    Considering “Shift Left Approach” for planning and executing accessibility tests

Preliminary testing with timely detection of accessibility issues can be the mantra for achieving accessibility goals. Project Managers must talk about them right from the planning phase and ensure that the team follows the same too.

• Get resources trained on Accessibility topics so that they are well-versed with accessibility guidelines.
• Ensure regular interactions and discussions with UX, Design and Content teams to incorporate accessibility in their designs.
• Choose free and open-source automation tools which can be easily integrated into the development workflows. This makes the life of a unit tester much easier as it can help the developers write stubs for testing at the design level itself.
• Organize periodic review sessions by the subject matter experts for early detection of issues.

This not only ensures product quality but also helps organizations save time and cost.

2.  Make products fully compliant with Accessibility Standards and Guidelines

Web Content Accessibility Guidelines (WCAG) are the technical standards defined and maintained by W3C for making web content accessible. These standards outline the principles, guidelines, testable success criteria, and techniques needed to make content accessible to a wider audience, including disabled people.

These are ACCEPTED WORLDWIDE and keep evolving continuously with the latest tech trends.

WCAG compliance is the best way to achieve compliance with most worldwide legislations like Section 508, ADA (The Americans with Disability Act), ACA (The Accessible Canada Act), EAA (European Accessibility Act), etc.

Compliance to these guidelines can also help avoid potential costly lawsuits and save brand image.

Do checkout the famous “Domino's Pizza” case!

They lost a legal case in US after one of their blind customers filed a complaint against their website and app for not being blind people friendly.

• The latest upgraded version (WCAG 2.1) has 17 new Success Criteria (With acceptance Level A, AA and AAA) designed majorly with a focus on improving smartphone accessibility among disabled users.
  
  
• The focus is gradually shifting towards not-so-common and unseen disabilities (such as Cognitive, Learning, Reading, Language, Motor, and Intellectual) apart from just the commonly known visual and hearing disabilities among users.
  While creating Success Criteria, designers are factoring in understanding specific user requirements from people with uncommon disabilities. Also, they are considering users with temporary and situational disabilities, along with the elderly and children as well.
  There is an overlap when we talk about WCAG 2.1 Success Criteria catering to the type of disability. Majorly, these are designed in such a way so that they can benefit a mixed type of disabled users.  For example,
  • Providing text alternatives to non-text content (audio and video) proves to be beneficial for users with visual and hearing impairments.  
  • Giving sufficient time allocation while filling forms or any place where user action is required comes under “No Timing” success criteria which can be beneficial for users with cognitive, learning, motor, and low vision disabilities.
  
  Below figure depicts the type of disabilities covered in WCAG2.1 Success Criteria & how much weightage they have.
  
  As observed, Visual disability problems are taken care of by maximum number of success criteria within WCAG2.1 guidelines. Nevertheless, new criteria are also being introduced on a gradual basis to cater to a larger group of people with all types of disabilities, like cognitive, learning, motor, language, hearing, etc.   

• From an execution perspective, the following WCAG Success Criteria can be automated using automation tools (Plug-ins and browser extensions) by integrating them with our development workflows (CI/CD pipelines and DevOps environments). They can also be integrated with tools and platforms like GitHub and JIRA for easy reporting. 
  
  WCAG Success Criteria:
  • Non-Text Content
  • Language of Page
  • Parsing
  • Contrast (Minimum)
  • Bypass Blocks
  • Name, Role, Value
  • Info and Relationships
  • Page Titled
  • Headings and Labels
  • Resize Text
  
  For all other criteria, human intervention is required in some way or the other and can be done with the support of manual testing tools only!
  
  
• As per a recently conducted Web AIM community 2022 accessibility survey, “low contrast” has emerged as the most common reason (86.3%) for WCAG2 failures. Often, the accessibility conformance issues fall under these categories as listed in the diagram below:

Accessibility across the web can be improved just by addressing these above-listed failures.

3.    Choose the right strategy for maximum test coverage

Till date, no artificial intelligence has been fully able to replicate human minds and none of the existing automation tools can simulate real user experience for testing.

Automation accessibility scans are prone to generate false positives as they fail to understand the design context and quality.

20-30% of Success Criteria are still not being covered by any of manual or automation tools, assistive technologies etc and they can be checked manually only.

Considering the above factors, we suggest choosing a Hybrid approach!

Human intervention is necessary to validate the descriptive content of a given web page and to validate the outcomes generated from accessibility automation tools. For WCAG compliance, combining automated accessibility tools with manual tools is the best approach.

Moreover, testing on real web and mobile devices with real users having disability proves to be quite beneficial in the long run. This also promotes empowering users with disabilities as part of their workforce within the organizations.

There are multiple ways to achieve web content accessibility through manual and automation testing.

While NO tool can guarantee full WCAG coverage, we can still best utilize the capabilities of freely available open-source tools (mix of manual and automation toolset) and come up with a testing strategy which can provide 70-80% of the coverage for WCAG guidelines and success criteria.  

Manual Testing

An extensive manual test check-up of website pages, web elements, mobile apps can be done against the web accessibility checklist. You can make use of real device cloud platforms available in the market for manual testing.

Apart from this, the following tools can also be used effectively in manual testing phase:

• Make use of online HTML/CSS Parsers, Bookmarklets, PDF accessibility checkers: Poor code can generate accessibility issues. Hence, it is always a good practice to get your code reviewed using these tools.
• Colour Contrast Analysers:  These tools can help in identifying colour and contrast related issues on the web and mobile product.
• Assistive Technology Integration: Using assistive technology tools like Screen Readers (NVDA, VoiceOver, Talkback), Speech recognition software, special adaptive keyboards can help identify accessibility issues to some extent.
• Keyboard-Only Accessibility: It is important to ensure that each element of the page is accessible through the keyboard. This will help make products accessible to users with mobility limitations. Check for manual inspection using keyboard shortcuts and keys like “Tab”, “Shift+Tab”, “Enter”, “Spacebar”, and “Arrow keys”.

Automatic Testing

Making use of free open-source Plug-ins and browser extensions tools like WAVE accessibility evaluation tool by WebAIM, aXe DevTools by Deque chrome extension, Accessibility Insight for Web, Google LightHouse etc. can definitely expedite the testing process and help automate accessibility testing.

Companies can also opt for paid tools available in the market, considering cost factors in mind. With more advanced AI solution integration happening with automation, tools are now promising advanced features, but this comes with an extra added cost for sure!

4.  Minimize QA effort by incorporating best practices during the Design phase

Last but not the least, to accelerate the testing process and to minimize accessibility issues at the testing phase, it is highly recommended that product designers or developers incorporate these best practices during the design phase itself. This will help them to create all-inclusive and disabled friendly products in the market, which are less error prone.

Conclusion

In this digital era, Accessibility Testing has a pivotal role to play for all business entities. Companies around the world can expand their horizon to a larger userbase by catering to disability-related issues. It’s important for businesses to implement best design practices to minimize QA effort, capitalize on right set of automation and manual tools for accessibility testing, and figure out a cost-effective way for its execution. They can leverage accessibility testing benefits and work towards developing an all-inclusive product design which is not only important for individuals but also for businesses and society at large. The emergence of accessibility-related laws being implemented regularly all across the world is another testimony to the growing importance of accessibility.

In today’s times, if your software does not meet accessibility guidelines, that would be the real disability.

References:

1. Disability (who.int)
2. Domino's Pizza app must be accessible to blind people - BBC News
3. WebAIM: The WebAIM Million - The 2022 report on the accessibility of the top 1,000,000 home pages
4. Accessibility Testing Market Size, Share and Forecast to 2031 | Straits Research
5. The Business Case for Digital Accessibility | Web Accessibility Initiative (WAI) | W3C
6. Accessibility in Design: Best Practices, Case Studies, and Useful Tips (springboard.com)
7. What Amazon Teaches Us About Accessibility - Retail TouchPoints
8. 33 Accessibility Statistics You Need to Know in 2021 (monsido.com)