Quality Assistance unveiled: Challenges and opportunities in the new testing role

Before we assess how quality is maintained today and how best it can be done, let us first consider and reflect on the circumstances we live in:

Modern software development is fast and agile, prioritizing people and communication over processes and tools. The concept of a Minimum Viable Product (MVP) reigns supreme, with end-users wielding significant influence over product direction. Iterative software development, enabled by continuous integration of tools, has revolutionized product release cycles, which often occur within weeks or even days now. This trend towards iterative software development underscores the importance of adaptability and responsiveness in today’s market. Amidst all these developments and rapid evolution, testing remains a critical, and, often, a cumbersome part of the development process.

Traditionally, the Quality Assurance (QA) role has been plagued by various challenges, chief among them being the ping-pong of responsibilities between developers and testers. This process, characterized by iterative bug-fixing and testing cycles, frequently leads to delays in software releases. As development teams race against the clock to meet deadlines, the QA phase often becomes a bottleneck, impeding progress and compromising the overall quality of the product. Moreover, the question of how to address the discovered bugs further complicates matters, as developers grapple over whether to prioritize bug fixing over developing new features.

In response to these challenges, the concept of Quality Assistance (QA) has emerged as a transformative approach in software testing. Unlike traditional QA, which focuses primarily on identifying and reporting defects, QA emphasizes a collaborative and proactive approach to quality assurance. By empowering all team members to contribute to the testing process, QA fosters a culture of shared responsibility and accountability, where quality becomes everyone's concern.

The pillars of Quality Assistance

So, what all should constitute as the key pillars of Quality Assistance? Let’s find out.

Heavy reliance on Test Automation

Test Automation, spanning across multiple levels from unit tests to end-to-end functional tests, forms the backbone of Quality Assistance. By automating all those repetitive tasks and regression tests, teams can release products with confidence and minimal human testing effort. Automation testing has already proven its mettle, by covering regression tests, to ensure successful releases. Test Automation must be implemented across multiple levels.

Begin with testing from a lower level, where unit tests should be implemented first, with a decent code coverage. Another developer or architect can review the written tests and approve the tests initially.

The next level of testing is integration testing, where one or more components interact with each other. This is where we can use mock systems to simulate the behaviour of external systems and make tests more reliable.

We can also automate tests at the system level. For this level, the most important user functional flows are automated. These tests are mainly regression tests, which execute in every release to make sure that the existing features are unaffected even as we introduce any new feature(s).

Teams can use these multiple automated feedback layers to release their products with confidence and with lesser human testing effort.

Figure 1: Test Pyramid

The Assistance role is to help the developers create these multi-level feedback systems. How can this be done?  Quality Assistance can contribute in:

• Helping developers create the most used and important business tests.
• Reviewing the test scripts and suggesting adding any excluded scenarios.
• Participating in writing end-to-end automation tests.
• Analyzing the completeness of the mock system of the external system.
• Analyzing the test data created in automation.
• Evaluating whether non-functional testing components, such as load and performance testing, security testing, or accessibility testing, need to be automated.

Automation helps developers ship product releases early, with minimal manual regression testing. Developers with quality assistance can concentrate more on the features that are currently being developed instead of having to work on fixing any already released features.

Pair Testing

Figure 2: A diagram of a workflow

Pair testing is akin to peer review, with a key difference being that in the former, the focus is on testing instead of review. Testers don’t just test the user stories in isolation. They sit with the developer and test the relevant feature, based on the requirement or user story.

The tester prepares a list of all the scenarios and the developer demonstrates the feature. They execute these scenarios together. Based on the testing outcome, they may perform one or two more rounds of pair testing. Pair testing also ensures whether enough scenarios have been automated at multiple levels.

Advantages of Pair Testing

• Developers learn how to perform procedural testing (Positive scenarios come first, followed by negative testing and other type of testing).
• Ensures quick and more practical testing.
• Defect-fixing time reduces significantly and there is no ping-pong of responsibility.
• Evaluates if a feature has been sufficiently automated at multiple levels, right from unit tests to end-to-end functional tests.

Developers often have more insights on whether new features can affect the existing features. Pair Testing addresses these issues easily while also allowing for executing automation simultaneously.

Pair Testing saves precious developer time by fixing bugs and testing the feature. Teams can also take advantage of working together as they can decide whether a bug needs to be fixed or deferred to the next release.

Testing sessions

After completing Pair Testing, this is the next stage where Testing is organized to target the entire feature. This is not procedural testing as much as it is exploratory testing. The test sessions aim to find any undiscovered defects, including usability. It also provides more clarity and confidence about determining whether the feature is ready for release. All team members, including POs, participate in this test session to get a first glimpse of the new feature.

A Quality Assistant moderates this session and is responsible for all preparations. Based on every team member’s skill, the QA wisely distributes the feature test areas. The Quality Assistant can also suggest different devices or browser to increase the testing scope. Any defects are raised by the team and the developer fixes them accordingly.

It is important that the entire team is keen to contribute to these test sessions. This is where a Quality Assistant can encourage exploratory testing.

Testing sessions give a team the confidence that the feature has been built as intended and is ready to ship. It also gives a good idea to figure out the effort needed to ship it to production. They can also discuss bugs, including the effort required to fix them.

Release early and frequently

Thanks to modern continuous integration pipelines and cloud infrastructure, it is far easier to release products today, as compared to earlier.

Having frequent releases helps in fixing defects quickly and in detecting any problems soon enough. It also reduces production down time significantly. A release is planned at least once in week.

The team checks for any blockers or critical defects in the current release. The release on production system is only to be done when deployment has been successful on the beta environment for at least one week.

The role of the test environment is pivotal in ensuring the success of a product. Developers often create new features in a production-like environment, where there are minimal structural differences between the production and development environments.

The team collectively decides on the frequency of releasing updates to production. Releases may occur weekly, bi-weekly, or even multiple times a day, depending on the project's needs.

Developing a strong Beta user community

Successful products are not a result of chance but of deliberate and continuous effort. Beta users can play a crucial role in this process. While it can be challenging to motivate people to test the same software continuously, we can encourage participation by offering incentives and recognizing their contributions. Beta systems help teams identify critical defects early, before they affect the production release.

For example, it's relatively easy to cultivate beta gaming communities due to their enthusiasm for trying out new features. Similarly, groups of users can be formed on the client or vendor side. However, it's essential to ensure that beta users feel rewarded for their contributions. A robust beta community aids in identifying hidden defects before the software goes into production, thus preserving the software manufacturer's reputation.

Automating Daily Defect Reports  

Timely communication of defects is crucial. The team receives Daily Defect Reports automatically via communication channels like Slack or Teams, allowing critical or blocker defects to be addressed promptly.

Developers actively discuss critical/blocker defects in daily meetings, prioritizing their resolution over ongoing feature development. Once defects are fixed, automation ensures they do not recur.

Release Go or No-Go

After analyzing the defect status of the target feature, including beta defects, the tester recommends whether the feature is ready for production release. This decision is made after completing testing sessions for the target feature and resolving all identified defects.

No testing on production

Production servers should not be altered to support testing activities. Testing on the production system should only occur by providing access as real users. This preventive measure aims to avoid introducing any defects into the live environment.

Finding the right Testing KPIs

Quality Assistance involves analyzing defect reports, release history, and identifying key performance indicators (KPIs). This is a regular activity, planned monthly. Testing KPIs may include defect distribution, defect-prone areas, average defect age, functional test coverage, code coverage, and defect rejection rate.

Quality Assistance testing activities in Agile rituals

While Agile methodologies come in many shapes and sizes, let’s keep things simple and just focus on the Scrum rituals as our example framework. Teams can integrate Quality Assistance (QA) activities into these rituals to ensure that testing is an ongoing process, which can seamlessly become an integral part of the development cycle.

Project kick-off

The Product Owner/client provides detailed information about the product. The subject matter experts in the team discuss about the future product. Architects imagine about the most well-suited technology and tools for the project. Designers pay more attention about the user designs and workflows. Quality Assistance must ask the right questions about testing.

Refinement

During refinement, the Quality Assistance team analyzes user stories for completeness, helps estimate testing effort, identifies testing needs, and reviews UI/UX concepts.

Retrospective

During retrospectives, the Quality Assistance team identifies areas for improvement in testing.

Daily status meeting

During daily status meetings, the Quality Assistance team schedules pair testing appointments and test sessions, provides brief overviews of critical and blocker defects, and outlines tasks for upcoming releases.

Review presentations

During review presentations, the Quality Assistance team shares testing KPI results, presents new features, motivates stakeholders to test new features, and shares reports on the performance of previous features.

Quality Assurance vs. Quality Assistance

The following table summarizes the difference between Quality Assurance and Quality Assistance: 

Conclusion

In conclusion, Quality Assistance represents a shift from traditional Quality Assurance roles and emphasizes the importance of collective responsibility for quality. While it offers numerous advantages, it also presents challenges. Success with Quality Assistance requires acquiring the necessary skills, garnering support from all team members, and fostering a strong beta user community.

Quality Assistance may be well-suited for small teams working closely together but may face challenges in larger enterprise setups with multiple agile teams working on a common product. Teams should prioritize learning from Quality Assistance and invest effort in cultivating a supportive beta community.