Streamlining Quality Assurance: The Power of Shift-Left Testing

Will you believe us if we say that a production release delay was caused by software testing rather than a lack of planning?

That’s correct! Software testing still has a set period in the conventional approach, which begins right before the production release. Inadequate code quality checks, emphasis on manual testing rather than automation testing, a delay in defect detection, significant architecture changes because of blockers and critical defects, and delayed bug fixing costs - all these contribute to projects ultimately running over budget or behind schedule.

According to the Systems Science Institute at IBM, the cost of fixing a defect is 100X in the maintenance phase, 15X in the testing phase, and 6.5X in the implementation phase as compared to the design phase.

Let’s take an example of a real-world production environment issue where the company had to bear the huge cost of bug fixing (around $17 million). Why? Only because that defect was found quite late after the production was done in the market. – The case of Samsung Note 7 battery explosion.

To overcome the challenges faced in traditional testing approach and with the adoption of practices like Agile methodologies, DevOps and CI/CD pipeline integration, the focus started to gradually shift from a traditional QA approach and organizations showed keen interest in introducing testing early in SDLC. This led to the invention of “Shift-Left Testing”.

The Shift-Left testing process entails shifting of testing activities to the left and beginning to consider testing at all stages of the process. It emphasizes early testing and testing everything while involving testers in every stage of the development cycle. It is a subset of the continuous testing approach, more applicable to Agile environments which lay emphasis on faster delivery with continuous quality. It requires continuous communication between the stakeholders.

The Shift-Left approach focuses on establishing “QA Quality Gates” in all phases of SDLC cycle starting from planning till deployment. The goal is to identify defects early in the product development lifecycle. The best way to Shift-Left is by allowing the QA teams to participate in activities starting from planning till deployment, when code is production ready.  

Let’s understand the importance of Shift-Left approach by examining the real-life case studies that demonstrate the benefits and outcomes of Shift-Left testing methodologies implemented for different clients from different domains. These case studies dictate the chronicles of a Shift-Left approach that helped businesses to achieve the quality and time-to-market goals.

Chronicle 1: Early testing assisted in overcoming issues caused by frequent changing requirements for an energy client.

The client was one of the global leaders in the energy domain working towards providing innovative connected technologies considering sustainability in mind. For this assignment, they were looking for a configuration device to be installed at multiple field locations which could help them reduce their configuration time which was at present too high (around 7-10 days), and it also involved a lot of integration with external services and applications.

Changing requirements, integration with external services, multiple team interaction, backlog prioritization and frequently changing release plan dates leading to low productivity were a few of the challenges faced by the clients.

To overcome these challenges, a cross functional scrum team was formed to adapt to such an environment. With early adoption of testing and ensuring test early and test often approach, team started to work towards formulating a testing strategy before the start of each sprint.

The team’s constant focus remained on automation, CI/CD pipeline for smooth delivery of code and a thorough review of requirements and design architecture done using regular “Story Grooming Sessions” before the sprint starts. The scrum team also took charge of organizing “Mini Story Grooming Sessions” while any sprint was in progress. These sessions ensured that all issues and concerns were being addressed and action taken upon, no gaps left out!

All these practices ensured frequent quality deliverables to the client and the results were great too! Our team was successful in reducing the configuration time from 7 days to just 15-20 minutes, which is a truly remarkable improvement.

The scrum team also put a strong emphasis on “Measuring the Metrics” that were identified and targeted for each completed sprint release. Following a comparison of these figures with the benchmark goals established by our engineering excellence team, the appropriate action was taken.

Fig.  1: Metrics in consideration and steps taken to accomplish goals

All these actions led to results that enhanced the client’s productivity and efficiency. 
Few of the results achieved by the team:

• Delivered ~20% additional v/s planned scope as compared to last releases.
• Helped client save ~180000 Euros in the financial year.
• 0 deployment issues through introduction of CI/CD 
• With automated API testing, saved around ~20% in every sprint spent on manual API testing.
• 0% defect leakage achieved through regular grooming sessions to avoid any gaps.
• By ensuring 90% backend Junit coverage, code quality was maintained.

Chronicle 2: Surge in cascading defects lowering the retail sector application quality.

Another success story presents the interesting benefits gained by US based B2B retail service provider of IT hardware to worldwide customers from different domains like Education, Defense, Health, etc. Nagarro is developing multiple applications for them to handle their critical business functional areas like sales, purchase, inventory management, administration, and microsite.

The sprint defects were expanding as the application grew (approximately 10-12 defects in 3 weeks). On a lighter note, the QA team was not to blame for the situation; rather, key contributing factors included the lack of communication between the engineering team and business stakeholders during requirement discussions, the implementation of application changes without impact analyses, less than 40%-unit test coverage, an unstable testing environment, and the absence of an automated regression test suite.

Measures that improved the quality in the project

First important thing that was required to make a connection between the business and the engineering team to eradicate ambiguous requirements, and this was solved by program “Walkthrough Wednesday” where anyone from engineering team member explains the team understanding of requirements to business every week.

Another initiative “WAS (Walk As User)”, where UAT team was responsible to review the designed wireframes for the changed requirements and to validate the impact on the business flow according to those new wireframes. It helped in solving the defects introduced to the design change.

The integration of SonarQube in the CI/CD pipeline helped to merge the clean and good quality code. The architect also had enough time to perform functional code reviews. But the real cherry on top was the fact that on integrating with JaCoCo, the code was not merged until the benchmark (minimum 80%) of unit test coverage was not reached.

Our QA experts implemented Nagarro’s A2A.DRIVEN – proven tool agnostic automation test accelerator for automation on UI and Web Services layer that helped in reducing the time spent on manual regression test cycles and automated the defect logging after automated test execution by integration with defect management tool.

Fig. 2 Numbers that proved the benefits in the project

Shift-Left approach  

Fig 3: Shift-Left approach in SDLC  

The points below offer a thorough explanation of what should be done to skew testing towards the left.

• Find Bugs Early: Put more effort into finding bugs during the early phases of SDLC cycle when requirements are being finalized and design is under consideration. Eventually you will have to put less effort into the later stages of Testing and Deployment phase.
• Design a Unified Test Strategy: Make QA responsible for designing a Unified Test Strategy and talk about resource planning, deciding upon testing tools and strategies, analyzing cost and budget constraints from a Testing perspective.
• Check Requirement Feasibility & Evaluate Risks: It is important for QA to do requirement feasibility check and ensure requirement testability status well in advance.
• Perform Static Code Analysis & Review: Design architecture & code issues prove to be a major bottleneck if not detected earlier. They can be due to poor choice of architectural frameworks, incorrect memory allocations and resource synchronization, deadlock situations, CPU spikes, inefficient usage of cache memory etc.
• Emphasize on the Unit Testing and API Testing Coverage: Shift-Left approach stresses focusing more on Unit & API Testing. Static code analysis and review sessions should be done in design and coding phase. This includes proper code review by peers and architect before merging to main code branch.
• Embrace Test Automation: The goal remains to embrace test automation tools early, focus on implementing CI/CD pipeline, automate the deployment process for smooth delivery to the customer, test each code increment too. Consider performance & security aspects well in advance at the design time itself. 
• Ensure Test Data Availability: Preparing quality test data is a challenge, especially with Agile development. The necessary test data must be prepared earlier, and faster. High quality test data is required to execute use cases and uncover failures. Test data management is a crucial task and organizations should opt for a good test data management tool. During this process, teams verify APIs, container configurations, and interactions between microservices.
• Change Team’s Mindset: Make a shift in the mindset of people too: A workforce ready to leave its comfort zone, who is open to accepting changes and is willing to give importance to QA activities at early stages is very much needed to adapt to Shift-Left Testing.

Collaboration with Nagarro’s Engineering Excellence Programme for successful Shift-Left implementation

The focus is always on implementing a Shift-Left testing approach for enhanced delivered quality to our clients. Nagarro’s Engineering Excellence Programme (EEP) also advocates the benefits of Shift-Left and supports us in this mission.

Nagarro’s brightest engineering minds focus on creating an exceptional customer experience. EEP is another arrow in Nagarro’s quiver that aims for innovation and ownership within the team itself by motivating them to define and drive their own success by making distance irrelevant between them and get rewarded in an agile way.

This program has established “Thinking Breakthrough” mindset among the team members to bring continual improvement via Engineering Excellence Journey (EEJ) framework in various excellence areas (like core engineering, customer satisfaction, innovation, business goals, learning and development etc.), designed to empower teammates to solve identified areas of issues/problems and gradually improve over them.

The EEP program identified various improvement areas to raise the application's overall quality with increased consistent delivery, optimum productivity, increased innovation, and happier customers and most of all, an overall excitement and continually improved outcome, again and again.

Interested to know more about EEP and how to shape the future of quality? Get in touch: aqt@nagarro.com