4 key considerations for an effective automation testing strategy

Considering the wide user base that a typical web application targets, the need to test software systems on many browsers, platforms and mobile devices has enhanced the testing scope dramatically. However, testing with only a manual approach is not only inefficient but error-prone.

An automation testing strategy for web applications is a must to enable quick time to market, maximize coverage of a wide testing scope and optimize cost. It also ensures that there is no room for error in the current and previously developed scope.

Here are four key considerations to keep in mind for an effective automation testing strategy.

• Previously developed scope

Automation today is an integral part of any testing strategy. Automated scripting is done in each sprint and an automation regression test suite is then gradually built. The automated scripts should not just validate the basic functionality of the web application but should validate the core business logic as well. Automated scripts of previously built scope are then integrated with build and deployment processes. Continuous Integration ensures that the previously developed scope is automatically regressed before we start testing for new functions.

To give an example from my own experience for a client in the US, a leading provider of nonprofit software and services, we have been building an automation regression test suite for many years. We have automated more than 3000 test cases using a keyword-driven automation framework.

• Quick time to market

While a DevOps practice like the one we have at Nagarro ensures that most of the manual tasks of development operations are automated for the release, by running tests against the codebase on a frequent basis after each code check-in, the testing team can provide rapid notifications for any defects, particularly those in unchanged areas of the code. This means that the software is rebuilt and tested every time the code is changed.

For one of our clients in the gaming domain in Europe, we used the entire DevOps tool chain including JIRA, Stash, Sonar Qube, Jenkins/Hudson, Selenium Web Driver and automated end-to-end release cycles. Then we coupled automated release cycles with automated regression suite resulting in shorter testing cycles (moved to fortnightly releases from the previous quarterly cycles) and hence reduced time to market.

• Cross browser testing

A cross browser testing strategy mandates the use of automation focusing on testing the business logic. We can use customized automation frameworks developed in-house to facilitate in parallel execution of test cases (for instance, if we run 3 tests at a time, then it would be able to finish the whole suite almost 3 times faster). Test cases are run against different browsers and platforms at the same time.

For another one of our clients, we enhanced the automation coverage from 0% to almost 80% and reduced the execution time for regression to 2 days from 4 weeks through automated parallel execution.

• Sensitive nature of user data

Sensitive nature of user data and unavailability of production data makes the entire automation testing strategy more challenging. Data such as customer’s personal information (phone, address, DOB etc.) and tax account related information (account no, name etc.) is highly confidential and can easily be misused. Automated data masking on production data can be used to ensure data generation for web based automation framework. In addition, automated data generation scripts can be written for some exceptional cases for which data in production is non-existent.

Finally, an effective automation strategy is one from which we can learn, that we can improve over time through predictive analytics and applied business knowledge.