The only thing that is constant in a software product is the continuous and rapid changes. The hurry to deliver something new and unique before someone else does in order to grab more business sometimes backfires due to the lack of quality. The same is true for bug fix release.
The question is not if quality should be ensured before any release or not but how it can be achieved within the timeframe. I have seen immense pressure on the developers and testers to release the fix quickly.
I remember a classic example, which can help us understand where and how test automation can make a difference:
One of our teams started working on a healthcare application with a team size of 5 developers and 2 test engineers. The application went live after one year of development and testing efforts. Stakeholders also had some plans for future releases with new features. Gradually, the customer base increased to around 250 licensed users within 4 months. Things were manageable until the first release.
After a few months of the first release, the team realized that they needed to reduce the testing time duration in order to release the urgent fixes as soon as possible and not doing this can cost them losing customer base. So, the product owner decided to double the testing team size. Increasing the testing team size reduced the test cycle time but it was not sufficient enough to meet the demand of rapid delivery of new features as testers were still spending most of the time in testing urgent fixes. Following stats show testing efforts required for different test cycles.
Manual Test execution efforts
Test cycles (Number of test cases) |
Efforts (in hours) |
| Full Regression (1000) | 160 |
| Partial regression (250) | 40 |
| Sanity (50) | 08 |
| Smoke (15) | 02 |
How Evon helped them delivering rapid releases?
The major pain area in the above story is the waiting time during the testing phase. The testing time could be reduced by including automated testing in different test cycles. Evon suggested to create a priority list of test case candidates for automation based on the following criteria:
- Tests that are business critical
- Tests that need to run for each patch release, minor release, and major release.
- Tests that are less frequent to changes in the UI.
- Tests that execute the same work-flow but use different data for its inputs for each test run.
- Tests that involve inputting large volumes of data, such as filling up very long forms.
- Tests that take a lot of effort and time when manual testing.
So, we started automating tests, first with smoke test suite then sanity, partial regression, and full regression. Gradually we were able to reduce the time spent during the testing phase.
Let's have a look at the comparative table to understand it:
Manual vs Automated test execution efforts |
||
|
Test cycles (Number of test cases) |
Manual testing efforts (in hours) |
Automation testing efforts (in hours) |
|
Full Regression (1000) |
160 |
16 |
|
Partial regression (250) |
40 |
4 |
|
Sanity (50) |
8 |
0.8 |
|
Smoke (15) |
2 |
0.2 |
Above statistics clearly show that test automation reduced test execution time significantly. Now let us have a look at the complete picture by including time spent on test case preparation. To understand let us compare testing efforts for the sanity test cycle.
Effort in hours / test case
|
Manual |
Automation |
|
|
Test case development |
0.35 |
2.00 |
|
Test case execution |
0.16 |
0.016 |
Manual vs Automated test (development/maintenance + execution) efforts |
||
|
The regression test suite (1000 test cases) |
Manual (in hours) |
Automation (in hours) |
|
1st test cycle |
510 |
2016 |
|
2nd test cycle onward |
160 |
16 |
Total manual testing efforts till 11th cycle= 2110
Total automation testing efforts till 11th cycle = 2160
So ROI started growing from 12th cycle onward.
We discussed the direct benefits of test automation above. There are a few considerable indirect benefits also listed below:
Other benefits of Test automation
Benefits |
Impact of test automation |
|
Confidence in product |
Test automation enhances confidence as it executes the test cases, in the same manner, every time. There is no chance of deviation or lack of concentration. |
|
Improved testing coverage |
Since automated tests take less time to execute, we generally need not compromise test cases in lack of time. Instead, we can cover more testing in lesser time. We can also achieve testing on different environments in lesser time. This results in enhanced test coverage. |
|
Improved path coverage |
There are paths in the flow of the application that is not always possible to cover by manual testing can be catered by applying test automation as once scripted, it will always go through that path. |
|
Resource dependency |
Automated testing requires very less product knowledge comparing to manual test resource. So it completely removes the dependency on manual test resource. The tests once automated will execute the same flow and test steps every time. |
|
Lesser human resources |
You don't need a lot of human resources. You just need test automation engineer to write and execute test scripts. You may need a manual test engineer(s) to test a few specific test cases that can not be automated. |
Why Choose Evon for Test Automation Services
Evon is a renowned software consulting company in India and is engaged in offering an array of software services. Quality assurance is one of the core areas of expertise of Evon, providing quality testing services to various happy clients across the globe for more than 10 years now.
Evon has the expertise in using various cutting-edge test automation tools available in the tech market. Since every test automation tool has its own pros and cons, expert testers at Evon first study the domain or the nature of a product and then pick the one that best meets the testing requirements of the product. Evon also helps you improve your overall test coverage by testing in and out of your software like the databases, UI, custom web design and development services or APIS, etc.
With significant advancements being made in smartphones, tablets and new operating systems, test automation has become imperative to keep the pace with ever-changing demands. This is why more and more businesses are now adopting automated testing to boost their development initiatives. It’s true that manual testing can’t be sidelined completely as it is still required to run some tests that are not possible with automated testing. But, this is where Evon helps you establish what all types of tests should be automated to accelerate the test cycle and roll out new versions of your product faster. Choosing Evon will guarantee you reliable support across various hardware and software, test infrastructure as well as high test coverage.
Contact us for any query related to test automation services, and we promise to get back to you shortly.