Manual Software Testing Versus Automated Testing
Guide to when to Use Manual Testing or When to Use Automated Tools
Jan 2, 2009
Traditionally the tasks of testing from planning through execution to reporting results have been a manual set of tasks. There are now software programs that can do some of the testing and consequently ease the burden. However, automated testing is not a silver bullet and a business needs to evaluate when to use automation and when to do manual testing.
Test Plan
Software testing must be planned. A key question and decision for the test plan is whether to use testing tools or test software for automated testing. To answer that question the following test tasks needs to be considered for both manual testing and automated testing:
- Test Planning
- Writing Test Cases
- Executing Test Cases
- Reporting the Results of Testing
Review Test Tasks for Test Automation
Reviewing the test tasks it is clear that test planning will be common to both manual testing and automated testing. Similarly writing test cases whether for manual execution or test tools needs to be done. The benefit of testing tools becomes apparent when actually executing tests, especially repeatedly. Since both the execution and the reporting of results can be automated then potentially considerable manual effort is saved.
Manual Testing Versus Automated Testing
It becomes clear that to evaluate the business benefit of manual testing versus automated testing requires careful consideration of:
- The effort required to produce written test cases for manual execution
- The effort required to produce test cases for test automation
- The probably number of times those test cases are executed for this test cycle
- The likelihood of reusing automated test cases for future test cycles
These can be determined as a set of numbers and consequently a comparison can be made or a cost benefit analysis. In general terms it makes sense to use testing tools if:
- The effort required to create test cases is not substantially more than to produce written test cases
- Test cases are likely to be used more than once in this test cycle
- It is probable that the test cases can be reused at once in the future with little or preferably no change
- Test tools have already been purchased and are in use
It does not make sense if:
- Cost benefit analysis of purchasing and using test software is not justified on project basis
- Effort or cost to produce test cases for testing tools significantly exceeds manual production
- Opportunity for repeated testing is minimal
- Future reuse is highly unlikely or requires substantial changes
Test Plan Decision on Test Automation
After consideration of these points a project decision can made as to whether to use manual testing or automated testing. There is no right answer of whether to use test automation or not, it can only be decided on a case by case basis. Or, by a business decision to invest in test software tools and to have subsequent projects then exploit that capability.
