In a tough economic situation, many software companies are focused on reducing their overheads. Unfortunately, software validation is often one of the first areas to feel this pressure. But with the explosion of hardware and software products and services in today’s hyper-connected world, producing high quality products with lower budgets is proving to be quite a challenge.
The execution and validation of manual/automated test cases can be a very time consuming procedure, especially for large pieces of software that involve thousands of test cases. Most software is released by making small changes in the code of the previous versions. If a set methodology is not followed, all test cases will have to be executed even though there may only be a small number of such cases that are affected, leading to release delays.
A solution to these problems and for lowering the operational cost of software validation is, refining the testing process by implementing Change Based Testing (CBT). CBT is a prioritization scheme that focuses on testing the changed portions of the software first, instead of testing the whole software. Using this new methodology, software quality can be significantly improved, while development time can also be reduced.
Test cases are written (manual or automated) to test the software. When a developer makes some changes in the code, and if the test cases relevant to the change are not known, the software has to be tested by running all the test cases again. Often, while attempting to run all test cases, some of them have to be dropped towards the end of the test cycle owing to a shortage in time, and sometimes they happen to be among the most relevant test cases, and quality is therefore impacted. With CBT, the developer or tester will know which test cases are relevant for testing, so they either commence with them, or run only these cases, thereby increasing software quality and decreasing testing time and cost.
To implement CBT, test cases must be mapped to the code. Test case to code mapping can be done manually, but it is time consuming. Alternatively, it can be done using instrumentation, which is used to gather run time test coverage information, i.e. test to code traceability. Code changes between two versions of the code, together with test to code traceability maps can be used to identify test cases that need to be repeated. .
When a software development team has an automated and comprehensive method such as CBT for preventing errors throughout the lifecycle of the software product, the team typically prevents hundreds or even thousands of errors from occurring throughout a project. As a result, there are significantly fewer errors to find and fix which translates to reduced costs and a more rapid and efficient development process.
Some CBT solutions available in the market support multiple programming languages, while others are restricted to a few specific languages. Here are some companies that provide a CBT solution:
Vector Software: This provider of software solutions for testing safety and mission-critical embedded applications recently announced the release of its CBT solution - VectorCAST 6.2. VectorCAST hosts new features that support CBT, intelligently indicating the sub-set of tests that are affected by each change, and thus test times are dramatically reduced.
Wind River Systems: Wind River provides embedded system software for intelligent connected systems. It has released Wind River Test Management 3.3 with the CBT feature. Release 3.3 promises a ten-time improvement in scalability which enables runtime test coverage for very large applications.“Reboot persistence” maintains coverage and performance data across complex tests. Reports have been improved for better management of Agile projects and test coverage trends. “Hybrid test type” facilitates migration from manual to automated testing besides supporting sophisticated device test sequences. “Analytics management” is available on test execution and runtime data for multi-device test beds.
Parasoft Corporation: Parasoft is an independent software vendor that develops automated defect prevention technologies. They support the Automated Defect Prevention methodology developed by Adam Kolawa. These technologies automate a number of defect prevention practices for Java, C and C++, and .NET.
CBT has been around for a while, and a lot of companies have toyed with CBT at one time or the other. However, some practical challenges such as managing hybrid code, handling kernel portions of embedded software, and dealing with distributed web applications limit its popularity. Finding ways to address these challenges will help software companies realize the power of change based testing.