In continuation with my earlier blog "Test models - The way forward", let us understand how ‘Model-based Testing’ is currently being leveraged across the industry.
Model-based Testing (MBT) is a technique for the automatic generation of test cases using models extracted from software artifacts (i.e. system requirements specifications). This approach helps in controlling the software quality and reducing the costs related to testing process, because test cases can be generated from the software artifacts produced throughout the software development process. MBT can use models developed during any software development phase to identify and construct the set of test cases.
The MBT strategy usually includes different levels of abstraction, a behavior model, the relationship between models and code, test case generation technology, the importance of selection criteria for test cases, and a discussion of what can or cannot be automated when it comes to testing.
MBT entails building the model, generating test cases, executing the tests, comparing actual output with expected output and deciding upon further actions (whether to modify the model, generate more test cases, stop testing, or estimate the software reliability [quality]).
The model used is an important element in the application of MBT, as it defines the limitation of each approach based on the information that it can represent about the software structure or behavior. Sometimes the model used for a specific application domain cannot be employed by another domain.
The main differences among MBT approaches are the criteria used to define the testing coverage (subset of test cases generated from a specific model) and test generation (steps to be accomplished). Each approach has its specific steps with different complexity or automation level.
The steps are:
- Modeling of Software Behavior;
- Applying Software Product Testing Coverage Criteria to select the set of test cases;
- Generation of test cases and expected results; and
- Execution of tests and comparison of obtained results against the expected results.
Each MBT approach has specific characteristics that make it different from other approaches and therefore a clear comparison cannot be made to define whether one approach is better than the other. The behavior model must be developed carefully. Its limitation and the restrictions must be respected during the modeling of the software under test. Moreover, the correctness of a model is fundamental to start the test case generation process. If the model is wrong, the generated tests will be invalid.
There is a promising future for MBT as software becomes even more ubiquitous and quality becomes the only distinguishing factor between brands. Modeling in general seems to be gaining favor; particularly in domains where quality is essential and less-than-adequate software is not an option.
It is quite intuitive to use MBT for new software development as the model evolves as the requirements are defined and elaborated. It is especially true for development where system behavior is already modeled using state diagrams. It is equally possible to leverage it for development needs where the software product and artifacts exist by leveraging some existing artifacts like use cases. We will soon discuss about the trends and current market scenarios in our next blog.