TestCon Europe 2020
Vilnius and Online
CEO of Software Testing Company & Books Author
4TestDev GmbH, Germany
István Forgács Ph.D. was originally a researcher at the Hungarian Academy of Sciences. He published 30+ scientific articles in leading international journals and conference proceedings. He was a speaker at several academic and industrial conferences.
He is the lead author of the book Practical Test Design published last year and the co-author of the book Agile Testing Foundations, An ISTQB Foundation Level Agile Tester guide. He is a member of the Agile WG of ISTQB. He is the founder and co-CEO of 4TestDev GmbH.
Dr. Forgács is the creator of the test first, codeless test automation tool Harmony.
How to Find Bugs, Which Haven’t Been Found by Others
Test design is not an easy task if you want to find tricky bugs without creating an enormous number of test cases.
In the presentation, first Istvan will classify the defects in a new way. The simplest case is when a single data parameter triggers a failure whenever a program point has been reached along any program path from the data entry point. It can occur, however, that the failure only happens if more parameters have specific values in common. If the number of parameters causing the failure together is two, then the bug is a second-order bug, etc.
Here come the combinatorial methods. However, even pair-wise testing is quadratic w.r.t. the code size. If you apply a simple technique and the number of test cases is 100, then combinatorial testing may require 10,000 test cases. It’s usually not possible to design and maintain so many test cases. Therefore, you will use it only for safety-critical or extremely risky parts of your code.
Combinative testing is a new linear, yet very efficient test design technique, introduced in our book Practical Test Design last year.
The rationale behind the combinative technique is to assure that for a single parameter each computation is calculated in more than one context. The basic variant of combinative testing is called diff-pair testing. Informally, it requires that each value p of any parameter P be tested with at least two different values q and r for any other parameters.
For example, if the parameter P has two possible values, let us say x and y, and there are two more parameters Q and R with values: Q: (1, 2) and R: (Y, N), then we have to test the value pairs (x, 1), (x, 2), (x, Y), (x, N).
Istvan will show examples for which equivalence partitioning combined with boundary value analysis fails to find the fault, but diff-pair testing always finds it. It’s very easy to extend BVA test cases so that both predicate and higher-order domain errors be found. I also show why this method is linear and thus can be used to find tricky bugs at a low cost.