Andrei Lascu and Alastair F. Donaldson and Tobias Grosser and Torsten Hoefler:
Metamorphic Fuzzing of C++ Libraries
(In IEEE International Conference on Software Testing, Verification and Validation, Jun. 2022)
Abstract
We present a method for automated metamorphic fuzzing of software libraries, implemented as an open-source tool, MF++, targeting C++ libraries. Our approach works by automatically synthesising equivalent sequences of calls to a library’s API based on a user-provided specification, in a randomized fashion. Equivalent call sequences are then tested using randomized inputs, and result mismatches reveal bugs in the library implementation. This is an instance of metamorphic testing: it avoids the oracle problem because we do not need to know the expected results of a set of equivalent call sequences, only that their results should match. Automated test case reduction can then be used to find minimized equivalent call sequences that trigger mismatches, as an aid to debugging. We evaluate MF++ with respect to four SMT solving libraries and two Presburger arithmetic libraries, leading to the discovery of 21 bugs. We have also successfully used MF++ and its test case reduction facilities to automatically generate small test cases that exercise source code not covered by the regression test suites of various libraries under test. Unlike most test case generation techniques, the tests we synthesise are equipped with an oracle by construction: the equivalence-based oracle offered by our metamorphic approach. We have submitted patches contributing new coverage-enhancing test cases to the isl, Yices2 and Z3 projects. The developers of these projects have accepted 21 tests based on our patches so far.
Documents
download article:
Recorded talk (best effort)
BibTeX
@inproceedings{, author={Andrei Lascu and Alastair F. Donaldson and Tobias Grosser and Torsten Hoefler}, title={{Metamorphic Fuzzing of C++ Libraries}}, year={2022}, month={Jun.}, booktitle={IEEE International Conference on Software Testing, Verification and Validation}, source={http://www.unixer.de/~htor/publications/}, }
