Stateful testing: Finding more errors in code and contracts

Yi Wei, Hannes Roth, Carlo A. Furia, Yu Pei, Alexander Horton, Michael Steindorfer, Martin Nordio, Bertrand Meyer

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

9 Citations (Scopus)

Abstract

Automated random testing has shown to be an effective approach to finding faults but still faces a major unsolved issue: how to generate test inputs diverse enough to find many faults and find them quickly. Stateful testing, the automated testing technique introduced in this article, generates new test cases that improve an existing test suite. The generated test cases are designed to violate the dynamically inferred contracts (invariants) characterizing the existing test suite. As a consequence, they are in a good position to detect new faults, and also to improve the accuracy of the inferred contracts by discovering those that are unsound. Experiments on 13 data structure classes totalling over 28,000 lines of code demonstrate the effectiveness of stateful testing in improving over the results of long sessions of random testing: stateful testing found 68.4% new faults and improved the accuracy of automatically inferred contracts to over 99%, with just a 7% time overhead.
Original languageEnglish
Title of host publication2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011, Proceedings
Pages440-443
Number of pages4
DOIs
Publication statusPublished - 1 Dec 2011
Externally publishedYes
Event2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011 - Lawrence, KS, United States
Duration: 6 Nov 201110 Nov 2011

Conference

Conference2011 26th IEEE/ACM International Conference on Automated Software Engineering, ASE 2011
CountryUnited States
CityLawrence, KS
Period6/11/1110/11/11

Keywords

  • automation
  • dynamic analysis
  • random testing

ASJC Scopus subject areas

  • Software

Cite this