Coverage Goal Selector for Combining Multiple Criteria in Search-Based Unit Test Generation

Zhichao Zhou, ZHOU Yuming (DR.), Chunrong Fang, Zhenyu Chen, Xiapu Luo, Jingzhu He, Yutian Tang

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Unit testing is critical to the software development process, ensuring the correctness of basic programming units in a program (e.g., a method). Search-based software testing (SBST) is an automated approach to generating test cases. SBST generates test cases with genetic algorithms by specifying the coverage criterion (e.g., branch coverage). However, a good test suite must have different properties, which cannot be captured using an individual coverage criterion. Therefore, the state-of-the-art approach combines multiple criteria to generate test cases. Since combining multiple coverage criteria brings multiple objectives for optimization, it hurts the test suites' coverage for certain criteria compared with using the single criterion. To cope with this problem, we propose a novel approach named smart selection. Based on the coverage correlations among criteria and the subsumption relationships among coverage goals, smart selection selects a subset of coverage goals to reduce the number of optimization objectives and avoid missing any properties of all criteria. We conduct experiments to evaluate smart selection on 400400 Java classes with three state-of-the-art genetic algorithms under the 22-minute budget. On average, smart selection outperforms combining all goals on 65.1%65.1% of the classes having significant differences between the two approaches. Secondly, we conduct experiments to verify our assumptions about coverage criteria relationships. Furthermore, we assess the coverage performance of smart selection under varying budgets of 55, 88, and 1010 minutes and explore its effect on bug detection, confirming the advantage of smart selection over combining all goals.

Original languageEnglish
Pages (from-to)854 - 883
Number of pages30
JournalIEEE Transactions on Software Engineering
Volume50
Issue number4
DOIs
Publication statusPublished - 1 Apr 2024

Keywords

  • SBST
  • software testing
  • test generation

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Coverage Goal Selector for Combining Multiple Criteria in Search-Based Unit Test Generation'. Together they form a unique fingerprint.

Cite this