Publication
Title
Speeding up mutation testing via the cloud : lessons learned for further optimisations
Author
Abstract
Background: Mutation testing is the state-of-the-art technique for assessing the fault detection capacity of a test suite. Unfortunately, it is seldom applied in practice because it is computationally expensive. We witnessed 48 hours of mutation testing time on a test suite comprising 272 unit tests and 5,258 lines of test code for testing a project with 48,873 lines of production code. Aims: Therefore, researchers are currently investigating cloud solutions, hoping to achieve sufficient speed-up to allow for a complete mutation test run during the nightly build. Method: In this paper we evaluate mutation testing in the cloud against two industrial projects. Results: With our proof-of-concept, we achieved a speed-up between 12x and 12.7x on a cloud infrastructure with 16 nodes. This allowed to reduce the aforementioned 48 hours of mutation testing time to 3.7 hours. Conclusions: We make a detailed analysis of the delays induced by the distributed architecture, point out avenues for further optimisation and elaborate on the lessons learned for the mutation testing community. Most importantly, we learned that for optimal deployment in a cloud infrastructure, tasks should remain completely independent. Mutant optimisation techniques that violate this principle will benefit less from deploying in the cloud.
Language
English
Source (book)
ESEM '18 : Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Oktober 11-12, 2018, Oulu, Finland
Publication
New york : Assoc computing machinery , 2018
ISBN
978-1-4503-5823-1
978-1-4503-5823-1
Volume/pages
(2018) , 9 p.
Article Reference
26
ISI
000469776800026
Medium
E-only publicatie
Full text (Publisher's DOI)
Full text (publisher's version - intranet only)
UAntwerpen
Faculty/Department
Research group
Project info
BOF Sabbatical Leave- Prof. Demeyer.
Publication type
Subject
Affiliation
Publications with a UAntwerp address
External links
Web of Science
Record
Identification
Creation 19.11.2018
Last edited 19.10.2021
To cite this reference