ABSTRACT

Metamorphic testing is an approach to both test case generation and test result verification. A central element is a set of metamorphic relations, which are necessary properties of the target function or algorithm in relation to multiple inputs and their expected outputs. Since its first publication, we have witnessed a rapidly increasing body of work examining metamorphic testing from various perspectives, including metamorphic relation identification, test case generation, integration with other software engineering techniques, and the validation and evaluation of software systems. In this paper, we review the current research of metamorphic testing and discuss the challenges yet to be addressed. We also present visions for further improvement of metamorphic testing and highlight opportunities for new research. CCS Concepts: Sofware and its engineering — Software verification and validation; Software testing and debugging Additional Key Words and Phrases: Metamorphic testing, metamorphic relation, test case generation, oracle problem

1. This research was supported in part by a linkage grant of the Australian Research Council (project ID LP100200208) and a grant of the General Research Fund of the Research Grants Council of Hong Kong (project no. 716612). Dave Towey acknowledges the financial support from the Artificial Intelligence and Optimisation Research Group of the University of Nothingham Ningbo China, the International Doctoral Innovation Centre, the Ningbo Education Bureau, the Ningbo Science and Technology Bureau, and the University of Nothingham. 2. It is with deep regret and sadness that we report the passing of the second author Fei-Ching Kuo on October 6, 2017. 3. (Corresponding author.) Department of Computer Science and Software Engineering, Swinburne University of Technology, John Street, Hawthorn, VIC 3122, Australia. Email: 4. Department of Computer Science and Software Engineering, Swinburne University of Technology, John Street, Hawthorn, VIC 3122, Australia. 5. College of Engineering & Science, Victoria University, Melbourne, VIC 8001, Australia. 6. School of Business IT and Logistics, RMIT University, Melbourne, VIC 3001, Australia. 7. School of Computer Science, University of Nottingham Ningbo China, Ningbo, Zhejiang 315100, China. 8. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong. 9. Institute of Cybersecurity and Cryptology, School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia.

EVERY VISITOR COUNTS: