The Symposium on Engineering Test Harness (TSETH '13),
in Proceedings of the 13th International Conference on Quality Software (QSIC '13),
IEEE Computer Society Press, Los Alamitos, CA, 245-252 (2013)

Oracles are Hardly Attain'd, And Hardly Understood:
Confessions of Software Testing Researchers 1

W.K. Chan 2 and T.H. Tse 3

[paper from IEEE Xplore | paper from IEEE digital library | technical report TR-2013-03]

 ABSTRACT

In software testing, a test oracle refers to the mechanism for determining whether the results of the software under test agree with the expected outcomes. To achieve this, we need a means to determine the expected outcomes, a means to gauge the actual results, and a means to decide whether the actual results agree with the expected outcomes. In real-life situations, however, a test oracle may not exist owing to a missing link in any of these aspects. In this paper, we summarize our research for the last 15 years on selected issues related to each of these aspects. We present the use of metamorphic testing, pattern classification, and formal object equivalence and nonequivalence to alleviate the problems.

Keywords: test oracle, test harness, metamorphic testing, pattern classifier, object equivalence and nonequivalence

1. This work is supported in part by the General Research Fund of the Research Grants Council of Hong Kong (project nos. 111410, 123512, 716612, and 717811) and a linkage grant of the Australian Research Council (project no. LP100200208).
2. Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Hong Kong.
Email:
3. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong.

 EVERY VISITOR COUNTS:

  Cumulative visitor count