ABSTRACT

In real life, a tester can only afford to apply one test case prioritization technique to one test suite against a service-oriented workflow application once in the regression testing of the application, even if it results in an adverse scenario such that the actual performance in the test session is far below average. It is unclear whether the factors of test case prioritization techniques known to be significant in terms of average performance can be extrapolated to adverse scenarios. In this paper, we examine whether such a factor or technique may consistently affect the rate of fault detection in both the average and adverse scenarios. The factors studied include prioritization strategy, artifacts to provide coverage data, ordering direction of a strategy, and the use of executable and non-executable artifacts. The results show that only a minor portion of the 10 studied techniques, most of which are based on the iterative strategy, are consistently effective in both average and adverse scenarios. To the best of our knowledge, this paper presents the first piece of empirical evidence regarding the consistency in the effectiveness of test case prioritization techniques and factors of service-oriented workflow applications between average and adverse scenarios. Keywords: XML-based factor; WS-BPEL; adaptation; adverse

1. This research is supported in part by the Early Career Scheme and the General Research Fund of the Research Grants Council of Hong Kong (project nos. 111410, 123512, 716612, and 717811). 2. Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Hong Kong. 3. (Corresponding author.) IBM Research – China, Beijing, China. Email: 4. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong.

EVERY VISITOR COUNTS: