Journal of Systems and Software 84 (4): 603-619 (2011)

XML-Manipulating Test Case Prioritization for XML-Manipulating Services 1

Lijun Mei 2 , W.K. Chan 3 , T.H. Tse 2 , and Robert G. Merkel 4

[paper from ScienceDirect | technical report TR-2010-11]

 ABSTRACT

A web service may evolve autonomously, making peer web services in the same service composition uncertain as to whether the evolved behaviors are compatible with its original collaborative agreement. Although peer services may wish to conduct regression testing to verify the agreed collaboration, the source code of the former service may be inaccessible to them. Owing to the black-box nature of peer services, traditional code-based approaches to regression testing are inapplicable. In addition, traditional techniques assume that a regression test suite for verifying a web service is available. The location to store a regression test suite is also a problem. On the other hand, we note that the rich interface specifications of a web service provide peer services with a means to formulate black-box testing strategies. In this paper, we provide a strategy for black-box service-oriented testing. We also formulate new test case prioritization strategies using tags embedded in XML messages to reorder regression test cases, and reveal how the test cases use the interface specifications of web services. We experimentally evaluate the effectiveness of these black-box strategies in revealing regression faults in modified WS-BPEL programs. The results show that the new techniques can have a high chance of outperforming random ordering. Moreover, our experiment shows that prioritizing test cases based on WSDL tag coverage can achieve a smaller variance than that based on the number of tags in XML messages in regression test cases, even though their overall fault detection rates are similar.

Keywords: test case prioritization; black-box regression testing; WS-BPEL; service testing; service-oriented testing

1. This work is supported in part by the General Research Fund of the Research Grant Council of Hong Kong (project no. 717308), a strategic research grant of City University of Hong Kong (project no. 7002464), and a discovery grant of the Australian Research Council (project no. DP0984760).
2. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong.
3. (Corresponding author.)
Department of Computer Science, City University of Hong Kong, Tat Chee Avenue, Hong Kong.
Email:
4. Centre for Software Analysis and Testing, Swinburne University of Technology, Hawthorn 3122, Australia.

 EVERY VISITOR COUNTS:

  Cumulative visitor count