Software Testing, Verification and Reliability 22 (4): 221-243 (2012) |
Zhi Quan Zhou 2 , ShuJia Zhang 3 , Markus Hagenbuchner 3 , T.H. Tse 4 , Fei-Ching Kuo 5, 6 , and T.Y. Chen 5
[paper from Wiley Online Library | technical report TR-2010-06]
ABSTRACT |
Search services are the main interface through which people discover
information on the internet.
A fundamental challenge in testing search services is the lack of oracles.
The sheer volume of data on the internet prohibits testers from
verifying the results.
Furthermore, it is difficult to objectively assess the ranking quality
because different assessors can have very different opinions on the
relevance of a Web page to a query.
This paper presents a novel method for automatically testing search
services without the need of a human oracle.
The experimental findings reveal that some commonly used search
engines, including Google, Yahoo!, and Live Search, are not as
reliable as what most users would expect.
For example, they may fail to find pages that exist in their own
repositories, or rank pages in a way that is logically inconsistent.
Suggestions are made for search service providers to improve their
service quality.
Keywords: software testing; metamorphic testing; verification and validation; internet search |
|
EVERY VISITOR COUNTS: |