The testing of large databases faces the test oracle problem, namely, that it is difficult to verify execution results against expected outcomes. Rigger and Su applied metamorphic testing through query partitioning and ternary logic partitioning techniques to alleviate the challenge. In Part (A) of our project, we conduct an in-depth investigation and have identified a gap between the two techniques. We propose a disjoint partitioning approach to address it. In Part (B), we conduct a comprehensive investigation into the metamorphic testing of DBMS by comparing disjoint partitioning with metamorphic relation output patterns (MROPs) by Segura et al. We propose an exhaustive collection of MROPs for DBMS. To the best of our knowledge, this is the first project to integrate in-depth and comprehensive approaches to tackle the diverse challenges in DBMS testing. In Part (C), we conduct an empirical case study of their applications to OceanBase, the DBMS associated with the world’s fastest online transaction processing system. Although OceanBase has been extensively tested and widely used in the industry, we have detected 12 hidden failures and 8 new crashes. Index Terms: test oracle, metamorphic testing, metamorphic relation output pattern, DBMS, SQL, OceanBase

1. This project was supported in part by an internship of the first author at Alibaba and Ant Group, China. 2. Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong. 3. (Corresponding author.) Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong. Email: 4. School of Computer Science and Software Engineering, University of Wollongong, Wollongong, NSW 2522, Australia, and Alibaba and Ant Group, Hangzhou, China.

EVERY VISITOR COUNTS: