IEEE/ACM 5th International Workshop on Metamorphic Testing
(MET '20),
Proceedings of the IEEE/ACM 42nd
International Conference on Software Engineering Workshops (ICSEW '20), ACM, New York, NY, pp. 388-395 (2020) |
Dickson T.S. Lee 2 , Zhi Quan Zhou 3 , and T.H. Tse 2
ABSTRACT |
Current research on the testing of machine translation software
mainly focuses on functional correctness for valid, well-formed
inputs.
By contrast, robustness testing, which involves the ability
of the software to handle erroneous or unanticipated inputs, is often
overlooked.
In this paper, we propose to address this important
shortcoming.
Using the metamorphic robustness testing approach,
we compare the translations of original inputs with those of followup
inputs having different categories of minor typos.
Our empirical
results reveal a lack of robustness in Google Translate, thereby
opening a new research direction for the quality assurance of neural
machine translators.
Keywords: Robustness testing, Oracle problem, Metamorphic testing, Metamorphic robustness testing, Machine translation, MT4MT |
|
EVERY VISITOR COUNTS: |