Sitar: Gui Test Script Repair

Abstract

The system tests of a GUI-based application require that the test cases, consisting of sequences of user actions / events, be executed and the output of the software verified. To allow a new automated test, such test cases are increasingly encoded as low-level test scripts, to be automatically reproduced using test harnesses. Every time the GUI changes, the widgets move, the windows merge, some scripts become unusable because they no longer encode the valid input sequences. In addition, because the software output may have changed, it is possible that your test-assertion oracles and checkpoints-encoded in the scripts no longer correctly check the desired GUI objects. Introducing ScrIpT repAireR (SITAR), a technique to automatically repair unusable low-level test scripts. SITAR uses reverse engineering techniques to create an abstract test for each script, maps it to an annotated event flow graph (EFG), uses reparative transformations and human input to repair the test, and synthesizes a new test script "repaired." During this process, SITAR also repairs the reference to the GUI objects used in the control points, producing a final test script that can be executed automatically to validate the revised software. SITAR amortizes the cost of human intervention in multiple scripts by accumulating human knowledge as annotations in the EFG. An experiment using QTP test scripts suggests that SITAR is effective because 41-89 percent of the unusable test scripts were repaired. The annotations significantly reduced the human cost after the 20 percent test scripts were repaired.