KORAT — A platform independent test automation tool by emulating keyboard/mouse hardware signals

Abstract

Software, ranging from firmware, BIOS, and embedded software to complex software products, can only be tested by designing test cases to go through code and then verifying the results with expected outcomes. When code is changed frequently, regression testing is critical to ensure that changes do not introduce new faults. However, depending on the input types of the system under test (SUT), regression tests often require testers to drive the SUT manually, mainly by keyboard and mouse. In the meantime, testers play an important role as test oracle to determine the correctness of a test run by observing if the SUT behaves abnormally. Regression tests can be automated by programming or adopting testing tools. The most cost-effective approach supported by some commercial testing tools is capturing the testing behaviors of a human tester and then replaying the tests to assert the correctness. Unfortunately, most capture/replay tools are designed for testing the software which must be executed under a general-purpose O.S. They are inapplicable to many software systems, such as embedded software, BIOS, etc. In this paper, a capture/replay testing tool called KORAT is proposed. KORAT adopts a hardware component to intercept and emulate keyboard/mouse signals to drive an SUT as if the SUT is interacting with a human. A tester can design and operate a test case on a correct SUT to record the behaviors into a KORAT test script, in which no programming skills are required. In a regression run, the test case is replayed and the correctness is asserted automatically by analyzing SUT's video output (aka, images) and sending keyboard and mouse signals smartly. The correctness of a replay run can be asserted by image recognition, optical character recognition (OCR), and ASCII string matching via networking. Since KORAT only interfaces the video output of a SUT, it is platform independent and non-intrusive; meaning there is no performance interference caused by KORAT's capture and replay. A real application of KORAT to BIOS regression testing of industrial computer (militarized computers) manufacturing is described.