Towards Faster Software Revision Testing

Software systems have been increasingly prevalent in all facets of our lives over the last few decades and play a critical role in modern living. They have a significant impact on the quality of our lives and provide tremendous convenience. However, software faults (also known as bugs) are unavoidable throughout the development of software systems that can have a substantial negative impact on the commercial company and result in significant losses. Numerous researchers have been working on this problem to test software systems during development and fix bugs after the software systems are established. However, due to the complexity of these systems, these approaches can be very time consuming. For example, mutation testing is an important component of software testing which can be very powerful to evaluate the quality of the test suite, but it can be extremely time consuming due to a large number of mutant execution. Also, Automated Program Repair (APR) techniques can reduce software debugging human efforts by advising plausible patches for buggy programs. However, the APR techniques need to repeatedly execute all the test suites to identify the plausible patches for the bugs under fixing. This process could be extremely costly. Therefore, it is essential to explore some approaches to speed up the processes of software testing and debugging. In this dissertation, we aim to speed up software testing and debugging via faster software revision testing. The idea is to decrease the testing time between different revisions to speed up software testing and debugging. We explored two scenarios in software testing during the evolution of software systems: mutation testing and behavioral backward incompatibilities (BBIs) detection. We applied regression test selection (RTS) techniques to speed up mutation testing for the first study. Our study showed that both file-level static and dynamic RTS could achieve efficient and precise mutation testing, providing practical guidelines for developers. We called the second BBIs detection technique DeBBI which can reduce the end-to-end testing time for detecting the first and average unique BBIs by 99.1% and 70.8% for JDK compared to naive cross-project BBIs detection. Additionally, we detected 97 BBI bugs including 19 that were previously confirmed as unknown bugs. Lastly, we explored the application in patch validation of APR technique to speed up software debugging. We treated every single patch as a revision to develop a unified on-the-fly patch validation framework, named UniAPR. Our study demonstrated that on-the-fly patch validation could often speed up state-of-the-art source-code-level APR by over an order of magnitude, enabling all existing APR techniques to explore a more extensive search space to fix more bugs in the near future.