Using fault injection for testing Linux kernel components

The paper considers methods of using fault injection for testing components of Linux kernel. The main goal of the methods is to check how the kernel behaves in abnormal situations such as lack of resources and hardware faults. Such situations happens quite rarely and unpredictably, that makes very difficult to detect and to localize bugs in the code responsible for their handling. The most widely-used approach to find the bugs is random fault injection during execution of normal tests. Random fault injection works much better than absence of abnormal testing at all, but it has a number of drawbacks. Its random nature does not allow to reproduce test scenarios and it does not allow to estimate absence of bugs even if all tests are passed.
The paper presents a method of systematic fault injection for robustness testing that also enables reproducibility of test scenarios out of box. The basic idea of the method is to detect all execution points where a fault can happen, to filter out the points on the base of some equivalence relation and to execute tests bringing code into the chosen execution points with injecting faults at that points.Applicability and efficiency of the approach depends on the equivalence relation. The equivalence relation can be defined using manual markup of important segments of tests or it can be defined completely automatically on the base of a set of stack traces describing a fault injection.Experimental data were collected using test suites for Linux kernel file system implementations augmented with random and systematic fault injection.The experiments demonstrate the efficiency of systematic approach and acknowledge benefits coming from its deterministic nature.