Searching System Call Information for Clues : The Effects of Intrusions on Processes

The United States Air Force extensively uses information systems as a tool managing and maintaining its information. The increased dependence on these systems in recent years has necessitated the need for protection front threats of information warfare and cyber terrorism. One type of protection utilizes intrusion detection systems to provide indications that intrusive behavior has occurred. Other types of protection may include packet filtering, cryptography and strong user authentication. Traditional approaches toward intrusion detection rely on features that are external to computer processes. By treating processes as black-boxes, intrusion detection systems may miss a wealth of information that could be useful for detecting intrusions. This thesis effort investigate the effectiveness of anomaly-based intrusion detection using system call information from a computational process. Previous work uses sequences of system calls to identity anomalies in processes. Instead of sequences of system calls, information associated with each system call is used to build a profile of normality that may be used to detect a process deviation. Such information includes parameters passed, results returned and the instruction pointer associated with the system call. Three methods of detecting deviations are evaluated for this problem. These include direct matching, relaxed matching and artificial immune system matching techniques. The test data used includes stack- based buffer overflows, heap-based buffer overflows and file binding race conditions. Results from this effort show that although attempted exploits were difficult to detect, certain actual exploits were easily detectable from system call information. In addition, each of the matching approaches provides some indication of anomalous behavior, however each has strengths and limitations. This effort is considered a piece of the defense-in- depth model of intrusion detection.Air Force Inst of Tech Wright-Patterson AFB OH School of Engineering and Management is the author of 'Searching System Call Information for Clues : The Effects of Intrusions on Processes', published 2003 under ISBN 9781423502838 and ISBN 1423502833.