Analysis of Automated Rootkit Detection Methodologies

The focus of this study was to identify, analyze, compare, and evaluate the effectiveness of rootkit detection methodologies. Specifically, two methodologies were studied in depth. The first is the heuristic of statically analyzing kernel module binaries, which attempts to determine whether or not a software module's behavior is malicious, prior to passing it to the operating system. The second methodology analyzed in this paper, the Strider Ghostbuster framework, compares what a computer system believes to be true (i.e., what modules are visible to the OS) to the absolute "truth," which is determined via low-level system programming. The expected results of this comparison should always be equal, unless a malicious tampering on the system is observed. After comparing the effectiveness of detection methodologies on a set of well-known (and publicly available) rootkits, including a very simple rootkit built by the author, the methodologies are compared and their effectiveness is evaluated.