Survey and comparison of malware analysis methods

Dr. Buttyán Levente

In recent years, the world has witnessed a series of high-profile targeted attacks against various targets, including organizations that operate critical infrastructures, or have an impact on critical infrastructure operations. These attacks showed that even the most secure networks can be compromised, and they induced an interesting discussion in the security industry and in the research community alike. An important lesson that the security community can learn from these incidents is that we must revisit some of the most fundamental assumptions which our systems rely on for security. In particular, one must make the assumption that motivated and resourceful attackers can compromise a system and gain access to its resources, and this may not be immediately detected by the system owner.

It is important to understand that widely used, commercially available security tools are not effective in detecting stealthy targeted attacks and previously unknown malware. One of the main reason for this is that there is a strong information asymmetry between attackers and defenders: attackers can have full knowledge of the defensive tools used by defenders, while defenders often lack the expertise and sufficient resources to acquire knowledge about the offensive tools and techniques. This means that attackers can optimize their attack strategy and tools in such a way that they maximize the probability of their success and minimize the probability of being identified. Practical examples show, for instance, that malware, such as Stuxnet, Duqu, and Flame, used in targeted attacks were optimized such that they evade detection by mainstream anti-virus products.

In order to detect previously unidentified malware, we advocate the use of behavior based anomaly detection on hosts, instead of relying on signature based detection. While high-profile malware employs various tricks in order to stay invisible, it must still make changes to the system state that leaves detectable traces in the behavior of the infected system. Our goal should be to identify those traces reliably, i.e., to decrease false positive detections, which is the major challenge of behavior based anomaly detection. To this end, in this report, we review the state-of-the-art on malware analysis, focusing on behavioral anomaly based malware detection approaches.

A teljes kutatási beszámoló letölthető innen (PDF)

Dr. Buttyán Levente, docens, BME Hálózati Rendszerek és Szolgáltatások tanszék,

2014. november 19.