Now, Near, Deep: The Power of Multi-Layered Malware Analysis & Detection

For malware authors, an important part of their strategy is to drown target organizations with a fire hose of constantly changing information. SOC teams struggle to keep pace with attackers’ ability to rapidly generate new malware variants, new URLs leading to infected websites, and new C2 (command & control) server locations to orchestrate attacks.

In response to these advanced tactics, forward-looking security teams are fighting back by combining the strengths of traditional reputation services and static analysis with the power and innovation of dynamic behavioral analysis. The end result is more comprehensive protection against malware threats, greater scalability to handle short-term spikes and long-term growth in malware-related activity, and more effective use of skilled personnel and security infrastructure.

Here’s how this emerging approach—described here by the shorthand ”Now, Near and Deep”—works in concept and in practice.


Reputation Scoring: Detecting known malicious files right NOW

As the first and fastest line of defense against malware, reputation scoring provides a very quick lookup, within milliseconds, of known good and known bad files and URLs. This allows SOC teams to quickly filter out the high volume of benign files that require no further attention, thereby freeing Tier 1 analysts and Tier 2 incident responders to focus on files that exhibit suspicious behavior, are known to be malicious, or are still unknown.


Static Analysis: Detecting active elements that could mean a threat is NEAR

During this second stage—which takes a few seconds—static analysis inspects suspect and unknown files and URLs to identify and extract active elements, such as embedded scripts, links, and macros, which could be part of a multistage malware threat. Files that lack active elements are considered safe and can be dismissed, while files that do have active elements are submitted to the Dynamic Analysis Engine for a deeper investigation. This sorting process greatly reduces the volume of potential threats that need to be scrutinized.


Dynamic Analysis: Digging DEEP to reveal and block malware behavior

In a process that typically takes two to three minutes, automated threat intelligence and detection tools can give SOC teams full visibility into malware behavior by leveraging sandboxing, where malware executes in a safe, controlled environment. Truly comprehensive analysis results will detail all the places malware tried to reach out to, all the files it tried to create, and all the registry keys it tried to touch or modify. This deep dive provides insight into the specific threat or attack, helping determine the best course of action for incident response.


Now, Near, Deep Architecture - VMRay

Figure 1: VMRay’s Now, Near, Deep Architecture


How Now-Near-Deep Works in Practice: Microsoft Word Files

Infected Word documents—distributed as email attachments—are a common vector for initiating attacks. They offer an instructive example of how a multi-layered approach to analysis and detection can identify and block threats that are missed by less comprehensive methods.

Stage 1: At this first stage of malware analysis, infected documents will be checked against the reputation engine to see if the file is already known as malicious.  However, by making slight alterations to existing, known malware, authors can easily embed into a document new, malicious variants that change the file size and resulting hash for the altered malware, thereby evading detection early on.

Stage 2: However, in the second stage of the process–when the infected document is passed on to the static analysis engine–the latter will detect and extract any active elements in the document, such as embedded scripts or URLs that could potentially trigger malicious activity. The file will then be submitted to the malware sandbox for further scrutiny. Documents that are determined to have no active elements can be considered safe, so the email and its attachments can be sent on their way without consuming further system resources.


 VMRay Static Analysis Engine

Figure 2: Detections from VMRay Analyzer’s Static Analysis Engine


Stage 3: During the third stage, dynamic analysis, the malware sandbox monitors interactions between those active elements and the operating system to distinguish between not suspicious and malicious behavior. Two examples of suspect behavior:

  • A macro that attempts to communicate with an unknown external server may be “calling home” to its command-and-control (C2) server to initiate the next stage of an attack.
  • An embedded URL that links to an unknown website may exist for the sole purpose of infecting computers that access that URL, however briefly.



Figure 3: Example of macro attempting to communicate with its C2 Server from VMRay Analyzer’s Dynamic Analysis Engine


Belt, Buttons & Suspenders

In both these cases, a second pass through the reputation engine—which did not pick up any threat on the first pass—would quickly identify whether that specific external server or the unfamiliar website had already been discovered and shut down by a previously targeted organization and blacklisted on threat intelligence feeds. This information would not only signal to responders to fix or block the infected email. It should also prompt them to search across the enterprise for other internal systems attempting to access those malicious resources, indicating how far an attack may have already spread.

As this scenario demonstrates, a Now-Near-Deep strategy is not simply a three-stage, linear filtering process. There’s also a belt-buttons-and-suspenders aspect, where a gap left by one or two of the analytical components will be identified and closed by the third.


Bringing It All Together in One Platform

Over the last year, we at VMRay have seen enterprises implementing some version of a Now-Near-Deep approach, combining two or all three elements of the architecture, typically relying on multiple vendors for delivery. VMRay has taken this trend one step further, integrating all three components into the VMRay Analyzer platform.

At the heart of VMRay’s solution is a groundbreaking malware analysis sandbox that utilizes agentless, hypervisor-based monitoring.  Grounded in years of research by the company’s co-founders, it embodies four differentiating traits:

  • Resistance to evasive malware
  • Full visibility into malware behavior
  • Precise, noise-free output
  • Scalability to support short-term spikes and long-term growth in malware activity.

In early 2017, VMRay introduced a built-in reputation engine that leverages threat intelligence from some of the most trusted providers in this market space. More recently, VMRay added a VMRay-developed static analysis engine, which is winning converts in its own right and as a complement to reputation and dynamic analysis tools.

Taken together, these capabilities empower security teams to handle larger analysis volumes, speed up detection, and improve the productivity and efficacy of security personnel and infrastructure.