The Need for Memory Dumping
Before deploying Windows malware, financially motivated attackers almost always obfuscate it. The obfuscation method can range from a quick manual obfuscation, to very complex commercial packers. Packing and obfuscation has become such a basic requirement for attackers that they often don’t even need to purchase a packer separately: commercial malware builders targeting Windows often already come with built-in packers, and generate obfuscated malware without any effort from the attacker.
A long time ago, traditional antivirus software used to mitigate packers by maintaining a separate unpacker against each common packer. Over time, packers have become too complex and diverse for this mitigation, and maintaining unpackers one-by-one is not a feasible solution anymore. Over time, security solutions (including antivirus software) have switched to more generic solutions, attempting to unpack in more generic ways and finding malware in memory.
This is where memory dumping becomes useful. The theory is simple: the malware needs to deobfuscate its code and data in order to use it, so we can just let the sample run in a sandbox, wait for it to do the deobfuscation for us, and then dump the code and data from the process memory. From good memory dumps we can reap the benefits of having access to deobfuscated code and data:
- signature-based malware detection and classification becomes possible,
- manual malware analysis becomes less time-consuming,
- malware configuration extraction becomes faster and more reliable.
Mistake #1: Dumping at the Wrong Time
Although the general idea of memory dumping is simple, the implementation is far from it.
One naive implementation of memory dumping is to dump the memory only when the process terminates or the sandbox run times out.
The issue with this is that once malware has executed some code or interpreted some data, it won’t necessarily keep it in memory any longer than necessary. Sometimes this is caused by the packer’s evasiveness, but often it’s just the nature of programming: since the sample doesn’t need the stored bytes anymore, it can free the memory or override it with something else. This happens less for code, but especially often for data, such as strings or malware configuration.
Creating memory dumps at the end of the process is helpful, but often it’s not enough.
Mistake #2: Dumping Too Much
The second naive implementation is to always dump everything: whenever a region is freed or overwritten, create a dump. This can generate gigabytes of useless data even for very simple samples, especially if they are using an underlying framework such as .NET or Java.
Unnecessary memory dumps have a cost on performance and usability.
- The malware process can’t properly run while its memory is being copied, so it’s paused for the time. This means that creating dumps too often slows down the execution of the sample inside the sandbox.
- Automated malware analysis systems create memory dumps so they can process them to detect and classify malware and extract useful data. Creating too large dumps can drastically slow down this processing.
- When using the memory dumps for manual malware analysis, the malware analyst doesn’t want to waste their time going through many memory dumps. In manual malware analysis the purpose of memory dumping is to allow the analyst to skip manual unpacking and speed up the analysis. For the technology to be useful, picking the right memory dumps needs to be significantly faster then just unpacking the sample manually.
VMRay’s Approach: Smart Memory Dump Triggers
For memory dumping to be optimal, the right data needs to be dumped, and as little other data as possible. Many malware analysis sandboxes offer memory dumping, but the implementation and quality of resulting memory dumps can be very different based on the algorithm used to figure out when to create a memory dump.
The VMRay Platform uses a set of triggers to define when to create memory dumps, and a set of limits that make the number of memory dumps manageable. Until now, the triggers were mostly focused on dumping code and mapped executables from memory, by tracking which memory regions are executed, overwritten or freed. To learn more about how we dump code, read our previous blog post on the topic.
The VMRay Platform 4.4 release extends the memory dump triggers with a focus on dumping memory regions which are not marked as executable, but are still likely to contain useful data, such as the malware’s configuration or strings used internally by malware. The configuration is how the malware typically stores its parameters that the attacker defined for the specific executable: what C2 URLs should be connected, which modules should be loaded, what cryptographic keys to use in communication, which process to inject into or where to place the autorun file. Malware does its best to hide and obfuscate this malicious configuration, but during its execution there is still a time when either the full decrypted configuration is in memory, or it can be found by little post-processing. Our new memory dumping triggers attempt to find these times.
Newly Added Triggers
To dump valuable data, we added three new generic triggers, and an option to manually find interesting memory regions with signatures.
The first trigger is related to network connections. When a process first attempts to make a network connection, we dump all of its tracked memory regions and even the stacks of its threads. This is because at the time the first C2 beacon is sent, malware is typically already unpacked in-memory. This trigger is also helpful because malware usually stores the URLs of C2 servers in its configuration, so by the time the first network connection happens it had to at least partially decrypt its configuration. So by dumping the memory at this time, we have a good chance to get either the full decrypted configuration or a state that is very close to it.
Figure 1: Allocated buffers of the sample were dumped when it first attempted to make a network connection. One of these buffers was detected in post-processing by YARA as Raccoon Stealer
The second trigger dumps injected processes more aggressively. Malware often injects into processes to hide from host-based detection, evade application allowlisting or gain access to privileged information within another program via function hooks. Injection can be effective in hiding from host-based detection, but in a sandbox the injection is monitored, and actually gives away that the injected process likely contains malware. To take advantage of this information, when the injected process terminates or the sandbox analysis ends, we create a complete dump of the injected process, including its non-executable regions.
Figure 2: Formbook injected into a process. When the injected process terminated, most of its buffers were dumped. YARA detected Formbook in one of these memory dumps
The third trigger dumps reflectively loaded .NET assemblies. Reflection in .NET is a powerful feature for programmers to inspect and load code at runtime. It is also often used by malware authors to conveniently load and execute an unpacked stage. By understanding how the .NET framework implements reflective loading, we are able to detect it while it happens purely from the hypervisor without modifying anything inside the VM. Besides the executable code, we dump the full loaded .NET assembly, which is often the fully unpacked malware.
Figure 3: Reflectively loaded assembly dumped, YARA detected it as the Redline Stealer
Filling Gaps with Signature-Based Smart Memory Dump Rules
The generic triggers are designed to work well for most malware, but no generic solution can cover every case. This is why together with the new triggers we also added a feature to scan memory during execution with memory dump rules. These rules are created by VMRay Labs, and we will continuously release new memory dump rules together with regular signature updates. This gives VMRay Labs the ability to react very quickly to gaps we find in memory dumping.
Under the hood, memory dump rules are based on YARA, but serve a different purpose than YARA rules designed for detection, have different requirements and are executed differently. For performance, memory dump rules do not decide that the memory region contains something malicious, instead they only detect that the region is worth dumping for later analysis. Further processing and detection happens later during post-processing, when time constrains are looser, and more complex logic can be executed without hurting performance.
Figure 4: Memory dump rule found strings in memory that often appear in Trickbot configurations, so it created a dump. The dump contains the decrypted configuration.
VMRay is very conscious about the performance of analysis. To generate excellent data for post-processing, we need malware to execute as much behavior as possible within the few minutes assigned for analysis. This also means that memory scanning needs to be extremely fast, to the point where usual YARA rules do not fulfill our performance criteria. To achieve the performance we want, we only use a small subset of features YARA makes possible, creating only very fast rules. Each rule also defines whether it should be matched on regions that are being freed, overwritten or on terminating processes. This avoids a lot of unnecessary scans, and keeps performance high.
Conclusion
VMRay Platform 4.4 extended our triggers to dump relevant data used by malware with minimal impact on analysis performance. This results in more malware detection, better family classifications, helping manual malware analysis and paving the way for malware configuration extraction.
Appendix: Samples Used in Examples
dab10e4773c0aa338199b6b595ddcd7cf9e0a17423ca0d488a0130a29048083d (Raccoon)
23062ba932165210ebb3ffcd15474e79f19e6ad74869ff43923a0795b5072ccd (Redline loader)
6810f77478c037f70109dfb99877a6dd3bd80496ee7cc304027f2803a2209ab5 (Formbook)
c7c8353673369dcd293b0160fd35a9143ceb4ee98ebb69d8617b596b5fd5244d (Trickbot)