Deep Instinct is revolutionizing cybersecurity with its unique Deep learning Software – harnessing the power of deep learning architecture and yielding unprecedented prediction models, designed to face next generation cyber threats.
Back to the Blog

Common Anti-Debugging Techniques in the Malware Landscape

Dec 27, 2017 |

This is the first article in our new series, “The Malware D.Igest”, in which each time a malware expert at Deep Instinct will cover another core topic in the world of malware analysis.

Malware authors have always looked for new techniques to stay invisible. This includes, of course, being invisible on the compromised machine, but it is even more important to hide malicious indicators and behavior during analysis. Malware authors assume that once the malicious file is out in the wild, its lifetime clock is ticking, and eventually, it will be detected by researchers and thoroughly analyzed.

To make the post-detection analysis more difficult, threat actors use various anti-analysis techniques, one of the more common ones is Anti-Debugging. Threat actors have proven to be more innovative not only in the malware they are creating, but also the techniques they are employing in order to evade detection and analysis by cybersecurity analysts and products. Anti-debugging, therefore, poses a hindrance for malware analysts as it can prolong the process of being able to reverse engineer the code and thus make it more difficult to decipher how it works. Once the malware realizes that it is running under a debugger, it can adjust its usual code execution path or modify the code to provoke a crash, which then hinders the analysts’ attempts to decipher it, all the while adding time and additional overhead to their efforts. These techniques pose a threat to companies with limited resources and cybersecurity technologies that are not capable of detecting their existence.

In this blog post, I will present some examples of popular anti-debugging techniques. The techniques covered here constitute an exhaustive list of the core concepts and are based on what the research team here at Deep Instinct encounters on a daily basis.

What is Anti-Debugging?

Debugging malware code enables a malware analyst to run the malware step by step, introduce changes to memory space, variable values, configurations and more. Therefore, if debugging is done successfully, it facilitates the understanding of the malware’s behavior, mechanisms and capabilities. For obvious reasons, this is something malware authors would want to prevent. Anti-Debugging techniques are meant to ensure that a program is not running under a debugger, and in the case that it is, to change its behavior correspondingly. In most cases, the Anti-Debugging process will slow down the process of reverse engineering, but will not prevent it.

PEB Structure

Process Environment Block (PEB) is a data structure that resides in the user space of each process and contains data about the related process. It is designed to be accessed by the WinAPI modules, but access is not limited exclusively to them; one can access the PEB directly from memory, as we will see next:

BeingDebugged – The easiest implementation and obvious technique is to check the value of the BeingDebugged field at PEB structure. There are many API calls (documented and undocumented – isDebuggerPresentCheckRemoteDebuggerPresentNtQueryInformationProcess) to check this value. In order to enhance evasion, it also can be checked manually without a WinAPI call; PEB can be accessed by the fs segment register at fs:[30h] (This register points to TIB (Thread Information Block) in x86 machines).

ProcessHeap – a flag under PEB that shows if the first heap memory space of the process was created in debug mode. It can be accessed at offset 0x18 (at PEB).

Residue in the system

A machine that contains debugging functionalities can be identified without special efforts. Like any other software, a debugger will leave a trace over the machine.

Registry keys – besides the obvious registry locations that most applications write themselves, there is a registry key that is set to the debugger program that should run when an error occurs (“HKLM\Software\Microsoft\Windows NT\CurrentVersion\AeDebug”).

FindWindow – WinAPI function that can recognize the presence of a debugger program (e.g “ollydbg.exe”). The same result can be reached with OpenProcess for processes or with FindFirstFile and FindNextFile for filesystem paths (for example in the %ProgramFiles% folder).

Identifying Breakpoints

A Breakpoint is probably the most important feature of a debugger; it enables the researcher to stop the execution of a program at a certain point of code.

Instructions scanning – A debugger’s breakpoint is implemented by inserting “0xCC” (“INT 3” in assembly) which means the software interrupt with the parameter ‘3’, A malware can scan itself to find this assembly instruction (for example with the assembly instruction “repne scasb”).

Calculate code checksum or hash – malware can calculate a checksum or hash of its code in run time to determine if it was patched or if a breakpoint was inserted (for instance, a value that was described before: ‘0xCC’), and therefore this can be used to identify debuggers as well.

The Debugger’s Environment

There are many techniques to identify running debugger’s context.

OutputDebugString is used to send a string to a debugger. If it fails, an error occurs. Malware can use SetLastError with a defined value, then run OutputDebugString (if it fails, it will overwrite the last error value), then check the last error with GetLastError.

Timing checks – when debugging in a single-step mode / setting a breakpoint, a lag occurs while running the executable. The malware can check for a timestamp and compare it to another one after a few malicious instructions, in order to check if there was a delay. The instruction rdtsc is used to get the number of ticks since the last system reboot and it rolls over after 49.7 days (will be placed as 64bit value in EDX:EAX 32bit registers). There is equivalent WinAPI functions for this test (GetTickCount / QueryPerformanceCounter).

Debugger vulnerabilities – a debugger is a program, and like any other program, it may have vulnerabilities. For instance, a famous one would be: crashing OllyDbg 1.1 by calling OutputDebugString with the value “%s”.

Image File Execution Options (IFEO)

IFEO is a registry key that is intended to run a particular program under a local debugger.

Key: “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\<particular_program>

Value: Debugger: <full-path to a local debugger> (REG_SZ)

A malware can check if there is any debugger configured manually on the machine, although debugging and Anti-Debugging is not the only use for this registry key. It is also easily implemented as a technique to launch malware, since Windows does not perform any check that the “debugger” is indeed a debugger. For this reason, a malware author can abuse it for running malware, for example, by just attaching to another process that is loaded by default with startup.

TLS Callback

TLS (Thread Local Storage) is a memory region that is local to a thread. It can be abused to run a function before the entry point of the PE (i.e in the initialization of a thread) – this technique is called TLS Callback. Mostly, debuggers start debugging from the entry point. This technique can be identified in the following manners:

* Statically by .tls section existence
* IDA Pro supports running from all the entry points (including TLS Callback)
* Debuggers can be configured to set a breakpoint before running TLS Callback
* All TLS callback functions are labeled with “TlsCallback” prepended

To sum up, Anti-Debugging can be reached by the essential features of the operating system or by the natural behavior of the debugger itself. Malware authors keep challenging researchers with new techniques in order to delay the reverse engineering process.

Stay tuned for more content from Deep Instinct’s malware experts.

New Call-to-action