Dec 27, 2017

This is the first article in our new series, “The Malware D.Igest”, in which each time a malware expert of ours will cover another core topic in the world of malware analysis.


Malware authors have always looked for new techniques to stay invisible. This includes, of course, being invisible on the compromised machine, but it is even more important to hide malicious indicators and behavior during analysis. Malware authors assume that once the malicious file is out in the wild, its lifetime clock is ticking and eventually it will be detected by researchers and thoroughly analyzed.

To make post-detection analysis more difficult, threat actors use various anti-analysis techniques, one of the more common ones is Anti-Debugging. In this blog post, I will present some examples of popular anti-debugging techniques. The techniques covered here constitute an exhaustive list of the main concepts and  are based on what the research team here at Deep Instinct encounters on a daily basis.

What is Anti-Debugging?

Debugging malware code enables a malware analyst to run the malware step by step, introduce changes to memory space, variable values, configurations and more. Therefore, if debugging is done successfully, it facilitates the understanding of the malware’s behavior, mechanisms and capabilities. For obvious reasons, this is something malware authors would want to prevent. Anti-Debugging techniques are meant to ensure that a program is not running under a debugger, and in the case that it is, to change its behavior correspondingly. In most cases, the Anti-Debugging process will slow down the process of reverse engineering, but will not prevent it.

PEB Structure

Process Environment Block (PEB) is a data structure which resides in the user space of each process, and contains data about the related process. It is designed to be accessed by the WinAPI modules, but access is not limited exclusively to them; one can access the PEB directly from memory, as we will see next:

BeingDebugged – The easiest implementation and obvious technique is to check the value of the BeingDebugged field at PEB structure. There are many API calls (documented and undocumented – isDebuggerPresentCheckRemoteDebuggerPresentNtQueryInformationProcess) to check this value. In order to enhance evasion, it also can be checked manually without a WinAPI call; PEB can be accessed by the fs segment register at fs:[30h] (This register points to TIB (Thread Information Block) in x86 machines).

ProcessHeap – a flag under PEB that shows if the first heap memory space of the process was created in debug mode. It can be accessed at offset 0x18 (at PEB).

Residue in the system

A machine that contains debugging functionalities can be identified without special efforts. Like any other software, a debugger will leave a trace over the machine.

Registry keys – besides the obvious registry locations that most applications write themselves, there is a registry key that is set to the debugger program that should run when an error occurs (“HKLM\Software\Microsoft\Windows NT\CurrentVersion\AeDebug”).

FindWindow – WinAPI function that can recognize the presence of a debugger program (e.g “ollydbg.exe”). The same result can be reached with OpenProcess for processes, or with FindFirstFile and FindNextFile for filesystem paths (for example in the %ProgramFiles% folder).

Identifying Breakpoints

A Breakpoint is probably the most important feature of a debugger; it enables the researcher to stop the execution of a program at a certain point of code.

Instructions scanning – A debugger’s breakpoint is implemented by inserting “0xCC” (“INT 3” in assembly) which means the software interrupt with the parameter ‘3’, A malware can scan itself to find this assembly instruction (for example with the assembly instruction “repne scasb”).

Calculate code checksum or hash – malware can calculate a checksum or hash of its code in run time to determine if it was patched or if a breakpoint was inserted (for instance, a value that was described before: ‘0xCC’), and therefore this can be used to identify debuggers as well.

The Debugger’s Environment

There are many techniques to identify running debugger’s context.

OutputDebugString is used to send a string to a debugger. If it fails, an error occurs. Malware can use SetLastError with a defined value, then run OutputDebugString (if it fails, it will overwrite the last error value), then check the last error with GetLastError.

Timing checks – when debugging in a single-step mode / setting a breakpoint, a lag occurs while running the executable. The malware can check for a timestamp and compare it to another one after a few malicious instructions, in order to check if there was a delay. The instruction rdtsc is used to get the number of ticks since the last system reboot and it rolls over after 49.7 days (will be placed as 64bit value in EDX:EAX 32bit registers). There is equivalent WinAPI functions for this test (GetTickCount / QueryPerformanceCounter).

Debugger vulnerabilities – a debugger is a program, and like any other program it may have vulnerabilities. For instance, a famous one would be: crashing OllyDbg 1.1 by calling OutputDebugString with the value “%s”.

Image File Execution Options (IFEO)

IFEO is a registry key that is intended to run a particular program under a local debugger.

Key: “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\<particular_program>

Value: Debugger: <full-path to a local debugger> (REG_SZ)

A malware can check if there is any debugger configured manually on the machine, although debugging and Anti-Debugging is not the only use for this registry key. It is also easily implemented as a technique to launch malware, since Windows does not perform any check that the “debugger” is indeed a debugger. For this reason, a malware author can abuse it for running malware, for example, by just attaching to another process that is loaded by default with startup.

TLS Callback

TLS (Thread Local Storage) is a memory region that is local to a thread. It can be abused to run a function before the entry point of the PE (i.e in the initialization of a thread) – this technique is called TLS Callback. Mostly, debuggers start debugging from the entry point. This technique can be identified in the following manners:

* Statically by .tls section existence
* IDA Pro supports running from all the entry points (including TLS Callback)
* Debuggers can be configured to set a breakpoint before running TLS Callback
* All TLS callback functions are labeled with “TlsCallback” prepended

To sum up, Anti-Debugging can be reached by essential features of the operating system or by the natural behavior of the debugger itself. Malware authors keep challenging researchers with new techniques in order to delay the reverse engineering process.

Stay tuned for more content from Deep Instinct’s malware experts.

New Call-to-action