Overview
In this post, IBM Security X-Force Red offensive hackers analyze how attackers, with elevated privileges, can use their access to stage Windows Kernel post-exploitation capabilities. Over the last few years, public accounts have increasingly shown that less sophisticated attackers are using this technique to achieve their objectives. It is therefore important that we put a spotlight on this capability and learn more about its potential impact. Specifically, in this post, we will evaluate how Kernel post-exploitation can be used to blind ETW sensors and tie that back to malware samples identified in-the-wild last year.
Intro
Over time, security mitigations and detection telemetry on Windows have improved substantially. When these capabilities are combined with well-configured Endpoint Detection & Response (EDR) solutions, they can represent a non-trivial barrier to post-exploitation. Attackers face a constant cost to develop and iterate on tactics, techniques, and procedures (TTPs) to avoid detection heuristics. On the Adversary Simulation team at IBM Security X-Force, we face this same issue. Our team is tasked with simulating advanced threat capabilities in some of the largest and most hardened environments. The combination of complex fine-tuned security solutions and well-trained Security Operations Center (SOC) teams can be very taxing on tradecraft. In some cases, the use of a specific TTP is made completely obsolete in the span of three to four months (usually tied to specific technology stacks).
Attackers may choose to leverage code execution in the Windows Kernel to tamper with some of these protections or to avoid a number of user-land sensors entirely. The first published demonstration of such a capability was in 1999 in Phrack Magazine. In the intervening years there have been a number of reported cases where Threat Actors (TAs) have used Kernel rootkits for post-exploitation. Some older examples include the Derusbi Family and the Lamberts Toolkit.
Traditionally these types of capabilities have mostly been limited to advanced TAs. In recent years, however, we have seen more commodity attackers use Bring Your Own Vulnerable Driver (BYOVD) exploitation primitives to facilitate actions on endpoint. In some instances, these techniques have been quite primitive, limited to simple tasks, but there have also been more capable demonstrations.
At the end of September 2022, researches from ESET released a white-paper about such a Kernel capability used by the Lazarus TA in a number of attacks against entities in Belgium and the Netherlands for the purpose of data exfiltration. This paper lays out a number of Direct Kernel Object Manipulation (DKOM) primitives that the payload uses to blind OS / AV / EDR telemetry. The available public research on these techniques is sparse. Gaining a more thorough understanding of Kernel post-exploitation tradecraft is critical for defense. A classic, naïve, argument often heard is that an attacker with elevated privileges can do anything so why should we model capabilities in that scenario? This is a weak stance. Defenders need to understand what capabilities an attacker has when they are elevated, which data sources remain reliable (and which don’t), what containment options exist and how advanced techniques could be detected (even if capabilities to perform those detections don’t exist). In this post I will focus specifically on patching Kernel Event Tracing for Windows (ETW) structures to render providers either ineffective or inoperable. I will provide some background on this technique, analyze how an attacker may manipulate Kernel ETW structures, and get into some of the mechanics of finding these structures. Finally, I will review how this technique was implemented by Lazarus in their payload.
ETW DKOM
ETW is a high-speed tracing facility built into the Windows operating system. It enables logging of events and system activities by applications, drivers, and the operating system, providing detailed visibility into system behavior for debugging, performance analysis, and security diagnostics.
In this section, I will give a high-level overview of Kernel ETW and its associated attack surface. This will be helpful to have a better understanding of the mechanics involved in manipulating ETW providers and the associated effects of those manipulations.
Kernel ETW Attack Surface
Researchers from Binarly gave a talk at BHEU 2021, which discussed the general attack surface of ETW on Windows. An overview of the threat model is pictured below.
Figure 1 – Veni, No Vidi, No Vici: Attacks on ETW Blind EDR Sensors (Binarly)
In this post, we focus on the Kernel space attack surface.
Figure 2 – Veni, No Vidi, No Vici: Attacks on ETW Blind EDR Sensors (Binarly)
This post considers only attacks within the first attack category shown in “Figure 2”, where tracing is either disabled or altered in some way.
As a cautionary note, when considering opaque structures on Windows it is always important to remember that these are subject to change, and in fact frequently do change across Windows versions. This is especially important when clobbering Kernel data, as mistakes will likely result in a Blue Screen of Death (BSoD), roll safe!
Initialization
Kernel providers are registered using nt!EtwRegister, a function exported by ntoskrnl. A decompiled version of the function can be seen below.
Figure 3 – nt!EtwRegister decompilation
Full initialization happens within the inner EtwpRegisterKMProvider function but there are two main takeaways here:
- The ProviderId is a pointer to a 16-byte GUID. This GUID is static across operating systems so it can be used to identify the provider that is being initialized.
- The RegHandle is a memory address that receives a pointer to an _ETW_REG_ENTRY structure on a successful call. This data structure and some of its nested properties provide avenues to manipulate the ETW provider as per the research from Binarly.
Let’s briefly list out the structures that Binarly highlighted on their slide in Figure 2.
ETW_REG_ENTRY
A full 64-bit listing of the _ETW_REG_ENTRY structure is shown below. Added details are available on Geoff Chappell’s blog here. This structure can also be further explored on the Vergilius Project.
// 0x70 bytes (sizeof)
// Win11 22H2 10.0.22621.382
struct _ETW_REG_ENTRY
{
struct _LIST_ENTRY RegList; //0x0
struct _LIST_ENTRY GroupRegList; //0x10
struct _ETW_GUID_ENTRY* GuidEntry; //0x20
struct _ETW_GUID_ENTRY* GroupEntry; //0x28
union
{
struct _ETW_REPLY_QUEUE* ReplyQueue; //0x30
struct _ETW_QUEUE_ENTRY* ReplySlot[4]; //0x30
struct
{
VOID* Caller; //0x30
ULONG SessionId; //0x38
};
};
union
{
struct _EPROCESS* Process; //0x50
VOID* CallbackContext; //0x50
};
VOID* Callback; //0x58
USHORT Index; //0x60
union
{
USHORT Flags; //0x62
struct
{
USHORT DbgKernelRegistration:1; //0x62
USHORT DbgUserRegistration:1; //0x62
USHORT DbgReplyRegistration:1; //0x62
USHORT DbgClassicRegistration:1; //0x62
USHORT DbgSessionSpaceRegistration:1; //0x62
USHORT DbgModernRegistration:1; //0x62
USHORT DbgClosed:1; //0x62
USHORT DbgInserted:1; //0x62
USHORT DbgWow64:1; //0x62
USHORT DbgUseDescriptorType:1; //0x62
USHORT DbgDropProviderTraits:1; //0x62
};
};
UCHAR EnableMask; //0x64
UCHAR GroupEnableMask; //0x65
UCHAR HostEnableMask; //0x66
UCHAR HostGroupEnableMask; //0x67
struct _ETW_PROVIDER_TRAITS* Traits; //0x68
};
ETW_GUID_ENTRY
One of the nested entries within _ETW_REG_ENTRY is GuidEntry, which is an _ETW_GUID_ENTRY structure. More information about this undocumented structure can be found on Geoff Chappell’s blog here and on the Vergilius Project.
// 0x1a8 bytes (sizeof)
// Win11 22H2 10.0.22621.382
struct _ETW_GUID_ENTRY
{
struct _LIST_ENTRY GuidList; //0x0
struct _LIST_ENTRY SiloGuidList; //0x10
volatile LONGLONG RefCount; //0x20
struct _GUID Guid; //0x28
struct _LIST_ENTRY RegListHead; //0x38
VOID* SecurityDescriptor; //0x48
union
{
struct _ETW_LAST_ENABLE_INFO LastEnable; //0x50
ULONGLONG MatchId; //0x50
};
struct _TRACE_ENABLE_INFO ProviderEnableInfo; //0x60
struct _TRACE_ENABLE_INFO EnableInfo[8]; //0x80
struct _ETW_FILTER_HEADER* FilterData; //0x180
struct _ETW_SILODRIVERSTATE* SiloState; //0x188
struct _ETW_GUID_ENTRY* HostEntry; //0x190
struct _EX_PUSH_LOCK Lock; //0x198
struct _ETHREAD* LockOwner; //0x1a0
};
TRACE_ENABLE_INFO
Finally, one of the nested entries within _ETW_GUID_ENTRY is ProviderEnableInfo which is a _TRACE_ENABLE_INFO structure. For more information about the elements of this data structure, you can refer to Microsoft’s official documentation and the Vergilius Project. The settings in this structure directly affect the operation and capabilities of the provider.
// 0x20 bytes (sizeof)
// Win11 22H2 10.0.22621.382
struct _TRACE_ENABLE_INFO
{
ULONG IsEnabled; //0x0
UCHAR Level; //0x4
UCHAR Reserved1; //0x5
USHORT LoggerId; //0x6
ULONG EnableProperty; //0x8
ULONG Reserved2; //0xc
ULONGLONG MatchAnyKeyword; //0x10
ULONGLONG MatchAllKeyword; //0x18
};
Understanding Registration Handle Usage
While some theoretical background is good, it is always best to look at concrete example usage to gain a deeper understanding of a topic. Let us briefly consider an example. Most critical Kernel ETW providers are initialized within, nt!EtwpInitialize, which is not exported. Looking within this function reveals about fifteen providers.
Figure 4 – nt!EtwpInitialize partial decompilation
Taking the Microsoft-Windows-Threat-Intelligence (EtwTi) entry as an example, we can check the global ThreatIntProviderGuid parameter to recover the GUID for this provider.
Figure 5 – EtwTi Provider GUID
Searching this GUID online will immediately reveal that we were able to recover the correct value (f4e1897c-bb5d-5668-f1d8-040f4d8dd344).
Let’s look at an instance where the registration handle parameter, EtwThreatIntProvRegHandle, is used and analyze how it is used. One place where the handle is referenced is nt!EtwTiLogDriverObjectUnLoad. From the name of this function, we can intuit that it is meant to generate events when a driver object is unloaded by the Kernel.
Figure 6 – nt!EtwTiLogDriverUnload decompilation
The nt!EtwEventEnabled and nt!EtwProviderEnabled functions are both called here passing in the registration handle as one of the arguments. Let’s look at one of these sub-functions to understand more about what is going on.
Figure 7 – nt!EtwProviderEnable decompilation
Admittedly this is a bit difficult to follow. However, the pointer arithmetic is not especially important. Instead, let’s focus on how this function processes the registration handle. It appears that the function validates a number of properties of the _ETW_REG_ENTRY structure and its sub-structures such as the GuidEntry property.
struct _ETW_REG_ENTRY
{
…
struct _ETW_GUID_ENTRY* GuidEntry; //0x20
…
}
And the GuidEntry->ProviderEnableInfo property.
struct _ETW_GUID_ENTRY
{
…
struct _TRACE_ENABLE_INFO ProviderEnableInfo; //0x60
…
}
The function then goes into similar level-based checks. Finally, the function returns true or false to indicate if a provider is enabled for event logging at a specified level and keyword. More details are available using Microsoft’s official documentation.
We can see that when a provider is accessed through its registration handle the integrity of those structures become very important to the operation of the provider. Conversely, if an attacker was able to manipulate those structures, they could influence the control flow of the caller to drop or eliminate events from being recorded.
Attacking Registration Handles
Looking back at Binarly’s stated attack surface and leaning on our light analysis, we can posit some strategies to disrupt event collection.
- An attacker can NULL the _ETW_REG_ENTRY pointer. Any functions referencing the registration handle would then assume that the provider had not been initialized.
- An attacker can NULL the _ETW_REG_ENTRY->GuidEntry->ProviderEnableInfo pointer. This should effectively disable the provider’s collection capabilities as ProviderEnableInfo is a pointer to a _TRACE_ENABLE_INFO structure which outlines how the provider is supposed to operate.
- An attacker can overwrite properties of the _ETW_REG_ENTRY->GuidEntry->ProviderEnableInfo data structure to tamper with the configuration of the provider.
- IsEnabled: Set to 1 to enable receiving events from the provider or to adjust the settings used when receiving events from the provider. Set to 0 to disable receiving events from the provider.
- Level: A value that indicates the maximum level of events that you want the provider to write. The provider typically writes an event if the event’s level is less than or equal to this value, in addition to meeting the MatchAnyKeyword and MatchAllKeyword criteria.
- MatchAnyKeyword: 64-bit bitmask of keywords that determine the categories of events that you want the provider to write. The provider typically writes an event if the event’s keyword bits match any of the bits set in this value or if the event has no keyword bits set, in addition to meeting the Level and MatchAllKeyword criteria.
- MatchAllKeyword: 64-bit bitmask of keywords that restricts the events that you want the provider to write. The provider typically writes an event if the event’s keyword bits match all of the bits set in this value or if the event has no keyword bits set, in addition to meeting the Level and MatchAnyKeyword criteria.
Kernel Search Tradecraft
We have a good idea now of what a DKOM attack on ETW looks like. Let’s assume that the attacker has a vulnerability that grants a Kernel Read / Write primitive, as the Lazarus malware does in this case by loading a vulnerable driver. What is missing is a way to find these registration handles.
I will outline two main techniques to find these handles and show the variant of one that is used by Lazarus in their Kernel payload.
Medium Integrity Level (MedIL) KASLR Bypass
First, it may be prudent to explain that while there is Kernel ASLR, this is not a security boundary for local attackers if they can execute code at MedIL or higher. There are many ways to leak Kernel pointers that are only restricted in sandbox or LowIL scenarios. For some background you can have a look at I Got 99 Problems But a Kernel Pointer Ain’t One by Alex Ionescu, many of these techniques are still applicable today.
The tool of choice here is ntdll!NtQuerySystemInformation with the SystemModuleInformation class:
internal static UInt32 SystemModuleInformation = 0xB;
[DllImport(“ntdll.dll”)]
internal static extern UInt32 NtQuerySystemInformation(
UInt32 SystemInformationClass,
IntPtr SystemInformation,
UInt32 SystemInformationLength,
ref UInt32 ReturnLength);
This function returns the live base address of all modules loaded in Kernel space. At that point, it is possible to parse those modules on disk and convert raw file offsets to relative virtual addresses and vice versa.
public static UInt64 RvaToFileOffset(UInt64 rva, List<SearchTypeData.IMAGE_SECTION_HEADER> sections)
{
foreach (SearchTypeData.IMAGE_SECTION_HEADER section in sections)
{
if (rva >= section.VirtualAddress && rva < section.VirtualAddress + section.VirtualSize)
{
return (rva – section.VirtualAddress + section.PtrToRawData);
}
}
return 0;
}
public static UInt64 FileOffsetToRVA(UInt64 fileOffset, List<SearchTypeData.IMAGE_SECTION_HEADER> sections)
{
foreach (SearchTypeData.IMAGE_SECTION_HEADER section in sections)
{
if (fileOffset >= section.PtrToRawData && fileOffset < (section.PtrToRawData + section.SizeOfRawData))
{
return (fileOffset – section.PtrToRawData) + section.VirtualAddress;
}
}
return 0;
}
An attacker can also load these modules into their user-land process using standard load library API calls (e.g., ntdll!LdrLoadDll). Doing so would avoid complications of converting file offsets to RVA’s and back. However, from an operational security (OpSec) point of view this is not ideal as it can generate more detection telemetry.
Method 1: Gadget Chains
Where possible, this is the technique that I prefer because it makes leaks more portable across module versions because they are less affected by patch changes. The downside is that you are reliant on a gadget chains existing for the object you want to leak.
Considering ETW registration handles, let’s take Microsoft-Windows-Threat-Intelligence as an example. Below you can see the full call to nt!EtwRegister.
Figure 8 – nt!EtwRegister full CALL disassembly
Here we want to leak the pointer to the registration handle, EtwThreatIntProvRegHandle. As seen loaded into param_4 on the first line of Figure 8. This pointer resolves to a global within the .data section of the Kernel module. Since this call occurs in an un-exported function, we are not able to leak its address directly. Instead, we have to look where this global is referenced and see if it is used in a function whose address are able to leak.
Figure 9 – nt!EtwThreatIntProvRegHandle references
Exploring some of these entries quickly reveals a candidate in nt!KeInsertQueueApc.
Figure 10 – nt!KeInsertQueueApc partial decompilation
This is a great candidate for a few reasons:
- nt!KeInsertQueueApc is an exported function. This means we can leak its live address using a KASLR bypass. Then we can use our Kernel vulnerability to read data at that address.
- The global is used at the start of the function. This is very helpful because it means we most likely won’t need to construct complex instruction parsing logic to find it.
Looking at the assembly shows the following layout.
Figure 11 – nt!KeInsertQueueApc partial disassembly
Leaking this registration handle then becomes straightforward. We read out an array of bytes using our vulnerability, and search for the first mov R10 instruction to calculate the relative virtual offset of the global variable. The calculation would be something like this:
Int32 pOffset = Marshal.ReadInt32((IntPtr)(pBuff.ToInt64() + i + 3));
hEtwTi = (IntPtr)(pOffset + i + 7 + oKeInsertQueueApc.pAddress.ToInt64());
With the registration handle, it is then possible to access the _ETW_REG_ENTRY data structure.
In general, such gadget chains can be used to leak a variety of Kernel data structures. However, it is worth pointing out that it is not always possible to find such gadget chains and sometimes gadget chains may have multiple complex stages. For example, a possible gadget chain to leak page directory entry (PDE) constants could look like this.
MmUnloadSystemImage -> MiUnloadSystemImage -> MiGetPdeAddress
In fact, a cursory analysis of ETW registration handles revealed that most do not have suitable gadget chains which can be used as described above.
Method 2: Memory Scanning
The other main option to leak these ETW registration handles is to use memory scanning, either from live Kernel memory or from a module on disk. Remember that when scanning modules on disk it is possible to convert file offsets to RVAs.
This approach consists of identifying unique byte patterns, scanning for those patterns, and finally performing some operations at offsets of the pattern match. Let’s take another look at nt!EtwpInitialize to understand this better:
Figure 12 – nt!EtwpInitialize partial decompilation
All fifteen of the calls to nt!EtwRegister are mostly bunched together in this function. The main strategy here is to find a unique pattern that appears before the first call to nt!EtwRegister and a second pattern that appears after the last call to nt!EtwRegister. This is not too complex. One trick that can be used to improve portability is to create a pattern scanner that is able to handle wild card byte strings. This is a task left to the reader.
Once a start and stop index have been identified, it is possible to look at all the instructions in-between.
- Potential CALL instructions can be identified based on the opcode for CALL which is 0xe8.
- Subsequently, a DWORD sized read is used to calculate the relative offset of the potential CALL instruction.
- This offset is then added to the relative address of the CALL and incremented by five (the size of the assembly instruction).
- Finally, this new value can be compared to nt!EtwRegister to find all valid CALL locations.
Once all CALL instructions have been found it is possible to search backward and extract the function arguments, first the GUID that identifies the ETW provider and second, the address of the registration handle. With this information in hand we are able to perform informed DKOM attacks on the registration handles to affect the operation of the identified providers.
Lazarus ETW Patching
I obtained a sample of the FudModle DLL mentioned in the ESET whitepaper and analyzed it. This DLL loads a signed vulnerable Dell driver (from an inline XOR encoded resource) and then pilots the driver to patch many Kernel structures in order to limit telemetry on the host.
Figure 13 – Lazarus FudModule hash
As the final part of this post, I want to review the strategy that Lazarus uses to find Kernel ETW registration handles. It is a variation on the scanning method we discussed above.
At the start of the search function, Lazarus resolves nt!EtwRegister and uses this address to start the scan.
Figure 14 – Lazarus FudModule partial ETW search decompilation
This decision is a bit strange because it relies on where that function exists in relation to where the function gets called. The relative position of a function in a module may vary from version to version since new code may be introduced, removed, or altered. However, because of the way modules are compiled, it is expected that functions maintain a relatively stable order. One assumes this is a search speed optimization.
When looking for references to nt!EtwRegister in ntoskrnl it appears that not many entries are missed using this technique. Lazarus may also have performed additional analysis to determine that the missed entries are not important or otherwise don’t need to be patched. The missed entries are highlighted below. Employing this strategy allows Lazarus to skip 0x7b1de0 bytes while performing the scan which may be a non-trivial amount if the scanner is slow.
Figure 15 – Instances of calls to nt!EtwRegister
Additionally, when starting the scan, the first five matches are skipped before starting to record registration handles. Part of the search function is shown below.
Figure 16 – Lazarus FudModule partial ETW search decompilation
The code is a bit obtuse, but we get the plot highlights. The code looks for calls to nt!EtwRegister, extracts the registration handle, converts this handle to the live address using a KASLR bypass, and stores the pointer in an array set aside for this purpose within a malware configuration structure (allocated on initialization).
Finally, let’s have a look at what Lazarus does to disable these providers.
Figure 17 – Lazarus FudModule NULL ETW registration handles
This mostly makes sense, what Lazarus does here is leak the global variable we saw earlier and then overwrite the pointer at that address with NULL. This effectively erases the reference to the _ETW_REG_ENTRY data structure if it exists.
I am not completely happy with the tradecraft shown for a few reasons:
- The payload does not capture provider GUID’s so it can’t make any intelligent decisions as to whether it should or should not overwrite the provider registration handle.
- The decision to start scanning at an offset inside ntoskrnl seems questionable because the offset of the scan may vary depending on the version of ntoskrnl.
- Arbitrarily skipping the first 5 matches seems equally questionable. There may be strategic reasons for this decision but a better approach is to first collect all providers and then use some programmatic logic to filter the results.
- Overwriting the pointer to _ETW_REG_ENTRY should work but this technique is a bit obvious. It would be better to overwrite properties of _ETW_REG_ENTRY or _ETW_GUID_ENTRY or _TRACE_ENABLE_INFO.
I re-implemented this technique for science; however, I made some adjustments to the tradecraft.
- A speed optimized search algorithm is used to find all 0xe8 bytes in ntoskrnl.
- Afterward, some post-processing is done to determine which of those are valid CALL instructions and their respective destinations.
- Not all calls to nt!EtwRegister are useful because sometimes the function is called with a dynamic argument for the registration handle. Because of this, some extra logic is needed to filter the remaining calls.
- Finally, all GUID’s are resolved to their human readable form and the registration handles are enumerated.
Overall, after adjustments, the above technique is clearly the best way to perform this type of enumeration. Since search time is negligible with optimized algorithms, it makes sense to scan the entire module on disk and then use some additional post-scan logic to filter out results.
ETW DKOM Impact
It is prudent to briefly evaluate how impactful such an attack could be. When provider data is reduced or eliminated entirely there is a loss of information, but at the same time not all providers signal security-sensitive events.
Some subset of these providers, however, are security-sensitive. The most obvious example of this is Microsoft-Windows-Threat-Intelligence (EtwTi) which is a core data source for Microsoft Defender Advanced Threat Protection (MDATP) which is now called Defender for Endpoint (it’s all very confusing). It should be noted that access to this provider is heavily restricted, only Early Launch Anti Malware (ELAM) drivers are able to register to this provider. Equally, user-land processes receiving these events must have a protected status (ProtectedLight / Antimalware) and be signed with the same certificate as the ELAM driver.
Using EtwExplorer it is possible to get a better idea of what types of information this provider can signal.
Figure 18 – ETW Explorer
The XML manifest is too large to include here in its entirety, but one event is shown below to give an idea of the types of data which can be suppressed using DKOM.
Figure 19 – EtwTi partial XML manifest
Conclusion
The Kernel has been and continues to be an important, contested, area where Microsoft and third-party providers need to make efforts to safeguard the integrity of the operating system. Data corruption in the Kernel is not only a feature of post-exploitation but also a central component in Kernel exploit development. Microsoft has made a lot of progress in this area already with the introduction of Virtualization Based Security (VBS) and one of its components like Kernel Data Protection (KDP).
Consumers of the Windows operating system, in turn, need to ensure that they take advantage of these advances to impose as much cost as possible on would-be attackers. Windows Defender Application Control (WDAC) can be used to ensure VBS safeguards are in place and that policies exist which prohibit loading potentially dangerous drivers.
These efforts are all the more important as we increasingly see commodity TAs leverage BYOVD attacks to perform DKOM in Kernel space.
Additional References
- Veni, No Vidi, No Vici: Attacks on ETW Blind EDR Sensors (BHEU 2021 Slides) – here
- Veni, No Vidi, No Vici: Attacks on ETW Blind EDR Sensors (BHEU 2021 Video) – here
- Advancing Windows Security (BlueHat Shanghai 2019) – here
- Exploiting a “Simple” Vulnerability – In 35 Easy Steps or Less! – here
- Exploiting a “Simple” Vulnerability – Part 1.5 – The Info Leak – here
- Introduction to Threat Intelligence ETW – here
- TelemetrySourcerer – here
- Data Only Attack: Neutralizing EtwTi Provider – here
- WDAC Policy Wizard – here
Learn more about X-Force Red here. Schedule a no-cost consult with X-Force here.
The post Direct Kernel Object Manipulation (DKOM) Attacks on ETW Providers appeared first on Security Intelligence.