Background
Endpoint Detection and Response (EDR) solutions have become essential in modern cybersecurity, providing constant monitoring of an organization’s endpoints—computers and servers. These tools track and analyze endpoint data, helping identify and mitigate threats in real-time. However, as cyberattacks grow more sophisticated, attackers are constantly finding ways to bypass or evade EDR monitoring.
A major vulnerability for many EDR solutions is their reliance on monitoring API calls, which track process actions on endpoints. To avoid detection, attackers often turn to one of the most effective evasion techniques: direct syscalls.
This post will explain the mechanism behind direct syscalls, how attackers use them to bypass EDRs, and how Cortex XDR’s kernel-mode monitoring is able to detect these stealthy attacks.
Windows System Call Flow
Before discussing direct syscalls, it’s important to understand how a typical system call (syscall) works in Windows.
A system call is an instruction that allows a user-mode application to request a service from the kernel, such as interacting with hardware, reading a file, or sending data over a network. Syscalls are initiated by invoking functions found in system libraries, such as ntdll.dll or win32u.dll.
In a conventional Windows syscall flow:
- A user-mode application makes a call to a high-level API like
CreateFileWinkernel32.dll. - This API function calls a lower-level function in
kernelbase.dll. - Eventually, the syscall reaches the Native API
NtCreateFileinntdll.dll, where the actual syscall takes place.
This process involves a system call stub that executes the syscall instruction, which then communicates with the kernel to perform the requested task.
How Most EDRs Monitor API Calls
Most EDR solutions monitor system calls and API calls by hooking into the Windows DLLs. One common technique is inline hooking, which intercepts calls to certain functions. Inline hooking modifies the prologue of functions like CreateFileW to redirect execution to an EDR proxy function. This proxy can then analyze or block the operation.
For example, the inline hook might redirect the execution flow from CreateFileW to an EDR function, which in turn calls the original function and allows the EDR to monitor the action.
How Attackers Bypass Most EDRs
Although EDR solutions effectively use inline hooking to monitor API calls, attackers have found ways to bypass these hooks. Many EDR bypass techniques involve manipulating the user-mode hooks that EDRs rely on, such as:
- Manual DLL Loading: Attackers can load a clean, unhooked DLL by using Reflective DLL Loading. This bypasses the EDR hooks by loading the DLL directly into memory, avoiding the hooks installed by EDRs.
- Cloning DLLs: By copying a hooked DLL to a new location and loading it under a different name, attackers can avoid detection since the EDR does not hook the new DLL.
- Direct Syscall: The most effective and stealthy technique is the direct syscall. By bypassing the user-mode hooks entirely, attackers can invoke syscalls directly from their own code, preventing the EDR from monitoring these actions. Instead of going through the Windows API and
ntdll.dll, attackers implement the syscall directly in their application using the appropriate syscall index. In this case, attackers call the syscall instruction with the correct index and parameters, transferring control directly to the kernel for execution. This makes it much harder for EDRs to detect malicious behavior.
Cortex XDR and Direct Syscall Detection
Unlike most EDR solutions, Cortex XDR employs kernel-mode syscall interception, which provides deeper visibility and greater resistance to user-mode hook bypass techniques. Cortex XDR uses the KTRAP_FRAME structure, which saves the state of the system during a syscall, including user-mode context and registers. This allows Cortex XDR to track syscalls directly from the kernel, even when the user-mode hooks are bypassed.
For each intercepted syscall, Cortex XDR extracts the RIP (return instruction pointer) from the KTRAP_FRAME and resolves it to the corresponding user-mode module using a component called ImageTracker. If the return address is not associated with ntdll.dll or win32u.dll (as is the case with direct syscalls), Cortex XDR flags the event and investigates further.
Are Direct Syscalls Malicious?
While direct syscalls can indeed be a sign of malicious activity, they are not inherently malicious. Legitimate software, including security products, gaming anti-cheat software, and Chromium-based applications, also use direct syscalls for legitimate purposes. This makes it difficult to differentiate between benign and malicious uses of direct syscalls based on individual events.
How Cortex XDR’s Analytics Engine Detects Malicious Direct Syscalls
To address the challenge of distinguishing legitimate from malicious direct syscalls, Cortex XDR employs its Analytics Engine. This learning mechanism aggregates real-time data from Cortex XDR agents to establish baselines for “normal” behavior. The engine uses these baselines to detect anomalous activity that deviates from established patterns.
The Analytics Engine answers key questions such as:
- How common is this direct syscall for the current environment?
- Is this process known to execute direct syscalls regularly?
- How often does a specific memory location trigger a direct syscall?
By analyzing these behaviors and aggregating local and global data, the Analytics Engine can detect unusual or malicious behavior with high confidence.
Real-Life Example – Lumma Stealer
A great example of how Cortex XDR detects direct syscalls in action is the Lumma Stealer malware. Lumma Stealer, an info-stealing tool, targets cryptocurrency wallets and 2FA browser extensions. Once executed, it loads shellcode into the mshtml.dll library and directly invokes system calls by exploiting direct syscall techniques. The malware uses the ntdll.dll library to extract the correct syscall index and then performs its attack by bypassing EDR monitoring.
Cortex XDR, using its advanced syscall detection and Analytics Engine, was able to flag the unusual behavior triggered by Lumma Stealer’s direct syscalls, enabling swift detection and response.
Conclusion
Direct syscalls represent a potent technique for attackers aiming to bypass traditional EDR solutions. By operating at the kernel level and avoiding user-mode hooks, they remain hidden from many monitoring tools. However, with Cortex XDR’s kernel-mode syscall detection and the power of the Analytics Engine, malicious direct syscalls can be detected even when conventional EDR methods fail. By leveraging advanced techniques such as KTRAP_FRAME analysis and real-time behavior profiling, Cortex XDR ensures that threats like Lumma Stealer are swiftly identified and neutralized.
For organizations looking to bolster their defenses against increasingly sophisticated attacks, embracing solutions that provide visibility at the kernel level, like Cortex XDR, is a critical step.
Cybersecurity for Business
Your business faces constantly evolving cyber threats that can jeopardize sensitive data, disrupt operations, and damage your reputation. Our cybersecurity for business solutions are tailored to meet the unique challenges of companies of all sizes, providing robust protection against malware, phishing, ransomware, and more.
Whether you’re a small startup or a large enterprise, we offer multi-license cybersecurity packages that ensure seamless protection for your entire team, across all devices. With advanced features like real-time threat monitoring, endpoint security, and secure data encryption, you can focus on growing your business while we handle your digital security needs.
Get a Free Quote Today! Safeguard your business with affordable and scalable solutions. Contact us now to request a free quote for multi-license cybersecurity packages designed to keep your company safe and compliant. Don’t wait—protect your business before threats strike!
