Malware analysts spend a significant amount of time deciding which signals from a sandbox run are worth keeping. When a sample is executed in a controlled environment, it can generate hundreds of measurable attributes covering file structure, registry edits, process behavior, and network traffic. Most of those attributes, however, add noise rather than clarity. A recent study works through this problem in detail, and the part that earns attention from working defenders is the feature selection process, not the deep learning model attached to it. The findings provide a structured approach to distinguishing Trojan activity from other types of malware and benign software, offering a practical checklist for threat hunting, EDR tuning, and detection rule writing.

What the study set out to do

The research team built a detection framework specifically for Windows-based IoT and industrial IoT gateways. These environments are often resource-constrained and run a mix of legacy and modern software, making them attractive targets for Trojan attacks that aim to establish persistent access, exfiltrate data, or serve as a foothold for lateral movement. To develop a robust detection method, the team assembled a dataset of 3,000 Windows executables. Each executable was run through the ANY.RUN sandbox, and behavioral, static, and network-level data were recorded for every sample. The samples were labeled as benign, suspicious, or malicious based on known threat intelligence and manual analysis. From the raw sandbox output, an initial pool of 146 features was extracted. Through a combination of domain expertise and automated feature selection techniques, this pool was reduced to a working set of just 33 features. A custom neural network—called TrDNN—was then trained to classify the samples, and its performance was compared against ten common machine learning and deep learning models, including random forests, support vector machines, and gradient boosting classifiers.

The classification results were strong, with high accuracy, precision, and recall. However, for a cybersecurity professional, the more useful material lies in how the 33 features were chosen and what those features reveal about current Trojan tradecraft. The study’s methodology demonstrates that careful feature engineering can yield a detection pipeline that is both effective and interpretable, even when the underlying model is relatively simple.

The feature set reads like a Trojan playbook

The retained features map directly to the stages of a Trojan compromise lifecycle. Understanding these stages helps defenders anticipate attacker actions and design detection rules that catch malicious behavior at multiple points in the kill chain.

Persistence mechanisms

Trojan malware typically needs to survive reboots and remain active on the system. The study identified several persistence indicators that were strong discriminators. These include registry autorun keys (such as those under Run and RunOnce), scheduled tasks created or modified by the sample, Windows service installation, and edits to the startup folder. These are classic persistence techniques, but their presence in combination with other signals provides a high-confidence indicator of Trojan activity.

Execution and evasion techniques

To execute payloads and avoid detection, Trojans often employ process injection into trusted processes like explorer.exe or svchost.exe. The study flagged such injection attempts, along with suspicious memory-allocation calls (e.g., VirtualAllocEx with write/execute permissions), hidden-window execution (using the SW_HIDE flag in CreateProcess), and User Account Control (UAC) tampering. These behaviors indicate attempts to run code stealthily or bypass privilege controls.

Command-and-control communication

Network-level signals were crucial for identifying active C2 activity. The researchers looked for low-jitter beaconing intervals—where the time between connections is unusually regular—and HTTP POST and PUT patterns that suggest data exfiltration. Encrypted outbound bursts and traffic concentrated on a small number of external endpoints also stood out. These patterns differ from normal web browsing or software updates, which tend to involve more diverse destinations and variable timing.

Binary-level indicators

Static features related to the executable file itself also proved valuable. PE header anomalies—such as unusual section names, alignment values, or checksum mismatches—were flagged. High section entropy (indicating packed or encrypted code) and the presence of unsigned executables in sensitive system directories (like System32 or SysWOW64) were additional strong signals. These static features are particularly useful because they can be detected before the code executes, enabling pre-emptive blocking.

What was excluded—and why it matters

The study’s explicit exclusions are equally informative. The team dropped features such as privilege-token manipulation, generic HTTP communication chains, and abuse of living-off-the-land binaries (LOLBins) like PowerShell, regsvr32, and wmic. At first glance, this seems counterintuitive because these behaviors are notoriously associated with malware. However, the researchers reasoned that these behaviors carry real weight in an investigation but are not specific enough to separate Trojans from other threat categories. Privilege escalation appears in ransomware and wiper attacks, generic HTTP traffic is used by worms and legitimate software updates, and LOLBins are commonly abused by red team tooling and even some benign administrative scripts. By excluding these common signals, the feature set becomes a sharper discriminator for Trojan activity alone.

This reasoning is a reminder that a signal common to many threat types can still be a poor discriminator for any single one. Defenders often struggle with alert fatigue because they rely on high-signal but low-specificity indicators. The study’s approach advocates for a more targeted feature selection that reduces noise and improves the signal-to-noise ratio for a specific malware family.

This catalog of top features is portable knowledge. The detection list works as a behavioral checklist for threat hunting, EDR tuning, and detection-rule writing—independent of any single machine learning model. Security teams can incorporate the 33 features into their existing detection pipelines, configure SIEM rules based on them, or use them as a starting point for custom behavioral analytics.

The deployment claims deserve a closer look

The researchers took their framework beyond the lab. They deployed it as a continuous monitoring loop driven entirely by Windows command-line tools. The script uses built-in utilities such as tasklist, netstat, and wmic to enumerate running processes, extract the 33 features, and pass the data to a trained model for classification. The system runs on a standard enterprise workstation with an Intel Core i7 processor and 32 GB of RAM, with no GPU or specialized hardware. The loop operates on a three-minute cycle, which was determined after stress testing to balance detection timeliness with system resource consumption.

That setup matters for environments with operator workstations, human-machine interfaces (HMIs), and supervisory control systems. In industrial settings, Windows is common, but spare compute capacity is often limited. A detection approach that runs on hardware already in the building lowers the barrier to adoption. It also avoids the need for costly additional infrastructure or cloud dependencies, which can be a concern in air-gapped or sensitive environments.

However, the deployment model raises questions about overhead. While the study reports stable operation, real-world enterprise environments may have hundreds or thousands of endpoints. Running a three-minute behavioral analysis loop on each endpoint could consume significant CPU cycles and generate large volumes of log data. The researchers did not provide benchmarks for network-level load or disk I/O, which would be important for scaling.

Where the limits sit

The researchers are direct about the constraints of their work. The dataset is moderate in size—3,000 samples—and comes from a single sandbox source. This raises the question of how well the model generalizes to samples it has never seen, especially those that are obfuscated, packed, or designed specifically to evade sandbox analysis. Trojans engineered to stay dormant may never surface during a given monitoring window, since the detection system depends on observing live behavior within a three-minute interval. Sophisticated malware that detects sandbox conditions—such as by checking for virtualized hardware or the presence of analysis tools—can suppress its activity entirely and feed the model misleading data. This is a well-known limitation of sandbox-based detection and applies equally to the proposed framework.

The platform constraint carries the most operational weight. The pipeline targets Windows. Many IoT devices run embedded Linux, real-time operating systems (such as VxWorks or FreeRTOS), or microcontroller firmware that does not support command-line scripts. The framework fits the Windows-heavy slice of an industrial environment but leaves the embedded layer for separate tooling. For organizations with mixed-architecture networks, this means maintaining at least two detection approaches, which increases complexity.

Additionally, the study did not address how the framework handles Trojans that use fileless execution or rely entirely on in-memory payloads. Fileless techniques are becoming more common and can bypass traditional file-based sandbox analysis. The feature set includes some process-level indicators, but it is unclear whether the sandbox environment captured enough of these sophisticated attacks to train the model effectively.

Disciplined feature work over bigger models

The transferable lesson here runs deeper than one model. The study achieved strong detection results not by using a massive dataset or a complex neural network, but through disciplined, domain-informed feature work that isolated behaviors specific to Trojan activity. The researchers focused on the lifecycle of a Trojan—from initial execution and persistence to C2 communication—and selected features that map directly to those phases. They then discarded any feature that fired across multiple threat categories, even if those features were technically interesting.

Defenders can apply that same thinking to their own pipelines: identify the signals tied to a threat’s lifecycle, discard the ones that fire across every category, and keep the detection logic understandable to the analysts who maintain it. This approach also facilitates better collaboration between detection engineers and incident responders, because the rule logic is transparent and grounded in observable behavior rather than opaque model weights.

In an era where the cybersecurity industry is increasingly focused on large language models and automated threat detection, the study serves as a reminder that sometimes the most effective improvements come from careful feature engineering rather than model complexity. By understanding what specific behaviors distinguish a Trojan from other malware, organizations can build detection systems that are both more accurate and easier to adjust as adversarial tradecraft evolves.

Source: Help Net Security News