arXiv subjects

Yanis Pavlidis

Publications and source records attributed to Yanis Pavlidis.

3 recordsLinked to original sources

arXiv2025

Democratizing ML for Enterprise Security: A Self-Sustained Attack Detection Framework

Despite advancements in machine learning for security, rule-based detection remains prevalent in Security Operations Centers due to the resource intensiveness and skill gap associated with ML solutions. While traditional rule-based methods offer efficiency, their rigidity leads to high false positives or negatives and requires continuous manual maintenance. This paper proposes a novel, two-stage hybrid framework to democratize ML-based threat detection. The first stage employs intentionally loose YARA rules for coarse-grained filtering, optimized for high recall. The second stage utilizes an ML classifier to filter out false positives from the first stage's output. To overcome data scarcity, the system leverages Simula, a seedless synthetic data generation framework, enabling security analysts to create high-quality training datasets without extensive data science expertise or pre-labeled examples. A continuous feedback loop incorporates real-time investigation results to adaptively tune the ML model, preventing rule degradation. This proposed model with active learning has been rigorously tested for a prolonged time in a production environment spanning tens of thousands of systems. The system handles initial raw log volumes often reaching 250 billion events per day, significantly reducing them through filtering and ML inference to a handful of daily tickets for human investigation. Live experiments over an extended timeline demonstrate a general improvement in the model's precision over time due to the active learning feature. This approach offers a self-sustained, low-overhead, and low-maintenance solution, allowing security professionals to guide model learning as expert ``teachers''.

cs.CR↗

arXiv2024

Facade: High-Precision Insider Threat Detection Using Deep Contextual Anomaly Detection

Insiders with privileged access have the power to cause great harm to their organization. Even a single insider threat incident can be catastrophic, resulting in both financial losses and reputation damage. These threats are some of the most difficult to detect, as attack activity is interspersed in large volumes of legitimate activity. Although it is a serious threat, the literature is sparse aside from a few studies with various limitations, preventing their practical deployment in large-scale organizations. We present Facade: fast and accurate contextual anomaly detection, a high-precision, deep-learning system that has served as the last line of defense against insider threats at Google since 2018. Facade is an innovative self-supervised system that detects suspicious events by considering the context surrounding each event, including relevant facts about the user and resource involved. It is built around a new multi-modal model that is trained on corporate document access, SQL table access, and HTTP/RPC request logs. To overcome the scarcity of labeled incident data, Facade employs a novel contrastive learning strategy trained exclusively on benign activity. Facade detects insider attackers with an extremely low false-positive rate, lower than 0.01%. For single rogue events, such as the illegitimate access to a sensitive document, the false-positive rate is as low as 0.0003%. To the best of our knowledge, Facade is the only proposed insider risk anomaly detection system with a false positive rate low enough for use in large corporate environments.

cs.CR↗

arXiv2024

Fine Grained Insider Risk Detection

We present a method to detect departures from business-justified workflows among support agents. Our goal is to assist auditors in identifying agent actions that cannot be explained by the activity within their surrounding context, where normal activity patterns are established from historical data. We apply our method to help audit millions of actions of over three thousand support agents. We collect logs from the tools used by support agents and construct a bipartite graph of Actions and Entities representing all the actions of the agents, as well as background information about entities. From this graph, we sample subgraphs rooted on security-significant actions taken by the agents. Each subgraph captures the relevant context of the root action in terms of other actions, entities and their relationships. We then prioritize the rooted-subgraphs for auditor review using feed-forward and graph neural networks, as well as nearest neighbors techniques. To alleviate the issue of scarce labeling data, we use contrastive learning and domain-specific data augmentations. Expert auditors label the top ranked subgraphs as ``worth auditing" or ``not worth auditing" based on the company's business policies. This system finds subgraphs that are worth auditing with high enough precision to be used in production.

cs.CR↗