Unsupervised Machine Learning with Splunk: the cluster command | by Alex Teixeira | Detect FYI

There are known knowns. There are things we know we know. We also know there are known unknowns. That is to say, we know there are some things we do not know. But there are also unknown unknowns, the ones we don’t know we don’t know. Secretary of Defense Donald Rumsfeld, 2002 To pick a use case from that, let's focus on web browsers for a moment. In large-scale environments, there might be thousands, if not, millions of new, unique HTTP User Agent (browser) strings every day. The issue is that it's not possible to simply pattern-match every new malicious browser string (Known Known). So what can be done instead? Splunk does provide commands such as stats, rare and top, which allow users to organize results based on least or most frequent values. However, simply stack counting (simple frequency analysis) does not apply to all cases, especially when the distinction lies in very small details. In this quick article I will introduce a simple yet powerful clustering technique available for

Unsupervised Machine Learning with Splunk: the cluster command Alex Teixeira 7 min read · Feb 19, 2024 -- 2 Listen Share If you are in cyber for long, you should have probably heard this one: There are known knowns . There are things we know we know. We also know there are known unknowns . That is to say, we know there are some things we do not know. But there are also unknown unknowns , the ones we don’t know we don’t know. Secretary of Defense Donald Rumsfeld , 2002 To pick a use case from that, let's focus on web browsers for a moment. In large-scale environments, there might be thousands,

saved by

• Cooper Edmunds

related reading

• Model-Assisted Threat Hunting (M-ATH) with the PEAK Framework | Splunksplunk.com
• Advanced threat detection with User and Entity Behavior Analytics (UEBA) in Microsoft Sentinel | Microsoft Learnlearn.microsoft.com
• Anomalies detected by the Microsoft Sentinel machine learning engine | Microsoft Learnlearn.microsoft.com
• Risk-Based Alerting: The New Frontier for SIEM | Splunksplunk.com
• Baseline Hunting with the PEAK Framework | Splunksplunk.com
• How rare is a rare HTTP agent? Context-rich alerts because of math – Opstune.comopstune.com
• Elevating Security Intelligence with Splunk UBA's Machine Learning Models | Splunksplunk.com
• clustering_algorithms_hartigan.pdfcs.columbia.edu
• Mediumdetect.fyi
• Time Series Clustering - Deriving Trends and Archetypes from Sequential Data | Towards Data Sciencetowardsdatascience.com
• GitHub - open-edge-platform/anomalib: An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference. · GitHubgithub.com
• GitHub - zhoushengisnoob/DeepClustering: Methods and Implements of Deep Clustering · GitHubgithub.com