The Transition from Monolithic SIEMs to Data Lakes for Security Monitoring

This generation of security analytics tools is based on a decoupled data architecture combining cloud storage, open data formats, and highly performant distributed query engines. While this provides improved performance, scalability, and new pricing models for security teams, it comes with a nuance in usability and technical understanding. This blog explores the transition from SIEM to security data lakes, discusses their key components, strategies for interoperability, and query engines for such workloads. The roles of detection engineers and security analysts are distinctly different and mirrored in the technologies that cater to each. Security analysts are often experts in understanding attacker TTPs, and detection engineers specialize in tool accuracy, performance, and capabilities to identify those TTPs. Thus, when responding to incidents, analysts need tools that provide exhaustive answers, and platforms like Splunk and Elastic enabled that by indexing all of the data and couplin

This generation of security analytics tools is based on a decoupled data architecture combining cloud storage, open data formats, and highly performant distributed query engines. While this provides improved performance, scalability, and new pricing models for security teams, it comes with a nuance in usability and technical understanding. This blog explores the transition from SIEM to security data lakes, discusses their key components, strategies for interoperability, and query engines for such workloads. The roles of detection engineers and security analysts are distinctly different and…

related reading

• Security is about data: how different approaches are fighting for security data and what the cybersecurity data stack of the future is shaping up to look likeventureinsecurity.net
• Security data lakehouse and modular designrippling.com
• What I Learned From The Modern Data Stack Conference 2021 - James Lejameskle.com
• The Evolution of the Modern Security Data Platformsoftwareanalyst.substack.com
• The Security Data Fabric Identity Crisisomeronsecurity.com
• The Security Data Fabric Identity Crisisomeronsecurity.com
• The Great Splunkbundlingrakgarg.substack.com
• The Splunk Effectomeronsecurity.com
• Protect your organization as SIEM vendor, technology and threat landscape changeslinkedin.com
• Why did we need to build our own SIEM?rippling.com
• The Two-Headed SIEM Monster - by Omer Singeromeronsecurity.com
• Is the SIEM dead? - CPO Magazinecpomagazine.com