The Butterfly Effect: The Changing Winds on the SIEM Market

The butterfly effect, a concept originating in chaos theory, describes how small changes can have a nonlinear impact on a complex system, such as the flap of a butterfly's wings causing a typhoon thousands of miles away.

The idea here is that small changes—the burgeoning amount of unstructured data being generated in the enterprise, the interminable event fatigue problem created by false positives in security information and event management (SIEM) solutions, and the global talent shortage in cybersecurity that makes finding affordable security operations center (SOC) analysts difficult—are ushering in a big change as the sun begins to set on SIEM technology.

While organizations historically sent device logs, security control events, and operating system and application logs to central log servers or SIEMs that end up eventually slowing the SIEM down due to over-burdened SQL databases, they are now rethinking this approach by instead sending logs and events into their existing data lakes for analysis by machine learning-powered analytics platforms.

Indeed, the sun is setting on the SIEM as a technology for monitoring the disparate events of security controls and application, operating system, and device logs in the enterprise. Chief information security officers (CISOs) want fewer cybersecurity controls that add to the existing noise and instead want something that can do a better job of finding the signal in the noise of what they are currently generating.

Enter security analytics platforms from companies such as SAS. Security analytics platforms apply machine learning models instead of patterns and signatures to find that signal in the noise within data at rest and leverage data streaming in real time using solutions such as StreamSets. Using machine learning, security analytics platforms take data at rest or in real time in enterprise data lakes and perform analytics on it in order to find that signal, addressing many of the pain points of legacy SIEM technology.

What, you don't think that unstructured data and its prevalence is that big of an issue?

According to IBM, 90% of the world’s data has been created in the last two years alone, and according to IDC, by 2025, 80% of the world's data will be unstructured. And for those not in the know, unstructured data is unable to be stored in traditional SQL databases, creating a challenge for organizations to search through it, edit it, and analyze it. The fact is, the world is moving to unstructured data. Organizations are now creating exabytes of unstructured data in their own data lakes that require a migration to NOSQL databases, such as Hadoop, Elasticsearch, and Cloudera—companies incontrovertibly capitalizing on this data challenge.