How a 5TB question brought a SOC to its knees
In my experience, there are some questions that will literally blow the CISO’s mind. Well, let me be clear: not literally, figuratively. Not the CISO, but her SOC team. And not their minds, but their SIEM.
Hyperbole aside, this happens regularly.
Recently a customer called this “the 5TB question.” Sounds geeky, but it makes sense. The CISO would get a “simple” question from the CEO or CIO that would require at least 5TB / day of additional data ingestion and analytics to answer. The first time it happened to him, it was the straw that broke the SOC’s back. He ripped out his legacy system and never looked back.
I started asking more customers about their “5TB / day nightmare,” and it turns out that this happens all the time and sometimes it’s more like 50 TBs:
- A huge European bank had a decent SIEM in the SOC. The CIO asked, “Can you guys detect credit card fraud?” Mind blown. Their SIEM was already slow, credit card data was 15TB / day. They needed to rethink.
- A giant clothing maker ingested lots of logs into a logging platform. Then the business asked if they could detect fraud and malware across all e-commerce sites. Mind blown. They had to go from 3TB / day to more than 50TB / day to solve this problem.
- A national defense organization with several disparate SIEMs was asked to detect foreign infiltration within 20 minutes of occurrence. Mind blown. They had to rethink their entire data collection strategy and ramp to 40TB / day.
So why is it, after all the work you’ve done, that adding much more data into your SIEM or logging platform will break things? Three inconvenient truths emerge, and the last one is the worst:
- Your current SIEM platform doesn’t scale.
- Your current platform is only good with core security logs.
- Your vendor will use this moment of stress to try and extort more cash from you.
As I said, the third reason is the worst. But it’s happening all the time.
We all know we’re just one meeting away from our own “5TB” question. Books have been written about the explosion of data. During this bizarre, ongoing COVID time, new data needs have emerged:
- Employees working from home during the pandemic are connecting to the company network. Is your security team logging this outside-the-security-perimeter activity for threats?
- Increasingly sophisticated hackers are attacking cloud environments. Are all of the organization’s rapidly growing cloud activities being logged and reviewed for threats?
- Nation-states are constantly trying to disrupt critical government, healthcare, industrial, and other systems. How do you know if your “secure systems” are really secure?
Today SOCs ingest data from core security systems—firewalls, IPS, identity—which is mostly manageable with the legacy SIEM solutions many organizations still use. But CEOs and CIOs now ask questions that go far beyond traditional security topics. Collecting and digging through ever-growing data logs to find answers is causing legacy security systems to collapse. The challenge facing security teams is how can they respond to these urgent questions, which require querying multiple terabytes of data to answer?
CISOs are facing demands to effectively log more and different types of data than ever before, especially in organizations accelerating their cloud shift. And these growing demands raise many concerns.
Security logging solutions have not kept pace with organizations’ need to collect the amounts of data required to answer these critical questions. This forces organizations to retain less historical data, reduce query performance, or spend significantly more on logging. None of these options is appealing.
Luckily, the answer isn’t so hard anymore. I’ve been in the logging space for 20 years across three different solutions, each leapfrogging the last. Today we’re helping the largest organizations to ramp fast to meet this need. Key is a cloud-native architecture, a friendly cost and price structure, and amazing analytics. We’re helping organizations ween off of punitive contracts and transition in a matter of weeks.
As a CISO, are you fully prepared to answer these mind-blowing 5-, 25-, or 50TB questions?
