Most security pros know the value of log data. Organizations collect metrics, logs, and events from some parts of the environment. But there is a big difference between monitoring and a true centralized log management. How can you measure the effectiveness of your current logging solution?
Here are four signs that it’s time to centralize log management in your organization:
- A growing number of a teams are using their own monitoring tools
- The number of new data sources are proliferating
- Hybrid environments are appearing quickly
- Your organization is going through a digital transformation
This post is based on content from the new Devo eBook The Shift Is On.
Teams are Using Their Own Monitoring Tools
User/desktop admins receive data from workstations and tablets; server admins receive logs from virtual and physical servers; network admins get flows from routers and switches, cloud admins use S3 buckets for cloud logs; and security personnel receive data from firewalls, IPS, and DLP solutions. But none of these teams has a single, correlated view for all of this data.
Inevitably a problem occurs, forcing you to go from tool to tool and team to team to chase down the issue. This is painfully time consuming, and it makes it difficult to correlate data from multiple sources to perform an accurate investigation. If you and your team find yourselves in an “all-hands-on-deck” situation looking at a bunch of different tools and struggling to resolve an issue, you need a centralized log management solution to pull all the metrics, events, logs, and traces together for faster, more effective correlation.
New Data Sources are Proliferating
Most organizations don’t even strive for 100% visibility. They set the bar much lower, for a variety of reasons. Sometimes network, server, and desktop teams don’t feel the need to share data. So, each only collects the data “they own.” This often results in crucial pieces slipping through the cracks, which feeds into the silo problem described above. Sometimes they just don’t know how to combine data from different sources that have different formats, such as collecting logs from a DB server and flows from a switch. Often, they don’t have the on-premises infrastructure to store all this data, or the operational resources to support that on-prem infrastructure.
If you and your team find yourselves saying “If only we had a way to collect and analyze data from ‘x,’” or “We have to decide to either collect data from ‘x’ or data from ‘y’ because we don’t have capacity for both,” then you probably have some serious gaps in visibility and need a SaaS-based CLM solution.
Hybrid Environments are Appearing Quickly
Too often, teams completely ignore private or public cloud environments, such as AWS or Azure. These aren’t considered “gaps in visibility” because organizations don’t think about these environments the same way they do traditional on-prem environments. And since many organizations have completely separate “CloudOps” teams that handle these environments, they may fall outside the purview of SecOps. In reality, this is just another silo—often one that is poorly instrumented, poorly secured, and without sufficient management. Many executives consider public cloud environments to be “the provider’s problem,” and never factor it into their overall strategies. This is a disaster waiting to happen. Incorporating multiple cloud environments into your centralized log management solution is crucial for avoiding major problems.
If you aren’t combining and correlating data from all environments together, then your organization would benefit from a centralized log management solution that will perform those important tasks.
Digital Transformation is Sweeping the Organization
Most digital transformation projects center on “data at rest.” This is data stored in databases, mainframes, and other monolithic structures that are “mined” for insights. But what about “data in motion”? Data in motion is data that flows across your network, in and out of your multiple cloud environments, and between systems. It is every bit as crucial as data at rest. It contains vital information about who accesses your services, how much data they send or receive, patterns of data access and usage, what type of devices they use, where data comes from, and where it goes.
This wealth of log data is created every second by every piece of infrastructure, and it often goes unused in digital transformation projects. Security organizations can make a vital contribution to digital transformation projects by leveraging a centralized log management solution as a data lake.
In the next post in the series, we’ll examine the evaluation criteria for choosing the right CLM solution for your business. But if you can’t wait to read more, download the eBook The Shift is On.
