Log management is the answer to all of your digital transformation woes. No, hear me out.
At its heart, log management is the (challenging) task of collecting and storing all machine-generated data from across your entire enterprise into a common repository. If this collection doesn’t happen, or if log collection is limited to certain datasets, there’s little chance of deriving the high value business intelligence you dream of. For businesses to be able and prevent service disruptions, detect threats, and map user web activity to CSAT they must rely on logs and it becomes especially important to make sure your log management tool is up for the task, especially at scale.
Not only is the breadth of data collected important, but so is its depth. To increase its value, you must be able and collect data at scale, store it all, and visualize it. In addition, you can gain insights by applying machine learning to your data. By leveraging both real-time and historical data to train ML models you can then apply those models against new unseen data to gain differentiated insights. When you arrive at the intersection of viewing all log data sources – coupled with historical and real-time streaming data – you will be able to gain differentiated intelligence that drives change for your business, whatever your goals and key performance indicators may be.
As an example of the benefits that can accrue to your business from log management let’s review three situations where log management is critical to driving business intelligence:
- Service delivery: Telefonica leveraged Devo for multiple service delivery improvements; for example, the business can proactively monitor errors per router brand, kernel version, firmware, and more. These parameters can be contextualized within the configuration management database to monitor; for example, high performing routers create fewer issues or have a lower rate of failure than others. This insight lets Telefonica reduce the volume of outages, automate configuration changes and increase quality of service (QoS) for customers. In another service improvement for customers, the help desk service group gets up to date information about the customer’s equipment, past and current issues, account parameters, historic behavior, and more. This proactive information lets the operator know immediately if the customer is experiencing service issues, and if so, whether it’s specific to their in-home equipment or is part of a broader issue affecting the region.
- Security: We’ve already covered the value and importance of enterprise machine data, so it shouldn’t be a surprise that it’s just as critical to leverage your data to tackle the toughest security challenges. More specifically, you must use real-time data to catch bad actors in the process of a breach, rather than hours, days, or even weeks later. Adding business intelligence from historical data provides an important level of context to the real-time stream to help you understand what happened, how much of the customer base was impacted, what was breached in the environment, and more.
- Web traffic: Regardless of industry, company, or use case, web traffic is an important metric. Not only can it provide breadcrumbs of information about sales or lead generation, when analyzed properly it can provide early insight into retail trends, for example. Your web traffic historical data and real-time data together can tell you about uptime, number and origination of users, etc. as well as how bottom lines are affected by data from revenue tracking systems, CSAT ratings, and more.
The beauty of log data is that it can be joined quickly with other data sets, such as revenue and web server errors. The possibilities are endless. But, what’s preventing companies from taking full advantage is that tools don’t work at the speed of human thought – lines of business want to be creative with data visualization and action, but many tools hinder that creativity due to operational issues such as the expense of storing data or latency in analyzing streaming data.
Log management requires central coordination and attention, which can often pose the biggest hurdles in large enterprises and in distributed IT models. As advanced DevOps teams built intelligent self-service applications for lines of business to improve customers’ experiences, the lack of an overarching view into all those applications has increased. Now, business and technology leaders are realizing the importance of that visibility, and the pendulum has swung in the other direction. Business intelligence can’t be achieved without historical and streaming log data and with a system that is fast enough to take timely action with ease.
