The cloud-native platform for centralized log management
Analytics, visualizations, and workflows purpose built for practitioners
Leading firms gaining more value from their machine data
Any source, any velocity – centralize logs, metrics, and traces for full visibility.
Close the gap between detection and response with an analyst-focused, cloud-native approach.
Understand complex environments with visual analysis and KPIs that matter most.
The most recent articles & research from Devo
You might have not noticed it, but we are on the verge of a disruptive change in the way we humans transport ourselves. In the next five to ten years the way we think about – and take – our daily commute, our holiday vacation, trips to the market – anything that involves transport over a distance too great to walk – will dramatically change. This new ‘mobility model’ will be driven by more than evolution in the auto industry: it will be driven by data. How effectively we use and manipulate data, and how we take action on data in real time, will play a critical role in this transformation.
Factors underlying the new mobility model include environmental awareness; increased adoption of the electric vehicle; advances in autonomous driving technologies, and societal changes to the notion of “car ownership.”
These factors implicitly rely on each other and, at the same time, depend inherently on data management.
Let’s discuss Autonomous Driving.
This is probably one of the most disruptive changes we will experience in the near term. Tesla and Uber’s aggressive approach to pushing this technology – for example, working with states, like Arizona, which mandate little oversight of the companies’ autonomous vehicle programs – along with more slow-moving strategies, such as Waymo’s, are examples of the inevitable future of the car industry.
The media is full of news on this topic, touting the achievements of autonomous driving technology while pushing apocalyptic reporting on recent fatalities caused by the failure of autonomous driving systems – or their human co-pilots. Rather than judge the readiness of current systems or the ethical questions of the technology, let’s assume for the context of this post that AD is working and is a reality.
The changes could be so radical that they would affect many aspects of our daily routine, even those that might seem unrelated. One example: Traditional radio stations attract millions of listeners who are drivers on their daily commutes. Listening to your favorite radio station is an activity you can do easily while focusing on driving. Imagine now that you don’t have to pay attention while driving anymore. You can start your working hours as soon as you get to your car. Will you still listen to the radio? Some studies say no.
AD will be disruptive in many other areas: how we plan our day, city planning, allocation of funds for road construction and maintenance, and more. As with other disruptive technologies, small changes to a system will create large downstream effects.
AD relies heavily on data to learn and “see” what is in the car’s environment. Regardless of the technical approach used to implement AD capabilities, e.g. camera-based systems vs Lidar or radar-based systems vs. all the previous technologies combined, estimates of the amount of data needed and produced by the system to work are staggering. Depending on the source, the figure of 1Gb of data/second/car appears to be reasonable.
Multiply this number by average commute time per day and the number of AD-enabled cars available, let’s say in 2025. We are talking about Exa, perhaps even Zettabyte-scale data.
The car industry may be one of the most important data producers in coming decades. Data’s relevance is directly related to road safety. Having potentially thousands of autonomous (and therefore data-driven) vehicles raises, besides the obvious opportunities (and the not so obvious), a lot of concerns. This is a multidisciplinary challenge:
(1) Engineers must know how to deal with huge volumes of relevant data in such a way that data is actionable. Understanding a car failure in real-time (or ahead of time) might save lives. It’s also important to understand the different data architecture options; should data be kept in isolated clusters? Should the architecture rely on a centralized data lake? What about retention periods – should the system keep all the data, or just data identified as relevant? But what you think is not relevant today might be extremely relevant later. It’s not only a matter of collecting the data but also being able to extract insights – not a trivial task.
(2) Cyber Security: We’ve all seen news stories about a car being hacked to the point where the driver could do nothing to control the vehicle. That was a few years ago, when a hacker had limited access to on-board systems (basically the OBD and USB ports, and the car-connected services). In the near future, cars will be (some already are) complex servers accessible through the Internet, with the same attack surface as any Internet-connected device. Early intrusion-detection techniques we use today for real-time threat detection must be implemented by the automotive industry. The challenge is that in most cases, cars are not designed with software security in mind.
(3) Legal: Who is liable when there is no driver to blame? In this scenario, the ability to collect data and analyze it “post-accident” is relevant. From the insurance company’s point of view, in a scenario where there is no driver, or the driver is just sharing a car (remember the concept of car ownership will probably change dramatically in the coming years), access to data is critical.
(4) Environmental issues: The rise of the Electric Vehicle (EV) is driven by the need to reduce CO2, NOx, HC and other atmospheric pollutants. Pollution is forcing governments at all levels to take restrictive measures to lower the level of these elements.
Before the EV takes significant market share from the Internal Combustion Engine vehicle (ICE), at least in the urban landscape, governments will continue to invest taxpayer money on measuring air quality and acting upon these readings. These readings, along with the data produced by the cars themselves, represent a gold mine for city planners.
(5) Business opportunities: Many business models will appear around this new mobility paradigm and the data vehicles will provide in real time. New insurance models, intelligent traffic jam managements, alternative unassisted parking spaces, quality of service, real time route optimization, vehicle to vehicle (V2V) protocols… these new needs in a new space all pivot around data.
To prepare for the huge transformation marked by autonomous driving and electric vehicles, AI and improvements in communication networks are necessary to enable this paradigm shift in transportation.
Solution providers in the data management space must be ready to provide the tools to enable this revolution.
Are you ready?
By Sergio Bellido
Sign up to stay informed with the latest updates from Devo.