IT Operations Analytics from the Source


trends.jpgOne of the top uses of big data today is IT operations analytics. This makes sense. By nature, IT components are designed to log all of their many status messages, and this information is generated with debug, tracking, and audit purposes in mind. The aggregate output, however, can be a logistical problem in itself. For each device, some poor sysadmin has to decide what level of logging is desired, and then live with the consequences of that decision. Set the logging threshold to "errors only" and important context will be missing when it's time to diagnose an issue. Set the logging criteria to "everything" and staggering amounts of data will be generated, often too much to process, and certainly much of little or no value. Limit the time period to an hour or a day, and the key information may have been overwritten by the time it's needed, and then the problem will have to be recreated.

 All these trade-offs become an order of magnitude worse when the issue is multi-level, multi-disciplinary, or multi-geography. Correlating system info from storage to server to networks to application to desktop/mobile device to end-user experience is certainly no easy task. Hence, big data and analytics on all that unstructured data from diverse sources. Perfect way to spot, understand, and remedy any subtle, rare, or complex systems problem. 

I've written a bit about this in challenge in the past, for example:, but this might be a matter of perspective. A number of IT operations analytics tools for troubleshooting and planning operate at different layers. Network analytics sees the world in terms of packets and flows. Application analytics sees code and functions. Server analytics sees resource utilization and local I/O. End-user device analytics sees their response (and wait) times. Yet almost any diagram of the technology stack will show that storage underlying everything. Storage is the source and destination of all data. Storage is where it starts and where the buck stops. Good place to analyze the buck.

Many enterprise storage systems have "phone home" functionality now, usually used by the vendor to shorten their support calls and determine who's to blame. It's rarer for the storage vendor to look beyond their own concerns however. One that does is Nimble Storage. Their InfoSight technology offers a great vantage point for addressing the many challenges that IT faces today, and probably again tomorrow, and that thing that happened just the one time a couple of months ago. InfoSight also has the advantage of seeing not just one company's experiences, but across all their customers. In this way, many issues can be solved before you ever pick up the phone to log a call with the helpdesk. This is a big deal for those who like IT planning instead of IT firefighting.


big data internet of things


Topics: Big Data Data Management Data Analytics IT operations analytics