[Partner Blog] Managing Data Chaos with Agile Analytics

If we examine the root cause of an IT organization’s pain, it continues to be the inability to integrate data from many disparate and operationally focused systems. A recent survey1 found that only 18% of the respondents had extensively integrated their analytics initiatives across operations and realized their objectives, which pointed to the diversity of datasets and formats and poor data quality. Indeed, data integration is causing the analytics burden to be placed on IT professionals.

Building Better Business Outcomes with Analytics
This doesn’t have to be the case. According to Gartner2, providing insights to many parts of the organization – including operations – is the role of the next generation analytics platform which will inevitably lead to IT transformation. Solving the data integration problem first, and with the correct platform, allows IT to arm its internal customers with self-service analytics and insights. This positions IT as a strategic player for building better business outcomes for the organization. It also allows organizations to experiment, fail, iterate, and succeed at their own pace, and according to their respective metrics. Solving the data problem alleviates the IT burden of providing analytics to everyone.

Overcoming the Data Integration Challenge
As we mentioned above, solving the data integration challenge is the foundation of an analytics strategy and platform. This is especially true if the goal is to de-centralize analytics from the IT organization closer to the operational end users. The integration platform must be versatile in ingesting any data type from any data source of any quality level. Bringing in data as it comes in real-time, and at scale, while integrating even the most complex data sources to gain operational efficiency is the goal.

In addition, the ability to integrate data at the edge – as close to the raw data as possible – will be a competitive advantage for industrial IT organizations to offer to their internal customers hoping to gain faster insights. Using the same data integration platform to integrate data in the data center and at the edge allows IT organizations to scale and standardize on a single data ingestion tool for all of their internal customers.

Leveraging a Purpose Built IIoT Analytics Framework
In order to be applicable across many organizations, the next generation data integration and analytics platform should contain a sophisticated analytics framework. The framework should include an entire library of algorithms and methods that can be exploited to detect, model, ingest, cleanse, reconcile, analyze, match, correlate, transform, and present data within the platform, as well as perform application-to-application and machine-to-machine operations. The library can be applied throughout the various stages of data ingestion and analysis allowing for a sophisticated approach to relationship analysis.

For example – manufacturing operations teams might be interested in outlier detection algorithm families to detect potential failure scenarios in critical plant floor equipment and perform preventive maintenance. Planning teams, on the other hand, might be interested in seasonal decomposition, smoothing and clustering algorithm families to more efficiently control inventory or intelligently prioritize capital expansion projects, all with statistically strong rationale. IT’s role in this future state is to provide a data integration and analytics platform that can handle the above examples and many more with little to no burden on its own staff.

Exposing Users to the Analytics
With the right analytics framework in place and relevant data integrated, exposing analytics to users becomes an easy and logical next step. By leveraging role-based access control and an easy-to-use interface, IT can expose varying and appropriate levels of the analytics to users of corresponding technical sophistication – operators versus process engineers versus data scientists. This is where the proverbial rubber meets the road in providing agile analytics to everyone.

In addition, both IT and OT users should be exposed to an integrated development environment that includes the tools, documentation, knowledge base, and sandbox needed to enable agile development, outside of the production system. A key element of the environment is a plugin library – foundational objects that allow users to build functionality and reports using building blocks without getting bogged down by coding or creating these from scratch. The development environment and plugin library round out what the next generation data integration and analytics platform should offer to fully empower IT’s internal customers for agile analytics.

Bringing Agile Analyics to Life
De-centralizing analytics away from IT to the people who are closest to the data and arming them with the knowledge of what they actually care about (actionable insights) is the future of data integration and the analytics platforms that will prove instrumental.

Download the White Paper


1Going Big: Why Companies Need to Focus on Operational Analytics, Capgemini Consulting Digital Transformation Institute, March 2016.

2Magic Quadrant for Business Intelligence and Analytics Platforms, Gartner, February 2016.

About Andrew Miller, Senior Sales Engineer at Bit Stew Systems
Andrew Miller is an outcome focused sales engineer with over five years in network consulting, and a decade in in the information technology industry. Andrew brings experience in international project and program management and technical team leadership and management. At Bit Stew, Andrew drives eastern US and European direct technical sales support activities for the company. Andrew also drives technical activity within the partner ecosystem development team and is responsible for partner recruitment, onboarding, and sales support.