As the core of any digital business, analytics is at a critical inflection point. Data complexity is increasing and business people across the enterprise are drowning in data. They’re struggling to identify what’s most important and the best actions to take.
Across the analytics stack, tools have become easier to use and more agile, enabling greater access and self-service. Many analytics processes, however, remain largely manual and prone to bias, including managing, preparing and analysing data. Building data science and machine learning (ML) or artificial intelligence (AI) models, interpreting results and making insights actionable require the same manual approaches they have for as long as 30 years.
It’s not possible for users to explore every possible combination and pattern using current approaches, let alone determine whether their findings are the most relevant, significant and actionable.
Relying on business users to find patterns, and data scientists to build and manage models manually, can result in them exploring their own biased hypotheses. They may miss key findings and draw incorrect or incomplete conclusions, adversely affecting decisions, actions and outcomes.
A new model — augmented analytics — is rapidly gaining traction. Central to this development is the use of ML automation/AI techniques to augment human intelligence and contextual awareness.
Augmented analytics uses ML to automate data preparation, insight discovery, data science, ML model development and insight sharing for a broad range of users. It uses AI and ML techniques, as well as natural language processing, to deliver analytics anywhere and for everyone in the organisation with less time, skill and interpretation bias of current manual approaches.
Next wave of analytics disruption
As businesses become inundated with data, augmented analytics becomes crucial for presenting only what’s important for users across the business in their context to act upon at that moment. It drives less biased decisions and more impartial contextual awareness — transforming how users interact with data, make decisions and act on insights.
As more organisations digitally transform, they want to expand use of data science and ML/AI. They want to leverage it to create new differentiated analytic applications and embed ML/AI into existing applications.
The scarcity of expert data science skills, however, has become a significant barrier. By automating many time consuming and bias prone tasks, augmented analytics expands those capabilities with more widely available skill sets — the business analyst and the application developer (a new breed of citizen data scientists). In addition to expert data scientists, augmented analytics allows these roles to generate insights and create augmented-analytics-assisted models to embed in applications.
Gartner predicts that augmented analytics will be a dominant driver of new purchases of analytics and business intelligence (BI) by 2020, as well as data science and ML platforms, and embedded analytics.
Augmented analytics capabilities will rapidly achieve mainstream adoption, as a key feature of data preparation, broader data management, modern analytics and BI, as well as data science and ML platforms.
More importantly, automated insights will also be embedded in enterprise applications and conversational chatbots for analytics to make insights available to more people across the organisation. This will reach beyond citizen and developer data scientists to enable operational workers to assist in business transformation.
How does it work?
Augmented analytics can reduce time consuming exploration and the identification of false or less relevant insights. It applies a range of algorithms and ensemble learning to data in parallel, generating the most important insights and explaining actionable findings to users. This reduces the risk of missing important insights in the data in comparison to manual exploration. It also optimises resulting decisions and actions.
It does however, require a focus and investment in data literacy throughout organisations, as insights are distributed to all employees so they know how to best act on the new insights.
Think of one of the most common cases of analytics – business people monitoring revenue and profit in their business by a number of dimensions. Typically, known sales and profit drivers are built into dashboards that business people access every day. When sales trends change, users have to explore those relationships to find the root cause of the change.
Is it pricing, promotions, packaging, product quality, the weather, a combination of these factors, or something else? What if the change is caused by new, unknown factors or new combinations of factors not included in the dashboard, or that the business person didn’t think of or have time to fully explore?
How often do business people draw suboptimal conclusions from their data? How often do they explore what they think are the key drivers or attributes of an outcome variable, and stop when they confirm their hypotheses? How many times might there be other, more important factors affecting the outcome variable that they have not thought to explore?
This is the root of the challenge with the current manual process for exploring data and building data science and ML models. The desire to overcome it will drive the transformational nature of augmented analytics as the next wave of market disruption.
Since automation will enable expert data scientists to focus on specialised problems and on operationalising and embedding enterprise-grade models into applications, only the most accurate and significant insights will be acted on by users. Expanded use of automation should also translate into fewer errors from the bias inherent in manual exploration.
If you’re planning to use augmented analytics to modernise solutions, start by exploring opportunities to complement existing data and analytics initiatives by piloting it for high-value business problems currently requiring time-consuming, manual analysis. Then you can expand from there.
Rita Sallam is a VP analyst and Gartner Fellow. Her focus includes data and analytics market trends and best practices, particularly how data management and analytics are being disrupted by AI-enabled augmented analytics. Rita will be presenting at the Gartner Data & Analytics Summit in Sydney, 18-19 February 2019.