In today's business environment, organizations are increasingly demanding advanced analytics that allow them to use large volumes and diverse types of data to discover patterns and anomalies and predict outcomes.
"Our research makes it very clear that advanced analytics is rapidly becoming fully integrated into the operations and decision-making processes at companies across many different industries," write Hurwitz & Associates COO and Principal Analyst Marcia Kaufman and Senior Analyst Daniel Kirsch in the research firm's recent Advanced Analytics: The Hurwitz Victory Index Report 2014.
"It is no longer sufficient for businesses to understand what has happened in the past, rather it has become essential to ask what will happen in the future, to anticipate trends and to take action that optimize results for business."
For instance, says Kirsch, drug stores are now turning to advanced analytics to help them predict the scope of flu season or allergy season six months in advance so they can more efficiently stock just the right amount of medication neither running short nor overstocking and taking up valuable shelf space that could be dedicated to something else.
Farms are leveraging advanced analytics to provide more insight into when to plant, how to optimize crop yields and when to harvest. Manufacturers are turning to advanced analytics to predict when a machine on the production floor is going to fail so they can perform preventative maintenance before a failure causes expensive unscheduled downtime.
Financial services firms are leveraging the technology to combat internal and external fraud. Professional sports teams are using the technology for all manner of things, including the use of sensors on athletes to create optimized workouts and predict injuries. The use cases are nearly limitless.
"Where we're really seeing catch on is predicting consumer behavior," Kirsch says. "Companies want to be able to provide customized offerings: You bought three sweaters, and people who bought those sweaters might be incentivized to buy this package of undershirts for 30 percent off. If you're getting that level of customized service, you're more likely to bite."
Offerings in the advanced analytics space are rapidly evolving to meet the changing needs of these organizations, Kirsch says. Here are 11 market trends that Hurwitz & Associates see dominating the advanced analytics space today.
1. Customers Seeking Integrated Hardware and Software for Analytics Workloads
Customers of advanced analytics are increasingly looking for hardware that is pre-integrated and optimized to run advanced analytic workloads, giving traditional vendors like SAP, IBM and SAS a leg-up in the market. In the report, Kaufman and Kirsch note that these hardware offerings allow users to scale support of big data and advanced analytics while maintaining high levels of speed and reliability.
"SAP offers an in-memory platform, HANA, which allows customers and partners to run InfiniteSight on hardware that is designed for high-speed and volume analytics," they write. "In addition, IBM's PureData System is an integrated system that is designed and optimized for operational analytics workloads. Customers can benefit from the increased reliability, scalability and speed of an integrated system SAS has partnered with database maker Teradata to offer a pre-integrated and optimized platform."
2. Vendors Are Packaging for Horizontal and Vertical Use Cases
Kirsch says customers are increasingly looking for end-to-end vertical or horizontal solutions and vendors are obliging with solutions specialized solutions for verticals like healthcare, finance and government and horizontal packages aimed at improving customer service, churn reduction or fraud prevention.
"The solutions come pre-integrated with best practices, data preparation automation and automation for model building, but also allow for some customization," Kaufman and Kirsch say. "Some examples of this packaging include SAS' customer intelligence platform that gives customers tools to personalize consumer experience and Pega's extensions for SAP and salesforce.com. Pega's offering allows customers to run business process management (BPM) and customer relationship management (CRM) analytics from specific data sources."
3. The R Open Source Programming Language Is Becoming Pervasive
R, an open source programming language for computational statistics, visualization and data is becoming a ubiquitous tool in advanced analytics offerings.
Kirsch says nearly every top vendor of advanced analytics has integrated R into their offering and so that they can now import R models. This allows data scientists, statisticians and other sophisticated enterprise users to leverage R within their analytics package.
One of the big beneficiaries of this trend, Kirsch says, is Revolution Analytics, the leading provider of enterprise support for R. Kaufman and Kirsch also point to advanced analytics firm Predixion, which is focused on extending R beyond data scientists and statisticians to business users through a wizard interface.
4. Python Is Opening the Door for General Purpose Programmers in Advanced Analytics
While R is typically the domain of data scientists who can develop complex analytics models using sophisticated deep data analytics and machine learning, the open source language Python is allowing the much larger body of general purpose programmers to get in on the act.
"While Python does not have the sophisticated deep data analytics and machine learning capabilities that R does, the community is working hard to develop more focused advanced analytics capabilities for Python," Kaufman and Kirsch say. "IBM and SAS both allow customers to integrate R and Python projects into larger projects."
5. Visual Interfaces Are Making Advanced Analytics More Accessible to Business Users
Data scientists are few and far between on the ground, and small and mid-sized enterprises in particular are struggling to create experienced analytics teams due to a lack of budget. At the same time, analytics is working its way into decision-making at all levels of the enterprise, making it more important than ever that business users be able to access data insights. That combination has advanced analytics vendors firmly focused on offering features that make their platforms easier for business users to use.
"For example, SAP is focusing on automating the predictive process, while Angoss offers a very visual interface for decision and strategy trees," Kaufman and Kirsch write. "SAS and IBM have released specific offerings aimed at business users. For example, SAS' Visual Analytics offering and IBM's Analytics Catalyst are both aimed at business users."
8 things to know about data analytics
6. Real-Time Data Streams and the Internet of Things Are Hot
The demand for analytics on real-time data streams is increasing rapidly as more and more devices connect to the Internet. By applying advanced analytics to streaming data, organizations can respond with much greater agility, whether that means providing personalized recommendations as you shop online, or monitoring a jet engine's key metrics to identify signs of failure before maintenance crews notice.
"Traditionally, the airline would rely on manually set thresholds and visual inspections," Kaufman and Kirsch say. "These thresholds might send an alert if the engine overheated, but will be unable to identify potential problems that result from the occurrence of several normally innocuous factors that when combined are problematic. Vendors are responding to the need to provide analytics on real-time data. SAS' Event Stream Processing Engine and IBM's InfoSphere Streams allow users to run analytics while data is in motion."
7. Data Visualization Is Becoming a Business Requirement
Data visualization is taking on an even more important role within organizations as they become flooded with streaming data, social media data, machine data and other large volumes of structured, semi-structured and unstructured data. Visualizations are necessary to help analysts uncover insights that would simply be impossible to spot in a vortex of data tables, spreadsheets and charts.
"Visualization might be the primary interface for the business users and might be a first step for the data scientist," Kaufman and Kirsch say. "To help bridge the gap between business users and data scientists, vendors are offering more visualization capabilities. Visualization capabilities can be customized for different user groups so that they can easily understand them. Some vendors are offering complex visualization products. For example, SAS has an in-memory based interactive visualization tool, SAS Visual Analytics. IBM's Rapidly Adaptive Visualization Engine (RAVE) is built into SPSS Analytic Catalyst and gives users suggestions for visualizations based on the data set. Other vendors, such as Megaputer, RapidMiner and StatSoft rely on visualization capabilities that are built into the core offering."
8. Organizations Are Infusing Big Data Analytics into All Decision-Making Activities
It is no longer enough for analytics to be managed solely through a statistics or data analysis department. Organizations want to make analytics part of the decision-making process across all areas, including marketing, sales, operations, finance and human resources.
"In order to improve customer engagement and optimize outcomes across all these functional areas, companies want to include more varieties of data in their analysis," Kaufman and Kirsch say. "For example, data types ranging from machine-generated and other sensor data to mobile and financial data feeds and social media data are typically included in big data analysis. These companies are looking to their vendors to support very large data sets."
Vendors are responding with holistic platforms that help integrate the process of big data analysis with analytics efforts across all areas of the organization. Kirsch points to IBM's SPSS Analytic Server, which helps companies get fast results for predictive analytics of big data, as an example.
9. Analytics Services Are Increasingly Hosted in the Cloud
Advanced analytics vendors are increasingly turning to the cloud to deliver analytics capabilities in a more affordable way, making them practical beyond the large enterprises able to afford the significant expense of complex, on-premise solutions.
"Some of these offerings are for specific use cases," Kaufman and Kirsch say. "For example, Angoss, Pega and SAP all offer salesforce.com applications through the AppExchange to perform analytics on CRM data. Angoss, IBM and SAS also offer more flexible software as Software as a Service (SaaS) that allows customers to do general-purpose analytics with cloud-based software."
10. In-Database Analytics Sidestep ETL Challenges
Performance, data governance and security can present serious challenges to performing advanced analytics on massive data sets. In-database analytics can ease many of these challenges by giving users the ability to deploy their models in the database itself rather than moving data to an analytics environment. By performing analytics on the data in place, the users can recognize performance and efficiency gains while simplifying security and data governance because the data never leaves the secure database.
"Many vendors are offering in-database capabilities for a number of data platforms, including Hadoop," Kaufman and Kirsch say. "IBM, SAS, RapidMiner, Revolution Analytics, Predixion, StatSoft, SAS and Angoss all support in-database mining. When evaluating a vendor based on in-database capabilities, it is important to investigate their support for the data platform your organization is using. Some vendors only support Hadoop while others support nearly every common data platform."
11. Companies Are Turning to Predictive Model Markup Language (PMML)
As companies make the switch from batch analytics to using real-time feedback to continuously improve the accuracy of their models, they are increasingly leveraging Predictive Model Markup Language (PMML). PMML is a standard for statistical and data mining models developed by the Data Mining Group (DMG), an independent consortium led by vendors. IBM and SAS are full members of the DMG, while SAP, StatSoft, RapidMiner and Angoss contributed to the development of PMML. Kirsch notes that the standard makes it easy to develop a model on one system with a particular application and then deploy it on a different system using a different application.
"These companies find that deploying models in applications with PMML helps to overcome delays and speed up the process of moving models more quickly into production," Kaufman and Kirsch say. "One of the major benefits of using PMML is that it eliminates the need for costly and time-consuming custom coding and proprietary processes."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.