It's no secret that many IT professionals in large enterprises look down at public clouds -- they don't consider them reliable, secure or safe. But that perception may be shifting, at least when it comes to big data workloads.
A new report released this week by Gigaom Research finds that 53 percent of large enterprises are either already leveraging public cloud resources for enterprise big data analytic needs (28 percent) or are planning to do so (25 percent). Only 13 percent of respondents say they would only use private data centers for all analytic processes.
"We expected many enterprises to be thinking about the cloud as a long-term thing," says Prat Moghe, founder and CEO of Cazena, a big data startup founded by former Netezza leaders, which commissioned the Gigaom report. "We found that a lot more enterprises are actively exploiting the cloud for their big data stack than the perception is."
The Key to Data Analysis Lies in the Cloud
"The cloud holds the keys to unlocking the next phase of big data, where analysis is completed on demand and immediately accessible to the primary users of this information -- the C-suite and key decision makers," he adds. "Understanding enterprise drivers as well as concerns around the cloud is vital to be able to develop a roadmap of strategic transformation of all enterprises."
In September, Gigaom Research surveyed more than 300 senior management and leaders in the U.S. at medium (500+ employees) to large enterprises (2,000+ employees) across IT and business roles, and across verticals.
The verticals that have most aggressively moved to embrace public cloud resources include: manufacturing (27 percent); technology, computing and consumer electronics (19 percent); and finance (14 percent). Those that "plan on leveraging public cloud resources but haven't started yet" most often come from three verticals:
- Technology, computing, consumer electronics (26 percent)
- Retail, wholesale (15 percent)
- Finance, insurance, real estate (15 percent)
"The survey participants looking to leverage the cloud for big data analytics intend to move large volumes of data to the cloud, with 92 percent of survey participants wanting to move more than a terabyte and 20 percent wanting to move more than 100 terabytes," Gigaom Research Analyst Lynn Langit, a big data and cloud architect, writes in the report. "The preferred starting point for enterprise big data projects migrating to the cloud are full or partial migrations on non-mission critical applications (65 percent)."
[ Related: Cloud Platform Simplifies Spark Clusters, Big Data Deployments ]
"They're not just thinking about playing in the cloud," Moghe adds. "They're exploring it as the future production platform for IT."
Analyzing External Data in the Cloud
Langit notes that the types of data-intensive workloads considered for cloud migrations most often are analytics -- real-time operational workloads (43 percent) or processing, such as batch processing with SQL (43 percent). Sandboxes for data discovery are also common (33 percent). Many of these projects include analyzing CRM data (53 percent), site analytics and log files (40 percent) and social media data (26 percent). Langit adds that much of this data is already external to the enterprise.
The preferred tooling for these projects are standard BI tools (55 percent) and analytic tools (49 percent).
"When you talk about why people want to move to the cloud and what they want to run in the cloud in terms of their big data applications, there is a lot of hype in the market about data science and data scientists and the ability of people to do predictive analytics," Moghe says. "When you ask people what workloads they want to run in the cloud and what's driving them to do big data in the cloud, the majority of the participants said they were really interested in exploring leveraging the cloud for their traditional analytics -- largely SQL-based. They want to leverage the cloud for their bread-and-butter analytics."
Despite the large percentage of enterprises that have either moved big data workloads to the public cloud or plan to do so, there are still significant barriers to adoption. Gigaom reports that security, privacy and complexity remain top enterprise concerns for data migration to the cloud. Fully 63 percent of respondents indicated security considerations where a key blocker for potential cloud-based implementations; 35 percent pointed to a lack of existing industry certifications (most commonly SOX or HIPAA); and 23 percent were concerned about the complexity of cloud and its impact on existing processes, tools and infrastructure.
Fifty-five percent of survey respondents say that "better understanding of the security posture of the cloud" would make them reassess their hesitation or refusal to implement analytic processes in the cloud.
"The key concern is transparency about public cloud vendor security and compliance practices," Langit says. "SOX and HIPPA are mentioned most often, but with an expressed need for more standards and understanding of how cloud vendors support these standards. Respondents appear to have a large amount of fear and confusion in this area and would welcome education about standards."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.