_500.jpg)
Raymie Stata, CEO of Altiscale
Organizations that want to run Apache Hadoop to analyze their big data without setting up a computer cluster can now procure the data processing framework as a service from a startup co-founded by the former CTO of Yahoo.
Altiscale aims to reduce much of the administrative overhead usually required to run the open source software.
"In all other Hadoop offerings, you have to worry about nodes. 'Do I have enough nodes?' 'Do I have too many nodes?' 'How do I get nodes need to get?'," said Raymie Stata, Altiscale CEO, and former CTO of Yahoo.
With the Altiscale Data Cloud, "There's no worrying about nodes. You submit jobs and they just run," Stata said.
Stata was the CTO at Yahoo who initially sponsored the use of Hadoop, which was first developed at Yahoo. After he left the company, he co-founded Altiscale in 2012 and has received US$12 million in investment from Sequoia Capital, General Catalyst and Accel Partners.
Altiscale employs other Yahoo Hadoop veterans as well. Altiscale CTO David Chaiken deployed Hadoop to drive all of Yahoo's advertising systems. Altiscale head of operations Charles Wimmer ran a 40,000 node multitenant Hadoop cluster at Yahoo.
The company has been running in stealth mode since last year, though now has opened its services for general availability.
The pricing structure is similar to most cell phone plans, meaning the customer pays a set monthly fee for a certain amount of usage and then is billed for any overages. The basic plan is 10TBs and 10,000 task hours for US$2,500 per month. Additional storage and compute can be purchased as well.
The company can handle jobs as small as a few gigabytes to as large as multiple terabytes. The service can scale to whatever size the customer needs. Clients submit jobs through Internet-accessible APIs (application programming interfaces), using the commands for Hadoop's YARN (Yet Another Resource Scheduler) and the HDFS (Hadoop File System).
Using Hadoop as a hosted service could offer a number of benefits over other approaches, Stata said.
Running Hadoop on premises would require provisioning a lot of equipment and requires a fair amount of expertise as well, which the average enterprise may not have, he said.
Many companies, such as IBM and Computer Sciences Corp., offered managed Hadoop deployments, but those can be expensive and may not offer help desk support.
Various IaaS (infrastructure-as-a-service) providers, such as Amazon Web Services or Microsoft Windows Azure, offer copies of the Hadoop distribution online, but these still require administrative expertise to run and may not be kept up to date. Also, a general purpose cloud tends not to be the most efficient way to use Hadoop and can be more expensive in the long run, Stata said.
Organizations that may already have some Hadoop clusters and need more capacity are the intended customers for Altiscale's service. "We can be additive to what they have," Stata said.
One early user has been online dining reservation service OpenTable, which uses the service to manage terabytes of data without dedicated operational support. Research company MarketShare also uses the service for its new MarketShare 360 program, which provides customers with information about where their brands are being discussed in different social media and traditional media outlets.
Altiscale is not the sole company offering a Hadoop service. VMware/EMC spinoff Pivotal also launched its PivotalOne PaaS (platform-as-a-service) offering late last year.
Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.