Over the past few years, Hadoop and big data have exploded onto the tech landscape, garnering attention on par with cloud and mobile. But that may all be about to change, says Mike Olson, co-founder, chairman and CSO of Hadoop distribution vendor Cloudera.
"I think that this year we're going to see Hadoop disappear," Olson said in his opening keynote at the O'Reilly Strata Conference + Hadoop World here in New York City's Jacob K. Javits Convention Center. "It's a pretty ridiculous statement in a venue like this."
When Olson spoke at the first Hadoop World in 2009 (the event merged with O'Reilly's Strata Conference in 2012), he spoke to an audience of just more than 400 in a New York City hotel. Today, just six years later, he spoke to an audience of nearly 5,000, with many more joining via livestream. So how can Hadoop -- the technology at the heart of an ecosystem of technologies transforming the way organizations interact with their data -- up and vanish?
"When I gave the first keynote at a Hadoop World event, nobody had heard about this delightful yellow elephant," he says, gesturing at the Apache Hadoop logo. "Over the years, we've switched the story we tell. We used to talk about those geeky projects because we had to explain what they were. Now we talk about what they're doing."
That's what he means when he says that Hadoop may just "disappear" this year. It will become a foundational layer of the data infrastructure powering the applications that allow you to turn data into insight.
For the most part, business users don't care whether the data they draw upon comes from a Teradata enterprise data warehouse or an Oracle database. The same will become true with Hadoop, Olson says.
"In a very real way, Teradata, EMC, Oracle and the rest have disappeared," he says. "Hadoop is making that transition now."
Even as Hadoop disappears, the market opportunity is expanding exponentially, he says, noting that Gartner research shows that 73 percent of enterprises today either use or plan to adopt applications on the Hadoop platform.
"Those businesses, by and large, don't want to build custom apps," Olson says. "I believe we are participating in the creation of a $1 trillion market."
Olson notes that the current market for relational databases and applications built on top of them is a US$100 billion business today. Many Cloudera customers have seen the data available to them increase by a factor of 1,000 in recent years. "It stands to reason that this will be 10 times more," he says.
"We get to build applications on [the Hadoop] foundation now that anyone can afford. Buyers are ready to adopt those applications on this foundation," he adds. "You should build it or, rather, we should all build it."
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.