A recent survey by Forrester found that 7 per cent of IT executives and 9 per cent of business leaders feel they have gained a true return on investment from Big Data. That means there's a lot more business can be doing to glean insights from the massive amount of data that's potentially available to them.
A panel discussion at Interop in Las Vegas featuring a combination of startup entrepreneurs and a representative from one of the leading big data companies concluded that the Big Data market is still in its early days, but the opportunity is huge.
End users have the ability to gain new levels of detail about their current operations, and tailor their products and services based on actual data that can help them make business decisions. A growing industry around big data analytics, data storage and data visualization tools is gradually growing as well to support these new use cases. "The market is evolving, but it's still complex and expensive," says Bruno Aziza, vice president of marketing for big data startup SiSense, a maker of Cloud-delivered business intelligence software for analyzing big data. "Businesses are starting to take control of the issue and figure out how they can use it."
[ MORE INTEROP:Interop 2013: The network has never been more important ]
The buzz about Big Data is being fueled by a variety of factors. Two of the biggest are the explosion of data available to companies and the maturation of data storage and analysis platforms. The Internet of things (IoT) represents a world where almost any type of device is connected to some sort of network, allowing it to create real-time data. Perhaps the biggest-name platform for handling this data is Hadoop, an open source distributed computing architecture that excels in handling massive data sets.
These two trends -- a massive increase in the amount of data available to businesses, and a platform for handling it -- has "democratized" Big Data, Aziza says. The type of data analytics that used to only be available to the biggest companies in the world can now be done by small companies. The rise of cloud-delivered services, like SiSense's plus the continued drop in the price to store information, combined with a management platform like Hadoop to manage it, means more businesses are holding on to more data.
"If I can store data at a commodity price, it starts to make sense to do it," Aziza says. "Data has value over time. In the past, I may say, 'I don't know if I'll have a need in the future for that data.'" Now, with platforms like Hadoop, customers can save the data in case they find a use case for it down the line. Having all that data, plus a platform to analyze it, means companies can run queries based on a whole set of data, instead of before just taking a subset of data to analyze.
There's also been a rise of new unstructured database options in the market. Many traditional enterprise databases have focused solely on structured data -- that is, the type of information that neatly fits into rows and columns, for example. More and more though, our world has complex data that does not nearly fit into categories and new types of databases focus on managing this unstructured data formats, like records from social media interactions, for example, by using NoSQL databases.
The rise of Big Data is a "paradigm shift" in the way companies can manage data, and many haven't caught on, says Justin Erickson, a product manager at Hadoop distribution company Cloudera. "If you're just taking your traditional data and putting it on this new platform, you're not using it right," he says -- that's just scratching the surface of potential use cases. Too often he sees companies data that they used to manage in SQL servers or other databases, and putting it in Hadoop clusters. But that's not nearly taking full advantage of the platform. Hadoop is optimized to handle queries and storage of massive amounts of data compared to traditional databases.
Some companies are starting to get the hang of the value big data can bring to their organization, says Stefan Groschupf, CEO of Datameer, another big data analytics startup. Expedia was able to more granularly track its Web traffic to glean additional insights into its customers to offer more targeted promotions, better place content on its site and even learn about visitors to the site who did not even make a purchase.
"Without doubling their data warehousing cots, Hadoop was the only way to manage all that additional data," says Groschupf. Online matchmaking site eHarmony, Groschupf says, uses Hadoop to run daily queries of its users to find the perfect matches for one another based on information customer supply to the site.
"It gives an opportunity for the little guys to catch up to the big guys," Aziza says. But given the results of that Forrester survey, it's still a vast majority of companies that have learned the tricks of the Big Data trade.
Network World senior writer Brandon Butler covers cloud computing and social collaboration. He can be reached at BButler@nww.com and found on Twitter at @BButlerNWW.
Read more about software in Network World's Software section.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.