Cost, integration issues halting big data projects

Big data projects are being held back by the high cost of setting up infrastructure to support the capturing of potentially hundreds of millions of data points each day.

A lack of a business case and the need to integrate data sources are also holding back adoption, according to the 2014 Australian Big Data and Analytics study, published on Tuesday by Telsyte.

The research firm spoke to 324 CIOs and senior IT executives in 30 industries for the study, which also found that 25 per cent of organisations with more than 20 staff are using big data applications and services.

Telsyte analyst Rodney Gedda said it can be costly to set up the required infrastructure for projects aimed at deriving business intelligence and analytics from high volume data sets.

“You’re not just capturing a few hundred data points every day, you are capturing millions, potentially hundreds of millions so you can fill up large storage arrays pretty quickly,” he told CIO Australia.

“There are tools out there that can help you manage that – there are also cloud services that can potentially ease the burden of an up-front capital expenditure,” he said.

Still, the research firm expects that 65 per cent of organisations with more than 20 staff will undertake big data projects by 2018.

This three-fold increase will be driven by several factors including more customer interactions occurring through digital channels, a more from reactive to “proactive or predictive” business intelligence and analytics, and better use of available real-time data from systems and devices, Telsyte said.

Gedda said with the perceived high cost of big data tools, the market is primed for low up-front cost options such as those offered by cloud providers and pay-per use options. This would help identify a return on big data investments.

Currently most big data storage is done on-premises, but around 30 per cent of organisations intend to move this to the cloud within the next 12 months, indicating a growing appetite and comfort with moving big data to the cloud, Telsyte said.

Meanwhile, more than 40 per cent of organisations are using big data tools for IT network monitoring and management, business intelligence, and sales and marketing applications.

The fastest growth areas for big data applications are telemetry network monitoring and customer behaviour measurement, the research firm said.

Follow CIO Australia on Twitter and Like us on Facebook… Twitter: @CIO_Australia, Facebook: CIO Australia, or take part in the CIO conversation on LinkedIn: CIO Australia

Follow Byron Connolly on Twitter:@ByronConnolly