Lack of data scientists could derail big data projects: IBM

Analysis of customer data is being used for competitive advantage by companies around the world, but a high number of vacancies for data scientists, particularly in the US, could mean challenges ahead as more companies look to use big data, according to the findings of a new IBM survey.

Big data to drive US$34 million global IT spend in 2013: Gartner

Speaking at the vendor’s Information On Demand 2012 conference in Las Vegas, IBM's business analytics information management foundation US vice president, Michael Schroeck, shared insights from the company's report called <i>Analytics: The real world use of big data.</i>

This survey was conducted in 130 countries with 1144 business and IT executives by IBM’s Institute for Business Value in partnership with the University of Oxford’s Said Business School in 2012.

According to the survey results, customer analytics are driving big data initiatives as companies indicated a need to understand and anticipate customer behaviour/preferences.

However, respondents also indicated that big data analysis requires strong analytic tools and staff capabilities.

"We found that the tools and technologies were advancing but the people skills required to use those tools are not keeping pace,” Schroeck said.

“In the US alone, there are over 300,000 openings for data scientists currently listed. This is also why we felt the need to partner with a university like Oxford on this study and work with universities around the world because it is the universities that are going to continue to define their data analytics curriculums.”

Big data roadmaps

Schoreck then turned to the survey’s findings on big data implementations, pilots and roadmaps by companies.

“What we found is that 24 per cent of the organisations surveyed have not yet formed projects or began investing in big data,” he said.

However, the majority of those organisations indicated that there were pockets of individuals who are educating themselves on big data, increasing organisation awareness and starting to look for business areas that can be enabled and supported by big data.

“Three out of four organisations have big data projects underway and one in four is either in pilot or production,” he said.

“Twenty-eight per cent of respondents are implementing pilots while 47 per cent [are] developing plans and roadmaps.”

Competitive advantage

According to the survey results, 63 per cent of respondents indicated they had gained a competitive advantage from using information and analytics.

“If we compare that to two years ago, only 37 per cent of respondents felt like they were getting competitive advantage from big data,” Schroeck said. “There has been a 70 per cent increase over two years.”

According to the survey, those organisations that implement big data solutions are 15 per cent more likely to gain a competitive edge than those who don't.

Hamish Barwick travelled to Information on Demand in Las Vegas as a guest of IBM

Follow Hamish Barwick on Twitter: @HamishBarwick

Follow CIO Australia on Twitter: @CIO_Australia