Enterprises confident about tackling big data Initiatives

In its second annual Big Data Insights and Opportunities study, CompTIA, the nonprofit association for the IT industry, found that 78 percent of organizations feel more positive about big data as a business initiative than they did last year and 42 percent said they were engaged in some sort of big data initiative already.

"Year over year, we confirmed that there is more awareness and familiarity with big data," says Tim Herbert, vice president of research and market intelligence at CompTIA and author of the study. "It has helped to drive more comfort with data initiatives among companies."

The study was conducted in two parts. In June 2013, CompTIA conducted an online survey of 500 U.S. business and IT executives responsible for technical or strategic decisions affecting data at their company. Then in April 2013, CompTIA surveyed 500 executives at U.S. IT firms that have some level of involvement with the IT channel.

Harnessing Data Will Make Companies Stronger

CompTIA's findings suggest that the increasing awareness and familiarity with big data is being driven by the executive suite: It found that eight in 10 executives agree with this sentiment: "If we could harness all our data, we'd be a much stronger company." And 75 percent agree that data is among their firm's greatest assets.

"Data has always been important in the business world, but the big data trend has elevated its importance, pushing companies to be smarter in how they manage and use data," Herbert says.

The drive seems to be paying dividends: 57 percent of survey respondents report that they are either very close or exactly where they want to be in data management and utilization, up from 37 percent a year ago. However, Herbert notes that leap forward may owe a great deal to increased awareness rather than rapidly improved data management and utilization.

To back up that interpretation, Herbert points to another finding: in the 2012 survey, only 16 percent of respondents felt their organizations harbored a high degree of data silos. In the 2013 survey, that number leapt to 29 percent.

"Companies are taking a harder look at where they are with data and whether they are positioned to undertake true data initiatives," Herbert says. "I don't think that in 12 months' time there's been an explosion of data silos."

And while a large number of companies feel they're very close to where they need to be with managing and using data, many also acknowledge that they need to do better in areas. Seventy-four percent of firms feel they need much better real-time analysis of incoming data; 74 percent still feel that converting volumes of data into actionable intelligence is a challenge; and 53 percent feel they have more data than they know what to do with.

Failure to Manage and Use Data Costs

What is clear, though, is that businesses largely agree that they cannot afford to poorly manage and use their data. They see the top five costs of failing to manage/use data as follows:

• Wasted time that could be spent in other areas of the business
• Internal confusion over priorities
• Inefficient or slow decision-making and a lack of agility
• Inability to effectively assess staff performance
• Lost sales and reduced margins due to operational inefficiencies

In most cases, with a few exceptions, the concerns about shortcomings managing and using data scale with the size of the organization.

"As businesses scale above 100 or so employees, complexities arise on a number of levels," Herbert says. "Consequently, medium-size businesses voice more concern over shortcomings in the management and utilization of data. For example, 46 percent of medium-size businesses (100 to 500 employees) cite confusion over internal priorities as a consequence of poor data management compared to 34 percent of small firms (less than 100 employees)."

"For IT solution providers or vendors working in the big data space, this should serve as an important reminder to connect data-related solutions to business objectives, emphasizing outputs over the nuts and bolts of the inputs."

Big Data Skills Gap Remains a Concern

While organizations are growing increasingly comfortable with big data, they still report a skills gap in the area. Six in 10 respondents told CompTIA they need to boost employee skill levels on the technical or business side of data management and analysis. Also, 66 percent plan to invest in training for current employees, while 43 percent intend to hire new workers with data-specific expertise.

"CompTIA's 2012 State of IT Skills Gaps study found companies currently place great importance on skills related to managing servers, data centers, storage and information," Herbert says. "The emerging areas of analytics and big data-specific tools such as Hadoop are lower on the list, which makes sense given the relative newness of these technologies. It's likely future iterations of CompTIA skills gaps research will show gains in importance of big data-related skill sets."

Among firms in the IT channel, CompTIA found an increasing number of big data-related offerings. Nearly one-third of channel partners in the study report providing big data application deployment or integration services, and Herbert says you can expect an additional 14 percent to offer big data consulting or advisory services over the next 12 months.

Thor Olavsrud covers IT Security, Big Data, Open Source, Microsoft Tools and Servers for CIO.com. Follow Thor on Twitter @ThorOlavsrud. Follow everything from CIO.com on Twitter @CIOonline, Facebook, Google + and LinkedIn. Email Thor at tolavsrud@cio.com

Read more about big data in CIO's Big Data Drilldown.