Driving big business with big data: Extracting value from information

Imagine this — a typical Tuesday morning, the corporate network is down. A few years ago, this would have sent offices into panic mode, leaving employees helpless to act until the IT department fixes the problem. Not anymore — today’s employees have their email and diaries on their smart phones, client lists on their Blackberrys and documents on their iPads. What started as organisations tolerating the odd iPhone and tablet at the workplace is rather reluctantly evolving to a Bring Your Own Technology (BYOT) culture.

Recent research by IDC reflects the trend of accelerated use of consumer-style technology for work purposes, although large organisations have not yet implemented programs to proactively manage, support and secure these technologies due to limited resources and the scope of end-user demands.

Nearly 28 per cent of Australian workers use iPhones for work, 25 per cent use iPads and other tablets, and over 20 per cent use Twitter and Facebook at work. IDC also estimates that over the next decade, IT managers will have ten times the number of servers to deal with, 50 times the amount of data, 75 times the number of files — and only one additional person to help them.

Data is being created and replicated at a rate faster than Moore’s Law predicted for computing power. It is not just the quantity of data but the nature of data that is changing. It comes in huge unstructured data sets, becoming known as ‘big data’ and replicates at phenomenal speed. Many IT managers may feel exasperated at the monumental task they face in simply taming the deluge of data, but there is potentially huge payback in the form of big data analytics.

So where is all this data coming from? In 2009, mobile internet users numbered 450 million. By 2013, this is predicted to rise to over one billion. Social media services generate vast quantities of unstructured data as online conversations proliferate. Use of social media applications in the Australian workplace varies from 20 per cent of employees to nearly 44 per cent of the workforce. Downloadable software applications allow people to do virtually everything from their smart phones, while geospatial applications like Google Maps generate vast quantities of transient data every day.

Just keeping up with the processing and storage requirements of this data tsunami is a weighty task in itself. Extracting meaningful intelligence from colossal amounts of data is an entirely different challenge. Big Data is where the real value lies for businesses and where forward-thinking IT professionals can prove their worth. However, Big data requires a different treatment and substantially more powerful technology to manage and manipulate than traditional architectures deliver.

So where should one start when it comes to extracting value from the digital universe? As a first step, it is crucial to devise business strategies to understand and interpret the data and define the resulting tangible business benefits. Then invest in the technologies needed to enable the collection, storing and analysis of the required data sets.

Big data analysis literally allows organisations to trend, cross-reference and correlate disparate sets of structured, semi-structured and unstructured data to find new competitive ‘sweet spots’. For example, mortgage companies could reduce lending risk by overlaying application data with foreclosure data in a geospatial context; retailers could aggregate social networking information, blog content and analyst research with socio-demographic data to identify buying trends and motivations for customer loyalty; law enforcement organisations could apprehend many more suspects by evaluating CCTV video footage, photographic evidence, police records, social media comments and other information sources all together.

Get it right and the potential gains are significant including faster routes to market, increased access to new customers, more insightful recruitment, improved marketing effectiveness and increased profitability.

Make no mistake. Business intelligence from big data is big business. It can turn IT from a cost centre into a profit centre. It can drive your differentiation. It could well determine the winners and losers over the next decade! Is that a possibility any IT professional can risk ignoring?

Clive Gold is the marketing CTO EMC Australia and New Zealand.