The torrents of data produced by social networks, sensors, supply chains and every imaginable device are creating new jobs. Gartner estimated this week that big data will create 1.9 million new jobs in the U.S. through 2015.
Michael Rappa saw this emerging trend and in 2007 became the founding director of the Institute for Advanced Analytics at North Carolina State University, which created a Master of Science in Analytics program, the first first academic program devoted to data analytics. Rappa continues as the Insitute's director. Universities around the U.S. are now establishing similar advanced degree programs.
In an interview this week, Rappa, who previously taught at MIT, explains what constitutes a big data job and the types of training people will need to get them.
What constitute a big data job? I'm not sure anyone can give a precise answer at this point. It would be misleading to describe big data positions simply in terms of a set of tools or programming languages.
It's not uncommon for employers to come to the Institute looking for big data talent without a written job description. But they know they need our graduates.
Those that do come with a job description in hand, describe a disparate array of positions up and down and across the organization. The common thread is the need for data savvy professionals who can draw meaningful insights from the flood of data pouring into the organization. The emerging role of "data scientist" aims to define such a person, but convergence on a single definition (especially from the point of educating one) may be premature.
Gartner predicts that the need to control 'big data' will create 1.9 million direct IT jobs in the U.S. through 2015, but there is only enough talent available to fill one third of them. What's your view of this forecast and assessment? Gartner is no doubt careful in formulating their estimate. It's a three-year horizon, so the groundwork on where we'll be in 2015 is being planned today.
It's not hard to get to a large number, in terms of jobs. It's the [prediction of a] shortfall in filling those jobs that will surprise people. Is the educational community failing us [as Gartner Research Director Peter Sondergaard suggests]? I tend to agree. We need to do more to align educational offerings with the rapidly evolving needs of the marketplace.
While we've shown we can deliver big data talent with our MSA degree -- we doubled enrollment this year to 80 -- we should be doing more and faster. If there were 10 institutes like mine each turning out 200 graduates a year -- still it would only represent about 1% of the number of students enrolled in MBA programs nationwide.
What advice would you give to undergraduates in science, technology, engineering and math programs to prepare themselves for a job in big data? Do they need an advanced degree? Undergraduate students do need to continue on to an advanced degree. The good news is you don't need a PhD. We've had great success at the masters level addressing the immediate need in the workplace.
My advice to undergraduates is to line-up your coursework with the necessary prerequisites in math, statistics and computer science, to prepare for graduate education. This means going beyond a year of calculus and into linear and matrix algebra. Don't stop with the mandatory course in probability and statistics, which is common with many majors. Take additional courses in areas like multivariate regression and statistical programming.
How can people already in the workforce put themselves in a position to pursue these jobs? Our model is to pull people out of the workforce for 10 months and immerse them in a rigorous and intensive training program. We turn learning into a full-contact sport and there's nothing like physical proximity to maximize learning. It works, and we've been equally successful with students in their twenties as with those in their fifties.
But not everyone can leave the workforce for 10 months. Some can pick-up the necessary skills on the job, in the right work environment where learning is encouraged. Professional certifications -- offered by vendors and professional societies -- can help workers demonstrate their knowledge and advance their career. Online learning may also prove helpful in allowing people to remain in the workplace and upgrade their knowledge. One can even participate in free online learning opportunities.
What skills or training would you consider core to any big data job? In my mind, big data isn't a new specialty or suite of tools we have to train people into, as much as it's a new organizational reality that everyone will need to adjust to occupationally.
How we train marketing people will change. How we train IT people will change. How we train supply chain people will change. And so on. Even how we train executives will change. Everyone across the board needs more formal training in statistical analysis, and it should start early in the education process. It would be valuable to develop interdisciplinary curricula around the emerging concept of "data science" as a way of blending elements of math and statistics and computer science.
Looking across the organization, some occupational roles will require additional computer and statistical programming skills, other roles will require new data management and data cleaning skills, and yet other roles will require skills in data visualization and interpretation.
Advanced degree programs are being created rapidly. What type of skills can someone gain from your program? What's unique about our Master of Science in Analytics is that we started from scratch to build a fully integrated learning experience from end-to-end, and we positioned employers as our customer.
Our goal was to directly address the employer need, in terms of the kind of talent they sought to hire. Technical skills are only one part of the package. Employers want people who understand the methods and applications of analytics, but also who are focused on the business problem (not the data alone), able to work in multi-functional teams, and who can effectively communicate insights to executives.
Our homegrown algorithm for creating an analytics professional -- both in terms of the content and structure of the program -- balances technical and tool skills with teamwork and communication skills. It's been a potent formula for us that yields phenomenal results in terms of student outcomes in just ten months.
What's the demand right now for graduates for your program? From my vantage point in analytics education, the "great recession" never happened.
Since founding the Institute in 2007, year after year demand for our graduates intensified. Students who graduated last May had an average of 16 initial job interviews, and over 80-percent had 2 or more job offers. Forty employers came to the Institute attempting to hire from a pool of 38 students. Three-quarters made an offer of employment, and half succeed in landing one or more graduates. However, the top six employers hired two-thirds of the class. That's why we've moved to double our enrollment this year. For the fifth consecutive year, over 90% of our students were employed by graduation. Average salaries continue to increase year after year.
Hardly a day goes by when I don't get a call or a message from an employer looking to hire a MSA graduate. Some of it has to do with our track record for producing analytics professionals that can hit the ground running, but there's no doubting that demand for talent is increasing with each year. Data is perhaps our most valuable renewable resource driving economic growth today.
Patrick Thibodeau covers SaaS and enterprise applications, outsourcing, government IT policies, data centers and IT workforce issues for Computerworld. Follow Patrick on Twitter at @DCgov, or subscribe to Patrick's RSS feed . His e-mail address is pthibodeau@computerworld.com.
Read more about big data in Computerworld's Big Data Topic Center.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.