For as long as there have been computer systems, there have been system administrators. These hardy souls are the glue of data centers, provisioning and managing systems: a mix of hardware, storage, OS, middleware, and application software. The best system administrators are like human Swiss Army Knives, armed with every skill needed to keep a system up and humming. In some respects, system administrators function like auto mechanics, needing keen diagnostic skills and repair capabilities to keep a complex mixture of disparate systems operating as a whole.
Of course, data centers have become far more complex over the past decade as systems have been deconstructed into functional components that are segregated into centralized groupings. For example, storage has, in many organizations, been migrated to centralized storage like a SAN or NAS. This has inevitably meant that personnel become more specialized in their tasks and skills. However, for every organization that has a separate storage group, there's another in which the arrival of centralized storage has just meant a new set of tasks heaped upon the sysadmin group.
Even in those IT organizations in which functions like networking and storage have been separated from system management functions, sysadmins still monitor, manage, and repair the software stack. IT organizations still rely on human insight, skill, and experience, to keep apps up and running.
I just read an IEEE article, though, that caused me to reconsider the future of sysadmins. The article, surprisingly, did not address developments in IT, but instead looked at the developments in automobiles; specifically, how autos are now rolling data centers in and of themselves. It notes that today's high-end cars (meaning, the low-end cars five to 10 years from now) have around 100 million lines of code in them distributed among 70 to 100 Electronic Control Units (ECU)-essentially, special purpose computers devoted to tasks like lighting, engine management, and yes (as Toyota is now grappling with), braking. Today's low-end cars have around 30 to 60 ECUs. In the near future, the article quotes research firm Frost and Sullivan, cars will contain 200 to 300 million lines of code.
As evidence of what cars have turned into, the author notes that his recent car came with a 500 page manual, along with a supplemental 200 page document describing the car's entertainment and GPS function. Clearly, this is not your father's Oldsmobile-and probably not even your Oldsmobile (or Honda, Ford, Audi, or whatever). Cars are becoming staggeringly complex admixtures of materials and software designed to move us safely, efficiently, and attractively around the earth.
It seems the genesis of the article is the Toyota recall dustup. For perhaps the first time, this issue has brought forward into people's consciousness just how much software is running in cars, or perhaps one could say, how much software is running cars. The potential for bugs or complex interactions between the software in different ECUs resulting in unexpected outcomes is obvious.
The point of this post is not to bewail the putative shortcomings of Toyota; that task has been done by elective representatives. I want to focus on what is almost a throwaway line in the article, near its end. Here is the quote:
Broy told me that more than 50 percent of the ECUs that mechanics replace in cars are technically error free: They exhibit neither a hardware nor a software problem. Mechanics replace the ECUs simply because they dont have a better way to fix them, he says.
The garages and the maintenance people are really at a point where repairing a car is too complex and demanding [for them], says Broy. Remote diagnostics and repair are likely to render mechanics obsolete for many tasks.
In the not-so-distant future, says Broy, when you have a problem with the computer system in your car, you will go to your garage, where your car will be connected to a network so that off-site OEM specialists can download data, do the analysis, and then upload a software correction.
The complexity of cars is outstripping the abilities of everyday mechanics. When confronting a problem too difficult to understand, they rip and replace. And in the future, on-site mechanics won't even control that. The car will be remotely diagnosed by a specialist, who will then direct the local mechanic regarding what parts to replace, adjustments to make, and so on. In other words, the role of the journeyman mechanic will bifurcate, with a few highly-skilled specialists operating from a central location, and many low-skilled part swappers based locally. The pretty good mechanic will have no place in this world.
If we turn to cloud computing, the same dynamics are in place. Software applications are becoming increasingly complex. Their scale is becoming staggeringly large (see this NYT article on Google's translation application). Diagnosing application issues is becoming harder and harder. Comprehending them requires specialized skills- indeed, may require a team, each with specialized skills.
It's not hard to envision, therefore, the death of the journeyman sysadmin. In the future, the specialized skills will be centralized-in large corporations, in a remote NOC; for SMBs, in a vendor or MSP location. In fact, in the future world of applications being delivered in virtual appliances, no local people may touch software products. Onsite personnel will focus on physical device placement (and replacement).
Obviously, this is an extreme picture, but not one that is out of the question. If you read last week's post on the changing nature of applications in a cloud world, it's obvious that the number of applications is going to skyrocket, and, to repeat what's already been stated, their complexity will skyrocket as well. And, before you reject this vision out of hand, remember that we've seen this happen before-in manufacturing. Factories used to be filled with hundreds of workers doing tasks by hand. Today's factories are stuffed with computerized machine tools and high degrees of automation. Far fewer workers occupy those factories-but the ones who do are highly skilled people who know how to run the computerized tools and manage the overall factory system.
Tomorrow's sysadmin won't be someone who knows enough about a bunch of different things and can write glue scripts. He or she will be a systems engineer, akin to a physician-someone doing assessment, diagnosis, and treatment of highly sophisticated software agglomerations. My only question: will there be enough such people available? No one has yet explored a future where computerization is so pervasive that the entire globe cannot graduate enough technical talent to manage it.
Bernard Golden is CEO of consulting firm HyperStratus, which specializes in virtualization, cloud computing and related issues. He is also the author of "Virtualization for Dummies," the best-selling book on virtualization to date.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.