A zero-day vulnerability in the popular FireEye security appliance was in the news several weeks ago, but it’s still worth discussing. That’s because some people in the security community were outraged that a security product could have an exploitable vulnerability. But why should products from security vendors be any different from other products? Because security vendors should know better? Please don’t tell me you’re going to trust your security career to that naive notion.
You shouldn’t have blind faith in anything you allow onto your network, and that includes security appliances. This was made amply clear to me a few years back, when a vendor of an email security appliance tried to convince me (as the CTO of a small company) to team up and help sell the appliance. I had our engineering team test the appliance, just as we would any product we were considering using or supporting. The team quickly found that the appliance was running an older SSH daemon that had known vulnerabilities. I notified the appliance team, and they sent back a “fixed”version that failed a second test a few days later. Needless to say, our partnership never happened.
In the FireEye vulnerability, the Apache network service was itself running as root, and there was a vulnerable PHP script that could be exploited, resulting in the attacker being able to attain root privileges on an affected system. That’s not good, but I don’t think it’s any worse for having been overlooked by a security vendor. Security will always fall short of perfection, as my personal mantra makes plain: There ain’t a horse that can’t be rode, and there ain’t a man that can’t be throwed.
And, yes, that applies to security products the same as it does to servers, applications and all the other things we allow on our networks. Here are a few things to bear in mind, in no particular order:
- Security products, even security appliances, are based on software. Just like any software, mistakes can and do happen. Trust, but verify.
- Security appliances should undergo rigorous security testing, just like any other system on a network for which you’re responsible.
- Minimize the attack surface when deploying security products. Consider security devices with dual network interfaces, one for production data and one for administrative data. The Web interface on the FireEye appliance may well have been better off on an administrative network segment, thereby removing the attack vector from your adversaries. The production interface should serve only mission-critical services.
- Security products should be regularly updated, just like any software. They need to be maintained, and not just for feature updates. Security product vendors push out patches from time to time that resolve security defects. (Apparently, this was the case with the recent FireEye vulnerability.) In consulting for various companies, I’ve often found security products that were several major releases behind the current shipping versions of the products. Whether this was due to budget, fear of breaking something or just plain laziness is moot.
- Don’t assume that outsourced security appliances are up to date. That’s foolish. At the end of the day, you are responsible for the security of your network. Verify that your security vendors are keeping things in ship shape.
- Watch the watchers. Even security devices can be attacked. You should be monitoring network traffic to and from them just as you would with any business application. If you’re seeing an uptick in HTTPS traffic to one of your security appliances, for example, that could be an indication of a problem.
- Make them invisible. When possible, your network monitoring devices should be invisible. I’m a big fan of connecting network monitors to networks using taps that prevent any outbound data from being sent onto the production network they monitor. This doesn’t make them immune to attack, but it does make the attack a heck of a lot more difficult. It’s the difference between a surveillance camera that everyone can see and a surveillance camera that is hidden from view. You’ll most definitely see different things when your adversaries don’t know they’re being watched.
Security appliances offer plenty of value. Since the FireEye incident, some in the security community have suggested we should ban them from our networks. That’s just silly. We should continue to use them, but proceed with caution. And don’t ever assume that a security product is more secure than any other type of product.
With more than 20 years in the information security field, Kenneth van Wyk has worked at Carnegie Mellon University's CERT/CC, the U.S. Deptartment of Defense, Para-Protect and others. He has published two books on information security and is working on a third. He is the president and principal consultant at KRvW Associates LLC in Alexandria, Va.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.