We've heard it many times in many forms -- expect to be breached, expect that you've been breached, expect that you are being breached.
The unfortunate reality is that most organizations don't even know that they've been compromised and therefore don't do anything to block spreading of the malware, control the damage, prevent loss of information, or even recover from the technical problems associated with the compromise.
IN PICTURES: The worst data breach incidents of 2012 -- so far
Shawn Henry, former executive assistant director (EAD) of the FBI and now president of CrowdStrike Services, told the 6,500-plus attendees of the recent Black Hat conference that the FBI has knocked on the doors of numerous companies to let them know their data had been discovered on the Internet (usually discovered in unrelated investigations). "Months, or even years later -- with unfettered access, and unbeknownst to the people that own the networks -- organizations are being alerted to being compromised and their data being stolen," said Henry. This is both shocking and unacceptable.
When people think FBI they often think about national security and nation-state adversaries. And there's no lack of speculation about these nation-states being the most threatening sources of these corporate attacks. This assessment doesn't come without cause. According to Henry, "dozens of our adversaries are extracting information from the U.S. every day, stealing corporate strategies, grabbing intellectual property, and looking for any competitive advantages they can find." [Also see: "Advanced persistent threats force IT to rethink security priorities"]
Henry also noted the threat implications where the U.S. critical infrastructure is concerned. "We're seeing an uptick in threats against industrial control systems (ICS), the devices that control the nation's critical infrastructure," Henry said.
The increase in attacks against ICS points to increased sophistication of the attackers. "Attacking a GE control system device is very different from attacking a website," said Francis Cianfrocca, CEO of Bayshore Networks. "It is easy to find a lot of effective material in the public domain to attack websites and enterprise apps, but the knowledge to attack ICS typically has been far less developed."
Critical infrastructure systems are generally much more open in design, and therefore, are much more vulnerable to attack than commercial/enterprise systems. Some might even say that ICS are wide open to attack because of their design and implementation. "Even though they are vulnerable, if you are going to attack ICS, you will require a lot of specialized knowledge as the devices and systems are often highly customized," Cianfrocca said. "All critical systems are different and use different protocols; vendors violate the protocols in different ways, essentially minimizing the hactivists and increasing the focus on nation states."
However, enterprises don't have the luxury of focusing solely on the nation-state adversaries as there are many more threats and adversaries they need to consider. Henry noted that organized crime is not too far behind the nation-state adversaries both in terms of skill and capability. And, once organized crime attackers get a few successes under their belts, funding is often not a problem either. "Businesses have formed that are offering 'hacking as a service,' and there are plenty of insiders and lone wolves taking legitimate jobs with a direct aim to extract sensitive information from the private sector," said Henry. [Also see: "EPA data breach highlights worrying trend"]
It's difficult for companies to protect themselves given the level of sophistication of many adversaries. "With the most sophisticated attacker at the threat controls, organizations won't stand a chance," said Phil Lieberman, president of Lieberman Software. "They would need to have NSA-like teams on staff -- or NSA-like partners -- if they are to prevent targeted attacks," Lieberman said. Henry shared a similar view, noting that a sophisticated adversary can and will easily jump over the fence -- hopping over or around the firewall with ease.
They're inside, now what?
Assuming the adversary makes it in, the question remains: How long after a breach occurs can the organization remediate and prevent further damage? This is where security response becomes critical. And whether it's done properly can make or break the bank. The response process can be broken down into four components:
- Know you've been compromised
- Get back online quick
- Stop the spread
- Detect the adversary (required) and track them down (optional)
* Identify the compromise: This could prove to be the most challenging of the steps as there is a small window of opportunity to spot the behaviors of an attacker between the point when the infection is first established and the point when the attacker finds its "hiding spots" within the network. As with most things, if you can detect the adversary's analysis of the network before the real damage is done, you are ahead of the game. The challenge: spotting it in time, if you can even spot it at all.
Security information and event management (SIEM) systems play an important role in this process, but historically haven't collected enough data in terms of both depth and speed in order to see these latent, stealthy attackers in the environment. While the SIEM system may bring in a better view of what may be happening on the network, the only real chance a company has for getting the data required is to leverage big data systems, processes and, of course ... big data feeds.
"Most organizations watch their network traffic for call-outs being made to the malware's command-and-control-center (CNC)," Cianfrocca said. "The problem with this method of detection is that the attackers space their communications out over time, making them nearly impossible to spot in the midst of other mounds of data."
So, if this method doesn't work well, what other options do we have? The answer may lie in the apps, more so than in the network.
Most of the nastiest and stealthiest malware will try to covertly figure out which app servers are vulnerable -- this is the activity organizations should be monitoring. The challenge here, of course, is that the attackers will instruct their code to use patterns of access crafted to look like normal activity of authorized users -- but the attackers are actually accessing these applications in subtly different ways.
"It's possible for organizations to monitor the application traffic, looking for certain combinations of error responses, anomalies in time patterns, variations in the spacing of access, etc.," Cianfrocca said. "This may be easier said than done though, as an organization will need to look for patterns of application access which are out of character for the app, which will require statistical modeling of how the app is supposed to perform."
While it may take some time for the industry to arrive, eventually we should be able to construct a statistical model for normal app use, collect that data, and use it to compare and detect any out-of-character accesses.
* Get back online: Once an attack has been spotted, the affected systems need to be brought back to known good states. This is where safe, secure backups are required; this is where a solid disaster recovery plan becomes invaluable.
"Organizations need to plan, create and be prepared to utilize secure, continuous backups," says Dmitriy Ayrapetov, director of product management at Dell SonicWALL. "Don't forget, the backups must have been scanned for previously compromised systems before being restored, otherwise you open things right back up again."
One option that is getting more attention of late is disaster recovery as a service (DRaaS). "We're seeing a lot of movement from internal recovery models to one in which the cloud is used to provide equivalent recovery capabilities delivered as a service," said Mike Gault, CEO of Guardtime. "When the data is being restored, organizations are demanding that the service providers maintain independent proof that their data has not been changed, manipulated or otherwise tampered with -- this must be achieved through the use of technology capable of delivering such proof."
* Stop the bleeding: While most organizations aren't willing to admit defeat, they will accept the fact that compromise is looming. So, is there anything to be done that can help protect an organization from widespread damage once an attacker is in? Maybe some of the advice from Black Hat can help.
"It is critical to compartmentalize the network with air gaps between the compartments," said Ayrapetov. "Once compartmentalized, organizations must apply the same level of security across each of the compartments to protect them from the other compartments -- just as you would for outside entry of the DMZ."
"The same compartmentalization requirement holds true for the wireless network," added Lieberman.
It is also critical that all information security and proper-use policies and rules are defined properly, implemented properly and regularly double-checked against the configurations. "Some security settings get turned down over time in order to enable business users and applications to operate," said Ayrapetov. "Sometimes it is good to turn up the volume on the view and to change the layout of the dashboard, even if it means seeing a lot of overwhelming data. This is where the compromise may be hiding."
Another tip Lieberman suggested is for organizations to ask themselves how far a breach could travel if it gets in. "Look at a potential breach from within and outside each compartment," said Lieberman. "What are your chances of keeping the infection from spreading beyond one compromised compartment to another compartment?"
* Detect the adversary: A primary goal of the security response plan should be to improve visibility throughout the process. This means leveraging centralized logs, tuning correlation engines so that they present solid information while reducing distracting false positives, and gathering external threat intelligence to help make sense of it all.
Unfortunately, it's not always that easy. There is often a lack of experience within the organization with respect to the entire incident response and incident handling framework. "Most organizations think that they can just 'handle it' when an incident occurs," said Stephen Grutzius, CMO at Cybersponse Inc. during a follow-up interview. "The root of the problem lies in the lack of knowledge including identification of systems; memory collection; malware detection and analysis; forensic imaging and analysis; and multi-department collaboration -- these all prevent effective, timely response."
"Companies should be prepared to create an investigation-ready environment," added Jim Aldridge, a manager at D.C.-based Mandiant. The plan should be formal yet flexible and it should let the smart people work. "The security response team should define playbooks that are meaningful, outcome-based, and provide clear metrics," Aldridge added. "Be prepared to share the information with anyone and everyone that can benefit and/or contribute."
Humans to the rescue?
People are looking for "drop-in security" where they can just install it, set a few dials, and move on, but it is important to separate hype from reality.
And the reality? Systems require human intervention. Scripts and rules are not enough. But, humans can't scale like computers can. The cloud only further exacerbates the problem.
"Most organizations don't have a dedicated forensics expert on staff," added Grutzius. "This makes it extremely difficult, if not impossible, to effectively triage and analyze a security event."
One of the more interesting takes on the human element compared to computer-only systems is Henry's description of a network-speed intelligence-sharing system:
- Human-to-human collaboration with little to no system automation involved is not acceptable as it can't scale
- Human-to-machine collaboration is irrelevant as the translations are not always accurate
- Machine-to-human is not enough either as humans are prone to make mistakes
"We need a machine-to-machine-to-human system," said Henry.
It's important to recognize that the determined/sophisticated attacker is also human and will often possess distinctive characteristics geared toward avoiding detection. The attacker will hide in the network for long periods of time -- trying to extend its reach and gain intelligence on a continual basis as a means to map out the network so it can take down parts of the network and access sensitive information when the timing is just right.
Regardless of how and where humans interact with machines, organizations must be prepared to deal with so much more than the breach itself.
Be prepared for more than data theft
Sure, data theft is a huge deal. But, data manipulation can be just as serious, if not more serious for some institutions. Consider an organization's scientific research data being unknowingly changed to throw a research project off-track. This could potentially destroy any chance the company has to succeed with a project. [Also see: "Is your intellectual property secure? Whitelisting can help secure against advanced persistent threats"]
"These days, you can't just protect the information from being viewed, you also need to protect it from being changed or modified," said Henry.
To this end, data integrity has become a component of a number of industry guidelines, government regulations and other internationally published standards. "Take financial institutions," says Gault. "They are bound to numerous data integrity requirements, so it is critical for these firms to have a secure audit trail around all aspects of their financial transaction histories; an audit trail that will stand the test of time, stand up in a court of law, hold up against regulatory scrutiny; an audit trail that cannot be manipulated by insiders, even when they maintain trusted access to core financial systems."
And of course when you have it all mapped out, it's important to execute the plan. "Companies must put their plans to the test, conducting table-top exercises for key scenarios the companies expect to encounter," Aldridge added.
As with anything specialized and complicated, practice makes perfect. Therefore, successfully getting through the response process takes practice. "You play the way you practice," said Henry during his keynote. "Training exercises are critical -- organizations need to conduct table-top exercises so that when things go bad within the process they know how to react to different situations."
When it comes to security response, it might be good advice to follow the guidance contained in the Ulysses S. Grant quote that Henry referenced at the conference: "The art of war is simple enough. Find out where your enemy is. Get at him as soon as you can. Strike him as hard as you can, and keep moving on."
An additional thought worth considering can also be attributed to Grant: "In every battle there comes a time when both sides consider themselves beaten, then he who continues the attack wins."
Don't give up. Don't give in. Don't get even. Just get back to business.
Sean Martin is a CISSP and the founder of security consulting, research and analysis firm imsmartin. Write him at sean@imsmartinc.com.
Read more about wide area network in Network World's Wide Area Network section.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.