We tested next generation firewalls by looking at seven separate areas that we felt would be important to network managers trying to deploy these products in enterprise networks.
We evaluated the basic firewall functionality of each product by examining the features for building security policy, VPNs, applying NAT and QoS, enabling dynamic routing, and supporting high availability. We tried to build VPNs with other IPSec-based firewalls and tried to synchronize each device with Cisco IOS BGP routers running both IPv4 and IPv6 protocols. We looked at global management features, where the vendor provided a global management system, and evaluated other enterprise areas of concern, such as VLAN and link aggregation support. We also looked at IPv6 support separately.
We investigated visibility features in the firewalls and any reporting or global management system provided by the vendor to see how well each product gave a view into the traffic flowing through the network. Since this is a next-generation test, we focused more on application identification than simple traffic statistics. We looked at differences between "on-box" and "external" reporting systems, and evaluated areas such as debugging and long-term reporting.
Next-gen firewalls: Off to a good start
When we tested next-generation firewall control features, we tried to understand the model for applying application identification and control features. Since next-generation firewalls all use categories to help group applications, we tried to evaluate whether these categories made sense and were at an appropriate level of granularity. Within applications, we looked for features such as the ability to block different sub-applications and directions (upload/download or read/post).
We looked into the application layer controls (such as blocking applications) and variations offered by different vendors, such as applying QoS on a particular application. We also looked at other next-generation features, such as controlling traffic based on username or group, IP address reputation, or geographic region.
Because SSL decryption is an important part of application visibility, we configured SSL decryption using our own certification authority and looked at how well this decryption worked. We evaluated important PKI features, such as handling of self-signed certificates and revoked certificates. We also looked at the features that each firewall offers to configure SSL decryption, such as exempting some traffic. Where these features existed, we tested them as well.
To test next generation application identification and control, we identified 40 separate applications in nine separate areas. We included commonly mentioned applications for next generation firewalls (such as Facebook, peer-to-peer networks, streaming video, and public webmail servers), along with enterprise applications (such as Microsoft Exchange, Terminal Services, VoIP, and Sharepoint). We also used very simple evasion to see if the next generation firewall could be easily fooled, and we scored those tests separately. We did not use elaborate evasion techniques.
We configured a client to use the applications, and tried to get the next-generation firewall to identify and block the application. All applications were "real" and running either on our test lab network or across the Internet. We mixed both public applications (such as Yahoo Mail and Google Mail) with private applications (such as Lotus Notes or Microsoft Exchange) to get an idea of how well the next generation firewall operated when tools such as URL filtering would not apply. We also tried to mix both web-based applications with non-web applications, such as Skype and BitTorrent.
When testing applications, we counted a partial blockage as a failure. For example, some of our firewalls were able to block some BitTorrent and Skype traffic, but as long as we were able to succeed in downloading the selected Torrent or making calls with Skype after a short delay, we didn't give the firewall credit for actually blocking traffic.
We evaluated IPS features for each firewall using both subjective measures, such as manageability and policy controls, and objective tests. We used a Mu Dynamics appliance running Mu Studio Security to test each firewall with published vulnerabilities. We tested the firewalls in two different configurations, one optimized to protect end users, and a second one optimized to protect servers. For each configuration, we sent a different set of about 1,000 vulnerabilities designed to attack first users, and then servers. For each vulnerability set (server attacking and client attacking), we created two policies for each firewall. One policy included all of the IPS signatures and the other one just had the subset of signatures marked as having the highest priority.
We tested URL filtering and anti-malware to determine how well each firewall would catch current viruses. We did not attempt a virus coverage test, but instead picked a small set of 10 viruses that were a few weeks old — long enough for each firewall to have signatures to catch them, but not so old that the signatures would have aged out. Then we tested each firewall to see how well it would catch and block those viruses across a variety of protocols, encryption methods, and evasion techniques (such as using non-standard ports).
Read more about wide area network in Network World's Wide Area Network section.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.