Independent test lab Broadband-Testing released a report on its performance validation of IT network firewall solutions from Cisco Systems, NetPilot, SonicWall and WatchGuard.


Using Spirent’s security solutions, the tests benchmarked the performance of the vendors’ devices against key criteria such as network and application traffic performance with and without attacks, to determine which solution was the best at providing an entry-level security product for enterprises.

The test report highlights key performance differences between the firewall products and concludes that there is a level of compromise involved in terms of performance when advanced functionality is enabled. This underscores the need for comprehensive testing that compares security device’s real-world performance and datasheet performance.

Differentiation among the four tested products appeared in security features, the variety of licensing options, differences in management GUIs, and competitive pricing. The report’s key findings include the following:

• In basic firewall mode, the firewall products in general were able to get close to their claimed performance figures and were perfectly reasonable for the target market
• With IDS/IPS functionality enabled, performance of the firewalls fell away markedly in all cases
• Differences in the products’ capabilities impacted their ability to prevent threats, depending on traffic conditions.

“Enterprises have much to lose by not taking the initiative to create and enforce a strict security regime. Instead they depend only on their security device vendors’ claims to assess the fortification of their network,” said Steve Broadhead, founder and director, Broadband-Testing. “The goal of this test was to look at different firewall solutions that sit at various price points and evaluate them on pure performance. Spirent’s solutions were instrumental in the testing, and were able to push the four products with realistic and scaling performance traffic and threat traffic. By testing the performance of the security devices and verifying against vendor claims, it was easy to identify areas of weakness in the network.”

Broadband-Testing utilized Spirent Studio for attack traffic with known vulnerabilities and Spirent Avalanche as a stateful application traffic generator. The tests were conducted using a realistic mix of traffic that a branch office may commonly encounter.

The complete report is available here (registration required).