The importance of data normalization in IPS
by Darren Suprina - Security Architect, Stonesoft - Monday, 7 January 2013.
To fully comprehend the importance of data normalization in an Intrusion Prevention System, it is first necessary to understand what data normalization is and what it does, how it accomplishes its goal, and why it is so integral to maintaining security against the advanced evasion techniques used today.

The critical importance of data normalization can also be seen while reviewing security failures and fundamental design flaws in many IPS devices that lack such normalization.

Data normalization explained

Data normalization is the process of intercepting and storing incoming data so it exists in one form only. This eliminates redundant data and protects the data’s integrity. The stored, normalized data is protected while any appearance of the data elsewhere is only making a reference to the data that is being stored and protected in the data normalizer.

The normalizer’s job is to patch up the incoming data stream to eliminate the risk of evasion as well as ambiguities. The monitor then views the data in its pure, protected and normalized form. Varying forms of normalization exist on levels of increasing complexity. The complexity is due to the set of requirements that must be met to achieve normalization. The most basic is known as First Normal Form, which is often abbreviated 1NF. It is followed by Second Normal Form, or 2NF, Third Normal Form, or 3NF and can continue increasing in forms and complexity as required or desired.

Normalization benefits

Normalization plays a key role in the security of a network, provided that normalization extends to every protocol layer. One of the major benefits is the forced integrity of the data as data normalization process tends to enhance the overall cleanliness and structure of the data. Normalization significantly contributes to the fortification of a network, especially in light of typical networks’ three main weak points: traffic handling, inspection and detection.

Where many IPS devices go wrong

When it comes to traffic handling, many IPS devices focus on throughput orientation for the most rapid and optimal inline performance. This process, while attractive for its rapidity, makes it impossible for full normalization to take place. The data traffic is then inspected without normalization, offering prime opportunities for infiltration to take place. One may agree that a rapid and optimal output performance is useless if the payload is riddled with malicious invaders.

When many IPS devices do employ normalization, they often rely on shortcuts that only implement partial normalization as well as partial inspection. This leaves gaps in the security and provides optimal opportunity for evasions. TCP segmentation handling is one example of such a process, as it is only executed in chosen protocols or ports and is drastically limited in its execution. Shortcut exploitation is a familiar evasion method and, with the proliferation of IPS devices that fail to perform full normalization, it is likely to remain that way due to its ease of execution.

Many IPS devices fall short in other areas, as well. They often perform only a single layer of analysis, execute traffic modifications and interpretations and rely on inspection of individual segments or pseudo-packets. Their detection methods are based on vulnerability and exploits, banner matching or shell detection. Their updates are generally delayed and their evasion coverage is extremely limited. Evasions can easily exploit the limited inspection scope by spreading attacks over segments or pseudo-packet boundaries.


Reactions to the IRS hack that impacted 100,000 people

Cybercriminals were able to successfully steal tax forms full of personal information of more than 100,000 taxpayers through IRS’ Get Transcript application. This data included Social Security information, date of birth and street address.

Weekly newsletter

Reading our newsletter every Monday will keep you up-to-date with security news.

Daily digest

Receive a daily digest of the latest security news.

Thu, May 28th