Over the past several years, a number of academic and commercial entities have conducted evaluations of various network intrusion detection (NID) software, to determine the overall effectiveness of each product and to compare the products to each other. Many system administrators and security analysts are also responsible for conducting their own evaluations of NID products, in order to choose a solution for deployment in their environments. NID evaluations typically include some rough indication of the relative quality of each product's signatures. However, high signature quality is critical to achieving a good NID solution, so the importance of accurately evaluating signature quality cannot be stressed strongly enough.

In this series of articles, we will present recommendations that will help you to evaluate NID signatures. As you shall see, properly testing NID signatures is a surprisingly complex topic. We will begin by discussing some of the basics of evaluating NID signature quality, and then look at issues relating to selecting attacks to be used in testing. Although you may not necessarily perform hands-on NID testing and evaluations, the information presented in this series of articles will give you the knowledge and the facts to get the most out of published reviews and comparisons of NID signatures. Note that we assume that the reader is already familiar with the basic concepts and principles of network intrusion detection.

[ Read more ]