An information gain based feature selection method and a network-based intrusion detection system framework utilizing anomaly detection using self organizing maps

Abstract

In this work, an information gain based feature selection method and a networkbased intrusion detection system utilizing anomaly detection using Self Organizing Maps (SOM) are proposed. KDD 99 (The International Knowledge Discovery and Data Mining Tools Competition 1999) is used for the feature selection and performance evaluation of the anomaly system. Feature selection method considers every combination of n feature groups as a unique feature and determines whether it is useful for the anomaly detection by calculating entropy of the each new feature. As the number of features in a group, namely n, goes up, both the number of the combinations and the time needed for calculating every new feature’s information gain increases, and it becomes computationally infeasible. To overcome this problem, a quantization method, which is also information gain based, is proposed. The quantization of the basic features makes possible of the calculations of the information gains of the new combinational features as the n increases. In the anomaly detection part of the work, multi number of SOMs, every one is specialized to detect an attack group, is proposed. The useful features for each SOM is determined according to proposed feature selection process, and the performance of the SOMs are calculated.