Improving text categorization performance by combining feature selection methods

Abstract

Even though the arrival of the machine learning methods in text categorization is one of the essential factors that improves the effectiveness of text categorization, high dimensionality is still a challenge for classification performance. There are several ways to reduce the dimension of input vector in classification and feature selection is one of the most popular and effective methods of reducing dimension. Various researches have been done to improve the performance of feature selection methods on text categorization but they mostly deal with how to advance the performance of the individual feature selection methods whereas we know that combining the outputs of multiple algorithms/classifiers is one of the promising strategies that has been studied extensively in information retrieval. With this motivation, we present a comprehensive analysis of the comparison between the feature selection methods and their varied binary combinations for text categorization with a comparative discussion. We analyze the performances of five common feature selection methods with their combinations on five standard datasets with varied skewness in both global and local policies by using SVM. Comparing the performance of the individual methods with the performance of the combination methods shows that combining two feature selection methods significantly improves the performance of the individual methods. In addition, rank combination achieves better performance in the case of global policy on the other hand score combination significantly achieves better performance in the case of local policy. In this thesis, the main concern is to investigate the effectiveness of combining the individual metrics on the performances of text categorization. Thus, we also propose new combination methods that some of them clearly outperform the success of the score and rank combinations.