Rule-extraction from support vector machines through consistency based feature selection approach

Ace p.

Order the writing of a tailor-made Computer science Term papers

Free quote online

Thesis Format .pdf

Rule-extraction from support vector machines through consistency based feature selection approach

Download

Read an extract

Reader
Abstract
Contents
Extract

Abstract

In this paper, a new approach is proposed to extract rules from the support vector machines through consistency based feature selection. To reduce the computational time and increase the performance of classification model, a two step classification approach is followed. First step includes feature selection using consistency based correlation metric approach. In second step, rules are generated based on most relevant and irredundant features. Feature selection approach proceeds as three steps. In the first step, irrelevant features are removed based on mutual information between each feature and the decision variable and secondly redundant features are removed through the pair wise correlation measure. Finally interactions among features are handled using efficient evaluation metric and search strategy with specially designed data structure. The above selected most discriminative feature subset is used for classification. The selected feature subset is classified using SVM. In this paper, the proposed rule extraction algorithm follows sequential covering approach. Rules are learned from the true positive support vectors of the most discriminative features, with the negative class being the default.

Abstract
Introduction
Feature extraction techniques
Rule extraction techniques
The proposed approach
1. Correlation based consistency measure (CBCM) - An algorithm
2. Definition (consistency contribution) or contribution
3. Rule generation
Experiment set up and results
Conclusion
References

Get this table of contents for free after login.

Extract

[...] 273- [9]Nahla Barakat and Andrew P.Bradely, “Rule extraction from SVM:A sequential Covering Approach”, IEEE Transactions on Knowledge and data engineering [10]Nahla Barakat and Joachim Diederich, ”Learning based Rule extraction from support vector machines”, Proc.Conf. Neuro Computing and Evolving Intelligence [11]X.Fu, C.Ongt, S.Keerthit, G.Hung,and L.Goh, ‘Extracting the Knowledge Embedded in Support Vector machines”, Proc IEEE Int'l Conf. Neural networks [12]L. Yu and H. Liu, “Efficient Feature Selection via Analysis of Relevance and Redundancy,” J. Machine Learning Research, vol pp. 1205-1224, Oct [13]Zheng Zhao and Huan Searching for Interacting Features” Proc. [...]

[...] Learning based approach utilize another machine learning technique which has explanation capability, to learn what the classifier has learned. Decompositional approaches looks into its individual components and then extract rules at the level of these components. This is the most transparent approach. The eclectic approach lies between learning based and decompositional approaches. Fig.1 CBCM algorithm A feature is relevant due to two reasons: it is strongly correlated with the target concept; or it forms a feature subset with other features and the subset is strongly correlated with the target concept. [...]

[...] Rule Generation In this paper, a method to extract the rules from the support vector machine using sequential covering approach is proposed. Based on the ordered search of most discriminative features rules are generated. Sequential covering approach learns one rule at a time which will explain the part of the available training set, one class at a time. The learned rule covers as many positive examples as possible. Such examples are removed from the training set and continue to learn new rules until all positive examples are covered. [...]

[...] The relevance of feature can be determined by the mutual information between decision and the feature. Feature selection methods based on relevance does not remove the redundant information existing among features. Redundant features along with irrelevant features affect the accuracy of the learning algorithms Some of the evaluation measures used to remove irrelevance and redundancy are consistency measure and correlation measure Feature redundancy is normally described in terms of feature correlation. It is widely accepted that two features are redundant to each other if their values are completely correlated. [...]

[...] From the results the need for feature interaction in subset selection are proposed to search for interacting features to measure feature relevance. The issues such as consistency measure is analyzed which handles the feature interaction and efficiently selects relevant features with reduced computational time. Our method demonstrates its efficiency and effectiveness for feature selection in supervised learning in domains where data contains many irrelevant and/or redundant features. The minimum reduced feature set from CBCM will give equivalent or improved classification performance than other algorithms. [...]

pdf