Incremental Naïve Bayesian Learning Algorithm Based On .

JOURNAL OF COMPUTERS, VOL. 9, NO. 8, AUGUST 20141967Incremental Naïve Bayesian Learning Algorithmbased on Classification Contribution DegreeShuxia Ren *, Yangyang LianSchool of Computer Science & Software Engineering, Tianjin Polytechnic University, Tianjin, Chinat rsx@126.com , sunshine lianyy@qq.comXiaojian ZouMilitary Transportation University, Tianjin, Chinaala 99@126.comAbstract—In order to improve the ability of graduallearning on the training set gotten in batches of NaiveBayesian classifier, an incremental Naïve Bayesian learningalgorithm is improved with the research on the existingincremental Naïve Bayesian learning algorithms. Aiming atthe problems with the existing incremental amendingsample selection strategy, the paper introduced the conceptof sample Classification Contribution Degree in the processof incremental learning, based on the comprehensiveconsideration about classification discrimination, noisy andredundancy of the new training data. The definition andtheoretical analysis of sample Classification ContributionDegree is given in this paper. Then the paper proposed theincremental Naïve Bayesian classification method based onthe Classification Contribution Degree. The experimentalresults show that the algorithm simplified the incrementallearning process, improved the classification accuracy ofincremental tion Degree, Naïve BayesianClassificationI. INTRODUCTIONData mining has been becoming more and morepopular with all walks of life, from manufacturing toservice industry, from medical diagnose to businessdecision, because its special advantage in discovering thepotential decision information and business model [1].Classification prediction is an important content in theresearch of data mining, which has been widely used inclassification prediction and pattern recognition.There are many classification methods such as decisiontree, Naïve Bayesian classifier, support vector machine,neural network and so on. Naïve Bayesian classificationmethod is widely used for its simple and efficiency. Itsclassification performance is comparative to neuralnetwork, decision tree and better than many otherclassifier in some cases [2]. The classification rulestrained on a large scale of dataset with strongcompleteness can accurately response model of thepractical problem. But in variety of applications such asmedical decision making, stock analysis, industrial 2014 ACADEMY PUBLISHERdoi:10.4304/jcp.9.8.1967-1974production, data are produced over time, and the trainingset are gained patched [3].In order to ensure the classifier gotten by the NaïveBayesian classification method more accuracy, theclassifier must be retrained again and again whenever thetraining set is updated as the time goes by. The problemsof wasting time and memory resources arise because ofrepeat training [4].Incremental learning can amend the current classifierwith knowledge learning from the new training set [5]. Itcan make full use of the previous trained classifier,making the previous work meaning. Meanwhile,incremental learning only needs new training set inmemory, making the amending process fast and efficient.Compared with training a new system, amending atrained system will save lots of resources like time, space,manpower and material. Moreover, incremental learningcan solve some problems of real-time application moreaccurately on the basis of small-scale study.Incremental learning for a mining algorithm, especiallythe classification mining algorithms, is a very importantability. Many studies of incremental learning ability weredown with many classification methods like RBF neuralnetwork, Support vector and k-Nearest Neighbor [6-9].And the applications of incremental classification focuson predicting bugs in software, pattern classification,mining of frequent time sequence, analysis of medicaldata features, collaborative filtering and so on [10-12].The research of incremental ability with naïveBayesian classifier was first proposed by Gong Xiujun.He presented the learning formula of incremental NaïveBayesian and selection strategy of amending samples [13].Many improving methods on the select range ofamending sample and sequence are made based on thispaper. There are three kinds of incremental NaïveBayesian classification algorithms.The first kind of algorithm uses all the samples in thenew training set for amending the current classifier, butprefer for samples whose sum loss of classification islesser, which was first proposed by Gong. The secondkind of algorithm predicts samples in the new training setwith the current classifier first, and uses samples that are

1968correctly classified for amending the current classifier[14]. The third kind of algorithm also predicts samples inthe new training set with the current classifier first, anduses samples that are wrongly classified to amend thecurrent classifier. In the process of amending, thesealgorithms still prefer for samples whose sum loss ofclassification is lesser [15]. Correctly classified samplesare samples whose predicted class labels are the samewith their own class labels. And wrongly classifiedsamples are samples whose predicted class labels aredifferent from their own class labels.The problems of three kinds of incremental NaïveBayesian classification algorithms are as follows:First, in the incremental learning process, thealgorithms do not fully consider the factor of datarepresentation, data redundancy and noise. Weakcharacterized data and redundant data will blur thecategory features. And the noisy data will decline theperformance of the classifier. Obviously, it is notadvisable to use all the new training samples to amendthe current classifier [16]. In fact, whether a new instanceis suit for amending the current classifier, we shouldconsider the data qualities that affect the predict result,the representativeness, discrimination degree ively, instead of whether the sample is rightlyor wrongly predicted.Second, when the order of samples is different, thelearning sequence obtained by the sum loss ofclassification is different with the same training set [17].There are two extreme cases for example. When samplesin new training set are descending according to theclassification sum loss, all samples will be used to amendthe current classifier. When samples in new training setare ascending according to the classification sum loss,only the first sample will be used to amend the currentclassifier.In order to overcome the above problems, the conceptof Classification Contribution Degree is presented rimination, redundancy and noisy of the data, andthen the Incremental Naïve Bayesian Learning Algorithmbased on Classification Contribution Degree (INB-CCD)is proposed. The algorithm analyzes the effect onclassification performance from the level of data quality,and avoids the dependence on sample order of learningresult.The rest of the paper is organized as follows. Section 2presents the Naïve Bayesian classification method andfurther gives the learning formula of incremental NaïveBayesian classification. In Section 3 the paper first givesthe definition and theoretical analysis of ClassificationContribution Degree, and then proposes the sampleselection strategy of Naïve Bayesian incremental learningand at last gives the medical data mining algorithm basedon Classification Contribution Degree. The effectivenessof the algorithm is studied in Section 4 throughcomparing its performance on five different medicaldatasets with the other three incremental learningalgorithms. Finally, the paper concludes in Section 5. 2014 ACADEMY PUBLISHERJOURNAL OF COMPUTERS, VOL. 9, NO. 8, AUGUST 2014II. LEARNING FORMULA OF INCREMENTAL NAÏVEBAYESIANA. Naïve Bayesian Classification MtheodNaïve Bayesian classification method is mainly basedon the Naïve Bayesian principle. So before theintroduction of incremental Naïve Bayesian formula, wefirst present the Naïve Bayesian theory here. It gives thecalculation of post probability according to the priorprobability. Prior probability is the probabilitydistribution of each class counted by the given datainformation. It contains two statistics: class priorprobability and conditional probability. Post probabilityis the probability of each unclassified sample belongingto some certain class.Suppose X { A1 , A2 ,., An } is a data sample with nfeatures, then post probability of sample X belonging toclass C is P (C X ) . The Naïve Bayesian formula ofcalculating the post probability is as follows:P( X C ) P(C )(1)P(C X ) P( X )Where, P(C) is the class prior probability of a samplebelonging to class C. P(X C) is conditional probability offeatures same with sample X among samples whose classlabel is C.The main idea of Naïve Bayesian classification methodis as follows:Suppose C1 , C2 ,., Cm represents the m different classes.For each test sample X, the classification methodcomputes the posterior probability P(C j X ) throughclass prior probability P (C C j ) and class conditionalprobability P( X C j ) . The value range of j is from 1 to m.And sample X belongs to the class whose posteriorprobability is the biggest of the all [18]. That is to say,Naïve Bayesian classification method classified thesample X as class Ci , when the sample X satisfied thefollowing expression:P(Ci X ) P(C j X ) , 1 i j m(2)We can learn from (1) that (2) equals to the followingexpression:P( X Ci ) P(Ci ) P( X C j ) P(C j )(3) P( X )P( X )Where, 1 i j mMoreover, (2) is equal to the following expression:(4)P( X Ci ) P(Ci ) P( X C j ) P(C j )Where, 1 i j mTherefore, the essence of training Naive Bayesianclassifier is to calculate two parameters using statisticalknowledge according to the training samples: priorprobability and conditional probability.B. Incremental Naïve Bayesian FormulaThe process of classification forecasting with NaïveBayesian classifier is as follows: