[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wekalist
Subject:    Re: [Wekalist] Can naive bayes model over-fit the data?
From:       Peter Reutemann <fracpete () waikato ! ac ! nz>
Date:       2006-02-27 21:33:15
Message-ID: 4403701B.3040302 () waikato ! ac ! nz
[Download RAW message or body]

> It seems for me that Naive bayes classifier(NBC) would not over-fit the 
> data (or over-fitting is not so serious compared with other classifiers 
> like C4.5, Neural Network, kNN, SVM etc.).  As NBC just estimate the 
> probability of each attribute value(with some smoothing method), and the 
> final model is linear. It seems it is hard for NBC to over-fit the data. 
> I am wondering whether there is any reference about the robuestness of 
> NBC.  Any idea?

Just had a chat with a colleague of mine and he says:
"If you really want to, you can make NB overfit by having a discrete 
attribute with many values and a class with many values and very few 
data points, say with a 100 valued attributed and a 100 valued class and 
100 data points. In most machine learning situations, you would expect 
NB to be quite robust. For example, Breiman in his paper on bagging 
refers to NB as a stable classifier and therefore not suited for bagging."

HTH

Cheers, Peter
-- 
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/     +64 (7) 838-4466 Ext. 5174

_______________________________________________
Wekalist mailing list
Wekalist@list.scms.waikato.ac.nz
https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
[prev in list] [next in list] [prev in thread] [next in thread]