[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wekalist
Subject:    Re: [Wekalist] CSF and thousands of attributes
From:       Mark Hall <mhall () cs ! waikato ! ac ! nz>
Date:       2003-12-14 20:47:34
[Download RAW message or body]

Hi,

There is a now a new version of CFS and the best first search class in 
the CVS repository for Weka that addresses these memory issues. The 
main problem was caused by using CFS in conjunction with best first 
search. CFS requires an n x n matrix of doubles to hold feature 
correlations and best first search uses a large cache to hold 
previously evaluated features.

The new version of CFS now uses 1/4 of the memory that it did 
previously and the best first search now has an option that allows the 
user to control the maximum size of the lookup cache. CFS should now be 
applicable to data sets with tens of thousands of features (it has been 
successfully run on a 16,000 feature data set inside of 1Gb of memory 
for the VM).

Cheers,
Mark.

On Wednesday, December 10, 2003, at 05:57  AM, JESUS S. AGUILAR-RUIZ 
wrote:

> I would like to know why CSF does not work when the number of 
> attributes is very large (about a few thousands).
> There are two datasets with that amount of attributes in 
> http://www.lsi.us.es/~aguilar/bioinformatics.html .
> CSF works with less than 1000 aproximately, but not with 4000 or 7000 
> (the sizes of these datasets).
> If someone can help me, I would be very grateful.
> Thanks
> _________________________________________________________
>  
> Jesus S. Aguilar-Ruiz, Ph.D.
> Department of Computer Science
> University of Seville, Spain
> http://www.lsi.us.es/~aguilar/
> _________________________________________________________
>  
>  
> _______________________________________________
> Wekalist mailing list
> Wekalist@list.scms.waikato.ac.nz
> https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist


_______________________________________________
Wekalist mailing list
Wekalist@list.scms.waikato.ac.nz
https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist

[prev in list] [next in list] [prev in thread] [next in thread]