[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gnuplot-info-beta
Subject:    Re: gnuplot slow on BIG files
From:       Daniel J Sebald <daniel.sebald () ieee ! org>
Date:       2005-05-30 17:15:44
Message-ID: 429B4A40.7030802 () ieee ! org
[Download RAW message or body]

Hans-Bernhard Broeker wrote:
> Daniel J Sebald wrote:
> 
>> Hans-Bernhard Broeker wrote:
> 
> 
>>> Necessity is not the issue --- convenience of maintaining the overall 
>>> structure of a very ancient code base is.  The code has always been 
>>> organized to read all data, then let the "every" filter decide which 
>>> points to actually use.
> 
> 
>> Actually, from what I remember, in datafile.c the "everypoint" 
>> variable is used to toss out points.  
> 
> 
> Yes.  But the question is: *when* does that happen: before sscanf()ing
> the values, or afterwards?  After a quick look into the current sources,
> it would seem that at least for 'matrix' files, it reads the entire 
> thing before applying 'every'.  That's probably the reason why 'every 
> 500:500' didn't achieve any speedup for Dimitrios.

Oh yeah, I see now... and my memory is coming back.  This is true in the case of 
binary data as well.  I can't recall if I originated the concept or carried it 
over from previous code, but the idea was to bring in all the data and create a 
data format similar to what the normal routines read, but in memory.  That could 
be changed I guess.  (But would have to think of the ramifications.)  The 
decimation could be done in the df_read_matrix() routine and then indicate to 
the main routine that "every" should be 1 instead of, say, 500.  I wouldn't call 
that an urgent change, however, seeing as I have a number of things to do right now.

Dan


-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005
_______________________________________________
gnuplot-beta mailing list
gnuplot-beta@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
[prev in list] [next in list] [prev in thread] [next in thread]