[prev in list] [next in list] [prev in thread] [next in thread] 

List:       koffice
Subject:    Re: kspread slowness with a lot of data
From:       Vladimir Dergachev <volodya () mindspring ! com>
Date:       2003-02-22 17:28:11
[Download RAW message or body]

>On Saturday 22 February 2003 03:35, Vladimir Dergachev wrote:
>> I would appreciate any comments/advice on the following problem:
>>
>> I have a CSV file with about 12000 lines and 4 columns. I would like to
>> make a chart using data from one of columns as X and from another as Y.
>> I would be nice if Y axis was logarithmic.
>
>Don't know if this is possible at the moment. KChart has some limitations.
>
>> Here are the things I tried (with Kspread 1.2.1) :
>>
>>          * kspread and OpenOffice Calc. Both are *very* slow and I was
>>            able to get a chart with Calc (but not kspread)
>
>Did you get a crash or why didn't you get a chart in KSpread?
>If crash, what is the backtrace?

No crash, I simply could not do it. It took too long to do every
operation.

>
>>          * there is an exact same delay (with 99% cpu consumption)
>>            when doing many other operations:
>>
>>                    o creating a chart in a separate sheet
>
>Don't know, why this should be the case. Sheet and Chart are seperate objects.
>
>>                    o right-clicking on a chart in a separate sheet
>
>Don't know, why this should be the case. Sheet and Chart are seperate objects.

I want to emphasize that the delay appears to be the same with the one
seen which saving the data.

Also I have tried your Ctrl-Shift-Arrow operations. With down arrow it
works great ! (fast and easy) Thanks !

But with up arrow I see the exact same delay. (I am putting the cursor in
cell G12645 and pressing Ctrl-Shift-Up Arrow).

The delay is very long, way longer than even pressing Ctrl-Shift-Down arrow
in an empty column (which highlights all 32000 cells).

As for memory this is not a big problem. With 7 columns filled with 12645
cells kspread SIZE (as reported with top) is 427Megs. This system has 1G
of RAM so no swap issues. Also 300 bytes * 12000 = 36megs - quite
acceptable.

One more thing - I just tried saving the file and I saw the memory usage
jump up and then down (but above what it was before).

Could it be that the delay is caused by kspread saving all data for Undo,
even though I am only inserting a chart in a different sheet ?

(No, I don't think it would be a good idea for me to take over a part..
but I would really like to trace down this slowness. Also I'll take a
look at the cell size issue, perhaps there is an easy way to fix this).

                      best

                         Vladimir Dergachev


>
>>                    o some highlighting operations
>
>This is known and we are working on it. Do you use zoom <> 100% here?

Possibly, I'll have to check. I did not change much from default settings.

>
>>
>> Problems:
>>
>>       * I do not know how to tell KChart which data to use
>
>Beside of deleting and creating a new KChart, there is no way to change
the
>data source at the moment. This is simply not implemented yet, any
volunteer?
>
>>       * I do not know how to populate 12000 cells in a column
>>         with =log(Ex) (with x varying through 12000 numbers)
>>         - clicking on a dot and dragging takes too long to
>>         paint 12000 cells.
>
>Yes this takes long.
>But as I do in other spreadsheet apps too, I always make:
>1.: ctrl-c to copy the cell
>
>2.: Then I use the cursor key functionality: ctrl - crsr
up/down/left/right.
>With this keys you can jump through whole blocks. If you don't know it
yet,
>just try it.
>It can also be used in combination with shift to select the cells.
>This is the way to select and cycle through big areas!
>
>3. When you selected the "empty" cells, just do the ctrl-v to paste it.
>
>This is much faster than scrolling with the mouse.
>
>>       * I would like to get rid of the annoying delay. It looks very
much
>>         like something Kspread calls often "just-in-case" and it
degrades
>>         perfomance with a lot of data. Perhaps this is garbage
collection
>>         of some sort or verification that all cells are up to date ?
>>         I would appreciate a hint on where to look in the source tree.
>
>We are not 100% sure what generates all the delays, but one issue is
>repainting. Repainting is already better in cvs (which will become
KSpread
>1.3).
>The other issue is most propably memory consumption. Currently the style
is
>saved in each cell seperately, which is not very useful for big sheets
(it
>allocates >300bytes per cell). We have ideas and Norbert is working on
it,
>but we cannot promise big improvement for 1.3.
>
>But one thing is rather clear: There is most propably no easy fix.
>
>If you want to take over this part, you are very much welcome :-)
>It's all about layout in ranges and style management.
>We will help you and give you our information we collected so far about
what
>should be done. You can also look into the archive of koffice-devel and
into
>cvs (there is a THOUGHTS file containing some information).
>
>Philipp
>____________________________________
>koffice mailing list
>koffice@mail.kde.org
>To unsubscribe please visit:
>http://mail.kde.org/mailman/listinfo/koffice
>
____________________________________
koffice mailing list
koffice@mail.kde.org
To unsubscribe please visit:
http://mail.kde.org/mailman/listinfo/koffice
[prev in list] [next in list] [prev in thread] [next in thread]