[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-bugs-dist
Subject:    [kio] [Bug 342056] Ridiculously slow file copy (multiple small files)
From:       Frank Reininghaus <frank78ac () googlemail ! com>
Date:       2015-02-03 22:05:07
Message-ID: bug-342056-17878-YPhNrWDpHA () http ! bugs ! kde ! org/
[Download RAW message or body]

https://bugs.kde.org/show_bug.cgi?id=342056

--- Comment #9 from Frank Reininghaus <frank78ac@googlemail.com> ---
(In reply to Dawit Alemayehu from comment #7)
> Notice that it takes a huge amount of time (9 secs) to sort such a large
> list

But the sorting has to be done in the application that uses KDirLister anyway
(at least if "Sort by Name" is used, which is true most of the time, I think).
And if the user tries to open a directory with 50 million files, then there
might be things that take even longer than 9 seconds of sorting in
KDirListerCache ;-) 

> and that would have to happen each time someone inserts a new record.

You don't have to sort the entire list every time a new record (or some more
new records) are inserted. One would first sort the new records, and then merge
the two sorted lists (old and new records). For a single new file, one could
just do a binary search to find the correct place in the list, which should be
quite fast.

BTW, if many files are deleted in a folder with 50 million files, then I guess
that looking these up in the unsorted list that we currently have would be a
huge problem. With a sorted list, it could be done quite efficiently.

Maybe I'll try to analyze this in more detail when I find some time.

(In reply to Alexander Nestorov from comment #8)
> Can I get this bug in a "Confirmed" state?

I haven't analyzed the problem (which is different from the problem which I
first thought you meant and which I discussed with Dawit) in detail yet, so I
can't really say how bad it is and if it should be considered a bug or rather
wishlist material (I'll let others decide that). 

It's definitely not "grave" though in my opinion. You've said yourself that
"this is not a common use case"! And a factor of 20 in the performance between
KIO and rsync does not really sound that bad to me. I have seen performance
issues that were *much* worse. Moreover, you cannot really compare rsync to KIO
because both are designed to do quite different things. I do agree though that
any optimizations in KIO would be welcome.

-- 
You are receiving this mail because:
You are watching all bug changes.
[prev in list] [next in list] [prev in thread] [next in thread]