[prev in list] [next in list] [prev in thread] [next in thread]
List: taglib-devel
Subject: Re: question and method about taglib performance
From: Scott Wheeler <wheeler () kde ! org>
Date: 2012-09-19 9:16:53
Message-ID: C2761188-BDE2-4D7B-B5EB-A35063245DBB () kde ! org
[Download RAW message or body]
On Sep 19, 2012, at 11:05 AM, "Yao Weijian (CM/ESW13-CN)" \
<Weijian.Yao@cn.bosch.com> wrote:
> We checked the issue and found that much time was wasted when file fopen \
> and fread. We think this is caused by fileIO. So we used mmap instread of \
> fopen/fread/fwrite when OS is linux, and got the result that the \
> performance was increased about 25%.
I'm pretty skeptical of that. Are you sure that it wasn't just being moved \
into either kernel execution or that the speedup was from warm disk \
buffers?
That said, if you've got a patch, it's easy enough for us to take a look. \
5000 files isn't an especially large collection though.
> But, as we all know, when using file mmap, the memory which the process \
> need is equal to the size of file. For example, when the filesize is 1G, \
> the process need 1G memory for file mmap. This is intolerable. So we add \
> a condition. When filesize is smaller than 10M, we use mmap, others we \
> use fopen which the taglib use now.
That's not actually true. It's needs that amount of *address space*, not \
physical memory. You can map files with mmap into memory that far exceed \
the physical memory. mmap just, well, maps, it doesn't copy the file into \
memory.
The actual limit that would be of concern would be the 4 GB limit on \
address space in 32-bit processes. In 64-bit processes you can map files \
as large as desired.
-Scott
_______________________________________________
taglib-devel mailing list
taglib-devel@kde.org
https://mail.kde.org/mailman/listinfo/taglib-devel
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic