[prev in list] [next in list] [prev in thread] [next in thread] 

List:       taglib-devel
Subject:    Re: question and method about taglib performance
From:       Scott Wheeler <wheeler () kde ! org>
Date:       2012-09-19 9:16:53
Message-ID: C2761188-BDE2-4D7B-B5EB-A35063245DBB () kde ! org
[Download RAW message or body]

On Sep 19, 2012, at 11:05 AM, "Yao Weijian (CM/ESW13-CN)" <Weijian.Yao@cn.bosch.com> \
wrote:

> We checked the issue and found that much time was wasted when file fopen and fread. \
> We think this is caused by fileIO. So we used mmap instread of fopen/fread/fwrite \
> when OS is linux, and got the result that the performance was increased about 25%.

I'm pretty skeptical of that.  Are you sure that it wasn't just being moved into \
either kernel execution or that the speedup was from warm disk buffers?

That said, if you've got a patch, it's easy enough for us to take a look.  5000 files \
isn't an especially large collection though.

> But, as we all know, when using file mmap, the  memory which the process need is \
> equal to the size of file. For example, when the filesize is 1G, the process need \
> 1G memory for file mmap. This is intolerable. So we add a condition. When filesize \
> is smaller than 10M, we use mmap, others we use fopen which the taglib use now.

That's not actually true.  It's needs that amount of *address space*, not physical \
memory.  You can map files with mmap into memory that far exceed the physical memory. \
mmap just, well, maps, it doesn't copy the file into memory.

The actual limit that would be of concern would be the 4 GB limit on address space in \
32-bit processes.  In 64-bit processes you can map files as large as desired.

-Scott
_______________________________________________
taglib-devel mailing list
taglib-devel@kde.org
https://mail.kde.org/mailman/listinfo/taglib-devel


[prev in list] [next in list] [prev in thread] [next in thread]