[prev in list] [next in list] [prev in thread] [next in thread]
List: freebsd-hackers
Subject: Re: Strange memory management with mmap()
From: Dmitry Sivachenko <trtrmitya () gmail ! com>
Date: 2015-07-22 12:47:12
Message-ID: 15DE3B94-3C09-4855-A274-D5655B049403 () gmail ! com
[Download RAW message or body]
> On 16 июля 2015 г., at 21:19, Dmitry Sivachenko <trtrmitya@gmail.com> wrote:
>
> >
> > On 16 июля 2015 г., at 18:42, Dmitry Sivachenko <trtrmitya@gmail.com> wrote:
> >
> > Hello!
> >
> > I am using FreeBSD-10-stable and writing a program that uses large data file via \
> > mmap() in read only mode. To be specific, I have 256GB RAM machine and typical \
> > size of data file is ~160GB (more than 1/2 of RAM and less that the whole RAM). \
> > There is no other programs running during the test.
> > Consider the following use case: I have two files on disk. I mmap() the first \
> > one and prefetch data to RAM (touch every page of the file). After that I expect \
> > all data to be cached in RAM and subsequent access will be fast.
> > Next I do munmap() on the first file, mmap() the second one and do the same test: \
> > prefetch data and expect it to be cached in RAM (and some of the pages belonging \
> > to the first file to be purged out, because size_of(file1)+size_of(file2) > \
> > size_of(RAM).
> > Please find my test program attached.
> >
> > I run the program with 2 files provided via command line (both about 160GB).
> > What I observe in real is:
> > -- before I run the program all RAM is in FREE state as reported by top(1).
> > -- after first prefetch() of the first file, all it's data goes to "Cache" state, \
> > RES column of the process remains the same (small)
> > -- second prefetch() works fast as expected, memory goes from Cache to Active \
> > state, RES column of the process grows up to match file size \
> > (SIZE==RES now)
> > -- now first prefetch() for second file starts: the remaining Free memory goes to \
> > Cache state, Active size still equals to first file size.
> > -- second prefetch() for second file works as slow as first one, like if nothing \
> > was cached in memory during the first prefetch() run, RES column does not change. \
> >
> > Here is the output:
> > % /tmp/a.out file1.dat file2.dat
> > file1.dat... First prefault time: 1235.747351 seconds
> > Second prefault time: 74.893323 seconds
> > Ok.
> > file2.dat... First prefault time: 1316.405527 seconds
> > Second prefault time: 1311.491842 seconds
> > Ok.
> >
>
>
I tried the same test program on Linux machine with similar hardware. It behaves \
like expected (second prefetch works very fast):
file1.dat... First prefault time: 2664.621088 seconds
Second prefault time: 1.969283 seconds
Ok.
file2.dat... First prefault time: 2917.009003 seconds
Second prefault time: 34.128762 seconds
Ok.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic