[prev in list] [next in list] [prev in thread] [next in thread] 

List:       glibc-help
Subject:    Possible malloc/mmap bug?
From:       matthewc () cse ! unsw ! edu ! au (Matthew Chapman)
Date:       2011-11-19 0:44:00
Message-ID: CAE2NNPZfeVGCVm26jttTwOsxTW2-vN15rR5NY=tsanVZH-FAAA () mail ! gmail ! com
[Download RAW message or body]

It should be possible to increase the number of mappings allowed per
process via /proc/sys/vm/max_map_count.  This would probably solve the
immediate problem, although I'd be somewhat concerned about the impact
on the performance of page faults, subsequent mmaps, etc. - remember
these mappings are stored in a red-black tree in the kernel.

Matt


On Sat, Nov 19, 2011 at 8:03 AM, Carlos O'Donell
<carlos@systemhalted.org> wrote:
> On Fri, Nov 18, 2011 at 3:33 PM, Charles Hines <hines@cert.org> wrote:
> > > Your application is a pathological case of VM fragmentation.
> > But in a 64 bit operating system w/ the amount of virtual address space in the \
> > Exabyte range, and the fact that while we are processing large amounts of data \
> > (pre-reduction) and us preparing batches to feed to the program that won't exceed \
> > (pre-reduction) the system RAM (48GB), it's not an issue here.
> 
> It's the same problem, but just takes longer to reach the limit of the
> kernel VM fragmentation.
> 
> ~~~
> ...
> -----
> > > > > > 
> > > > > > ERROR: malloc returned NULL at call number 65569 <<<<<
> > > > > > 
> -----
> total allocated: 32.27 GB, total allocated after reallocs: 0.25 GB
> [elapsed time: 32 sec]
> malloc_stats() data:
> Arena 0:
> system bytes ? ? = ? ? 663552
> in use bytes ? ? = ? ? 135696
> Total (incl. mmap):
> system bytes ? ? = ?549068800
> in use bytes ? ? = ?548540944
> max mmap regions = ? ? ?65536
> max mmap bytes ? = ?548405248
> -----
> ~~~
> 1321649768.153047 mmap(NULL, 532480, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f113ffe3000
> 1321649768.153132 mremap(0x7f113ffe3000, 532480, 8192, MREMAP_MAYMOVE)
> = -1 ENOMEM (Cannot allocate memory)
> ...
> 1321649768.153170 brk(0) ? ? ? ? ? ? ? ?= 0x19f0000
> 1321649768.153197 brk(0x1a92000) ? ? ? ?= 0x1a92000
> 1321649768.153415 brk(0x1a12000) ? ? ? ?= 0x1a92000
> ...
> 1321649768.155301 mmap(NULL, 1048576, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> 1321649768.155323 mmap(NULL, 134217728, PROT_NONE,
> MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
> allocate memory)
> 1321649768.155343 mmap(NULL, 67108864, PROT_NONE,
> MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
> allocate memory)
> 1321649768.155362 mmap(NULL, 134217728, PROT_NONE,
> MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
> allocate memory)
> 1321649768.155382 mmap(NULL, 67108864, PROT_NONE,
> MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot
> allocate memory)
> 1321649768.155410 write(1, ">>>>>\n", 6) = 6
> 1321649768.155439 write(1, ">>>>> ERROR: malloc returned NUL"..., 61) = 61
> 1321649768.155461 write(1, ">>>>>\n", 6) = 6
> 1321649768.155480 write(1, "-----\n", 6) = 6
> ...
> ~~~
> * mremap fails to find a new mapping for the pages.
> * glibc starts trying to use kernel brk call to get new memory but
> that fails too.
> * final set of mmap allocations fails and glibc gives up.
> 
> Fix your application or fix the kernel.
> 
> Cheers,
> Carlos.
> 


[prev in list] [next in list] [prev in thread] [next in thread]