[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ast-developers
Subject:    [ast-developers] Re: Return of the SIGSEGV handler (vmbest.c saga) ...
From:       Roland Mainz <roland.mainz () nrubsig ! org>
Date:       2012-04-30 19:05:14
Message-ID: CAKAoaQkCBKfZzSHSqbFKZ_gnos3B6MHJQEQQGiNFfcD8akoQ3A () mail ! gmail ! com
[Download RAW message or body]

On Thu, Apr 26, 2012 at 7:15 PM, Glenn Fowler <gsf@research.att.com> wrote:
> On Thu, 26 Apr 2012 01:38:08 +0200 Roland Mainz wrote:
>> [AFAIK this is an old issue which somehow reappeared...]
>
>> A while ago we found that libast's allocator creates a SIGSEGV handler
>> for each chunk of memory obtained from the system to protect itself
>> against getting wallpoed with a SIGSEGV if first tries to use a newly
>> obtained memory page in cases when the kernel no longer has any
>> physical memory left to actually map the page, e.g.
>> like this:
>> -- snip --
>> mmap(0x00000000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON,
>> -1, 0) = 0xFFFFFFFF53800000
>> sigaction(SIGSEGV, 0xFFFFFFFF7FFFE320, 0xFFFFFFFF7FFFE428) = 0
>> <newly obtained memory page is accessed here. If memory is not
>> available the system will wallop the process with a SIGSEGV>
>> sigaction(SIGSEGV, 0xFFFFFFFF7FFFE320, 0xFFFFFFFF7FFFE428) = 0
>> -- snip --
>
>> This makes sense if an operating system allows such overcommitment of
>> memory but it doesn't make much sense if the OS doesn't allow this...
>> like in Solaris case (maybe AIX, too...).
>> The fix was to #ifdef-out the handler for Solaris but it seems the
>> wrong cpp symbol was used, e.g. |_SUNOS| instead of |__SunOS|.
>> The following patch will fix this issue:
>> -- snip --
>> diff --git a/src/lib/libast/vmalloc/vmbest.c b/src/lib/libast/vmalloc/vmbest.c
>> index c291409..ff75b0a 100644
>> --- a/src/lib/libast/vmalloc/vmbest.c
>> +++ b/src/lib/libast/vmalloc/vmbest.c
>> @@ -1111,7 +1111,7 @@ done:
>>  #endif
>>  #endif
>
>> -#if _SUNOS /* sunos guarantees that brk-addresses are valid */
>> +#if __SunOS /* sunos guarantees that brk-addresses are valid */
>>  #define        chkaddr(a,n)    (0)
>
>>  #else /* make sure that allocated memory are addressable */
>> @@ -1140,7 +1140,7 @@ static int chkaddr(Vmuchar_t* addr, size_t nsize)
>
>>         return rv;
>>  }
>> -#endif /*_SUNOS*/
>> +#endif /*__SunOS*/
>
>>  #if _mem_win32 /
>> -- snip --

>
> thanks Roland
> I really don't like #if's on os's/systems
> but I can't think of a way to reliably iffe overcommittment

Erm... there is AFAIK no way... and the issue is a bit more complex:
1. Linux allows processes to overcommit their allocations... but there
is a "knob" (I don't know where) to turn it "off"
2. Solaris does not allow overcommitment by default but there is a
semi-secret kernel tuneable to treat more or less all allocations as
|MAP_NORESERVE| (which means: This mapping doesn't have to be backed
by physical storage (e.g. memory or swap)). Even worse, the stack is
AFAIK always allocated with |MAP_NORESERVE|. But by default (if you
use |brk()|, |mmap()/MAP_ANON| or |mmap(),/dev/zero|) all allocations
are guranteed to be backed by physical storage
3. There are 3rd-party shared objects (which intercept calls like
|mmap()|, |open()| etc.) which can be used via LD_PRELOAD which either
enable or disable physical backing of memory pages.

Sometimes I wish there would be a |mmap()|-flag |MAP_RESERVE| which
would explicitly ask for physical storage backing for the particular
allocation... ;-/

> I put in a change that assumes innocence until proven guilty
>
>        #if __linux__
>        /* check for mem overcommit */
>        #else
>        #endif
>
> right now I only know of linux

Thanks... :-)

> do you know any others?

AFAIK no... but I suspect that all SystemV derivates are on the "safe"
(by default) side...

----

Bye,
Roland

----

Bye,
Roland
-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 3992797
 (;O/ \/ \O;)

_______________________________________________
ast-developers mailing list
ast-developers@research.att.com
https://mailman.research.att.com/mailman/listinfo/ast-developers
[prev in list] [next in list] [prev in thread] [next in thread]