[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hpux-devtools
Subject:    Re: HPUX-DEVTOOLS: Out of memory situations
From:       Mike Stroyan <stroyan () hpstryn ! fc ! hp ! com>
Date:       2001-10-23 17:44:56
[Download RAW message or body]

On Tue, Oct 23, 2001 at 10:40:53AM +0200, Claus Brod wrote:
> Mike Stroyan wrote:
> >  HP-UX is usually conservative about making certain there is enough
> >swap space to cover the needs of all allocations in a process.  The
> >resource requirements are checked and reserved when you exec, sbrk, mmap
> >or shmget.  There are a few exceptions where a memory reference needs to
> >allocate swap, so a process could run out of swap.
> >1: Stack growth.  (You would want to use sigaltstack to catch that signal.)
> >2: Writing to text areas in an EXEC_MAGIC program, (linked with -N).
> >3: Data access when using "chatr +z enable" or "chatr +dz enable" to set
> >   lazy swap allocation.
> >4: Mmap address access after using mmap with MAP_NORESERVE flag.
> 
> If I understand this correctly, those four cases describe when a process
> could run into signals when referencing memory, right? If so, then our
> code should be safe because none of the above applies (well, except if
> there is an infinite recursion and stack overflow due to a bug in some
> algorithm). I am *assuming* that the process will receive a signal in
> all these cases which it then has a chance to catch. Is that a valid
> assumption?

  Those are all the cases that I know of that could get a SIGBUS signal
because of when accessing running out of swap when referencing memory.
That stack growth one could conceivably hit an application very early
if the system was very close to running out of swap.  You could catch
it as soon as you set up sigaltstack and a SIGBUS signal handler.

> There is a dubious statement in the HP-UX Memory Management White Paper
> at http://docs.hp.com./hpux/onlinedocs/os/11i/mem_mgt.html, in the
> section "Reservation of Physical Swap Space":
> 
> "... If the system runs out of swap space but needs to write a process'
> page(s) to a swap device, it has no alternative but to kill the process. 
> To alleviate this problem, HP-UX reserves swap at the time the process
> is 
> forked or exec'd. When a new process is forked or executed, if 
> insufficient swap space is available and reserved to handle the entire 
> process, the process may not execute."
> 
> Two questions about this:
> 
> - They say that HP-UX reserves swap a fork/exec time to "alleviate"
>   the issue. I would interpret this to mean that this strategy
>   helps to reduce the number of cases where a process must be
>   killed, but my command of the English language is limited, so
>   I might have misunderstood. Any comments?

  It certainly reduces the number cases when a memory reference could
run out of swap.  The white paper seems to have been written before the
"chatr +z enable" option was implemented.  It does not discuss that option.

> - I *think* the above statement only applies for the time when
>   a new process starts. Once it is running, it will call
>   sbrk() which either reserves swap space and thereby makes
>   sure that subsequent memory modifications will succeed
>   (unless lazy swap is enabled) OR returns with an error 
>   indicating that it cannot reserve more virtual memory,
>   allowing the process to handle the out of memory
>   situation.  Am I hallucinating?

  It is saying that swap area is reserved for areas like bss and
initialized data at start time.  It is also reserved before returning
from brk/sbrk.

> Oh well, so I still have a contradiction here. Two people somehow
> remember that there is at least an off chance that the OS will select a
> process and kill it when the memory situation becomes really, really
> tight. On top of this, there is this slightly scary note in the white
> paper. On the other hand, all other documents and all experts
> essentially do not say anything about killing processes - which is in
> line with observations for a test program I have written. I am not sure
> what to make of this.

  It is not that the OS will select a process at random.  If a process
is in one of those four situations and tries to tries to use an address
that requires more swap than the system has it will receive a SIGBUS.
The signal will go to the process that was trying to access a bad
address.  The address is bad because the system can't get the swap space
to back it up.

> Is it possible that old versions of HP-UX behaved differently in
> emergency situations? What about HP-UX on good ol' s300 systems?

  I don't recall any version of HP-UX that defaulted to lazy swap
allocation for the data area.  That used to be a default on SGI IRIX.

> >  There is documentation of the lazy swap feature at
> >http://docs.fc.hp.com./hpux/onlinedocs/B3782-90716/00/00/33-con.html
> 
> That seems to be an HP-internal server, but I found a document on lazy
> swap on docs.hp.com as well, tnx!

  I'll just have to stop using that local mirror.  I keep forgetting to
take out the ".fc" part of the URL.  That URL works outside HP if you
drop the ".fc".

-- 
Mike Stroyan, mike_stroyan@fc.hp.com
 _________________________________________________________________
 To leave this mailing list, send mail to majordomo@cxx.cup.hp.com
    with the message UNSUBSCRIBE hpux-devtools
 _________________________________________________________________

[prev in list] [next in list] [prev in thread] [next in thread]