[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: About memory allocation failures....
From:       Kuba Ober <kuba () mareimbrium ! org>
Date:       2002-02-04 17:33:54
[Download RAW message or body]

> > > Don't bother. By the time new or malloc starts to return null your
> > > process has proably been killed already. Or you ran out of memory one
> > > allocation earlier which wasn't checked and caused your program to
> > > core-dump already.
> > >
> > > Unless you need to allocate several MBs of memory at once there is
> > > little practical use in checking the allocations.
> >
> > I'm sorry, I just don't agree here. I think all new/mallocs should be
> > check. A core dump, although practical for a programmer, is not practical
> > for an end-user. This reminds me of the early Microsoft days where apps
> > would simply fail with no meaningful message. An attempt to tell the user
> > that memory has run out can at least tell the user the app failed for
> > memory reasons, not something else.

A core dump can be post-processed by a generic application and system logs 
can be written to. Unless you run QNX or a similar system, there's no way to 
be sure that any dialog box will get displayed. Remember that memory 
allocation failure (assuming no limits/quotas which are useless anyway) means 
that typically:
- most of disk/network buffers and caches have been reclaimed as memory to be 
used by process resident sets
- swap is full with process code & data pages
- there's just a little (probably a few megabytes at most in large systems) 
physical memory left for kernel allocations, and only for these (i.e. 
user-space-originating allocations don't use it)
- there's not much to do but to either start killing processes (OOM killer), 
or to stop some processes in hopes that some other processes will give up 
their memory, which requires additional tools that I'm unaware of (it's 
doable, and possibly quite simple to implement, but I doubt of its 
usefullness)

> On MS Windows 2.x and 3.0, you could perhaps have a chance to get a dialog
> box, because a dialog box with a "STOP" was displayed at all costs, even
> under memory pressure.
>
> But here you have no such backdoors.
>
> One possibility is that you have bumped against a quota. In this case, you
> will simply not get anymore memory.
>
> Or you have exhausted all the memory (+ swap) of your system. In this case,
> the OOM (Out Of Memory) killer will kill your application without previous
> notice. And as far as I know, not only Linux 2.4 has a OOM killer.
>
> If the OOM killer kills another application than the one it should, (my)
> experience has shown that it was mostly kdeinit that was hurt. So you will
> not have KDE anymore to do anything! If your program is still running after
> that...
>
> And if your system has no OOM killer, you would have the same problem as
> with a quota and, seconds later, your system would probably crash.

Or, rather than crashing the processes will just get killed because of null 
pointer dereferencing. That's almost as good as OOM killer, and it's built 
into any virtual-memory-based unix derivative!

> All these reason makes that only if the memory that you have tried to get
> is big enough, you would get a chance to display something. However do not
> forget that your OS, X Window, (a few components of) KDE and your
> application are fighting about the rest of memory that you have (even just
> to display a dialog box.)

Yep: what you'd need to do is to grab a large chunk of memory in advance, and 
then when memory is low here's what happens:
0a. You've got some virtual memory allocated by the kernel (vmalloc or 
similar), and you've touched the memory (say written it all over with 0xF8 or 
somesuch ;-)
0b. You're sure you've not allocated it using standard C/C++ library calls 
(new/malloc). Otherwise it makes no sense. Allocations using new/malloc 
typically just enlarge your application's heap, and since these blocks aren't 
at the end of the heap, you don't decrease your process' heap anyway by 
freeing them - the pages will still eat up swap just as they did. There are 
ways around this by using clever memory allocation libraries, but the 
standard ones aren't so clever AFAIK.
Actually, if the pages were untouched, they may not even use (nor "protect by 
preallocation") *any* swap space nor physical memory space, save for page 
tables. Thus, freeing the memory and then using it actually enlarges your 
process' swap+physical memory requirements (as the unused pages have to be 
"materialized" somewhere, that is either in physical memory or in swap) !!!!!
1. You free the memory
2. The kernel gets some spare free pages, that typically makes a few of the 
waiting processes runnable, scheduler is called and you loose your execution 
context
3. After some time you reclaim execution context, but all the free pages were 
taken by whatever requirements the processes that have just executed their 
time slices had
4. You try to display a dialog box, but the freed memory might have as well 
not existed. This proves that the whole exercise is pointless.

I'm just reiterating why preallocation is useless, and why allocation 
checking is useless, too! If your language of choice (say C++ or Java) 
supports memory-allocation exceptions, that's the way to go as these come 
with negligible overhead if your application supports exceptions anyway, and 
they are handled automatically, and you only need to write one global handler 
in your app with hopes it will get a chance to actually do some work. 
Otherwise any explicit checks are totally, wholly useless, a waste of time, 
and a *bad* thing to do to the open-source community - better spend your time 
shrinking your code.

Cheers,
Kuba
 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
[prev in list] [next in list] [prev in thread] [next in thread]