'Re: [openMosix-general] Allocation failures'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openmosix-general
Subject:    Re: [openMosix-general] Allocation failures
From:       Jürgen_Knödlseder <Jurgen.Knodlseder () cesr ! fr>
Date:       2007-06-04 16:24:02
Message-ID: 568B0202-1C0C-45C4-BFC9-A1A6B5571F69 () cesr ! fr
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]


... at least it's enabled in my kernel, but I can't tell you if this  
will solve your problem.

Le 4 juin 07 à 18:18, David Brodbeck a écrit :

> It looks like the kernel was compiled with 4 GB support, so that  
> should be OK.
>
> I wonder if enabling HIGHMEM I/O Support would help?  I'm  
> suspicious I'm running out of space in the low memory area, so  
> maybe shifting the I/O buffers to high memory would mitigate the  
> problem.
>
>
> On Jun 4, 2007, at 6:23 AM, Jürgen Knödlseder wrote:
>
>> What memory option did you use to compile the kernel? I was told that
>> 64 GB HIMEM is not supported, so I'm now running with the 4 GB option
>> (which in fact does not allow me to use the full memory of my
>> machines :-( Since I used the 4 GB option I did not encounter any
>> problems anymore (I has stability problems with 64 GB ...)
>>
>> Jürgen
>>
>> Le 4 juin 07 à 11:54, Ralf Oelschlaegel a écrit :
>>
>>> David Brodbeck schrieb:
>>>> I'm managing an OpenMosix cluster, running kernel 2.4.26-om1 on
>>>> Debian Sarge.  After the head node has been up for a couple of  
>>>> weeks
>>>> I start seeing a lot of allocation failure messages in the kernel
>>>> logs.  The situation eventually deteriorates to the point where the
>>>> machine is unusable -- all attempts to launch processes end with  
>>>> fork
>>>> reporting allocation failures.  The problem does not appear to be
>>>> memory exhaustion; the machine has 4 GiB of RAM and never uses more
>>>> than 100 megabytes or so of swap.
>>>>
>>>> Has anyone else seen this problem?  Is this the infamous Linux  
>>>> 2.4.x
>>>> 'memory fragmentation' issue rearing its ugly head?  Is there a  
>>>> fix?
>>> we use the same kernel on SuSE8.2 system and we have the same
>>> problem. After one
>>> month I have to reboot the master node and I think the problem is
>>> memory
>>> fragmantation (see memory/rawmemory on master by mosmon).
>>> Our "workaround" is checkpoint/restart for long running jobs.
>>>
>>> Ralf
>>>
>>>
>>> -------------------------------------------------------------------- 
>>> --
>>> ---
>>> This SF.net email is sponsored by DB2 Express
>>> Download DB2 Express C - the FREE version of DB2 express and take
>>> control of your XML. No limits. Just data. Click to get it now.
>>> http://sourceforge.net/powerbar/db2/
>>> _______________________________________________
>>> openMosix-general mailing list
>>> openMosix-general@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/openmosix-general
>>
>>
>> --------------------------------------------------------------------- 
>> ----
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> openMosix-general mailing list
>> openMosix-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/openmosix-general
>
> David Brodbeck
> Information Technology Specialist 3
> Computational Linguistics
>
>
>
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/ 
> _______________________________________________
> openMosix-general mailing list
> openMosix-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/openmosix-general


[Attachment #5 (unknown)]

<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: \
after-white-space; ">... at least it's enabled in my kernel, but I can't tell you if \
this will solve your problem.<DIV><BR><DIV><DIV>Le 4 juin 07 à 18:18, David Brodbeck \
a écrit :</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><DIV>It \
looks like the kernel was compiled with 4 GB support, so that should be \
OK.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>I wonder if enabling \
HIGHMEM I/O Support would help?  I'm suspicious I'm running out of space in the low \
memory area, so maybe shifting the I/O buffers to high memory would mitigate the \
problem.</DIV><DIV><BR class="khtml-block-placeholder"></DIV><BR><DIV><DIV>On Jun 4, \
2007, at 6:23 AM, Jürgen Knödlseder wrote:</DIV><BR \
class="Apple-interchange-newline"><BLOCKQUOTE type="cite"><DIV style="margin-top: \
0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">What memory option \
did you use to compile the kernel? I was told that <SPAN \
class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">64 GB HIMEM is not \
supported, so I'm now running with the 4 GB option <SPAN \
class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">(which in fact does not \
allow me to use the full memory of my <SPAN class="Apple-converted-space"> \
</SPAN></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; ">machines :-( Since I used the 4 GB option I did not encounter any \
<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">problems anymore (I has \
stability problems with 64 GB ...)</DIV><DIV style="margin-top: 0px; margin-right: \
0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">Jürgen</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Le 4 juin 07 à 11:54, Ralf \
Oelschlaegel a écrit :</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></DIV> <BLOCKQUOTE \
type="cite"><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; ">David Brodbeck schrieb:</DIV> <BLOCKQUOTE type="cite"><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">I'm managing an OpenMosix cluster, running kernel 2.4.26-om1 on</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">Debian Sarge.<SPAN class="Apple-converted-space">  </SPAN>After the head node has \
been up for a couple of weeks</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">I start seeing a lot of allocation failure \
messages in the kernel</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">logs.<SPAN class="Apple-converted-space">  \
</SPAN>The situation eventually deteriorates to the point where the</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">machine is unusable -- all attempts to launch processes end with fork</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">reporting allocation failures.<SPAN class="Apple-converted-space">  </SPAN>The \
problem does not appear to be</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">memory exhaustion; the machine has 4 GiB of \
RAM and never uses more</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">than 100 megabytes or so of swap.</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">Has anyone else seen this problem?<SPAN \
class="Apple-converted-space">  </SPAN>Is this the infamous Linux 2.4.x</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">'memory fragmentation' issue rearing its ugly head?<SPAN \
class="Apple-converted-space">  </SPAN>Is there a fix?</DIV> </BLOCKQUOTE><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">we \
use the same kernel on SuSE8.2 system and we have the same <SPAN \
class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">problem. After \
one</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; ">month I have to reboot the master node and I think the problem is \
<SPAN class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">memory</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">fragmantation (see memory/rawmemory on master by mosmon).</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">Our "workaround" is checkpoint/restart for long running jobs.</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; ">Ralf</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; \
"><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">----------------------------------------------------------------------<SPAN \
class="Apple-converted-space"> </SPAN></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">---</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">This SF.net email is sponsored by DB2 Express</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Download DB2 Express C - \
the FREE version of DB2 express and take</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">control of your XML. No \
limits. Just data. Click to get it now.</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="http://sourceforge.net/powerbar/db2/">http://sourceforge.net/powerbar/db2/</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">_______________________________________________</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">openMosix-general mailing \
list</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; "><A \
href="mailto:openMosix-general@lists.sourceforge.net">openMosix-general@lists.sourceforge.net</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="https://lists.sourceforge.net/lists/listinfo/openmosix-general">https://lists.sourceforge.net/lists/listinfo/openmosix-general</A></DIV> \
</BLOCKQUOTE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; min-height: 14px; "><BR></DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; \
"><BR></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; ">-------------------------------------------------------------------------</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">This SF.net email is sponsored by DB2 Express</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Download DB2 Express C - \
the FREE version of DB2 express and take</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">control of your XML. No \
limits. Just data. Click to get it now.</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="http://sourceforge.net/powerbar/db2/">http://sourceforge.net/powerbar/db2/</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">_______________________________________________</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">openMosix-general mailing \
list</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; "><A \
href="mailto:openMosix-general@lists.sourceforge.net">openMosix-general@lists.sourceforge.net</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="https://lists.sourceforge.net/lists/listinfo/openmosix-general">https://lists.sourceforge.net/lists/listinfo/openmosix-general</A></DIV> \
</BLOCKQUOTE></DIV><BR><DIV> <SPAN class="Apple-style-span" style="border-collapse: \
separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; \
letter-spacing: normal; line-height: normal; text-align: auto; \
-khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: \
auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: \
0px; "><DIV>David Brodbeck</DIV><DIV>Information Technology Specialist \
3</DIV><DIV>Computational Linguistics</DIV><DIV><BR \
class="khtml-block-placeholder"></DIV><DIV><BR \
class="khtml-block-placeholder"></DIV><BR class="Apple-interchange-newline"></SPAN> \
</DIV><BR><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; \
margin-left: 0px; ">-------------------------------------------------------------------------</DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">This SF.net email is sponsored by DB2 Express</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Download DB2 Express C - \
the FREE version of DB2 express and take</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">control of your XML. No \
limits. Just data. Click to get it now.</DIV><DIV style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="http://sourceforge.net/powerbar/db2/____________________________________________ \
___">http://sourceforge.net/powerbar/db2/_______________________________________________</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; \
">openMosix-general mailing list</DIV><DIV style="margin-top: 0px; margin-right: 0px; \
margin-bottom: 0px; margin-left: 0px; "><A \
href="mailto:openMosix-general@lists.sourceforge.net">openMosix-general@lists.sourceforge.net</A></DIV><DIV \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A \
href="https://lists.sourceforge.net/lists/listinfo/openmosix-general">https://lists.sourceforge.net/lists/listinfo/openmosix-general</A></DIV> \
</BLOCKQUOTE></DIV><BR></DIV></BODY></HTML>



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

_______________________________________________
openMosix-general mailing list
openMosix-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/openmosix-general


[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic