[prev in list] [next in list] [prev in thread] [next in thread] 

List:       apache-modperl
Subject:    Re: Mod_Perl and MaxRequestsPerChild
From:       "Mark Maunder" <mmaunder () gmail ! com>
Date:       2007-10-16 19:39:25
Message-ID: dc9d21e80710161239q6f076ev66aee8e0c1c7e80f () mail ! gmail ! com
[Download RAW message or body]

Also, just some additional data for the archives and discussion: It
seems (in my case) that prefork is much more memory hungry than
worker. Here are the details:

I switched to a prefork model from worker MPM. As I mentioned before
my load avg seemed to drop, but memory usage went up. It continued to
rise as my processes grew. My mod_perl app works with some fairly
large data structures and AFAIK perl doesn't like to free memory back
to the OS once it's allocated it, so the processes tend to grow for
the first few hours of the server being up and then the plateau and
grow about 1 meg per day (a slow leak I think).

I brought up my server with prefork and only 150 children. As the
processes started growing my box ran out of memory, started swapping
heavily and I had to shut down and revert to worker.

I'm back on worker and I have a full 250 threads with much lower memory usage.

My unscientific data is as follows:

When i was running with prefork, each process was 29 Megs and there
were 150 of them. That's 4.3 Gigs and my box only has 2 Gigs so
apparently copy-on-write was in effect and some of that was shared.

Now that I'm back to threads, my largest process is 200Megs and there
are 5 of them (four seem to be consuming memory, so I assume the 5th
is the parent?). So the total memory consumed without sharing is 1
Gig.

So even without copy-on-write it seems that worker MPM is much more
memory efficient in my case. This may be something specific to my
config because my app is smaller than most apps I've written.

BTW, most of my app is written as handlers and all modules are loaded
in startup.pl.

Regards,

Mark.





On 10/16/07, Mark Maunder <mmaunder@gmail.com> wrote:
> Thanks guys. I assumed worker would save me memory because threads are
> light(er)weight. I'm busy trying prefork now and my load avg seems to
> have dropped slightly, but my memory usage seems to have gone up
> somewhat.
>
> Just read your comments in the archives on copy-on-write Perrin - I
> had no idea it doesn't work for threads. Thanks for that!
>
> http://www.jsw4.net/info/listserv_archives/mod_perl/07-wk22/msg00022.html
>
> However, my original problem may still exist and I have more data now.
> I noticed the number of httpd children just went up from 150 to 152
> and is holding there and I also have two messages in my error log that
> look like the following:
>
> (70007)The timeout specified has expired: ap_get_brigade failed during
> prefetch, referer: *snip*
>
> I've looked at my logs going back the last 3 days and this message
> appears several times over the last 3 days. It's the only message out
> of the ordinary. I've googled this message before and haven't found
> much of an explanation.
>
> Have any of you seen this before? Any idea what causes it? I've
> confirmed it appears whether I'm running as worker or prefork.
>
> Thanks again for your help so far!
>
> Mark.
>
>
> On 10/16/07, Perrin Harkins <perrin@elem.com> wrote:
> > On 10/16/07, Clinton Gormley <clint@traveljury.com> wrote:
> > > Unless you have a really good reason to use worker, on linux, the
> > > recommended MPM is worker.
> >
> > I'm sure you meant to say prefork there.  And I agree, if you're
> > concerned about memory, don't use threads.  Prefork will save you
> > memory because of copy-on-write.
> >
> > - Perrin
> >
>
[prev in list] [next in list] [prev in thread] [next in thread]