[prev in list] [next in list] [prev in thread] [next in thread]
List: opensolaris-opengrok-discuss
Subject: Re: [opengrok] Opengrok 11.1 Memory Leak
From: Rodrigo Chiossi <rodrigochiossi () gmail ! com>
Date: 2013-01-17 12:01:48
Message-ID: CAOH+zhjx3vecKuBO8jYUFb++-FPEVgM2wk=mPeCUSo5m5AxNPg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I have a production server running with 150Gb source + indexes. The server
has 16Gb RAM and moderate load. I haven't experienced any memory issues so
far. Actually, that RAM is mostly used for indexing. For daily operation,
it hardly gets to the 6GB mark.
On Thu, Jan 17, 2013 at 6:15 AM, Lubos Kosco <Lubos.Kosco@oracle.com> wrote:
> On 16.1.2013 23:46, Devlin, Alex wrote:
>
> > Thanks for the quick replies Vladimir and Lubos!
> >
> > It sounds like this problem is unique to our configuration. I'm using
> > mat, jmap, and jmx to try and track down any useful information now. Also,
> > one other question: How much RAM does your inhouse machine have? Our
> > production version currently has 8gb but we're concerned that this may not
> > be enough (even without the leak).
> >
>
> (not sure if jmx will help ;) )
>
> Well, due to zfs and other(file serving) purpose of that box the memory of
> opengrok server is 48GB
> It's a busy server too, hence tomcat6 was tuned a bit.
> I can see the tomcat6 instance currently eats 5G of ram (so it might have
> been set to more than 4G mem).
> Note that we talk about 70GB of sources indexed. Indexes themselves are
> around 85GB.
>
> As noted previously I will try to decrease this memory footprint (on the
> cost of disk I/O), so I hope next version will dramatically decrease its
> memory usage(indexing might take 5% longer though, will see once I have
> some benchmarks, it could even stay the same, depending on OS caches and
> how good new lucene 4.0 indexing is).
>
> hth
>
> L
>
>
> > Thanks again,
> > -Alex
> >
> > -----Original Message-----
> > From: opengrok-discuss-bounces@**opensolaris.org<opengrok-discuss-bounces@opensolaris.org>[mailto:
> > opengrok-discuss-**bounces@opensolaris.org<opengrok-discuss-bounces@opensolaris.org>]
> > On Behalf Of Vladimir Kotal
> > Sent: Wednesday, January 16, 2013 1:40 AM
> > To: opengrok-discuss@opensolaris.**org <opengrok-discuss@opensolaris.org>
> > Subject: Re: [opengrok] Opengrok 11.1 Memory Leak
> >
> > On 01/16/13 09:45, Lubos Kosco wrote:
> >
> > <snip>
> >
> > Get to 0.11.1 , I think no reindex needed, it should contain some
> > > fixes we use 0.11.1 inhouse (not with perforce, but with 4 other scms)
> > > and don't see any of this afaik (Vlada, any comments from your side?)
> > >
> > It's been very solid. We run 0.11.1 under 64-bit Tomcat6 and JavaDB (both
> > shipped with Solaris 11+) and serving variety of
> > Mercurial,Teamware,SCCS,svn,**CVS repositories (mirrored and indexed
> > daily). It's been running like this since 0.11.1 came out.
> >
> > It looks like this right now:
> >
> > $ svcs -p tomcat6
> > STATE STIME FMRI
> > online Dec_14 svc:/network/http:tomcat6
> > Dec_14 902 java
> > $ ps -yfl -p 902
> > S UID PID PPID C PRI NI RSS SZ WCHAN STIME TTY
> > TIME CMD
> > S webservd 902 1 0 40 20 4926672 5704004 ? Dec 14 ?
> > 148:04 /usr/jdk/instances/jdk1.6.0/**bin/amd
> >
> >
> > No sight of memory leaks, the last restart was purely administrative.
> >
> >
> > v.
> > ______________________________**_________________
> > opengrok-discuss mailing list
> > opengrok-discuss@opensolaris.**org <opengrok-discuss@opensolaris.org>
> > http://mail.opensolaris.org/**mailman/listinfo/opengrok-**discuss<http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss>
> > ______________________________**_________________
> > opengrok-discuss mailing list
> > opengrok-discuss@opensolaris.**org <opengrok-discuss@opensolaris.org>
> > http://mail.opensolaris.org/**mailman/listinfo/opengrok-**discuss<http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss>
> >
>
> ______________________________**_________________
> opengrok-discuss mailing list
> opengrok-discuss@opensolaris.**org <opengrok-discuss@opensolaris.org>
> http://mail.opensolaris.org/**mailman/listinfo/opengrok-**discuss<http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss>
>
[Attachment #5 (text/html)]
I have a production server running with 150Gb source + indexes. The server has 16Gb \
RAM and moderate load. I haven't experienced any memory issues so far. Actually, \
that RAM is mostly used for indexing. For daily operation, it hardly gets to the 6GB \
mark.<br>
<br>On Thu, Jan 17, 2013 at 6:15 AM, Lubos Kosco <span dir="ltr"><<a \
href="mailto:Lubos.Kosco@oracle.com" \
target="_blank">Lubos.Kosco@oracle.com</a>></span> wrote:<br><div \
class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 16.1.2013 23:46, Devlin, Alex wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> Thanks for the quick replies Vladimir and Lubos!<br>
<br>
It sounds like this problem is unique to our configuration. I'm using mat, jmap, \
and jmx to try and track down any useful information now. Also, one other question: \
How much RAM does your inhouse machine have? Our production version currently has 8gb \
but we're concerned that this may not be enough (even without the leak).<br>
</blockquote>
<br></div>
(not sure if jmx will help ;) )<br>
<br>
Well, due to zfs and other(file serving) purpose of that box the memory of opengrok \
server is 48GB<br> It's a busy server too, hence tomcat6 was tuned a bit.<br>
I can see the tomcat6 instance currently eats 5G of ram (so it might have been set to \
more than 4G mem).<br> Note that we talk about 70GB of sources indexed. Indexes \
themselves are around 85GB.<br> <br>
As noted previously I will try to decrease this memory footprint (on the cost of disk \
I/O), so I hope next version will dramatically decrease its memory usage(indexing \
might take 5% longer though, will see once I have some benchmarks, it could even stay \
the same, depending on OS caches and how good new lucene 4.0 indexing is).<br>
<br>
hth<div class="HOEnZb"><div class="h5"><br>
L<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <br>
Thanks again,<br>
-Alex<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:opengrok-discuss-bounces@opensolaris.org" \
target="_blank">opengrok-discuss-bounces@<u></u>opensolaris.org</a> [mailto:<a \
href="mailto:opengrok-discuss-bounces@opensolaris.org" \
target="_blank">opengrok-discuss-<u></u>bounces@opensolaris.org</a>] On Behalf Of \
Vladimir Kotal<br>
Sent: Wednesday, January 16, 2013 1:40 AM<br>
To: <a href="mailto:opengrok-discuss@opensolaris.org" \
target="_blank">opengrok-discuss@opensolaris.<u></u>org</a><br>
Subject: Re: [opengrok] Opengrok 11.1 Memory Leak<br>
<br>
On 01/16/13 09:45, Lubos Kosco wrote:<br>
<br>
<snip><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> Get to 0.11.1 , I think no reindex needed, it should contain \
some<br> fixes we use 0.11.1 inhouse (not with perforce, but with 4 other scms)<br>
and don't see any of this afaik (Vlada, any comments from your side?)<br>
</blockquote>
It's been very solid. We run 0.11.1 under 64-bit Tomcat6 and JavaDB (both shipped \
with Solaris 11+) and serving variety of Mercurial,Teamware,SCCS,svn,<u></u>CVS \
repositories (mirrored and indexed daily). It's been running like this since \
0.11.1 came out.<br>
<br>
It looks like this right now:<br>
<br>
$ svcs -p tomcat6<br>
STATE STIME FMRI<br>
online Dec_14 svc:/network/http:tomcat6<br>
Dec_14 902 java<br>
$ ps -yfl -p 902<br>
S UID PID PPID C PRI NI RSS SZ WCHAN STIME TTY<br>
TIME CMD<br>
S webservd 902 1 0 40 20 4926672 5704004 ? Dec 14 ?<br>
148:04 /usr/jdk/instances/jdk1.6.0/<u></u>bin/amd<br>
<br>
<br>
No sight of memory leaks, the last restart was purely administrative.<br>
<br>
<br>
v.<br>
______________________________<u></u>_________________<br>
opengrok-discuss mailing list<br>
<a href="mailto:opengrok-discuss@opensolaris.org" \
target="_blank">opengrok-discuss@opensolaris.<u></u>org</a><br> <a \
href="http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss" \
target="_blank">http://mail.opensolaris.org/<u></u>mailman/listinfo/opengrok-<u></u>discuss</a><br>
______________________________<u></u>_________________<br>
opengrok-discuss mailing list<br>
<a href="mailto:opengrok-discuss@opensolaris.org" \
target="_blank">opengrok-discuss@opensolaris.<u></u>org</a><br> <a \
href="http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss" \
target="_blank">http://mail.opensolaris.org/<u></u>mailman/listinfo/opengrok-<u></u>discuss</a><br>
</blockquote>
<br>
______________________________<u></u>_________________<br>
opengrok-discuss mailing list<br>
<a href="mailto:opengrok-discuss@opensolaris.org" \
target="_blank">opengrok-discuss@opensolaris.<u></u>org</a><br> <a \
href="http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss" \
target="_blank">http://mail.opensolaris.org/<u></u>mailman/listinfo/opengrok-<u></u>discuss</a><br>
</div></div></blockquote></div><br>
_______________________________________________
opengrok-discuss mailing list
opengrok-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/opengrok-discuss
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic