[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensolaris-opengrok-discuss
Subject:    [opengrok] Re: [website-discuss] Re: src.opensolaris.org bad search
From:       Alan Burlison <Alan.Burlison () sun ! com>
Date:       2007-02-01 16:44:44
Message-ID: 45C218FC.5050109 () sun ! com
[Download RAW message or body]

Darren J. Moffat wrote:

> Whats the current status of opengrok indexing on opensolaris.org ?
> 
> What is included ? What is not included ? Has any attempt been made
> at deploying a separate opengrok for the "canoncial"  consolidation
> repos vs project ones ?
> 
> I've noticed that there doesn't seem to be consistency at the moment
> about what is and is not index - and thats the worst situation of
> all!

As you've probably noticed we had quite a few stability problems with 
opengrok - see opengrok-discuss 
(http://mail.opensolaris.org/pipermail/opengrok-discuss/2007-January/000157.html) 
for details.

We've removed the Apache/mod_jk front-end, and put a robots.txt in place 
to reduce the hammering that googlebot was giving the site - googlebot 
was crawling the site 24x7, at about 4-5 pages a second, and was 
responsible for 60% of our load as a result.

We are going to rebuild the indexes to just include the "canonical" 
distributions so the project gates won't clutter up the search results, 
and Trond is going on to make opengrok play nicer with multiple source 
trees as well, but it isn't finished yet.

The other issue is one of index distribution - we build the OG indexes 
on one machine and then transfer them over to the two machines that 
actually provide the OG webservice.  Unfortunately you can't do this 
reliably on the fly, so we are going to have to copy the indexes over, 
then bounce the webservices to pick up the new indexes.  Again, Trond is 
working on getting a better solution for this, but it isn't finished yet.

If anyone can hack Java and wants to help, Trond is the person to talk 
to :-)

-- 
Alan Burlison
--
_______________________________________________
opengrok-discuss mailing list
opengrok-discuss@opensolaris.org
http://opensolaris.org/mailman/listinfo/opengrok-discuss
[prev in list] [next in list] [prev in thread] [next in thread]