[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: solr starting time takes too long
From:       Erick Erickson <erickerickson () gmail ! com>
Date:       2013-05-31 12:11:18
Message-ID: CAN4YXvf0b3YUan=5qADJmm=NO-PeO7BkS9AsDHTw9HfrC6vrGQ () mail ! gmail ! com
[Download RAW message or body]

Right, the Wiki is a bit out of date. What's going to happen (I hope)
is that SOLR-4779 will allow you to share the complete configuration,
schema and solrconfig rather than try to share them individually. This
is consistent with the SolrCloud way of having named config sets.

Essentially, you'll point a core at a "conf" directory (which will
probably be located in <solrhome>/configs/conf1,
<solrhome>/config/conf2 etc) and multiple cores can point to the same
conf directory. Then both schema and solrconfig (and stopwords and all
the rest of the stuff in the conf dir) will be shared amongst all
cores that specify the same dir.

At least that's the plan..

Thanks for this feedback btw, it's good to have some field evidence of
where the time is actually spent.

Best
Erick

On Thu, May 30, 2013 at 6:24 PM, Zhang, Lisheng
<Lisheng.Zhang@broadvision.com> wrote:
> Hi Eric,
>
> Thanks very much for helps (I should have responded sooner):
>
> 1/ My problem in 3.6 turned out to be much related to the fact I did not share schema,
>    after using shareSchema, the start time is reduced up to 80% (to my great surprise,
>    previously I thought burden is most in solrconfig).
>
> 2/ I just upgraded to solr 4.3, but somehow I did not see all the fixes mentioned in
>    the WIKI (like shareConfig), I saw the resolution is "Won't fix", do you have plan
>    to put the fix into next release?
>
> Thanks and best regards, Lisheng
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Wednesday, May 22, 2013 4:57 AM
> To: solr-user@lucene.apache.org
> Subject: Re: solr starting time takes too long
>
>
> Zhang:
>
> In 3.6, there's really no choice except to load all the cores on
> startup. 10 minutes still seems excessive, do you perhaps have a
> heavy-weight firstSearcher query?
>
> Yes, soft commits are 4.x only, so that's not your problem.
>
> There's a shareSchema option that tries to only load 1 copy of the
> schema that should help, but that doesn't help with loading
> solrconfig.xml.
>
> Also in the 4.3+ world there's the option to lazily-load cores, see:
> http://wiki.apache.org/solr/LotsOfCores for the overview. Perhaps not
> an option, but I thought I'd mention it.
>
> But I'm afraid you're stuck. You might be able to run bigger hardware
> (perhaps you're memory-starved). Other than that, you may need to use
> more than one machine to get fast enough startup times.
>
> Best,
> Erick
>
> On Wed, May 22, 2013 at 3:27 AM, Zhang, Lisheng
> <Lisheng.Zhang@broadvision.com> wrote:
>> Thanks very much for quick helps! I searched but it seems that
>> autoSoftCommit is solr 4x feature and we are still using 3.6.1?
>>
>> Best regards, Lisheng
>>
>> -----Original Message-----
>> From: Carlos Bonilla [mailto:carlosbonilla84@gmail.com]
>> Sent: Wednesday, May 22, 2013 12:17 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: solr starting time takes too long
>>
>>
>> Hi Lisheng,
>> I had the same problem when I enabled the "autoSoftCommit" in
>> solrconfig.xml. If you have it enabled, disabling it could fix your problem,
>>
>> Cheers.
>> Carlos.
>>
>>
>> 2013/5/22 Zhang, Lisheng <Lisheng.Zhang@broadvision.com>
>>
>>>
>>> Hi,
>>>
>>> We are using solr 3.6.1, our application has many cores (more than 1K),
>>> the problem is that solr starting took a long time (>10m). Examing log
>>> file and code we found that for each core we loaded many resources, but
>>> in our app, we are sure we are always using the same solrconfig.xml and
>>> schema.xml for all cores. While we can config schema.xml to be shared,
>>> we cannot share SolrConfig object. But looking inside SolrConfig code,
>>> we donot use any of the cache.
>>>
>>> Could we somehow change config (or source code) to share resource between
>>> cores to reduce solr starting time?
>>>
>>> Thanks very much for helps, Lisheng
>>>
[prev in list] [next in list] [prev in thread] [next in thread]