[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lucene-dev
Subject:    [jira] [Commented] (SOLR-11423) Overseer queue needs a hard cap (maximum size) that clients respect
From:       "Erick Erickson (JIRA)" <jira () apache ! org>
Date:       2017-09-30 15:05:00
Message-ID: JIRA.13106124.1506713711000.245328.1506783900145 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/SOLR-11423?page=com.atlassian.jira.plugin. \
system.issuetabpanels:comment-tabpanel&focusedCommentId=16187121#comment-16187121 ] 

Erick Erickson commented on SOLR-11423:
---------------------------------------

What version of Solr? There has been some work done recently to reduce the number of \
tasks the Overseer needs to process and I'm wondering if your observations include \
those changes or not.

> Overseer queue needs a hard cap (maximum size) that clients respect
> -------------------------------------------------------------------
> 
> Key: SOLR-11423
> URL: https://issues.apache.org/jira/browse/SOLR-11423
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public) 
> Components: SolrCloud
> Reporter: Scott Blum
> Assignee: Scott Blum
> 
> When Solr gets into pathological GC thrashing states, it can fill the overseer \
> queue with literally thousands and thousands of queued state changes.  Many of \
> these end up being duplicated up/down state updates.  Our production cluster has \
> gotten to the 100k queued items level many times, and there's nothing useful you \
> can do at this point except manually purge the queue in ZK.  Recently, it hit 3 \
> million queued items, at which point our entire ZK cluster exploded. I propose a \
> hard cap.  Any client trying to enqueue a item when a queue is full would throw an \
> exception.  I was thinking maybe 10,000 items would be a reasonable limit.  \
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]