[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: Standardized index metrics (Was: Constantly high disk read access (40-60M/s))
From:       Michael Sokolov <msokolov () safaribooksonline ! com>
Date:       2014-11-29 19:27:58
Message-ID: 547A1E3E.4040204 () safaribooksonline ! com
[Download RAW message or body]

On 11/29/14 1:30 PM, Toke Eskildsen wrote:
> Michael Sokolov [msokolov@safaribooksonline.com] wrote:
> > I wonder if there's any value in providing this metric (total index size
> > - stored field size - term vector size) as part of the admin panel?  Is
> > it meaningful?  It seems like there would be a lot of cases where it
> > could give a good rule of thumb for memory sizing, and it would save
> > having to root around in the index folder.
> At Lucene/Solr Revolution, I talked with Alexandre Rafalovitch about this. We know \
> (https://lucidworks.com/blog/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/) \
> that we cannot get the full picture of an index, but it is a weekly occurrence on \
> this mailing list that people asks questions where it helps to have a gist of the \
> index metrics and how the index is used. 
> Some sort of "Copy the content of this concentrated metrics box, when you need to \
> talk with other people about your index"-functionality in the admin panel might \
> help with this. To get an idea of usage, it could also contain a few non-filled \
> fields, such as "peak queries per second" or "typical queries". 
> - Toke Eskildsen
Yes - the cautions about the need for prototyping are all very well, but 
even if you take that advice, and build a prototype, it's not clear how 
to tell whether your setup has enough memory or not. You can add more 
and measure response times, but even then you only have a gross 
measurement, and no way of knowing where, in detail, the memory is being 
used.  Also, you might be able to improve your system to make better use 
of memory with more precise information. It seems like we ought to be 
able to monitor a running system, observe its memory requirements over 
time, and report on those.

-Mike


[prev in list] [next in list] [prev in thread] [next in thread]