[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-dev
Subject:    Re: Cocoon-2.1.9 vs. SOLR-20 & SOLR-30
From:       "Yonik Seeley" <yonik () apache ! org>
Date:       2006-11-22 20:11:12
Message-ID: c68e39170611221211x6f98bec6m7edda521631228f8 () mail ! gmail ! com
[Download RAW message or body]

On 11/22/06, Walter Underwood <wunderwood@netflix.com> wrote:
> > I took pains to make things streamable.. I'd hate to discard that.
> > How do other servers handle streaming back a response and hitting an error?
>
> Does Lucene access fetch information from disk while we iterate
> through the search results?

Yes.

Originally, all the documents were retrieved up-front, and the
response writer didn't even have access to the IndexReader.  After
seeing some users ask for some fields of *all* the documents in an
index on a different search product, I decided I'd better add
streamability to avoid OOM errors. A secondary consideration was
improving latency of the first document to the client when there are a
large number to be returned.

So Solr currently only records the ids (the internal integer lucene
docid) and optionally scores for documents to be returned.  During
response writing, the document for each id is read (which may involve
going to disk) right before it is written to the output stream.

-Yonik
[prev in list] [next in list] [prev in thread] [next in thread]