[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lucene-user
Subject:    Re: Fast access to a random page of the search results.
From:       Erik Hatcher <erik () ehatchersolutions ! com>
Date:       2005-02-28 16:59:54
Message-ID: d08003cfff6c6ece3420f8f8c97895b0 () ehatchersolutions ! com
[Download RAW message or body]

On Feb 28, 2005, at 10:39 AM, Stanislav Jordanov wrote:
> > What did you do in your private investigation?
> 1. empirical tests with an index of nearly 75,000 docs (I am attaching 
> the test source)

Only certain (.txt?) attachments are allowed to come through on the 
mailing list.

> > Sorted by descending relevance (the default), or in some other way?
> In some other way - sorted by some column (asc or desc - doesn't 
> matter)

Using IndexSearcher(query, sort)?

>  > If a search is fast enough, as you report, then you can simply start
> > your access to Hits at the appropriate spot.  For the current systems
> > I'm working on, this is the approach I've used - start iterating hits
> > at (pageNumber - 1) * numberOfItemsPerPage.
> >
> > Is that approach insufficient?
> I'm afraid this is not sufficient;
> Either I am doing something wrong,
> or it is not that simple:
> following is a log from my test session;
> It appears that IndexSearcher.search(...) finishes rather fast
> compared to the time it takes to fetch the last document from the Hits 
> object.

I assume you are only accessing the documents you wish to display 
rather than all of them up to where you need.   Also keep in mind that 
accessing a Document is when the document is pulled from the index.  If 
you have a large amount of data in a document it will take a 
corresponding amount of time to load it.  You may need to restructure 
what you store in a document to reduce the load times.  Or perhaps you 
need to investigate the (is it in the codebase already?) patch to load 
fields lazily upon demand instead.

	Erik

>

> The log starts here:
>
> pa
>
> Found 74222 document(s) that matched query 'pa'
>
> Sorting by "sfile_name"
>
> query executed in 16ms
>
> Last doc accessed in 375ms
>
> us
>
> Found 74222 document(s) that matched query 'us'
>
> Sorting by "sfile_name"
>
> query executed in 31ms
>
> Last doc accessed in 219ms
>
> 1
>
> Found 74222 document(s) that matched query '1'
>
> Sorting by "sfile_name"
>
> query executed in 15ms
>
> Last doc accessed in 235ms
>
> 5
>
> Found 74222 document(s) that matched query '5'
>
> Sorting by "sfile_name"
>
> query executed in 422ms
>
> Last doc accessed in 219ms
>
> 6
>
> Found 72759 document(s) that matched query '6'
>
> Sorting by "sfile_name"
>
> query executed in 344ms
>
> Last doc accessed in 250ms
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]