[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lucene-dev
Subject:    [jira] Updated: (LUCENE-769) [PATCH] Performance improvement for
From:       "Artem Vasiliev (JIRA)" <jira () apache ! org>
Date:       2007-01-31 22:20:05
Message-ID: 32484946.1170282005820.JavaMail.jira () brutus
[Download RAW message or body]


     [ https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]

Artem Vasiliev updated LUCENE-769:
----------------------------------

    Attachment: selfContained.patch

Removed several inner classes and documents cache from StoredFieldSortFactory. Now \
the whole class is pretty clean and simple. Checked the timings in the test - they \
remain pretty much the same.  

Checked it in Sharehound with more large index (~1mln documents), it sorts resultset \
with 4000 docs in ~0,2s now taking 70M RAM, that's fine to me. 

With standart new Sort(field, false) it takes (for the first search on a field) about \
30-40s and quite a lot of memory (after several sorted searches different fields it \
took about 500M).

> [PATCH] Performance improvement for some cases of sorted search
> ---------------------------------------------------------------
> 
> Key: LUCENE-769
> URL: https://issues.apache.org/jira/browse/LUCENE-769
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Artem Vasiliev
> Attachments: DocCachingSorting.patch, DocCachingSorting.patch, \
> IndexReaderUtils.java, QueryFilter.java, selfContained.patch, selfContained.patch, \
> selfContained.patch, selfContained.patch, StoredFieldSorting.patch 
> 
> It's a small addition to Lucene that significantly lowers memory consumption and \
> improves performance for sorted searches with frequent index updates and relatively \
> big indexes (>1mln docs) scenario. This solution supports only single-field sorting \
> currently (which seem to be quite popular use case). Multiple fields support can be \
> added without much trouble. The solution is this: documents from the sorting set \
> (instead of given field's values from the whole index - current FieldCache \
> approach) are cached in a WeakHashMap so the cached items are candidates for GC.  \
> Their fields values are then fetched from the cache and compared while sorting.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]