[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: Trying to get ALL scores from a previous search in a custom search component ("last-components")
From:       Erick Erickson <erickerickson () gmail ! com>
Date:       2014-11-29 16:57:45
Message-ID: CAN4YXvfN09VqtjHdHoyYOcSo=mz2O=kg-0Ggh90fd-r-jaU2Lg () mail ! gmail ! com
[Download RAW message or body]

Note that what I said doesn't apply if you use the underlying
_index_ reading capabilities in Lucene. It's a bit more complicated
than just fetching the doc, but it's all memory-resident thus fast.
You need to unInvert the index though, and DocValues may
be helpful here.

Best,
Erick

On Fri, Nov 28, 2014 at 1:26 PM, Darin Amos <darincs@gmail.com> wrote:
> Thanks for the advice,
> 
> I will take a look to see if there is some tuning I can do here, I am not terribly \
> concerned about that yet anyway. 
> My concern still remains with how can I get the scores of the entire matched set. \
> Maybe it is not possible, or perhaps I need to write my own query/match/scorer to \
> do this. 
> Curious is anyone else has tried to implement a similar rollup type search \
> component before. 
> Thanks!
> 
> Darin
> 
> > On Nov 28, 2014, at 4:11 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> > 
> > Because you're fetching and decompressing the doc from disk. Grouping etc.
> > Do their work from _indexed_ terms, which are already in memory. Two
> > different things.
> > 
> > If I'm reading this right on a quick scan...
> > 
> > Best
> > Erick
> > On Nov 28, 2014 10:21 AM, "Darin Amos" <darincs@gmail.com> wrote:
> > 
> > > Hi Eric,
> > > 
> > > I am curious why this would b considered an anti-patern to check a stored
> > > valued for every matching document. Is this not what the facet query
> > > component is doing anyway so it can get the total counts?
> > > 
> > > Grouping doesn't solve the issue because again, I will only see groups for
> > > the items returned, not all matching documents.
> > > 
> > > I am using a product that has SOLR 4.3 embedded into it, so I cannot
> > > upgrade to 4.9 or 4.10 and take advantage of the parent/child feature added
> > > recently.
> > > 
> > > Thanks!
> > > 
> > > Darin
> > > 
> > > > On Nov 28, 2014, at 12:03 PM, Erick Erickson <erickerickson@gmail.com>
> > > wrote:
> > > > 
> > > > Does grouping work for you here? Because
> > > > even if you solve this problem, if I'm reading
> > > > this right you're going to fetch stored values
> > > > for every doc that matches the query, which
> > > > is an anti-pattern big-time, consider *:*....
> > > > 
> > > > Of course I did a very quick skim, so maybe
> > > > I'm all wet....
> > > > 
> > > > Best,
> > > > Erick
> > > > 
> > > > On Thu, Nov 27, 2014 at 5:28 PM, Darin Amos <darincs@gmail.com> wrote:
> > > > > Hello,
> > > > > 
> > > > > I am trying to implement a Rollup Search component on a version of SOLR
> > > that exists previously to the parent/child additions, so I am trying to
> > > implement my own. The searches will be executed exclusively against the
> > > child documents, and I want to "rollup" those child documents into the
> > > parent documents.
> > > > > 
> > > > > The interface is going to allow the user to add the following
> > > parameters to the SOLR query:
> > > > > 
> > > > > &rollup=true&rollup.parentField=id&rollup.childField=parentId
> > > > > 
> > > > > My code so far is below. What I have works so far, except my second
> > > parent query loses the order. I would like to be able to sort my parent
> > > query by the score of the previous child search. Perhaps I would take the
> > > highest score from all children (haven't decided yet). My problem however
> > > is that I don't know how I can get the score from all the hits in the
> > > original search, just what is returned. If my child query gets 10,000 hits,
> > > but only return 100 records, I can't get all the scores I need.
> > > > > 
> > > > > Does anyone have any recommendations?
> > > > > 
> > > > > Thanks!!
> > > > > Darin
> > > > > 
> > > > > 
> > > > > //Loop through all the records and look for the
> > > parent reference field
> > > > > Set<String> parentRefs = new HashSet<String>();
> > > > > DocIterator docSetIterator =
> > > rb.getResults().docSet.iterator();
> > > > > while(docSetIterator.hasNext()){
> > > > > int docInt = docSetIterator.next();
> > > > > String fieldValues[] =
> > > rb.req.getSearcher().doc(docInt).getValues(childFieldName);
> > > > > 
> > > > > for(String fieldValue : fieldValues){
> > > > > if(fieldValue != null &&
> > > fieldValue.length() > 0 && !parentRefs.contains(fieldValue)){
> > > > > 
> > > parentRefs.add(fieldValue);
> > > > > }
> > > > > }
> > > > > }
> > > > > 
> > > > > //Build a boolean query of term queries
> > > > > BooleanQuery parentQuery = new BooleanQuery();
> > > > > Iterator<String> parentIdIterator =
> > > parentRefs.iterator();
> > > > > while(parentIdIterator.hasNext()){
> > > > > String parentId =
> > > parentIdIterator.next();
> > > > > TermQuery termQuery = new TermQuery(new
> > > Term(parentFieldName, parentId));
> > > > > parentQuery.add(termQuery,
> > > BooleanClause.Occur.SHOULD);
> > > > > }
> > > > > 
> > > > > 
> > > > > DocList parentList =
> > > searcher.getDocList(parentQuery, new ArrayList<Query>(), null, 0, 100, 1);
> > > //TODO: use correct start/end/flags later...
> > > > > 
> > > > > //Add parent results
> > > > > ResultContext resultContext = new ResultContext();
> > > > > resultContext.docs = parentList;
> > > > > resultContext.query = parentQuery;
> > > > > rb.rsp.add("parents", resultContext);
> > > > > rb.rsp.getToLog().add("hits", parentList.matches());
> > > 
> > > 
> 


[prev in list] [next in list] [prev in thread] [next in thread]