[prev in list] [next in list] [prev in thread] [next in thread] 

List:       solr-user
Subject:    Re: Highlighting for non-stored fields
From:       Erick Erickson <erickerickson () gmail ! com>
Date:       2010-10-31 1:05:59
Message-ID: AANLkTi=5SKc5o4p22CbYFVux_0nsZEcn+3EquP3pVN3e () mail ! gmail ! com
[Download RAW message or body]


Also, consider what you'd be reconstructing from if you could try it. The
indexed data has been transformed by, say, stemming, casing, etc. So
any attempt to reconstruct the fields for highlighting would necessarily
show the transformed version, which would not be pleasing. Plus you could
have synonyms in there...

I've used a variant on Pradeep's suggestion with good results...

Best
Erick

On Tue, Oct 26, 2010 at 3:44 PM, Phong Dais <phong.gdais@gmail.com> wrote:

> Thanks for the insight.
> This is definitely a feasible solution because I only need to highlight
> when
> the user open the document.
> I guess the easiest way I can do this is to "reuse" the solr code (with
> some
> modification) in my own application.
>
> On Tue, Oct 26, 2010 at 2:35 PM, Pradeep Singh <pksinghus@gmail.com>
> wrote:
>
> > Another way you can do this is - after the search has completed, load t=
he
> > field in your application, write separate code to reanalyze that
> > field/document, index it in RAM, and run it through highlighter classes=
.
> > All
> > this as part of your web application outside of Solr. Considering the
> size
> > of your data it doesn't look advisable to store it because then you wou=
ld
> > be
> > almost doubling the size of your index (if you are looking to highlight
> on
> > a
> > field then it's probably going to be full of content).
> >
> > -Pradeep
> >
> > On Tue, Oct 26, 2010 at 8:32 AM, Phong Dais <phong.gdais@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I understand that I need to store the fields in order to use
> highlighting
> > > "out of the box".
> > > I'm looking for a way to highlighting using term offsets instead of t=
he
> > > actual text since the text is not stored.  What am asking is is it
> > possible
> > > to modify the response (thru custom implementation) to contain
> > highlighted
> > > offsets instead of the actual matched text.  Should I be writing my o=
wn
> > > DefaultHighlighter?  Or overiding some of its functionality?  Can thi=
s
> be
> > > done this way or am I way off?
> > >
> > > BTW, I'm using solr-1.4.
> > >
> > > Thanks,
> > > P.
> > >
> > > On Tue, Oct 26, 2010 at 9:25 AM, Israel Ekpo <israelekpo@gmail.com>
> > wrote:
> > >
> > > > Check out this link
> > > >
> > > > http://wiki.apache.org/solr/FieldOptionsByUseCase
> > > >
> > > > You need to store the field if you want to use the highlighting
> > feature.
> > > >
> > > > If you need to retrieve and display the highlighted snippets then t=
he
> > > > fields
> > > > definitely needs to be stored.
> > > >
> > > > To use term offsets, it will be a good idea to enable the following
> > > > attributes for that field  termVectors termPositions termOffsets
> > > >
> > > > The only issue here is that your storage costs will increase becaus=
e
> of
> > > > these extra features.
> > > >
> > > > Nevertheless, you definitely need to store the field if you need to
> > > > retrieve
> > > > it for highlighting purposes.
> > > >
> > > > On Tue, Oct 26, 2010 at 6:50 AM, Phong Dais <phong.gdais@gmail.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I've been looking thru the mailing archive for the past week and =
I
> > > > haven't
> > > > > found any useful info regarding this issue.
> > > > >
> > > > > My requirement is to index a few terabytes worth of data to be
> > > searched.
> > > > > Due to the size of the data, I would like to index without storin=
g
> > but
> > > I
> > > > > would like to use the highlighting feature.  Is this even possibl=
e?
> > >  What
> > > > > are my options?
> > > > >
> > > > > I've read about termOffsets, payload that could possibly be used =
to
> > do
> > > > this
> > > > > but I have no idea how this could be done.
> > > > >
> > > > > Any pointers greatly appreciated.  Someone please point me in the
> > right
> > > > > direction.
> > > > >
> > > > >  I don't mind having to write some code or digging thru existing
> code
> > > to
> > > > > accomplish this task.
> > > > >
> > > > > Thanks,
> > > > > P.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > =B0O=B0
> > > > "Good Enough" is not good enough.
> > > > To give anything less than your best is to sacrifice the gift.
> > > > Quality First. Measure Twice. Cut Once.
> > > > http://www.israelekpo.com/
> > > >
> > >
> >
>


[prev in list] [next in list] [prev in thread] [next in thread]