[prev in list] [next in list] [prev in thread] [next in thread]
List: lucene-user
Subject: Re: SearcherManager not seeing changes in IndexWriteral and
From: Michael McCandless <lucene () mikemccandless ! com>
Date: 2018-11-12 13:49:28
Message-ID: CAL8PwkbxR6=x1irV2aVLLwMZaVWZfYYKvtg-f3j4ziJQ9=Qe+A () mail ! gmail ! com
[Download RAW message or body]
Thanks for bringing closure, Boris.
Mike McCandless
http://blog.mikemccandless.com
On Mon, Nov 12, 2018 at 7:13 AM Boris Petrov <boris_petrov@live.com> wrote:
> Hello,
>
> OK, so actually this appears to be a bug in our code - Lucene is searching
> correctly, we were doing something wrong with the result after that. Sorry
> all for the confusion! Thanks for the support!
> ________________________________
> From: Erick Erickson <erickerickson@gmail.com>
> Sent: Friday, November 9, 2018 6:16 PM
> To: java-user
> Subject: Re: SearcherManager not seeing changes in IndexWriteral and
>
> You might be able to do this with a couple of threads in a single
> program, but certainly up to you.
>
> Best,
> Erick
> On Fri, Nov 9, 2018 at 7:47 AM Boris Petrov <boris_petrov@live.com> wrote:
> >
> > Well, while debugging I put a bunch of println's which showed the
> > expected order. And, besides, I've written the code, I know that writing
> > to the index happens way before searching in it - the test makes sure of
> > that.
> >
> > If you think there indeed might be some problem, I'll try to reproduce
> > it in a small example. But I will try only the obvious (which perhaps
> > you've tried a million times and have tests for) - I'll have one thread
> > write to the index, another (which starts after the first) search in it
> > and I'll create a bash script that runs the program until it fails (what
> > I currently do with our test). I'll do this beginning of next week.
> >
> > Thank you for the support!
> >
> > On 11/9/18 5:37 PM, Erick Erickson wrote:
> > > If it's hard to do in a single thread, how about timestamping the
> > > events to insure that they happen in the expected order?
> > >
> > > That would verify the sequencing is happening as you expect and
> > > _still_ not see the expected docs...
> > >
> > > Assuming it does fail, do you think you could reduce it to a
> > > multi-threaded test case?
> > >
> > > Best,
> > > Erick
> > > On Fri, Nov 9, 2018 at 7:03 AM Boris Petrov <boris_petrov@live.com>
> wrote:
> > >> Well, it's a bit involved to try it in a single thread as I've
> > >> oversimplified the example. But as far as I understand this should
> work,
> > >> right? So something else is wrong? Committing the writer and then
> > >> "maybeRefreshBlocking" should be enough to have the changes visible,
> yes?
> > >>
> > >> On 11/9/18 4:45 PM, Michael Sokolov wrote:
> > >>> That should work, I think, but if you are serializing these threads
> so
> > >>> that they cannot run concurrently, maybe try running both operations
> > >>> in a single thread, at least as a test.
> > >>> On Fri, Nov 9, 2018 at 9:16 AM Boris Petrov <boris_petrov@live.com>
> wrote:
> > >>>> If you mean the synchronization of the threads, it is not in the
> > >>>> example, but Thread 2 is *started* after Thread 1 finished
> executing the
> > >>>> code that I gave as an example. So there is happens-before between
> them.
> > >>>> If you mean synchronization on the Lucene level - isn't that what
> > >>>> "maybeRefreshBlocking" should do?
> > >>>>
> > >>>> On 11/9/18 3:29 PM, Michael Sokolov wrote:
> > >>>>> I'm not seeing anything there that would synchronize, or
> serialize, the
> > >>>>> read after the write and commit. Did you expect that for some
> reason?
> > >>>>>
> > >>>>> On Fri, Nov 9, 2018, 6:00 AM Boris Petrov <boris_petrov@live.com
> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>
> > >>>>>> I'm using Lucene version 7.5.0. We have a test that does
> something like:
> > >>>>>>
> > >>>>>> Thread 1:
> > >>>>>>
> > >>>>>> Field idStringField = new StringField("id", id,
> > >>>>>> Field.Store.YES);
> > >>>>>> Field contentsField = new TextField("contents",
> reader);
> > >>>>>> Document document = new Document();
> > >>>>>> document.add(idStringField);
> > >>>>>> document.add(contentsField);
> > >>>>>>
> > >>>>>> writer.updateDocument(new Term(ID_FIELD, id),
> document);
> > >>>>>> writer.flush(); // not sure this flush is needed?
> > >>>>>> writer.commit();
> > >>>>>>
> > >>>>>> Thread 2:
> > >>>>>>
> > >>>>>> searchManager.maybeRefreshBlocking();
> > >>>>>> IndexSearcher searcher = searchManager.acquire();
> > >>>>>> try {
> > >>>>>> QueryParser parser = new QueryParser("contents",
> analyzer);
> > >>>>>> Query luceneQuery = parser.parse(queryText);
> > >>>>>> ScoreDoc[] hits = searcher.search(luceneQuery,
> > >>>>>> 50).scoreDocs;
> > >>>>>> } finally {
> > >>>>>> searchManager.release(searcher);
> > >>>>>> }
> > >>>>>>
> > >>>>>> Thread 1 happens before Thread 2.
> > >>>>>>
> > >>>>>> Sometimes, only sometimes, the commit from thread 1 is not
> *immediately*
> > >>>>>> visible in Thread 2. If I put a "Thread.sleep(1000)" it always
> works.
> > >>>>>> Without it, sometimes the search is empty. I'm not sure if I'm
> doing
> > >>>>>> something wrong or this is a bug?
> > >>>>>>
> > >>>>>> Thanks!
> > >>>>>>
> > >>>>>>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > >> For additional commands, e-mail: java-user-help@lucene.apache.org
> > >>
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic