[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-pim
Subject:    Re: [Kde-pim] Nepomuk feeder queuing problem
From:       Michael Jansen <info () michael-jansen ! biz>
Date:       2012-01-17 16:37:19
Message-ID: 5167438.bH38HL3LKq () gambit
[Download RAW message or body]

On Monday, January 16, 2012 06:41:33 PM Christian Mollekopf wrote:
> Hey Sebastian,
> =

> I don't see the problem with this as this timer just kickstarts the
> processing and can be called as many times as you want. The ItemQueue
> does the actual feeding of the data. The processing of the ItemQueue is
> triggered by the processItem() function which simply does nothing if
> called repeatedly (mRunningJobs > 0). processItem() is also the function
> which gets called if you call setOnline() or addItem(), but that
> shouldn't hurt as far as I can see.
> =

> So there should be at maximum two nepomuk jobs running at a time (one
> per ItemQueue).
> =

> Are you sure virtuoso is really going crazy? While indexing virtuoso
> takes all the cpu power it can get, but that seems normal to me. Also
> the indexing can take veeery long, so it's quite possible that users
> think virtuoso just went crazy.

No. I am sometimes sitting besides my desktop working on my laptop and enjo=
y =

virtuoso going crazy and stopping by just listening into the fan. So i put =
a =

top up and glimpsed over to see what happens.

It will go to 80-180% cpu for some time, then go back to normal, nearly all=
 of =

the time. It never stops doing that here. Usually i was able to find out  t=
he =

culprit by just looking at top. Another process was usually right behind =

virtuoso in cpu usage.

We fixed most of these problems.

This time it is different. Only virtuoso does it. Some akonadi process seem=
 to =

hover at 3-5% for me but not really sure if there is any relation.

I have no clue if it is related. But on session login i enjoy virtuoso,  =

nepomukstorage, kontact and akonadi_nepomukfeeder go onto a minutes long cp=
u =

and io burning session. I plan to have a look during the next days what is =
the =

reason for that.

Mike
> =

> Note that some merges of indexed items still fail (I believe it has to
> do with the same email address appearing several times in an email),
> maybe this could trigger a problem?
> I could send you a testitem which triggers the problem in case you wish
> (it contains private data and is therefore not suitable to be uploaded
> on a bugtracker).
> =

> Also it seems that some items get regularly reindexed because some
> property of an akonadi-item changed (which always triggers a full
> reindexing). I'm not yet sure why that happens exactly though. This just
> to say that the feeder can produce similar load to the initial indexing
> also after the initial indexing has finished.
> =

> So I think there are lots of improvements to be made regarding
> performance, but I'm not aware of a problem that overloads nepomuk.
> =

> Thanks for looking into this.
> =

> Cheers,
> Christian
> =

> On Mon, Jan 16, 2012, at 06:01 PM, Sebastian Tr=FCg wrote:
> > Hi Christian,
> > =

> > there seems to be a problem with the Nepomuk feeder in KDEpim. The
> > reports about virtuoso going crazy pile up and now I ran into the same
> > problem.
> > Thus, I had a look at the feeder and found a potential problem:
> > =

> > You use a single shot timer to continue the indexing. This timer tells
> > the queue to continue the processing of the items. This is all fine
> > internally. But both setOnline() and addItem() start the timer without
> > checking if any job has already been started. Thus, another job will be
> > started. This can potentially lead to tons of nepomuk jobs running which
> > could in fact make virtuoso go crazy as there is no real protection
> > against such an "attack" in Nepomuk.
> > =

> > Please have a look and tell me if I read the code correctly.
> > =

> > This is something we should sort out before the final tag.
> > =

> > Cheers,
> > Sebastian
> =

> _______________________________________________
> KDE PIM mailing list kde-pim@kde.org
> https://mail.kde.org/mailman/listinfo/kde-pim
> KDE PIM home page at http://pim.kde.org/
-- =

Michael Jansen
http://michael-jansen.biz
_______________________________________________
KDE PIM mailing list kde-pim@kde.org
https://mail.kde.org/mailman/listinfo/kde-pim
KDE PIM home page at http://pim.kde.org/
[prev in list] [next in list] [prev in thread] [next in thread]