[prev in list] [next in list] [prev in thread] [next in thread]
List: nepomuk
Subject: Re: [Nepomuk] KMail completion spams nepomuk, which isn't parallelized
From: Ignacio Serantes <kde () aynoa ! net>
Date: 2012-07-09 22:17:43
Message-ID: CAKbQbAprw8JQ_UthpTtH1rs=BTbrAG==e7zFUy43+C9Hw8J_kQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi,
You don't see my last query I put in IRC before I leave because you are not
connected. Please try this one:.
SELECT DISTINCT ?email ?fullname
WHERE {
?r a nco:Contact .
{
?r nco:hasEmailAddress ?v0 . ?v0 nco:emailAddress ?v1 .
FILTER(REGEX(STR(?v1), "^vau", "i")) .
} union {
?r nco:fullname ?v . FILTER(REGEX(STR(?v), "^fau", "i")) .
} union {
?r nco:nameFamily ?v . FILTER(REGEX(STR(?v), "^fau", "i")) .
} union {
?r nco:nameGiven ?v . FILTER(REGEX(STR(?v), "^fau", "i")) .
}
OPTIONAL { ?r nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . } .
OPTIONAL { ?r nco:fullname ?fullname . } .
}
I'm not sure about the nco:hasEmailAddress relation because I'm not using
akonady but based in the original query and in our previous test this one
must work.
We could optimize the query using bif:contains() instead REGEX,
it's theoretically fast because uses indexes, but we lost the possibility
to query using only one character, three are then the minimum, and for an
incremental search in a contact list with only 500?, 1000? records its a
big loss.
On Fri, Jul 6, 2012 at 5:46 PM, David Faure <faure@kde.org> wrote:
> Typing 10 letters in the kmail composer "To" field, leads to 10
> autocompletion nepomuk queries,
> all of which create a different thread in the nepomukqueryservice, and
> they pile up, each waiting
> for the previous one to finish, because they use the same mutex-protected
> socket for the
> connection to virtuoso...
>
> All the threads in `nepomukservicestub nepomukqueryservice` have this bt:
>
> Thread 5 (Thread 0x7fffea00e700 (LWP 23920)):
> #0 0x00007ffff4361cd9 in syscall () from /lib64/libc.so.6
> #1 0x00007ffff78f1652 in _q_futex (addr=0x7fffe4003220, op=0, val=2,
> timeout=0x0, addr2=0x0, val2=0) at thread/qmutex_unix.cpp:99
> #2 0x00007ffff78f170a in QMutexPrivate::wait (this=0x7fffe4003220,
> timeout=-1) at thread/qmutex_unix.cpp:113
> #3 0x00007ffff78ebe2a in QMutex::lock (this=0x7fffe4003038) at
> thread/qmutex.cpp:164
> #4 0x00007ffff0440b56 in Soprano::Socket::lock (this=0x7fffe4003020) at
> /d/kde/src/4/soprano/client/socket.cpp:117
> #5 0x00007ffff0441217 in Soprano::SocketStream::SocketStream
> (this=0x7fffea00d8a0, dev=0x7fffe4003020) at
> /d/kde/src/4/soprano/client/socketstream.cpp:36
> #6 0x00007ffff0444f13 in Soprano::Client::ClientConnection::executeQuery
> (this=0x7fffe40035c0, modelId=343926100, query="select distinct ?email
> ?fullname where { ?r a nco:Contact . ?r nco:hasEmailAddress ?v . ?v
> nco:emailAddress ?email . { FILTER regex( str(?email), \"\\\\bfaure\",
> \"i\") . ?r nco:fullname ?fullname } union "...,
> type=Soprano::Query::QueryLanguageSparql, userQueryLanguage="") at
> /d/kde/src/4/soprano/client/clientconnection.cpp:227
> #7 0x00007ffff04476b7 in Soprano::Client::ClientModel::executeQuery
> (this=0x7fffe4003110, query="select distinct ?email ?fullname where { ?r a
> nco:Contact . ?r nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . {
> FILTER regex( str(?email), \"\\\\bfaure\", \"i\") . ?r nco:fullname
> ?fullname } union "..., language=Soprano::Query::QueryLanguageSparql,
> userQueryLanguage="") at /d/kde/src/4/soprano/client/clientmodel.cpp:101
> #8 0x00007ffff60b631a in Nepomuk2::MainModel::executeQuery
> (this=0x7fffe4002d60, query="select distinct ?email ?fullname where { ?r a
> nco:Contact . ?r nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . {
> FILTER regex( str(?email), \"\\\\bfaure\", \"i\") . ?r nco:fullname
> ?fullname } union "..., language=Soprano::Query::QueryLanguageSparql,
> userQueryLanguage="") at
> /d/kde/src/4/nepomuk-core/libnepomukcore/resource/nepomukmainmodel.cpp:182
> #9 0x00007fffeba31eee in Nepomuk2::Query::SearchRunnable::run
> (this=0x7a4620) at
> /d/kde/src/4/nepomuk-core/services/queryservice/searchrunnable.cpp:89
> #10 0x00007ffff78e12fd in QThreadPoolThread::run (this=0x7a4960) at
> concurrent/qthreadpool.cpp:107
> #11 0x00007ffff78f1f28 in QThreadPrivate::start (arg=0x7a4960) at
> thread/qthread_unix.cpp:307
> #12 0x00007ffff7631f05 in start_thread () from /lib64/libpthread.so.0
> #13 0x00007ffff436510d in clone () from /lib64/libc.so.6
>
> Thread 5 is looking for "faure"
> Thread 4 is looking for "faur"
> Thread 3 is looking for "fau"
> Guess what I was typing ;)
> Thread 2 is doing another query, but also waiting on the socket.
> Thread 1 is in the event loop.
>
> Is there a way to cancel such queries? Surely kmail should abort the
> running query when making another one,
> it makes no sense to keep looking for "foo" when the user has replaced it
> with "bar".
>
> The other conclusion from this is... what's the point in having threads in
> the query service
> if they all wait on each other anyway due to the shared socket?
> This can lead to a very long delay, e.g. when sending an email (again),
> because it has
> to wait for all the previous queries to finish first.
>
> --
> David Faure, faure@kde.org, http://www.davidfaure.fr
> Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5
> _______________________________________________
> Nepomuk mailing list
> Nepomuk@kde.org
> https://mail.kde.org/mailman/listinfo/nepomuk
>
--
Best wishes,
Ignacio
[Attachment #5 (text/html)]
Hi,<div><br></div><div>You don't see my last query I put in IRC before I leave \
because you are not connected. Please try this one:.</div><div><br></div><div> <p \
style="margin:12px 0px 0px"><font face="courier new, monospace">SELECT DISTINCT \
?email ?fullname</font></p> <p style="margin:0px"><font face="courier new, \
monospace">WHERE {</font></p> <p style="margin:0px"><font face="courier new, \
monospace"> ?r a nco:Contact .</font></p> <p style="margin:0px"><font face="courier \
new, monospace"> {</font></p> <p style="margin:0px"><font face="courier new, \
monospace"> ?r nco:hasEmailAddress ?v0 . ?v0 nco:emailAddress ?v1 . \
FILTER(REGEX(STR(?v1), "^vau", "i")) . </font></p> <p \
style="margin:0px"><font face="courier new, monospace"> } union {</font></p> <p \
style="margin:0px"><font face="courier new, monospace"> ?r nco:fullname ?v . \
FILTER(REGEX(STR(?v), "^fau", "i")) .</font></p> <p \
style="margin:0px"><font face="courier new, monospace"> } union { </font></p> <p \
style="margin:0px"><font face="courier new, monospace"> ?r nco:nameFamily ?v . \
FILTER(REGEX(STR(?v), "^fau", "i")) .</font></p> <p \
style="margin:0px"><font face="courier new, monospace"> } union {</font></p> <p \
style="margin:0px"><font face="courier new, monospace"> ?r nco:nameGiven ?v . \
FILTER(REGEX(STR(?v), "^fau", "i")) . </font></p> <p \
style="margin:0px"><font face="courier new, monospace"> } </font></p> <p \
style="margin:0px"><font face="courier new, monospace"><br></font></p> <p \
style="margin:0px"><font face="courier new, monospace"> OPTIONAL { ?r \
nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . } .</font></p> <p \
style="margin:0px"><font face="courier new, monospace"> OPTIONAL { ?r nco:fullname \
?fullname . } .</font></p> <p style="margin:0px 0px 12px"><font face="courier new, \
monospace">}</font></p></div><div><div>I'm not sure about the nco:hasEmailAddress \
relation because I'm not using akonady but based in the original query and in our \
previous test this one must work.</div>
<div><br></div><div>We could optimize the query using bif:contains() instead REGEX, \
it's theoretically fast because uses indexes, but we lost the possibility to \
query using only one character, three are then the minimum, and for an incremental \
search in a contact list with only 500?, 1000? records its a big loss.</div>
<div><br><div class="gmail_quote">On Fri, Jul 6, 2012 at 5:46 PM, David Faure <span \
dir="ltr"><<a href="mailto:faure@kde.org" \
target="_blank">faure@kde.org</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
Typing 10 letters in the kmail composer "To" field, leads to 10 \
autocompletion nepomuk queries,<br> all of which create a different thread in the \
nepomukqueryservice, and they pile up, each waiting<br> for the previous one to \
finish, because they use the same mutex-protected socket for the<br> connection to \
virtuoso...<br> <br>
All the threads in `nepomukservicestub nepomukqueryservice` have this bt:<br>
<br>
Thread 5 (Thread 0x7fffea00e700 (LWP 23920)):<br>
#0 0x00007ffff4361cd9 in syscall () from /lib64/libc.so.6<br>
#1 0x00007ffff78f1652 in _q_futex (addr=0x7fffe4003220, op=0, val=2, timeout=0x0, \
addr2=0x0, val2=0) at thread/qmutex_unix.cpp:99<br> #2 0x00007ffff78f170a in \
QMutexPrivate::wait (this=0x7fffe4003220, timeout=-1) at \
thread/qmutex_unix.cpp:113<br> #3 0x00007ffff78ebe2a in QMutex::lock \
(this=0x7fffe4003038) at thread/qmutex.cpp:164<br> #4 0x00007ffff0440b56 in \
Soprano::Socket::lock (this=0x7fffe4003020) at \
/d/kde/src/4/soprano/client/socket.cpp:117<br> #5 0x00007ffff0441217 in \
Soprano::SocketStream::SocketStream (this=0x7fffea00d8a0, dev=0x7fffe4003020) at \
/d/kde/src/4/soprano/client/socketstream.cpp:36<br> #6 0x00007ffff0444f13 in \
Soprano::Client::ClientConnection::executeQuery (this=0x7fffe40035c0, \
modelId=343926100, query="select distinct ?email ?fullname where { ?r a \
nco:Contact . ?r nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . { FILTER \
regex( str(?email), \"\\\\bfaure\", \"i\") . ?r nco:fullname \
?fullname } union "..., type=Soprano::Query::QueryLanguageSparql, \
userQueryLanguage="") at \
/d/kde/src/4/soprano/client/clientconnection.cpp:227<br>
#7 0x00007ffff04476b7 in Soprano::Client::ClientModel::executeQuery \
(this=0x7fffe4003110, query="select distinct ?email ?fullname where { ?r a \
nco:Contact . ?r nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . { FILTER \
regex( str(?email), \"\\\\bfaure\", \"i\") . ?r nco:fullname \
?fullname } union "..., language=Soprano::Query::QueryLanguageSparql, \
userQueryLanguage="") at \
/d/kde/src/4/soprano/client/clientmodel.cpp:101<br>
#8 0x00007ffff60b631a in Nepomuk2::MainModel::executeQuery (this=0x7fffe4002d60, \
query="select distinct ?email ?fullname where { ?r a nco:Contact . ?r \
nco:hasEmailAddress ?v . ?v nco:emailAddress ?email . { FILTER regex( str(?email), \
\"\\\\bfaure\", \"i\") . ?r nco:fullname ?fullname } union \
"..., language=Soprano::Query::QueryLanguageSparql, \
userQueryLanguage="") at \
/d/kde/src/4/nepomuk-core/libnepomukcore/resource/nepomukmainmodel.cpp:182<br>
#9 0x00007fffeba31eee in Nepomuk2::Query::SearchRunnable::run (this=0x7a4620) at \
/d/kde/src/4/nepomuk-core/services/queryservice/searchrunnable.cpp:89<br> #10 \
0x00007ffff78e12fd in QThreadPoolThread::run (this=0x7a4960) at \
concurrent/qthreadpool.cpp:107<br> #11 0x00007ffff78f1f28 in QThreadPrivate::start \
(arg=0x7a4960) at thread/qthread_unix.cpp:307<br> #12 0x00007ffff7631f05 in \
start_thread () from /lib64/libpthread.so.0<br> #13 0x00007ffff436510d in clone () \
from /lib64/libc.so.6<br> <br>
Thread 5 is looking for "faure"<br>
Thread 4 is looking for "faur"<br>
Thread 3 is looking for "fau"<br>
Guess what I was typing ;)<br>
Thread 2 is doing another query, but also waiting on the socket.<br>
Thread 1 is in the event loop.<br>
<br>
Is there a way to cancel such queries? Surely kmail should abort the running query \
when making another one,<br> it makes no sense to keep looking for "foo" \
when the user has replaced it with "bar".<br> <br>
The other conclusion from this is... what's the point in having threads in the \
query service<br> if they all wait on each other anyway due to the shared socket?<br>
This can lead to a very long delay, e.g. when sending an email (again), because it \
has<br> to wait for all the previous queries to finish first.<br>
<span><font color="#888888"><br>
--<br>
David Faure, <a href="mailto:faure@kde.org" target="_blank">faure@kde.org</a>, <a \
href="http://www.davidfaure.fr" target="_blank">http://www.davidfaure.fr</a><br> \
Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5<br> \
_______________________________________________<br> Nepomuk mailing list<br>
<a href="mailto:Nepomuk@kde.org" target="_blank">Nepomuk@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/nepomuk" \
target="_blank">https://mail.kde.org/mailman/listinfo/nepomuk</a><br> \
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Best \
wishes,<div>Ignacio</div><div><br></div><br> </div></div>
_______________________________________________
Nepomuk mailing list
Nepomuk@kde.org
https://mail.kde.org/mailman/listinfo/nepomuk
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic