[prev in list] [next in list] [prev in thread] [next in thread]
List: nepomuk
Subject: [Nepomuk] Re: Virtuoso eating up CPUs
From: Will Stephenson <wstephenson () kde ! org>
Date: 2010-12-15 10:44:48
Message-ID: 4D089C20.1020505 () kde ! org
[Download RAW message or body]
On 07/12/10 09:02, Will Stephenson wrote:
> On Sunday 12 September 2010 22:18:27 Vishesh Handa wrote:
>> Virtuoso goes crazy with some queries. :-/ We know about it, and can
>> reproduce it. It's not a 4.5.1 issue, it's just a virtuoso thing.
>>
>> @Sebastian : Weren't your userVisibility patches supposed to fix this?
>
> Is there an upstream bug report for this that I can pursue? I'm still seeing
> this with 4.6 beta2.
>
> What techniques are there to check the state of the server to describe these
> 'virtuoso 100% cpu' cases? I'd like to have a bit more ability to support
> virtuoso than just telling people to restart the server.
So attached are several isql status() calls when virtuoso is going ape.
The active query seems to be the same each time.
Will
["virtuoso_log_runaway.txt" (text/plain)]
hq000006@HQVMT4XX20:~/Downloads> isql -H localhost -S 1112 -U dba -P \
dbaConnected to OpenLink Virtuoso
Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver
OpenLink Interactive SQL (Virtuoso), version 0.9849b.
Type HELP; for help and EXIT; to exit.
SQL> status();
REPORT
VARCHAR
_______________________________________________________________________________
OpenLink Virtuoso Server
Version 06.01.3127-pthreads for Linux as of Dec 10 2010
Started on: 2010/12/14 20:03 GMT+60
Lite Mode
Database Status:
File size 251658240, 30720 pages, 13356 free.
48200 buffers, 10228 used, 0 dirty 0 wired down, repl age 0 0 w. io 4 \
w/crsr. Disk Usage: 10220 reads avg 2 msec, 0% r 0% w last 0 s, 6118 \
writes, 53 read ahead, batch = 157. Autocompact 92 in 70 out, 23% saved.
Gate: 225 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
Log = /home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.trx, \
87 bytes 17350 pages have been changed since last backup (in checkpoint \
state) Current backup timestamp: 0x0000-0x00-0x00
Last backup date: unknown
Clients: 72 connects, max 10 concurrent
RPC: 16553 calls, -63 pending, 1 max until now, 0 queued, 18 burst reads \
(0%), 3 second brk=74076160 Checkpoint Remap 0 pages, 0 mapped back. 396 s \
atomic time. DB master 30720 total 13356 free 0 remap 0 mapped back
temp 256 total 251 free
Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
Currently 2 threads running 0 threads waiting 0 threads in vdb.
Pending:
Client 1112:72: Account: dba, 202 bytes in, 254 bytes out, 1 stmts.
PID: 13482, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-69: Account: dba, 656 bytes in, 331 bytes out, 1 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-71: Account: dba, 2970 bytes in, 8795 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-5: Account: dba, 71335 bytes in, 20798 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-2: Account: dba, 39410 bytes in, 1220768 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-4: Account: dba, 11324 bytes in, 4070 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-8: Account: dba, 2026090 bytes in, 972443 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Running Statements:
Time (msec) Text
601781 sparql select count(distinct ?r) as ?cnt where { { ?r ?v1 ?v2 \
. ?v2 bif:contain 20 status()
Hash indexes
69 Rows. -- 21 msec.
SQL> status();
REPORT
VARCHAR
_______________________________________________________________________________
OpenLink Virtuoso Server
Version 06.01.3127-pthreads for Linux as of Dec 10 2010
Started on: 2010/12/14 20:03 GMT+60
Lite Mode
Database Status:
File size 251658240, 30720 pages, 13356 free.
48200 buffers, 10229 used, 0 dirty 0 wired down, repl age 0 0 w. io 3 \
w/crsr. Disk Usage: 10221 reads avg 2 msec, 0% r 0% w last 216 s, 6118 \
writes, 53 read ahead, batch = 157. Autocompact 92 in 70 out, 23% saved.
Gate: 225 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
Log = /home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.trx, \
87 bytes 17350 pages have been changed since last backup (in checkpoint \
state) Current backup timestamp: 0x0000-0x00-0x00
Last backup date: unknown
Clients: 72 connects, max 10 concurrent
RPC: 16555 calls, -63 pending, 1 max until now, 0 queued, 18 burst reads \
(0%), 3 second brk=74076160 Checkpoint Remap 0 pages, 0 mapped back. 396 s \
atomic time. DB master 30720 total 13356 free 0 remap 0 mapped back
temp 256 total 251 free
Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
Currently 2 threads running 0 threads waiting 0 threads in vdb.
Pending:
Client 1112:72: Account: dba, 314 bytes in, 4260 bytes out, 1 stmts.
PID: 13482, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-69: Account: dba, 656 bytes in, 331 bytes out, 1 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-71: Account: dba, 2970 bytes in, 8795 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-5: Account: dba, 71335 bytes in, 20798 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-2: Account: dba, 39410 bytes in, 1220768 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-4: Account: dba, 11324 bytes in, 4070 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-8: Account: dba, 2026090 bytes in, 972443 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Running Statements:
Time (msec) Text
818773 sparql select count(distinct ?r) as ?cnt where { { ?r ?v1 ?v2 \
. ?v2 bif:contain 2 status()
Hash indexes
69 Rows. -- 3 msec.
SQL> status();
REPORT
VARCHAR
_______________________________________________________________________________
OpenLink Virtuoso Server
Version 06.01.3127-pthreads for Linux as of Dec 10 2010
Started on: 2010/12/14 20:03 GMT+60
Lite Mode
Database Status:
File size 251658240, 30720 pages, 13356 free.
48200 buffers, 10245 used, 0 dirty 0 wired down, repl age 0 0 w. io 5 \
w/crsr. Disk Usage: 10241 reads avg 2 msec, 0% r 0% w last 128 s, 6131 \
writes, 53 read ahead, batch = 157. Autocompact 92 in 70 out, 23% saved.
Gate: 225 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
Log = /home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.trx, \
87 bytes 17350 pages have been changed since last backup (in checkpoint \
state) Current backup timestamp: 0x0000-0x00-0x00
Last backup date: unknown
Clients: 72 connects, max 10 concurrent
RPC: 16557 calls, -63 pending, 1 max until now, 0 queued, 18 burst reads \
(0%), 3 second brk=74076160 Checkpoint Remap 0 pages, 0 mapped back. 396 s \
atomic time. DB master 30720 total 13356 free 0 remap 0 mapped back
temp 256 total 251 free
Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
Currently 2 threads running 0 threads waiting 0 threads in vdb.
Pending:
Client 1112:72: Account: dba, 426 bytes in, 8269 bytes out, 1 stmts.
PID: 13482, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-69: Account: dba, 656 bytes in, 331 bytes out, 1 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-71: Account: dba, 2970 bytes in, 8795 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-5: Account: dba, 71335 bytes in, 20798 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-2: Account: dba, 39410 bytes in, 1220768 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-4: Account: dba, 11324 bytes in, 4070 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-8: Account: dba, 2026090 bytes in, 972443 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Running Statements:
Time (msec) Text
947529 sparql select count(distinct ?r) as ?cnt where { { ?r ?v1 ?v2 \
. ?v2 bif:contain 2 status()
Hash indexes
69 Rows. -- 3 msec.
SQL> status();
REPORT
VARCHAR
_______________________________________________________________________________
OpenLink Virtuoso Server
Version 06.01.3127-pthreads for Linux as of Dec 10 2010
Started on: 2010/12/14 20:03 GMT+60
Lite Mode
Database Status:
File size 251658240, 30720 pages, 13356 free.
48200 buffers, 10245 used, 0 dirty 1 wired down, repl age 0 0 w. io 5 \
w/crsr. Disk Usage: 10241 reads avg 2 msec, 0% r 0% w last 240 s, 6131 \
writes, 53 read ahead, batch = 157. Autocompact 92 in 70 out, 23% saved.
Gate: 225 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
Log = /home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.trx, \
87 bytes 17350 pages have been changed since last backup (in checkpoint \
state) Current backup timestamp: 0x0000-0x00-0x00
Last backup date: unknown
Clients: 72 connects, max 10 concurrent
RPC: 16559 calls, -63 pending, 1 max until now, 0 queued, 18 burst reads \
(0%), 3 second brk=74076160 Checkpoint Remap 0 pages, 0 mapped back. 396 s \
atomic time. DB master 30720 total 13356 free 0 remap 0 mapped back
temp 256 total 251 free
Lock Status: 0 deadlocks of which 0 2r1w, 0 waits,
Currently 2 threads running 0 threads waiting 0 threads in vdb.
Pending:
Client 1112:72: Account: dba, 538 bytes in, 12278 bytes out, 1 stmts.
PID: 13482, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-69: Account: dba, 656 bytes in, 331 bytes out, 1 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 1 threads.
Locks:
Client 1112:1:-71: Account: dba, 2970 bytes in, 8795 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-5: Account: dba, 71335 bytes in, 20798 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-2: Account: dba, 39410 bytes in, 1220768 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-4: Account: dba, 11324 bytes in, 4070 bytes out, 0 stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Client 1112:1:-8: Account: dba, 2026090 bytes in, 972443 bytes out, 0 \
stmts.
PID: 2877, OS: unix, Application: unknown, IP#: 127.0.0.1
Transaction status: PENDING, 0 threads.
Locks:
Running Statements:
Time (msec) Text
1187815 sparql select count(distinct ?r) as ?cnt where { { ?r ?v1 ?v2 \
. ?v2 bif:contain 2 status()
Hash indexes
69 Rows. -- 3 msec.
SQL> quit;
hq000006@HQVMT4XX20:~/Downloads> cat /tmp/virtuoso_Ti2877.ini
[Database]
DatabaseFile=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.db
ErrorLogFile=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.log
TransactionFile=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.trx
xa_persistent_file=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso.pxa
[TempDatabase]
DatabaseFile=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso-temp.db
TransactionFile=/home/hq000006/.kde4/share/apps/nepomuk/repository/main/data/virtuosobackend/soprano-virtuoso-temp.trx
MaxCheckpointRemap=1000
[Parameters]
LiteMode=1
ServerPort=1112
DisableTcpSocket=1
PrefixResultNames=0
ServerThreads=100
NumberOfBuffers=48200
MaxDirtyBuffers=32133
SchedulerInterval=5
FreeTextBatchSize=1000
CheckpointInterval=10
MinAutoCheckpointSize=200000
hq000006@HQVMT4XX20:~/.kde4/share/config> cat nepomukserverrc
[Basic Settings]
Start Nepomuk=true
[Service-nepomukstrigiservice]
autostart=false
[main Settings]
Maximum memory=512
Storage Dir[$e]=$HOME/.kde4/share/apps/nepomuk/repository/main/
Used Soprano Backend=virtuosobackend
_______________________________________________
Nepomuk mailing list
Nepomuk@kde.org
https://mail.kde.org/mailman/listinfo/nepomuk
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic