[prev in list] [next in list] [prev in thread] [next in thread]
List: qgis-developer
Subject: Re: [Qgis-developer] Performance of QGIS expressions "attribute" and "get_feature"
From: Stefan Ziegler <stefan.ziegler.de () gmail ! com>
Date: 2015-12-31 9:18:50
Message-ID: CAJvo+1_idfjkmf+HehNCGY=_yiL=d1v208GTi+oFd3Zc1=tBoA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
As far as I understand: for every feature of the layer intersecting the map
canvas there is a openCursor and closeCursor command. Without the
expression you only have one openCursor and closeCursor command for the
layer. Don't know if this is necessary but it seems to slow it down.
regards
Stefan
On Thu, Dec 31, 2015 at 9:38 AM, Stefan Ziegler <stefan.ziegler.de@gmail.com
> wrote:
> I looked at the debug output. This time I did not add a virtual field but
> only using the expression in the rule based renderer:
>
> src/core/qgsvectorlayerrenderer.cpp: 94: (QgsVectorLayerRenderer) [3ms]
> rendering v2:
> Rule-based renderer:
> RULE - scale [0,0] - filter - symbol []
> RULE - scale [0,0] - filter
> attribute(get_feature('einzelobjekte_einzelobjekt','t_id',flaechenelement_von),'art')
> = 'unterirdisches_Gebaeude' - symbol FILL SYMBOL (1 layers) color
> 121,174,87,255
>
> src/core/qgsmaprenderercustompainterjob.cpp: 100: (start) [0ms] Rendering
> prepared in (seconds): 0.004
> src/core/qgsmaprenderercustompainterjob.cpp: 232: (doRender) [0ms]
> [thread:0x23feda0] Starting to render layer stack.
> src/core/symbology-ng/qgsrulebasedrendererv2.cpp: 839: (startRender) [0ms]
> [thread:0x23feda0] zLevel 0 -> 0
> src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [2ms]
> [thread:0x23feda0] Starting read-only transaction: 90310
> src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [2ms]
> [thread:0x23feda0] Starting read-only transaction: 90310
> src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [9ms]
> [thread:0x23feda0] Committing read-only transaction
> src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [1ms]
> [thread:0x23feda0] Starting read-only transaction: 90310
> src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [9ms]
> [thread:0x23feda0] Committing read-only transaction
> src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [1ms]
> [thread:0x23feda0] Starting read-only transaction: 90310
> src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [10ms]
> [thread:0x23feda0] Committing read-only transaction
> src/providers/postgres/qgspostgresfeatureiterator.cpp: 242: (fetchFeature)
> [1ms] [thread:0x23feda0] Finished after 2 features
> src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [0ms]
> [thread:0x23feda0] Committing read-only transaction
> src/core/qgsmaprenderercustompainterjob.cpp: 261: (doRender) [3ms]
> [thread:0x23feda0] Done rendering map layers
> src/core/qgsmaprenderercustompainterjob.cpp: 272: (drawLabeling) [0ms]
> [thread:0x23feda0] Draw labeling start
> src/core/qgsvectorlayer.cpp: 320: (drawLabels) [0ms] [thread:0x23feda0]
> Starting draw of labels: einzelobjekte_flaechenelement20151230200140594
> src/core/qgsmaprenderercustompainterjob.cpp: 299: (drawLabeling) [0ms]
> [thread:0x23feda0] Draw labeling took (seconds): 0
> src/core/qgsmaprenderercustompainterjob.cpp: 266: (doRender) [0ms]
> [thread:0x23feda0] Rendering completed in (seconds): 0.038
> src/core/qgsmaprenderercustompainterjob.cpp: 201: (futureFinished) [1ms]
> QPAINTER futureFinished
> src/core/qgsmaprendererjob.cpp: 327: (cleanupJobs) [0ms] caching image for
> einzelobjekte_flaechenelement20151230200140594
> src/core/qgsmaprenderersequentialjob.cpp: 121: (internalFinished) [0ms]
> SEQUENTIAL finished
> src/gui/qgsmapcanvas.cpp: 698: (rendererJobFinished) [0ms] CANVAS finish! 1
> src/core/qgsmaprenderercustompainterjob.cpp: 41:
> (~QgsMapRendererCustomPainterJob) [3ms] QPAINTER destruct
> src/core/qgsmaprenderersequentialjob.cpp: 38:
> (~QgsMapRendererSequentialJob) [4ms] SEQUENTIAL destruct
>
> For few features visible in the mapcanvas it is fast. But really slows
> down when zooming out. I checked the "execute expression on server side
> option". With this enabled I would assume that
> qgspostgresfeatureiterator.cpp would fetch only 1 feature if there is only
> 1 feature visible in the mapcanvas (according with the filter/expression)?
> But it alwas fetches all possible features in the map canvas (two in my
> case).
>
> regards
> Stefan
>
>
> On Wed, Dec 30, 2015 at 9:34 PM, Stefan Ziegler <
> stefan.ziegler.de@gmail.com> wrote:
>
>> Two postgres layers with an index on the fields.
>>
>> Do I see some debug messages (when compiled with DEBUG)?
>>
>> On Wed, Dec 30, 2015 at 9:27 PM, Nyall Dawson <nyall.dawson@gmail.com>
>> wrote:
>>
>>> On 31 December 2015 at 06:18, Stefan Ziegler
>>> <stefan.ziegler.de@gmail.com> wrote:
>>> > Hi Nyall
>>> >
>>> > I compared 2.12 with current master. But I don't see an improvement.
>>> Opening
>>> > the attribute table takes around 10 seconds for layers with approx.
>>> 2000
>>> > features with one virtual field (with the attribute(get_feature())
>>> > expression). Rendering is also really slow when using rule based
>>> renderer on
>>> > the virtual field.
>>>
>>> What's the data source for each layer? If it's in a database (eg
>>> postgres), is there a suitable index on the joined field?
>>>
>>> Nyall
>>>
>>>
>>>
>>> >
>>> > regards
>>> > Stefan
>>> >
>>> > On Wed, Dec 30, 2015 at 7:22 PM, Nyall Dawson <nyall.dawson@gmail.com>
>>> > wrote:
>>> >>
>>> >>
>>> >> On 31 Dec 2015 2:01 AM, "Stefan Ziegler" <stefan.ziegler.de@gmail.com
>>> >
>>> >> wrote:
>>> >> >
>>> >> > Hi
>>> >> >
>>> >> > I add a virtual field to a layer with e.g. following expression
>>> >> >
>>> >> > attribute(get_feature('pnf_pnf','t_id',afrom),'year')
>>> >> >
>>> >> > This works really great. Then I'm using this new virtual field for
>>> >> > filtering the features in the rule based renderer. How does affect
>>> this the
>>> >> > rendering performance? It seems that it really slows down rendering
>>> even
>>> >> > with layers with only approx 600 features each.
>>> >> >
>>> >> > If so, can the speed improved with e.g. some to develop caching
>>> >> > mechanism?
>>> >>
>>> >> Try with the current master version - this should be MUCH faster now.
>>> I'd
>>> >> love to hear any feedback of how you find the changes improve (or
>>> don't
>>> >> improve!) your situation.
>>> >>
>>> >> Nyall
>>> >>
>>> >> >
>>> >> > regards
>>> >> > Stefan
>>> >> >
>>> >> > _______________________________________________
>>> >> > Qgis-developer mailing list
>>> >> > Qgis-developer@lists.osgeo.org
>>> >> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> >> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>> >
>>> >
>>>
>>
>>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div><div>As far as I understand: for every feature of the layer \
intersecting the map canvas there is a openCursor and closeCursor command. Without \
the expression you only have one openCursor and closeCursor command for the layer. \
Don't know if this is necessary but it seems to slow it \
down.<br><br></div>regards<br></div>Stefan <br></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 31, 2015 at 9:38 AM, \
Stefan Ziegler <span dir="ltr"><<a href="mailto:stefan.ziegler.de@gmail.com" \
target="_blank">stefan.ziegler.de@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div><div>I looked at the debug output. \
This time I did not add a virtual field but only using the expression in the rule \
based renderer:<br><br>src/core/qgsvectorlayerrenderer.cpp: 94: \
(QgsVectorLayerRenderer) [3ms] rendering v2:<br> Rule-based renderer:<br>RULE - \
scale [0,0] - filter - symbol []<br> RULE - scale [0,0] - filter \
attribute(get_feature('einzelobjekte_einzelobjekt','t_id',flaechenelement_von),'art') \
= 'unterirdisches_Gebaeude' - symbol FILL SYMBOL (1 layers) color \
121,174,87,255<br><br>src/core/qgsmaprenderercustompainterjob.cpp: 100: (start) [0ms] \
Rendering prepared in (seconds): \
0.004<br>src/core/qgsmaprenderercustompainterjob.cpp: 232: (doRender) [0ms] \
[thread:0x23feda0] Starting to render layer \
stack.<br>src/core/symbology-ng/qgsrulebasedrendererv2.cpp: 839: (startRender) [0ms] \
[thread:0x23feda0] zLevel 0 -> 0<br>src/providers/postgres/qgspostgresconn.cpp: \
992: (openCursor) [2ms] [thread:0x23feda0] Starting read-only transaction: \
90310<br>src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [2ms] \
[thread:0x23feda0] Starting read-only transaction: \
90310<br>src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [9ms] \
[thread:0x23feda0] Committing read-only \
transaction<br>src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [1ms] \
[thread:0x23feda0] Starting read-only transaction: \
90310<br>src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [9ms] \
[thread:0x23feda0] Committing read-only \
transaction<br>src/providers/postgres/qgspostgresconn.cpp: 992: (openCursor) [1ms] \
[thread:0x23feda0] Starting read-only transaction: \
90310<br>src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [10ms] \
[thread:0x23feda0] Committing read-only \
transaction<br>src/providers/postgres/qgspostgresfeatureiterator.cpp: 242: \
(fetchFeature) [1ms] [thread:0x23feda0] Finished after 2 \
features<br>src/providers/postgres/qgspostgresconn.cpp: 1010: (closeCursor) [0ms] \
[thread:0x23feda0] Committing read-only \
transaction<br>src/core/qgsmaprenderercustompainterjob.cpp: 261: (doRender) [3ms] \
[thread:0x23feda0] Done rendering map \
layers<br>src/core/qgsmaprenderercustompainterjob.cpp: 272: (drawLabeling) [0ms] \
[thread:0x23feda0] Draw labeling start<br>src/core/qgsvectorlayer.cpp: 320: \
(drawLabels) [0ms] [thread:0x23feda0] Starting draw of labels: \
einzelobjekte_flaechenelement20151230200140594<br>src/core/qgsmaprenderercustompainterjob.cpp: \
299: (drawLabeling) [0ms] [thread:0x23feda0] Draw labeling took (seconds): \
0<br>src/core/qgsmaprenderercustompainterjob.cpp: 266: (doRender) [0ms] \
[thread:0x23feda0] Rendering completed in (seconds): \
0.038<br>src/core/qgsmaprenderercustompainterjob.cpp: 201: (futureFinished) [1ms] \
QPAINTER futureFinished<br>src/core/qgsmaprendererjob.cpp: 327: (cleanupJobs) [0ms] \
caching image for einzelobjekte_flaechenelement20151230200140594<br>src/core/qgsmaprenderersequentialjob.cpp: \
121: (internalFinished) [0ms] SEQUENTIAL finished<br>src/gui/qgsmapcanvas.cpp: 698: \
(rendererJobFinished) [0ms] CANVAS finish! \
1<br>src/core/qgsmaprenderercustompainterjob.cpp: 41: \
(~QgsMapRendererCustomPainterJob) [3ms] QPAINTER \
destruct<br>src/core/qgsmaprenderersequentialjob.cpp: 38: \
(~QgsMapRendererSequentialJob) [4ms] SEQUENTIAL destruct<br><br></div>For few \
features visible in the mapcanvas it is fast. But really slows down when zooming out. \
I checked the "execute expression on server side option". With this enabled \
I would assume that qgspostgresfeatureiterator.cpp would fetch only 1 feature if \
there is only 1 feature visible in the mapcanvas (according with the \
filter/expression)? But it alwas fetches all possible features in the map canvas (two \
in my case).<br><br></div>regards<span class="HOEnZb"><font \
color="#888888"><br></font></span></div><span class="HOEnZb"><font \
color="#888888">Stefan \
<br><div><div><div><div><br></div></div></div></div></font></span></div><div \
class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div \
class="gmail_quote">On Wed, Dec 30, 2015 at 9:34 PM, Stefan Ziegler <span \
dir="ltr"><<a href="mailto:stefan.ziegler.de@gmail.com" \
target="_blank">stefan.ziegler.de@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Two postgres layers with an index on the \
fields.<br><br></div>Do I see some debug messages (when compiled with DEBUG)? \
<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec \
30, 2015 at 9:27 PM, Nyall Dawson <span dir="ltr"><<a \
href="mailto:nyall.dawson@gmail.com" \
target="_blank">nyall.dawson@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">On 31 December 2015 at 06:18, Stefan Ziegler<br> <span><<a \
href="mailto:stefan.ziegler.de@gmail.com" \
target="_blank">stefan.ziegler.de@gmail.com</a>> wrote:<br> > Hi Nyall<br>
><br>
> I compared 2.12 with current master. But I don't see an improvement. \
Opening<br> > the attribute table takes around 10 seconds for layers with approx. \
2000<br> > features with one virtual field (with the attribute(get_feature())<br>
> expression). Rendering is also really slow when using rule based renderer on<br>
> the virtual field.<br>
<br>
</span>What's the data source for each layer? If it's in a database (eg<br>
postgres), is there a suitable index on the joined field?<br>
<span><font color="#888888"><br>
Nyall<br>
</font></span><div><div><br>
<br>
<br>
><br>
> regards<br>
> Stefan<br>
><br>
> On Wed, Dec 30, 2015 at 7:22 PM, Nyall Dawson <<a \
href="mailto:nyall.dawson@gmail.com" \
target="_blank">nyall.dawson@gmail.com</a>><br> > wrote:<br>
>><br>
>><br>
>> On 31 Dec 2015 2:01 AM, "Stefan Ziegler" <<a \
href="mailto:stefan.ziegler.de@gmail.com" \
target="_blank">stefan.ziegler.de@gmail.com</a>><br> >> wrote:<br>
>> ><br>
>> > Hi<br>
>> ><br>
>> > I add a virtual field to a layer with e.g. following expression<br>
>> ><br>
>> > attribute(get_feature('pnf_pnf','t_id',afrom),'year')<br>
>> ><br>
>> > This works really great. Then I'm using this new virtual field \
for<br> >> > filtering the features in the rule based renderer. How does \
affect this the<br> >> > rendering performance? It seems that it really \
slows down rendering even<br> >> > with layers with only approx 600 features \
each.<br> >> ><br>
>> > If so, can the speed improved with e.g. some to develop caching<br>
>> > mechanism?<br>
>><br>
>> Try with the current master version - this should be MUCH faster now. \
I'd<br> >> love to hear any feedback of how you find the changes improve \
(or don't<br> >> improve!) your situation.<br>
>><br>
>> Nyall<br>
>><br>
>> ><br>
>> > regards<br>
>> > Stefan<br>
>> ><br>
>> > _______________________________________________<br>
>> > Qgis-developer mailing list<br>
>> > <a href="mailto:Qgis-developer@lists.osgeo.org" \
target="_blank">Qgis-developer@lists.osgeo.org</a><br> >> > List info: <a \
href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" rel="noreferrer" \
target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br> \
>> > Unsubscribe: <a \
href="http://lists.osgeo.org/mailman/listinfo/qgis-developer" rel="noreferrer" \
target="_blank">http://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br> \
><br> ><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
[Attachment #6 (text/plain)]
_______________________________________________
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic