[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postgresql-general
Subject:    Re: [HACKERS] Improving executor performance
From:       Craig Ringer <craig () 2ndquadrant ! com>
Date:       2016-06-29 22:45:36
Message-ID: CAMsr+YHMOgZMas=Y1H9gKQ90rsLBgjeE8WcFKAHbYm-PtXmw-A () mail ! gmail ! com
[Download RAW message or body]

On 30 June 2016 at 02:32, Andres Freund <andres@anarazel.de> wrote:

> 
> Hi,
> 
> On 2016-06-28 10:01:28 +0000, Rajeev rastogi wrote:
> > > 3) Our 1-by-1 tuple flow in the executor has two major issues:
> > 
> > Agreed, In order to tackle this IMHO, we should
> > 1. Makes the processing data-centric instead of operator centric.
> > 2. Instead of pulling each tuple from immediate operator, operator can
> push the tuple to its parent. It can be allowed to push until it sees any
> operator, which cannot be processed without result from other operator.
> > More details from another thread:
> > 
> https://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB77159A9B904@szxeml521-mbs.china.huawei.com
>  
> I doubt that that's going to be ok in the generic case (memory usage,
> materializing too much, "bushy plans", merge joins)


Yeah. You'd likely start landing up with Haskell-esque predictability of
memory use. Given how limited and flawed work_mem handling etc already is,
that doesn't sound like an appealing direction to go in. Not without a
bunch of infrastructure to manage queue sizes and force work into batches
to limit memory use, anyway.

-- 
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


[Attachment #3 (text/html)]

<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 30 June 2016 at \
02:32, Andres Freund <span dir="ltr">&lt;<a href="mailto:andres@anarazel.de" \
target="_blank">andres@anarazel.de</a>&gt;</span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><br> Hi,<br>
<span class=""><br>
On 2016-06-28 10:01:28 +0000, Rajeev rastogi wrote:<br>
&gt; &gt;3) Our 1-by-1 tuple flow in the executor has two major issues:<br>
&gt;<br>
&gt; Agreed, In order to tackle this IMHO, we should<br>
&gt; 1. Makes the processing data-centric instead of operator centric.<br>
&gt; 2. Instead of pulling each tuple from immediate operator, operator can push the \
tuple to its parent. It can be allowed to push until it sees any operator, which \
cannot be processed without result from other operator.<br> &gt; More details from \
another thread:<br> &gt; <a \
href="https://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB77159A9B904@szxeml521-mbs.china.huawei.com" \
rel="noreferrer" target="_blank">https://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB77159A9B904@szxeml521-mbs.china.huawei.com</a><br>
 <br>
</span>I doubt that that&#39;s going to be ok in the generic case (memory usage,<br>
materializing too much, &quot;bushy plans&quot;, merge \
joins)</blockquote><div><br></div><div>Yeah. You&#39;d likely start landing up with \
Haskell-esque predictability of memory use. Given how limited and flawed work_mem \
handling etc already is, that doesn&#39;t sound like an appealing direction to go in. \
Not without a bunch of infrastructure to manage queue sizes and force work into \
batches to limit memory use, anyway.</div><div><br></div></div>-- <br><div \
class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div \
dir="ltr">  Craig Ringer                                     <a \
href="http://www.2ndQuadrant.com/" \
target="_blank">http://www.2ndQuadrant.com/</a><br>  PostgreSQL Development, 24x7 \
Support, Training &amp; Services<br></div></div></div></div> </div></div>



[prev in list] [next in list] [prev in thread] [next in thread]