[prev in list] [next in list] [prev in thread] [next in thread]
List: pgsql-hackers
Subject: Re: [HACKERS] Parallel Seq Scan
From: Amit Kapila <amit.kapila16 () gmail ! com>
Date: 2015-03-31 12:54:59
Message-ID: CAA4eK1+FMoXOhc7WER2kO1JOCZMugWyuZJO=RFzBMO7Vj_M0Xg () mail ! gmail ! com
[Download RAW message or body]
On Mon, Mar 30, 2015 at 8:11 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Mar 27, 2015 at 2:34 AM, Amit Kapila <amit.kapila16@gmail.com>
wrote:
> > The reason of this problem is that above tab-completion is executing
> > query [1] which contains subplan for the funnel node and currently
> > we don't have capability (enough infrastructure) to support execution
> > of subplans by parallel workers. Here one might wonder why we
> > have choosen Parallel Plan (Funnel node) for such a case and the
> > reason for same is that subplans are attached after Plan generation
> > (SS_finalize_plan()) and if want to discard such a plan, it will be
> > much more costly, tedious and not worth the effort as we have to
> > eventually make such a plan work.
> >
> > Here we have two choices to proceed, first one is to support execution
> > of subplans by parallel workers and second is execute/scan locally for
> > Funnel node having subplan (don't launch workers).
>
> It looks to me like the is an InitPlan, not a subplan. There
> shouldn't be any problem with a Funnel node having an InitPlan; it
> looks to me like all of the InitPlan stuff is handled by common code
> within the executor (grep for initPlan), so it ought to work here the
> same as it does for anything else. What I suspect is failing
> (although you aren't being very clear about it here) is the passing
> down of the parameters set by the InitPlan to the workers.
It is failing because we are not passing InitPlan itself (InitPlan is
nothing but a list of SubPlan) and I tried tried to describe in previous
mail [1] what we need to do to achieve the same, but in short, it is not
difficult to pass down the required parameters (like plan->InitPlan or
plannedstmt->subplans), rather the main missing part is the handling
of such parameters in worker side (mainly we need to provide support
for all plan nodes which can be passed as part of InitPlan in readfuncs.c).
I am not against supporting InitPlan's on worker side, but just wanted to
say that if possible why not leave that for first version.
[1]
I have tried to evaluate what it would take us to support execution
of subplans by parallel workers. We need to pass the sub plans
stored in Funnel Node (initPlan) and corresponding subplans stored
in planned statement (subplans) as subplan's stored in Funnel node
has reference to subplans in planned statement. Next currently
readfuncs.c (functions to read different type of nodes) doesn't support
reading any type of plan node, so we need to add support for reading all
kind
of plan nodes (as subplan can have any type of plan node) and similarly
to execute any type of Plan node, we might need more work (infrastructure).
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
[Attachment #3 (text/html)]
<div dir="ltr">On Mon, Mar 30, 2015 at 8:11 PM, Robert Haas <<a \
href="mailto:robertmhaas@gmail.com">robertmhaas@gmail.com</a>> \
wrote:<br>><br>> On Fri, Mar 27, 2015 at 2:34 AM, Amit Kapila <<a \
href="mailto:amit.kapila16@gmail.com">amit.kapila16@gmail.com</a>> wrote:<br>> \
> The reason of this problem is that above tab-completion is executing<br>> \
> query [1] which contains subplan for the funnel node and currently<br>> > \
we don't have capability (enough infrastructure) to support execution<br>> \
> of subplans by parallel workers. Here one might wonder why we<br>> > \
have choosen Parallel Plan (Funnel node) for such a case and the<br>> > reason \
for same is that subplans are attached after Plan generation<br>> > \
(SS_finalize_plan()) and if want to discard such a plan, it will be<br>> > much \
more costly, tedious and not worth the effort as we have to<br>> > eventually \
make such a plan work.<br>> ><br>> > Here we have two choices to proceed, \
first one is to support execution<br>> > of subplans by parallel workers and \
second is execute/scan locally for<br>> > Funnel node having subplan (don't \
launch workers).<br>><br>> It looks to me like the is an InitPlan, not a \
subplan. There<br>> shouldn't be any problem with a Funnel node having an \
InitPlan; it<br>> looks to me like all of the InitPlan stuff is handled by common \
code<br>> within the executor (grep for initPlan), so it ought to work here \
the<br>> same as it does for anything else. What I suspect is failing<br>> \
(although you aren't being very clear about it here) is the passing<br>> down \
of the parameters set by the InitPlan to the workers.<div><br></div><div style>It is \
failing because we are not passing InitPlan itself (InitPlan is</div><div \
style>nothing but a list of SubPlan) and I tried tried to describe in \
previous</div><div style>mail [1] what we need to do to achieve the same, but in \
short, it is not</div><div style>difficult to pass down the required parameters \
(like plan->InitPlan or</div><div style>plannedstmt->subplans), rather the \
main missing part is the handling</div><div style>of such parameters in worker side \
(mainly we need to provide support</div><div style>for all plan nodes which can be \
passed as part of InitPlan in readfuncs.c).</div><div style>I am not against \
supporting InitPlan's on worker side, but just wanted to</div><div style>say \
that if possible why not leave that for first \
version.</div><div><div><br></div><div><br></div><div>[1]</div><div><div \
style="font-size:13px">I have tried to evaluate what it would take us to support \
execution</div><div style="font-size:13px">of subplans by parallel workers. We need \
to pass the sub plans</div><div style="font-size:13px">stored in Funnel Node \
(initPlan) and corresponding subplans stored</div><div style="font-size:13px">in \
planned statement (subplans) as subplan's stored in Funnel node</div><div \
style="font-size:13px">has reference to subplans in planned statement. Next \
currently</div><div style="font-size:13px">readfuncs.c (functions to read different \
type of nodes) doesn't support</div><div style="font-size:13px">reading any type \
of plan node, so we need to add support for reading all kind</div><div \
style="font-size:13px">of plan nodes (as subplan can have any type of plan node) and \
similarly</div><div style="font-size:13px">to execute any type of Plan node, we might \
need more work (infrastructure).</div><div><div class="gmail_signature"><br>With \
Regards,<br>Amit Kapila.<br>EnterpriseDB: <a href="http://www.enterprisedb.com/" \
target="_blank">http://www.enterprisedb.com</a></div></div></div></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic