[prev in list] [next in list] [prev in thread] [next in thread]
List: pgsql-hackers
Subject: Re: [HACKERS] GiST subsplit question
From: Alexander Korotkov <aekorotkov () gmail ! com>
Date: 2012-05-30 20:35:44
Message-ID: CAPpHfdsfRpuCAJzJpyW95yvqycqp1zdQ5rktW2Vmj3ryeBu49g () mail ! gmail ! com
[Download RAW message or body]
On Wed, May 30, 2012 at 11:21 PM, Jeff Davis <pgsql@j-davis.com> wrote:
> I looked for the follow-up commit to support subsplit in the contrib
> modules, figuring that would answer some questions, but I couldn't find
> it.
>
> The part that's confusing me is that the commit message says: "pickSplit
> should set spl_(l|r)datum_exists to 'false'", but I don't see any
> picksplit method that actually does that in contrib, nor in the sample
> in the docs.
>
The only picksplit implementation I know to support secondary split
is fallbackSplit in gistproc.c :). I didn't understand how secondary split
works until I get deep into GiST code.
The code in that area is a bit difficult to follow, so it's not obvious
> to me exactly what is supposed to happen.
>
When GiST split index tupes by first column, it find if some tuples can be
equally placed to any of groups. If so it calls picksplit for second column
with only that tuples. But we already now that some keys of second column
should be placed to particular groups, because some tuples are already
unambiguously
placed to the groups. GiST calculates union of that keys of second column
for each group and pass it as ldatum and rdatum to pisckplit function of
second column. In order to indicate such secondary split GiST set
ldatum_exists and rdatum_exists flags. If picksplit function support
secondary split, it should join given keys to existing ldatum and rdatum
and set off ldatum_exists and rdatum_exists flags. If picksplit function
don't support secondary split than in leave ldatum_exists and rdatum_exists
as is, and GiST decide how to join picksplit result with existing groups
itself by using penalty function. Also it is possible than only one of
ldatum_exists
and rdatum_exists flags is set. It indicated that there aren't not null
keys of second column which unambiguously join group which flag isn't set.
In this case pisksplit function which support secondary split can form one
of ldatum or rdatum (which flag isn't set) without limitations.
> So, do we demote that message to a DEBUG1? Or do we make it more clear
> what the authors of a specific picksplit are supposed to do to avoid
> that problem? Or am I misunderstanding something?
>
+1 for demote message to DEBUG1. I think it shouldn't be so noisy, it just
indicates something could be improved.
Also I think we defenitely need to document secondary split. Now it's no
chances to understand without reverse engeneering it from code.
------
With best regards,
Alexander Korotkov.
[Attachment #3 (text/html)]
<div>On Wed, May 30, 2012 at 11:21 PM, Jeff Davis <span dir="ltr"><<a \
href="mailto:pgsql@j-davis.com" target="_blank">pgsql@j-davis.com</a>></span> \
wrote:</div><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">I looked for the follow-up commit to support subsplit in the \
contrib<br> modules, figuring that would answer some questions, but I couldn't \
find<br> it.<br>
<br>
The part that's confusing me is that the commit message says: "pickSplit<br>
should set spl_(l|r)datum_exists to 'false'", but I don't see \
any<br> picksplit method that actually does that in contrib, nor in the sample<br>
in the docs.<br></blockquote><div> </div><div>The only picksplit implementation I \
know to support secondary split is fallbackSplit in gistproc.c :). I didn't \
understand how secondary split works until I get deep into GiST code.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex"> The code in that area is a bit \
difficult to follow, so it's not obvious<br> to me exactly what is supposed to \
happen.<br></blockquote><div><br></div><div>When GiST split index tupes by first \
column, it find if some tuples can be equally placed to any of groups. If so it calls \
picksplit for second column with only that tuples. But we already now that some keys \
of second column should be placed to particular groups, because some tuples are \
already <span style="font-size:13px;text-align:left;font-family:Arial,sans-serif">unambiguously \
placed to the groups. GiST calculates union of that keys of second column for each \
group and pass it as ldatum and rdatum to pisckplit function of second column. In \
order to indicate such secondary split GiST set ldatum_exists and rdatum_exists \
flags. </span><span style="font-size:13px;text-align:left;font-family:Arial,sans-serif">If \
picksplit function support secondary split, it should join given keys to existing \
</span><span style="font-size:13px;text-align:left;font-family:Arial,sans-serif">ldatum \
and </span><span style="font-size:13px;text-align:left;font-family:Arial,sans-serif">rdatum \
and set off </span><span \
style="font-size:13px;text-align:left;font-family:Arial,sans-serif">ldatum_exists and \
rdatum_exists flags. If </span><span \
style="font-size:13px;text-align:left;font-family:Arial,sans-serif">picksplit \
function don't support secondary split than in leave </span><span \
style="font-size:13px;text-align:left;font-family:Arial,sans-serif">ldatum_exists and \
rdatum_exists as is, and GiST decide how to join picksplit result with existing \
groups itself by using penalty function. Also it is possible than only one of \
</span><span style="font-size:13px;text-align:left;font-family:Arial,sans-serif">ldatum_exists \
and rdatum_exists flags is set. It indicated that there aren't not null keys of \
second column which </span><span \
style="font-size:13px;text-align:left;font-family:Arial,sans-serif">unambiguously \
join group which flag isn't set. In this case pisksplit function which support \
secondary split can form one of ldatum or rdatum (which flag isn't set) without \
limitations.</span></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex"> So, do we demote that message to a DEBUG1? Or do we \
make it more clear<br> what the authors of a specific picksplit are supposed to do to \
avoid<br> that problem? Or am I misunderstanding \
something?<br></blockquote><div><br></div><div>+1 for demote message to DEBUG1. I \
think it shouldn't be so noisy, it just indicates something could be \
improved.</div><div>Also I think we defenitely need to document secondary split. Now \
it's no chances to understand without reverse engeneering it from code.</div>
<br>------<br>With best regards,<br>Alexander Korotkov.</div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic