[prev in list] [next in list] [prev in thread] [next in thread]
List: pgsql-hackers
Subject: Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization
From: Mark Dilger <markdilger () yahoo ! com>
Date: 2013-12-31 21:51:08
Message-ID: 1388526668.53184.YahooMailNeo () web125403 ! mail ! ne1 ! yahoo ! com
[Download RAW message or body]
The BDR documentation http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf
says,
"Physical replication forces us to use just one
node: multi-master required for write scalability"
"Physical replication provides best read scalability"
I am inclined to agree with the second statement, but
I think my proposal invalidates the first statement, at
least for a particular rigorous partitioning over which
server owns which data.
In my own workflow, I load lots of data from different
sources. The partition the data loads into depends on
which source it came from, and it is never mixed or
cross referenced in any operation that writes the data.
It is only "mixed" in the sense that applications query
data from multiple sources.
So for me, multi-master with physical replication seems
possible, and would presumably provide the best
read scalability. I doubt that I am in the only database
user who has this kind of workflow.
The alternatives are ugly. I can load data from separate
sources into separate database servers *without* replication
between them, but then the application layer has to
emulate queries across the data. (Yuck.) Or I can use
logical replication such as BDR, but then the servers
are spending more effort than with physical replication,
so I get less bang for the buck when I purchase more
servers to add to the cluster. Or I can use FDW to access
data from other servers, but that means the same data
may be pulled across the wire arbitrarily many times, with
corresponding impact on the bandwidth.
Am I missing something here? Does BDR really provide
an equivalent solution?
Second, it seems that BDR leaves to the client the responsibility
for making schemas the same everywhere. Perhaps this is just
a limitation of the implementation so far, which will be resolved
in the future?
On Tuesday, December 31, 2013 12:33 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
Mark Dilger wrote:
> This is not entirely "pie in the sky", but feel free to tell me why this is crazy.
Have you seen http://wiki.postgresql.org/wiki/BDR ?
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[Attachment #3 (text/html)]
<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, \
Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:12pt"><div \
id="yiv0614783746"><div><div \
style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, \
Helvetica, Arial, Lucida Grande, sans-serif;font-size:12pt;">The BDR documentation \
http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf<br>says,<br><br><span \
class="tab"> "</span>Physical replication forces us to use just \
one<br><span class="tab"> node: multi-master required for \
write scalability"<br><br><span class="tab"> "Physical replication \
provides best read scalability"<br><br>I am inclined to agree with the second \
statement, but<br>I think my proposal invalidates the first statement, at<br>least \
for a particular rigorous partitioning over which<br>server owns which \
data.<br><br>In my own workflow, I load lots of data from \
different<br>sources. The partition the data loads into depends on<br>which \
source it came from, and it is never mixed or<br>cross referenced in any operation \
that writes the data.<br>It is only "mixed" in the sense that applications \
query<br>data from multiple sources.<br><br>So for me, multi-master with physical \
replication seems<br>possible, and would presumably provide the best<br>read \
scalability. I doubt that I am in the only database<br>user who has this kind \
of workflow.<br><br>The alternatives are ugly. I can load data from \
separate<br>sources into separate database servers *without* replication<br>between \
them, but then the application layer has to <br>emulate queries across the \
data. (Yuck.) Or I can use<br>logical replication such as BDR, but then \
the servers<br>are spending more effort than with physical replication,<br>so I get \
less bang for the buck when I purchase more<br>servers to add to the cluster. \
Or I can use FDW to access<br>data from other servers, but that means the same \
data<br>may be pulled across the wire arbitrarily many times, with<br>corresponding \
impact on the bandwidth. <br><br>Am I missing something here? Does BDR really \
provide<br>an equivalent solution?<br clear="none"></span></span><span \
id="yiv0614783746yui_3_13_0_ym1_10_1388511900644_14"></span><br clear="none"><div \
class="yiv0614783746yahoo_quoted" \
id="yiv0614783746yui_3_13_0_ym1_10_1388511900644_10" style="display: block;"> Second, \
it seems that BDR leaves to the client the responsibility<br clear="none">for making \
schemas the same everywhere. Perhaps this is just<br clear="none">a limitation \
of the implementation so far, which will be resolved<br clear="none">in the \
future?<br clear="none"><br clear="none"> <br clear="none"> <div \
class="yiv0614783746yqt1651168966" id="yiv0614783746yqt19811"><div \
class="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32299" \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32359" \
style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, \
sans-serif;font-size:12pt;"> <div \
class="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32300" \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32358" \
style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, \
sans-serif;font-size:12pt;"> <div dir="ltr" \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32364"> <font \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32363" face="Arial" size="2"> On \
Tuesday, December 31, 2013 12:33 PM, Alvaro Herrera <alvherre@2ndquadrant.com> \
wrote:<br clear="none"> </font> </div> <div class="yiv0614783746y_msg_container" \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32357">Mark Dilger wrote:<div \
class="yiv0614783746yqt8331675798" id="yiv0614783746yqtfd92252"><br clear="none">> \
This is not entirely "pie in the sky", but feel free to tell me why this is \
crazy.</div><br clear="none"><br clear="none">Have you seen <a rel="nofollow" \
shape="rect" id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32362" target="_blank" \
href="http://wiki.postgresql.org/wiki/BDR">http://wiki.postgresql.org/wiki/BDR \
</a>?<br clear="none"><br clear="none">-- <br clear="none">Álvaro Herrera \
<a rel="nofollow" shape="rect" \
id="yiv0614783746yui_3_13_0_ym1_1_1388511900644_32360" target="_blank" \
href="http://www.2ndquadrant.com/">http://www.2ndQuadrant.com/</a><br \
clear="none">PostgreSQL Development, 24x7 Support, Training & Services<br \
clear="none"><br clear="none"><br clear="none">-- <br clear="none">Sent via \
pgsql-hackers mailing list (<a rel="nofollow" shape="rect" \
ymailto="mailto:pgsql-hackers@postgresql.org" target="_blank" \
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br \
clear="none">To make changes to your subscription:<br clear="none"><a rel="nofollow" \
shape="rect" target="_blank" \
href="http://www.postgresql.org/mailpref/pgsql-hackers">http://www.postgresql.org/mailpref/pgsql-hackers</a><div \
class="yiv0614783746yqt8331675798" id="yiv0614783746yqtfd28820"><br \
clear="none"></div><br clear="none"><br clear="none"></div> </div> </div></div> \
</div> </div></div></div></div></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic