[prev in list] [next in list] [prev in thread] [next in thread]
List: postgresql-general
Subject: Re: [HACKERS] Time-Delayed Standbys
From: Fabrízio_de_Royes_Mello <fabriziomello () gmail ! com>
Date: 2013-11-29 20:34:51
Message-ID: CAFcNs+oNDqVQ7GZXGxCLiGk2o1xdy=FnhR4QAMjJ_BvfR8BmAg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Fri, Nov 29, 2013 at 5:49 AM, KONDO Mitsumasa <
kondo.mitsumasa@lab.ntt.co.jp> wrote:
>
> Hi Royes,
>
> I'm sorry for my late review...
>
No problem...
> I feel potential of your patch in PG replication function, and it might
be something useful for all people. I check your patch and have some
comment for improvement. I haven't executed detail of unexpected sutuation
yet. But I think that under following problem2 is important functionality
problem. So I ask you to solve the problem in first.
>
> * Regress test
> No problem.
>
> * Problem1
> Your patch does not code recovery.conf.sample about recovery_time_delay.
> Please add it.
>
Fixed.
> * Problem2
> When I set time-delayed standby and start standby server, I cannot access
stanby server by psql. It is because PG is in first starting recovery which
cannot access by psql. I think that time-delayed standby is only delayed
recovery position, it must not affect other functionality.
>
> I didn't test recoevery in master server with recovery_time_delay. If you
have detail test result of these cases, please send me.
>
Well, I could not reproduce the problem that you described.
I run the following test:
1) Clusters
- build master
- build slave and attach to the master using SR and config
recovery_time_delay to 1min.
2) Stop de Slave
3) Run some transactions on the master using pgbench to generate a lot of
archives
4) Start the slave and connect to it using psql and in another session I
can see all archive recovery log
> My first easy review of your patch is that all.
>
Thanks.
Regards,
--
Fabr=EDzio de Royes Mello
Consultoria/Coaching PostgreSQL
>> Timbira: http://www.timbira.com.br
>> Blog sobre TI: http://fabriziomello.blogspot.com
>> Perfil Linkedin: http://br.linkedin.com/in/fabriziomello
>> Twitter: http://twitter.com/fabriziomello
[Attachment #5 (text/html)]
<div dir="ltr"><div class="gmail_extra">On Fri, Nov 29, 2013 at 5:49 AM, KONDO \
Mitsumasa <<a href="mailto:kondo.mitsumasa@lab.ntt.co.jp">kondo.mitsumasa@lab.ntt.co.jp</a>> \
wrote:<br>><br>> Hi Royes,<br>><br> > I'm sorry for my late \
review...<br>><br><br>No problem...<br><br> <br>> I feel potential of your \
patch in PG replication function, and it might be something useful for all people. I \
check your patch and have some comment for improvement. I haven't executed detail \
of unexpected sutuation yet. But I think that under following problem2 is important \
functionality problem. So I ask you to solve the problem in first.<br> ><br>> * \
Regress test<br>> No problem.<br>><br>> * Problem1<br>> Your patch does \
not code recovery.conf.sample about recovery_time_delay.<br>> Please add \
it.<br>><br><br></div><div class="gmail_extra"> Fixed.<br></div><div \
class="gmail_extra"><br><br>> * Problem2<br>> When I set time-delayed standby \
and start standby server, I cannot access stanby server by psql. It is because PG is \
in first starting recovery which cannot access by psql. I think that time-delayed \
standby is only delayed recovery position, it must not affect other \
functionality.<br> ><br>> I didn't test recoevery in master server with \
recovery_time_delay. If you have detail test result of these cases, please send \
me.<br>><br><br></div><div class="gmail_extra">Well, I could not reproduce the \
problem that you described.<br> <br></div><div class="gmail_extra">I run the \
following test:<br><br></div><div class="gmail_extra">1) Clusters<br></div><div \
class="gmail_extra">- build master<br></div><div class="gmail_extra">- build slave \
and attach to the master using SR and config recovery_time_delay to 1min.<br> \
</div><div class="gmail_extra"><br></div><div class="gmail_extra">2) Stop de \
Slave<br><br></div><div class="gmail_extra">3) Run some transactions on the master \
using pgbench to generate a lot of archives<br><br></div><div class="gmail_extra"> 4) \
Start the slave and connect to it using psql and in another session I can see all \
archive recovery log<br></div><div class="gmail_extra"><br><br>> My first easy \
review of your patch is that all.<br>><br><br></div> <div \
class="gmail_extra">Thanks.<br><br>Regards,<br><br>--<br>Fabrízio de Royes \
Mello<br>Consultoria/Coaching PostgreSQL<br>>> Timbira: <a \
href="http://www.timbira.com.br">http://www.timbira.com.br</a><br>>> Blog sobre \
TI: <a href="http://fabriziomello.blogspot.com">http://fabriziomello.blogspot.com</a><br>
>> Perfil Linkedin: <a \
href="http://br.linkedin.com/in/fabriziomello">http://br.linkedin.com/in/fabriziomello</a><br>>> \
Twitter: <a href="http://twitter.com/fabriziomello">http://twitter.com/fabriziomello</a></div>
</div>
["time-delayed-standby-v2.patch" (text/x-diff)]
diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml
index c0c543e..641c9c6 100644
--- a/doc/src/sgml/recovery-config.sgml
+++ b/doc/src/sgml/recovery-config.sgml
@@ -135,6 +135,27 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # \
Windows </listitem>
</varlistentry>
+ <varlistentry id="recovery-time-delay" xreflabel="recovery_time_delay">
+ <term><varname>recovery_time_delay</varname> (<type>integer</type>)</term>
+ <indexterm>
+ <primary><varname>recovery_time_delay</> recovery parameter</primary>
+ </indexterm>
+ <listitem>
+ <para>
+ Specifies the amount of time (in milliseconds, if no unit is specified)
+ which recovery of transaction commits should lag the master. This
+ parameter allows creation of a time-delayed standby. For example, if
+ you set this parameter to <literal>5min</literal>, the standby will
+ replay each transaction commit only when the system time on the standby
+ is at least five minutes past the commit time reported by the master.
+ </para>
+ <para>
+ Note that if the master and standby system clocks are not synchronized,
+ this might lead to unexpected results.
+ </para>
+ </listitem>
+ </varlistentry>
+
</variablelist>
</sect1>
diff --git a/src/backend/access/transam/recovery.conf.sample \
b/src/backend/access/transam/recovery.conf.sample index 5acfa57..97cc7af 100644
--- a/src/backend/access/transam/recovery.conf.sample
+++ b/src/backend/access/transam/recovery.conf.sample
@@ -123,6 +123,17 @@
#
#trigger_file = ''
#
+# recovery_time_delay
+#
+# By default, a standby server keeps restoring XLOG records from the
+# primary as soon as possible. If you want to delay the replay of
+# commited transactions from the master, specify a recovery time delay.
+# For example, if you set this parameter to 5min, the standby will replay
+# each transaction commit only whe the system time on the standby is least
+# five minutes past the commit time reported by the master.
+#
+#recovery_time_delay = 0
+#
#---------------------------------------------------------------------------
# HOT STANDBY PARAMETERS
#---------------------------------------------------------------------------
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index de19d22..714b1bd 100755
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -218,6 +218,8 @@ static bool recoveryPauseAtTarget = true;
static TransactionId recoveryTargetXid;
static TimestampTz recoveryTargetTime;
static char *recoveryTargetName;
+static int recovery_time_delay = 0;
+static TimestampTz recoveryDelayUntilTime;
/* options taken from recovery.conf for XLOG streaming */
static bool StandbyModeRequested = false;
@@ -730,6 +732,7 @@ static void readRecoveryCommandFile(void);
static void exitArchiveRecovery(TimeLineID endTLI, XLogSegNo endLogSegNo);
static bool recoveryStopsHere(XLogRecord *record, bool *includeThis);
static void recoveryPausesHere(void);
+static void recoveryDelay(void);
static void SetLatestXTime(TimestampTz xtime);
static void SetCurrentChunkStartTime(TimestampTz xtime);
static void CheckRequiredParameterValues(void);
@@ -5474,6 +5477,19 @@ readRecoveryCommandFile(void)
(errmsg_internal("trigger_file = '%s'",
TriggerFile)));
}
+ else if (strcmp(item->name, "recovery_time_delay") == 0)
+ {
+ const char *hintmsg;
+
+ if (!parse_int(item->value, &recovery_time_delay, GUC_UNIT_MS,
+ &hintmsg))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("parameter \"%s\" requires a temporal value", "recovery_time_delay"),
+ hintmsg ? errhint("%s", _(hintmsg)) : 0));
+ ereport(DEBUG2,
+ (errmsg("recovery_time_delay = '%s'", item->value)));
+ }
else
ereport(FATAL,
(errmsg("unrecognized recovery parameter \"%s\"",
@@ -5623,7 +5639,8 @@ exitArchiveRecovery(TimeLineID endTLI, XLogSegNo endLogSegNo)
* We also track the timestamp of the latest applied COMMIT/ABORT
* record in XLogCtl->recoveryLastXTime, for logging purposes.
* Also, some information is saved in recoveryStopXid et al for use in
- * annotating the new timeline's history file.
+ * annotating the new timeline's history file; and recoveryDelayUntilTime
+ * is updated, for time-delayed standbys.
*/
static bool
recoveryStopsHere(XLogRecord *record, bool *includeThis)
@@ -5633,6 +5650,9 @@ recoveryStopsHere(XLogRecord *record, bool *includeThis)
TimestampTz recordXtime;
char recordRPName[MAXFNAMELEN];
+ /* Clear any previous recovery delay time */
+ recoveryDelayUntilTime = 0;
+
/* We only consider stopping at COMMIT, ABORT or RESTORE POINT records */
if (record->xl_rmid != RM_XACT_ID && record->xl_rmid != RM_XLOG_ID)
return false;
@@ -5643,6 +5663,11 @@ recoveryStopsHere(XLogRecord *record, bool *includeThis)
recordXactCommitData = (xl_xact_commit_compact *) XLogRecGetData(record);
recordXtime = recordXactCommitData->xact_time;
+
+ if (recovery_time_delay > 0)
+ recoveryDelayUntilTime =
+ TimestampTzPlusMilliseconds(recordXactCommitData->xact_time,
+ recovery_time_delay);
}
else if (record->xl_rmid == RM_XACT_ID && record_info == XLOG_XACT_COMMIT)
{
@@ -5650,6 +5675,11 @@ recoveryStopsHere(XLogRecord *record, bool *includeThis)
recordXactCommitData = (xl_xact_commit *) XLogRecGetData(record);
recordXtime = recordXactCommitData->xact_time;
+
+ if (recovery_time_delay > 0)
+ recoveryDelayUntilTime =
+ TimestampTzPlusMilliseconds(recordXactCommitData->xact_time,
+ recovery_time_delay);
}
else if (record->xl_rmid == RM_XACT_ID && record_info == XLOG_XACT_ABORT)
{
@@ -5832,6 +5862,48 @@ SetRecoveryPause(bool recoveryPause)
}
/*
+ * When recovery_time_delay is set, we wait long enough to make sure we are
+ * at least that far behind the master.
+ */
+static void
+recoveryDelay(void)
+{
+ /* main delay loop */
+ while (true)
+ {
+ long secs;
+ int microsecs;
+
+ ResetLatch(&XLogCtl->recoveryWakeupLatch);
+
+ /* might change the trigger file's location */
+ HandleStartupProcInterrupts();
+
+ if (CheckForStandbyTrigger())
+ break;
+
+ TimestampDifference(GetCurrentTimestamp(), recoveryDelayUntilTime,
+ &secs, µsecs);
+
+ /*
+ * Wait for difference between GetCurrentTimestamp() and
+ * recoveryDelayUntilTime
+ */
+ if (secs > 0 || microsecs > 0)
+ {
+ elog(DEBUG2, "recovery delay %ld seconds, %d milliseconds",
+ secs, microsecs / 1000);
+
+ WaitLatch(&XLogCtl->recoveryWakeupLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
+ secs * 1000L + microsecs / 1000);
+ }
+ else
+ break;
+ }
+}
+
+/*
* Save timestamp of latest processed commit/abort record.
*
* We keep this in XLogCtl, not a simple static variable, so that it can be
@@ -6723,6 +6795,18 @@ StartupXLOG(void)
break;
}
+ /*
+ * If we've been asked to lag the master, pause until enough
+ * time has passed.
+ */
+ if (recovery_time_delay > 0)
+ {
+ recoveryDelay();
+
+ if (CheckForStandbyTrigger())
+ break;
+ }
+
/* Setup error traceback support for ereport() */
errcallback.callback = rm_redo_error_callback;
errcallback.arg = (void *) record;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic