[prev in list] [next in list] [prev in thread] [next in thread]
List: postgresql-admin
Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: could not get WAL end positi
From: John Scalia <jayknowsunix () gmail ! com>
Date: 2016-12-05 20:51:02
Message-ID: 38BC73D1-59DB-4E17-A252-C8DE9EE147CE () gmail ! com
[Download RAW message or body]
The only way to increase the segment size is to adjust the setting in the source code \
and recompile the server. There are no runtime adjustments available to do this.
Sent from my iPad
> On Dec 5, 2016, at 3:41 PM, Keith <keith@keithf4.com> wrote:
>
>
>
> > On Mon, Dec 5, 2016 at 3:27 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> \
> > wrote: Hi John,
> >
> >
> >
> > Thank you for the feed back. Presently it is taking around 1hr to finish the \
> > backups. We have the wal_keep_segments set to 50. Is there way to adjust the \
> > segment size to be more than 16MB? At times I notice in the pg_xlog, there are \
> > 10-15 segments on an invocation point.
> >
> >
> >
> >
> > Thanx,
> >
> > Sam
> >
> >
> >
> > From: John Scalia [mailto:jayknowsunix@gmail.com]
> > Sent: Monday, December 05, 2016 1:52 PM
> > To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
> > Cc: pgsql-admin@postgresql.org
> > Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: \
> > could not get WAL end position from server: FATAL: requested WAL segment"
> >
> >
> > More than likely your keep_wal_segments setting is too low. How long is the \
> > pg_basebackup taking? Remember that segments are generated about every 5 minutes, \
> > so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, \
> > you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.
> > --
> >
> > Jay
> >
> >
> >
> > Sent from my iPad
> >
> >
> > On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> \
> > wrote:
> > Hi
> >
> >
> >
> > I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we \
> > do the postgres backups we are noticing the following errors
> >
> >
> > pg_basebackup: could not get WAL end position from server: FATAL: requested WAL \
> > segment 000000010000020B00000040 has already been removed
> >
> >
> > But when I look into pg_xlog, I notice
> >
> >
> >
> > 000000010000020B00000040.00000118.backup
> >
> >
> >
> >
> >
> > May I know how to resolve this error? Are the backups done with these errors, \
> > recoverable?
> >
> >
> >
> >
> > Thanks in advance for your support.
> >
> >
> >
> > Thanx,
> >
> > Sam
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
> If you're using pgbasebackup just for creating consistent backups, use the \
> --xlog-method=strean (-Xs) to ensure you get all the WAL files necessary for a \
> consistent backup. This avoids having to set wal_keep_segments high enough to \
> account for the time length of your entire backup. Note you will need an extra WAL \
> stream for this setting, so ensure max_wal_senders is set high enough on the \
> master.
> Now if you're going to be using this backup to create a slave, you must get that \
> slave up and running before the master recycles all the WAL files necessary for \
> replication to be able to catch up from when the backup stopped. Otherwise you will \
> see that error you shared in your original post. That's when you may need to adjust \
> wal_keep_segments. Either that or you should set up an archive_command on the \
> master to keep a secondary copy of all WAL files.
> Keith
[Attachment #3 (text/html)]
<html><head><meta http-equiv="content-type" content="text/html; \
charset=utf-8"></head><body dir="auto"><div>The only way to increase the segment size \
is to adjust the setting in the source code and recompile the server. There are no \
runtime adjustments available to do this.<br><br>Sent from my iPad</div><div><br>On \
Dec 5, 2016, at 3:41 PM, Keith <<a \
href="mailto:keith@keithf4.com">keith@keithf4.com</a>> \
wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr"><br><div \
class="gmail_extra"><br><div class="gmail_quote">On Mon, Dec 5, 2016 at 3:27 PM, \
Tharmarajah, Sam <span dir="ltr"><<a href="mailto:sambavan.tharmarajah@usask.ca" \
target="_blank">sambavan.tharmarajah@usask.ca</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex">
<div lang="EN-CA">
<div class="gmail-m_9029006977082092381WordSection1">
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Hi \
John,<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="color:rgb(31,73,125)"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span style="color:rgb(31,73,125)">Thank you for the feed back. \
Presently it is taking around 1hr to finish the backups. We have the \
wal_keep_segments set to 50. Is there way to adjust the segment size to be more than \
16MB? At times I notice in the pg_xlog, there are 10-15 segments on an invocation \
point.<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="color:rgb(31,73,125)"><u></u> <u></u></span></p> <p \
class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p> \
<p class="MsoNormal"><span \
style="color:rgb(31,73,125)">Thanx,<u></u><u></u></span></p> <p \
class="MsoNormal"><span style="color:rgb(31,73,125)">Sam<u></u><u></u></span></p> <p \
class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p> \
<div> <div style="border-width:1pt medium medium;border-style:solid none \
none;border-color:rgb(225,225,225) -moz-use-text-color \
-moz-use-text-color;padding:3pt 0cm 0cm"> <p class="MsoNormal"><b><span \
lang="EN-US">From:</span></b><span lang="EN-US"> John Scalia [mailto:<a \
href="mailto:jayknowsunix@gmail.com" target="_blank">jayknowsunix@gmail.com</a><wbr>] \
<br> <b>Sent:</b> Monday, December 05, 2016 1:52 PM<br>
<b>To:</b> Tharmarajah, Sam <<a href="mailto:sambavan.tharmarajah@usask.ca" \
target="_blank">sambavan.tharmarajah@usask.ca</a><wbr>><br> <b>Cc:</b> <a \
href="mailto:pgsql-admin@postgresql.org" \
target="_blank">pgsql-admin@postgresql.org</a><br> <b>Subject:</b> Re: [ADMIN] Req. \
for some help with resolving the error "pg_basebackup: could not get WAL end position \
from server: FATAL: requested WAL segment"<u></u><u></u></span></p> </div>
</div><div><div class="gmail-h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">More than likely your keep_wal_segments setting is too low. How \
long is the pg_basebackup taking? Remember that segments are generated about every 5 \
minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 \
hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of \
backup.<span style="font-size:12pt"><u></u><u></u></span></p> </div>
<div id="gmail-m_9029006977082092381AppleMailSignature">
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div id="gmail-m_9029006977082092381AppleMailSignature">
<p class="MsoNormal">Jay<u></u><u></u></p>
</div>
<div id="gmail-m_9029006977082092381AppleMailSignature">
<p class="MsoNormal"><br>
<br>
Sent from my iPad<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt"><br>
On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <<a \
href="mailto:sambavan.tharmarajah@usask.ca" \
target="_blank">sambavan.tharmarajah@usask.ca</a><wbr>> wrote:<u></u><u></u></p> \
</div> <blockquote style="margin-top:5pt;margin-bottom:5pt">
<div>
<p class="MsoNormal">Hi <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I am a newbie to postgres. We are on postgres 9.1 running on \
RedHat 6. When we do the postgres backups we are noticing the following \
errors<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">pg_basebackup: could not get WAL end position from server: \
FATAL: requested WAL segment 000000010000020B00000040 has already been \
removed<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">But when I look into pg_xlog, I notice <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">000000010000020B00000040.<wbr>00000118.backup<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">May I know how to resolve this error? Are the backups done with \
these errors, recoverable?<u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <p \
class="MsoNormal"> <u></u><u></u></p> <p class="MsoNormal">Thanks in advance for \
your support.<u></u><u></u></p> <p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanx,<u></u><u></u></p>
<p class="MsoNormal">Sam<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</blockquote>
</div></div></div>
</div>
</blockquote></div><br></div><div class="gmail_extra">If you're using pgbasebackup \
just for creating consistent backups, use the --xlog-method=strean (-Xs) to ensure \
you get all the WAL files necessary for a consistent backup. This avoids having to \
set wal_keep_segments high enough to account for the time length of your entire \
backup. Note you will need an extra WAL stream for this setting, so ensure \
max_wal_senders is set high enough on the master.<br><br></div><div \
class="gmail_extra">Now if you're going to be using this backup to create a slave, \
you must get that slave up and running before the master recycles all the WAL files \
necessary for replication to be able to catch up from when the backup stopped. \
Otherwise you will see that error you shared in your original post. That's when you \
may need to adjust wal_keep_segments. Either that or you should set up an \
archive_command on the master to keep a secondary copy of all WAL files. \
<br><br></div><div class="gmail_extra">Keith<br></div></div> \
</div></blockquote></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic