[prev in list] [next in list] [prev in thread] [next in thread]
List: postgresql-hackers
Subject: Re: block-level incremental backup
From: Ibrar Ahmed <ibrar.ahmad () gmail ! com>
Date: 2019-08-31 19:40:28
Message-ID: CALtqXTcWKK93aYc=j+QgMZHcfG8t=XzX+FGn5UyvwEwkyWdbTA () mail ! gmail ! com
[Download RAW message or body]
On Sat, Aug 31, 2019 at 7:59 AM Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Aug 29, 2019 at 10:41 AM Jeevan Ladhe
> <jeevan.ladhe@enterprisedb.com> wrote:
> > Due to the inherent nature of pg_basebackup, the incremental backup also
> > allows taking backup in tar and compressed format. But, pg_combinebackup
> > does not understand how to restore this. I think we should either make
> > pg_combinebackup support restoration of tar incremental backup or
> restrict
> > taking the incremental backup in tar format until pg_combinebackup
> > supports the restoration by making option '--lsn' and '-Ft' exclusive.
> >
> > It is arguable that one can take the incremental backup in tar format,
> extract
> > that manually and then give the resultant directory as input to the
> > pg_combinebackup, but I think that kills the purpose of having
> > pg_combinebackup utility.
>
> I don't agree. You're right that you would have to untar (and
> uncompress) the backup to run pg_combinebackup, but you would also
> have to do that to restore a non-incremental backup, so it doesn't
> seem much different. It's true that for an incremental backup you
> might need to untar and uncompress multiple prior backups rather than
> just one, but that's just the nature of an incremental backup. And,
> on a practical level, if you want compression, which is pretty likely
> if you're thinking about incremental backups, the way to get that is
> to use tar format with -z or -Z.
>
> It might be interesting to teach pg_combinebackup to be able to read
> tar-format backups, but I think that there are several variants of the
> tar format, and I suspect it would need to read them all. If someone
> un-tars and re-tars a backup with a different tar tool, we don't want
> it to become unreadable. So we'd either have to write our own
> de-tarring library or add an external dependency on one.
Are we using any tar library in pg_basebackup.c? We already have the
capability
in pg_basebackup to do that.
> I don't
> think it's worth doing that at this point; I definitely don't think it
> needs to be part of the first patch.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
>
--
Ibrar Ahmed
[Attachment #3 (text/html)]
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Sat, Aug 31, 2019 at 7:59 AM Robert Haas <<a \
href="mailto:robertmhaas@gmail.com">robertmhaas@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Thu, Aug 29, 2019 \
at 10:41 AM Jeevan Ladhe<br> <<a href="mailto:jeevan.ladhe@enterprisedb.com" \
target="_blank">jeevan.ladhe@enterprisedb.com</a>> wrote:<br> > Due to the \
inherent nature of pg_basebackup, the incremental backup also<br> > allows taking \
backup in tar and compressed format. But, pg_combinebackup<br> > does not \
understand how to restore this. I think we should either make<br> > \
pg_combinebackup support restoration of tar incremental backup or restrict<br> > \
taking the incremental backup in tar format until pg_combinebackup<br> > supports \
the restoration by making option '--lsn' and '-Ft' exclusive.<br> \
><br> > It is arguable that one can take the incremental backup in tar format, \
extract<br> > that manually and then give the resultant directory as input to \
the<br> > pg_combinebackup, but I think that kills the purpose of having<br>
> pg_combinebackup utility.<br>
<br>
I don't agree. You're right that you would have to untar (and<br>
uncompress) the backup to run pg_combinebackup, but you would also<br>
have to do that to restore a non-incremental backup, so it doesn't<br>
seem much different. It's true that for an incremental backup you<br>
might need to untar and uncompress multiple prior backups rather than<br>
just one, but that's just the nature of an incremental backup. And,<br>
on a practical level, if you want compression, which is pretty likely<br>
if you're thinking about incremental backups, the way to get that is<br>
to use tar format with -z or -Z.<br>
<br>
It might be interesting to teach pg_combinebackup to be able to read<br>
tar-format backups, but I think that there are several variants of the<br>
tar format, and I suspect it would need to read them all. If someone<br>
un-tars and re-tars a backup with a different tar tool, we don't want<br>
it to become unreadable. So we'd either have to write our own<br>
de-tarring library or add an external dependency on one. \
</blockquote><div><br></div><div>Are we using any tar library in pg_basebackup.c? We \
already have the capability</div><div>in pg_basebackup to do that. \
</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px \
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> I \
don't<br> think it's worth doing that at this point; I definitely don't \
think it<br> needs to be part of the first patch.<br>
<br>
-- <br>
Robert Haas<br>
EnterpriseDB: <a href="http://www.enterprisedb.com" rel="noreferrer" \
target="_blank">http://www.enterprisedb.com</a><br> The Enterprise PostgreSQL \
Company<br> <br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" \
class="gmail_signature"><div dir="ltr"><div>Ibrar Ahmed</div></div></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic