[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postgresql-admin
Subject:    Re: How to get a more RSYNC compatible output of pg_dump?
From:       Jeff Janes <jeff.janes () gmail ! com>
Date:       2022-05-19 3:46:37
Message-ID: CAMkU=1xmXSp-enz-EhLi0jL0-NJPFNxx3CEx7UJOh=EAxdLSig () mail ! gmail ! com
[Download RAW message or body]

>
> Though, with the large dumps it seems to me that with every slight
> change in the actual data the entire dump gets downloaded again. I'm
> already using uncompressed dumps in the hope that the output is more
> stable and RSYNC better able to recognize unchanged parts. But I guess
> that most changes in the dumped data simply result in all subsequent
> data being that misplaced compared to what RSYNC reads against, that
> it's like downloading the whole file again in the end.
>

For me the rsync differential algorithm on dump files just works with no
special preparations.  I think you need to investigate this from the rsync
side, not PostgreSQL.  For example, if you change a dump file just by
adding or deleting one character near the beginning, how does rsync respond
to that?

Cheers,

Jeff

[Attachment #3 (text/html)]

<div dir="ltr"><div dir="ltr"><br></div><div class="gmail_quote"><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><br> Though, with the large dumps it seems to me \
that with every slight<br> change in the actual data the entire dump gets downloaded \
again. I&#39;m<br> already using uncompressed dumps in the hope that the output is \
more<br> stable and RSYNC better able to recognize unchanged parts. But I guess<br>
that most changes in the dumped data simply result in all subsequent<br>
data being that misplaced compared to what RSYNC reads against, that<br>
it&#39;s like downloading the whole file again in the \
end.<br></blockquote><div><br></div><div>For me the rsync differential algorithm on \
dump files just works with no special preparations.   I think you need to investigate \
this from the rsync side, not PostgreSQL.   For example, if you change a dump file \
just by adding or deleting one character near the beginning, how does rsync respond \
to that?  </div><div><br></div><div>Cheers,</div><div><br></div><div>Jeff</div></div></div>




[prev in list] [next in list] [prev in thread] [next in thread]