[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postgresql-hackers
Subject:    Re: Different compression methods for FPI
From:       Andrey Borodin <x4mmm () yandex-team ! ru>
Date:       2021-05-31 7:33:44
Message-ID: 3252174C-C21D-4711-A038-5230DE465BAE () yandex-team ! ru
[Download RAW message or body]



> 25 мая 2021 г., в 12:26, Michael Paquier <michael@paquier.xyz> \
> написал(а): 
> On Mon, May 24, 2021 at 11:44:45PM -0500, Justin Pryzby wrote:
> > The goal is to support 2+ "methods" (including "none"), which takes 4 bits, so
> > may as well support 3 methods.
> > 
> > - uncompressed
> > - pglz
> > - lz4
> > - zlib or zstd or ??
> 
> Let's make a proper study of all that and make a choice, the only
> thing I am rather sure of is that pglz is bad compared to all the
> others.  There is no point to argue as long as we don't know if any of
> those candidates are suited for the job.

There's a lot of open studies like [0,1].
In short, Lz4 is fastest codec. Zstd gives better compression at cost of more CPU \
usage[2]. Zlib is not tied to Facebook, however, it's slower and have smaller \
compression ratio than Zstd. Zstd is actively developed.

There is Google's Brotli codec. It is comparable to Zstd.

What kind of deeper study do we want to do?
Would it make sense to run our own benchmarks?
Or, perhaps, review other codecs?

In my opinion, anything that is sent over network or written to block storage \
deserves Zstd-5 compression. But milage may vary.

Thanks!

Best regards, Andrey Borodin.


[0] https://indico.cern.ch/event/695984/contributions/2872933/attachments/1590457/2516802/ZSTD_and_ZLIB_Updates_-_January_20186.pdf
 [1] https://facebook.github.io/zstd/
[2] Zstd gives significantly better compression at cost of little more CPU usage. But \
they both have points on Pareto frontier.


[prev in list] [next in list] [prev in thread] [next in thread]