[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: Btrfs by default, the compression option
From:       Chris Murphy <lists () colorremedies ! com>
Date:       2020-07-08 17:08:53
Message-ID: CAJCQCtQhddkRFcgni+EBH-hq+LKDVtdREGUa9KAq4h78xW8JiQ () mail ! gmail ! com
[Download RAW message or body]

On Wed, Jul 8, 2020 at 5:13 AM Kamil Paral <kparal@redhat.com> wrote:
> 
> On Wed, Jul 8, 2020 at 11:22 AM Dominique Martinet <asmadeus@codewreck.org> wrote:
> > 
> > Please test, but if a file is deemed not compressible (based on, not
> > sure what? the first few blocks?) then it will be stored in the
> > non-compressed version.
> > You can check with compsize after the fact if the file had been
> > compressed or not.
> > 
> > This should be true unless the compress-force mount option is used, even
> > the chattr is only a hint
> 
> 
> OK, this is an interesting piece of information I wasn't aware of. In this case, if \
> the btrfs heuristics work OK in the majority of cases, the performance hit might \
> not be there, as I feared. Some thorough investigation would be needed, though.

There is a cheap estimator of how well blocks will compress and Btrfs
will do an early bail, and not compress if there's no good gain. It's
common to see mixed compression on files.

I expect beefy CPU systems, including gaming systems, to have the same
or better read/write performance using mount option compress=zstd:1.
Where I've seen equal or better read performance, there can be a write
performance drop if the IO storage has been upgraded. Sample size 1,
and the workload was kernel compiling.

The mount option method is file system wide, and permits level to be
specified (except lzo). Whereas using the XATTR is
subvolume/directory/file granularity, and works by inheritance when
set on a directory, it doesn't yet support levels. Upstream
development tells me it's straightforward to include the level in the
XATTR - but that is an RFE so we can't plan for it just yet. The
default level for zstd is 3. The read time performance is about the
same, but the write time performance takes a bigger hit. This isn't a
big deal if we're "curating" specific directories that are likely to
have their write performance limited by, for example internet
bandwidth, or perception wise that the offline update takes however
many tens of seconds longer.

I also tentatively agree that many users' drives are not likely to see
their drive lifetime writes exceeded without compression. But if they
can save ~50% of the space without a read time performance hit (or
even a gain) that's a win. For folks compiling, that needs more
testing. It might work out in favor of some cases and not others - and
if they do a lot of writes the write amplification reduction is more
meaningful. And also some low end very high density chip SSDs can have
low write endurance these days.

SD Card and eMMC  it's a win for sure. Also an argument could be made
do use Btrfs+compression on USB sticks. This class of flash will just
return garbage if they encounter uncorrectable errors - rather than a
discrete read error. In this case, Btrfs refuses to hand over the
corrupt data, in normal operation. A good question is, whether the
desktop should warn that the file is corrupt, and then permit a
degraded read somehow to still get the file off the media. It might
imply necessary desktop integration.


-- 
Chris Murphy
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



[prev in list] [next in list] [prev in thread] [next in thread]