[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: F34 Change: Reduce installation media size by improving the compression ratio of SquashFS filesy
From:       Zbigniew =?utf-8?Q?J=C4=99drzejewski-Szmek?= <zbyszek () in ! waw ! pl>
Date:       2020-09-15 10:03:23
Message-ID: 20200915100323.GQ10369 () in ! waw ! pl
[Download RAW message or body]

On Tue, Sep 15, 2020 at 11:18:04AM +0200, Bohdan Khomutskyi wrote:
> Hello everyone,
> 
> Thanks for sharing your ideas and comments about this change.
> 
> Thanks to Mohan Boddu, I got RawHide DVD and netinstall images of
> RawHide with the optimization features enabled. Those test composes
> are available at the following locations:
> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200828.n.0/
> (Plain SquashFS)
> 
> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200908.n.1/compose/Server/x86_64/iso/
>  (Plain SquashFS and different xz compression options)
> 
> To select images from the test composes, I applied a patch from
> https://github.com/rhinstaller/anaconda/pull/2829, so you can
> download and run those images yourself to test the change:
> 
> https://drive.google.com/drive/folders/1tGsoqFMg2A6dQZHfuQNb9uDqYu9XEiPI
> 
> The result of benchmark will supplement already available data I
> previously posted for Fedora Live images at
> https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
> 
> As a reminder, there are 2 levels for this optimization:
> 
> 1. Making the SquashFS filesystem on the DVD plain (i.e. without
> embedded EXT4 inside it) -- has the suffix plain-xz-128k.iso

This sounds excellent. We should get both better time and space efficiency.

> 2. In addition to #1, using a different compression parameters for
> the SquashFS filesystem on the installation medium -- has the suffix
> plain-xz-1M.iso

And this ones trades times for space. Considering that the space
difference is only ~50 MB, I don't think it's worth it, for all the
reasons outlined before.

You haven't really answered the "why" part: why is it so important to
save 50MB? And why is the effect on QA less important?

Zbyszek


> I propose to apply both of optimizations, using the highest possible
> compression ratio supported by SquashFS. The highest compression
> ratio in SquashFS corresponds to xz level 1 (out of 9 available
> presets).
> 
> Making the first change will reduce both x86-64 Boot and the x86-64
> DVD ISO by approximately 24 MiB. With both changes combined, the
> reduction of size will be 70 MiB.
> 
> Because the SquashFS filesystem on the Live installation medium is
> of bigger size, storage savings on the Live images are even higher
> (I documented it in
> https://fedoraproject.org/wiki/Changes/OptimizeSquashFS)
> 
> On 05/09/2020 12:43, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Aug 27, 2020 at 11:13:26AM -0400, Ben Cotton wrote:
> > > https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
> > ...
> > > Based on the results above, I'd suggest selecting the following
> > > ''optimal configuration'': XZ algorithm, with block size of 1MiB and
> > > without BCJ filter (plain xz -b 1M, without -Xbcj x86).
> > > On the right, you can see the impact of the compression algorithms on
> > > installation time.
> > > 
> > > As can be seen from the picture on the right hand side, selecting
> > > 'plain xz -b 1M configuration' has minimal impact on the installation
> > > time and CPU usage during the installation. The compression will
> > > result in +6.51% or, in real terms, +24.94s additional installation
> > > time, compared to the savings of 142 MiB on the installation media,
> > > == Benefit to Fedora ==
> > > * Reduction of the installation media size and the cost of storing and
> > > distributing Fedora.
> > > * Reduction of the CPU usage at build time. Depending on which
> > > compression parameters chosen.
> > Hi Bohdan,
> > 
> > I think there's a misalignment of priorities.
> > 
> > My evaluation is the following: users won't care. The image size difference
> > is not big enough for people to notice. OTOH, slower installation will
> > impact QA and VM installations. We're doing more and more automated
> > installations and tests, and this change impacts those tests negatively.
> > 
> > > This increase in installation time will be compensated by the change
> > > in the installer:https://github.com/rhinstaller/anaconda/pull/2292
> > This PR is very interesting. But it seems that the more we optimize
> > things, the more slow decompression will be noticable. I.e. in some
> > way, right now, the slow decompression is obscured by the slow IO
> > speed, multiple levels of block device, or slow processing of the
> > uncompressed data. Any time the input or output speed is improved,
> > slow compression will be more of a bottleneck. So the approach of
> > increasing XZ compression levels has bad perspectives.
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



[prev in list] [next in list] [prev in thread] [next in thread]