[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: F34 Change: Reduce installation media size by improving the compression ratio of SquashFS filesy
From:       John Reiser <jreiser () bitwagon ! com>
Date:       2020-09-04 4:16:27
Message-ID: 48e4ce45-7363-2af5-04ae-8a171571e025 () bitwagon ! com
[Download RAW message or body]

On 2020-09-01 at 12:13 UTC, Kamil Paral wrote:
    [[snip]]
>  I'd like to ... hugely speed up the installation instead
    [[snip]]

Zstd is faster than xz at de-compression, but a much larger speed improvement
would be to parallelize and pipeline "rpm --install".  This would benefit
both install and the creation of 'mock' build environments.

Topologically sort the .rpm packages by dependency.  All packages within
a given tier (tier 0 is the packages that have no dependencies, tier 1 is
the packages that have no uninstalled dependencies after installing tier 0,
etc.) can be installed in parallel.  Even for scriptlets that might force
serialization [such as selinux], all processing before the first Write-to-
destination-filesystem can be parallel.  For any individual .rpm, the stages
of fetch the package (from network, USB flash memory, DVD, etc.), decompress,
and process the contained files, can be pipelined at the block level (4KiB to
64KiB) so that the memory cache (L3 on Intel x86_64 CPUs) suffers a Read miss
only rarely.

But I repeat myself, from 2002 (18 years ago):
    https://bugzilla.redhat.com/show_bug.cgi?id=71184
    RFE: 2X to 6X faster install via software pipeline and controlled memory footprint

Side note: Inexpensive USB flash memory storage does not have to be slow.
Reading from my 128GB flash memory device "ID 03f0:da07 HP, Inc" is competitive
with reading from a spinning harddrive:

 From a USB 3.0 port the flash drive gives:
    # dd if=/dev/sde1 of=/dev/null bs=1M count=2000
    2000+0 records in
    2000+0 records out
    2097152000 bytes (2.1 GB, 2.0 GiB) copied, 13.8432 s, 151 MB/s

 From a USB 2.0 port the flash drive gives:
    # dd if=/dev/sde1 of=/dev/null bs=1M count=2000
    2000+0 records in
    2000+0 records out
    2097152000 bytes (2.1 GB, 2.0 GiB) copied, 55.0733 s, 38.1 MB/s


_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

[prev in list] [next in list] [prev in thread] [next in thread]