[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fedora-devel-list
Subject:    Re: Small rant: installer environment size
From:       Adam Williamson <adamwill () fedoraproject ! org>
Date:       2022-12-10 16:48:56
Message-ID: 1ba5e43b20eca996b9f1a4cebd8352c75b808573.camel () fedoraproject ! org
[Download RAW message or body]

On Sat, 2022-12-10 at 11:59 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> On Fri, Dec 09, 2022 at 03:08:19PM -0700, Chris Murphy wrote:
> > 
> > 
> > On Fri, Dec 9, 2022, at 7:30 AM, Ray Strode wrote:
> > > Hi,
> > > 
> > > On Thu, Dec 8, 2022 at 2:55 PM Adam Williamson
> > > <adamwill@fedoraproject.org> wrote:
> > > > This is the direction Daniel was thinking down. I'm waiting for someone
> > > > with more expertise to reply, but I suspect the reply is going to be
> > > > along the lines of "yes, we *can* do that, but it's somewhat tricky
> > > > work that involves thinking about lots of paths that aren't obvious,
> > > > and somebody would need to dedicate their time to working on that".
> > > Presumably we could package the firmware separately and just unpack it
> > > into place from a udev rule when the hardware is detected?
> > > 
> > > But first, do we actually know this is a problem?
> > > I think you're saying squashfs loads the whole decompressed image into
> > > memory, but my expectation prior to your mail was that it performs I/O
> > > on the usb stick (with a cache in between).  If my intuition was right
> > > and files only hit ram when accessed, then it seems like this is
> > > pretty much not an issue, right?
> > 
> > From a certain point of view there's a potential inefficiency with squashfs reads \
> > in that there's a minimum block size that it needs to read in order decompress \
> > its 128 KiB block. It's possible quite a lot of what's decompressed isn't \
> > (immediately) needed. But it's still a random access file system. It's not \
> > necessary to read the whole image into RAM. 
> > Repo metadata is the big hit for netinstall because it's downloaded into /tmp \
> > which is tmpfs. And DVD already has repomd on it, and only downloads more if you \
> > enable some other repo. Live doesn't need repomd. 
> > So initially netinstaller uses less memory up until the the Anaconda language \
> > selection screen appears, at which point it starts background downloading repomd. \
> > It quickly catches up to, and surpasses, Live media memory consumption.
> 
> We *could* do something about repo metadata: only install the "main" metadata,
> and not the "filepath" metadata. This would reduce the metadata size by ~80%.
> It'd also have huge benefits for speed: on small dnf operations a significant
> portion of time is spent just loading metadata.
> 
> We had some discussions on this a few years ago, but this never went anywhere.
> (I did a quick search, but can't find the ticket now.)  But the idea was that
> dnf would load "main" metadata by default, and then load "filepath" metadata
> if needed and restart the transaction. Our Packaging Guidelines forbid filepath
> dependencies (except for a small set of directories which are in the "main" part),
> so normal rpm transactions don't need the "filepath" metadata. It is only
> needed for non-confirming rpms and when the user does something like
> 'dnf install /some/strange/path'.

AIUI this is already a part of dnf5.
-- 
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
 Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


[prev in list] [next in list] [prev in thread] [next in thread]