[prev in list] [next in list] [prev in thread] [next in thread]
List: gentoo-dev
Subject: Re: Re[2]: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
From: Thomas Gall <thomasagall () gmail ! com>
Date: 2024-03-31 1:41:05
Message-ID: CA+O5tps8Eb0PGBiDcSwmYoh7cZeC51noF_8T4uXhj+SqFkZoPQ () mail ! gmail ! com
[Download RAW message or body]
A decompression implementation all in rust it would seem.
https://github.com/gendx/lzma-rs
On Sat, Mar 30, 2024 at 12:36 PM Eddie Chapman <eddie@ehuk.net> wrote:
> Stefan Schmiedl wrote:
> > ------ Original Message ------
> >
> >> From "Eddie Chapman" <eddie@ehuk.net>
> >>
> > To gentoo-dev@lists.gentoo.org
> > Date 30.03.2024 16:17:19
> > Subject Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
> >
> >> Michał Górny wrote:
> >>
> >>> On Sat, 2024-03-30 at 14:57 +0000, Eddie Chapman wrote:
> >>>
> >>>
> >>>> Note, I'm not advocating ripping xz-utils out of tree, all I'm
> >>>> saying is wouldn't it be nice if there were at least 2 alternatives
> >>>> to choose from? That doesn't have to be disruptive in any way,
> >>>> people who wish to continue using and trusting xz-utils should be
> >>>> able to continue to do so without any friction whatsoever.
> >>>
> >>> So, you're basically saying we should go out of our way, recompress
> >>> all distfiles using two alternative compression formats, increase
> >>> mirror load four times and add a lot of complexity to ebuilds, right?
> >>>
> >>> --
> >>> Best regards,
> >>> Michał Górny
> >>
> >> Yes that's a very good point, that was something I was wondering in
> >> weighing up both sides, what the costs would be practically, as I don't
> >> know the realities of running Gentoo infrastructure. And maybe the
> >> costs is just too high of a price to pay.
> >>
> >> I wonder if increased use of git repos rather than distributed tarballs
> >> could be part of a solution to those issues, although that could put
> >> quite a storage burden on every user. Unless they were all shallow git
> >> pulls and the user could optionally choose to tar up the git directory
> >> after clone with compression. But yes granted then there is even more
> >> ebuild complexity.
> >>
> > Huh ... I read your original message as
> >
> > "wouldn't it be nice to have at least 2 alternative [implementations of
> > xz-utils] to choose from"
> >
> > As long as the file format itself is not inherently messed up, the
> > archives could stay as .xz, only a "minimal" unxz (similar to unrar)
> would
> > be required to access the contents.
> >
> > Regards,
> > s.
>
> I see, no, I originally meant to have two compression/decompression
> formats; LZMA (xz) and another. But yes the way you understood it is
> interesting. Initially I dismissed this idea as would take too long for a
> new tool to reach stability. But yes what you suggest is it could be a
> very simple implementation that only does decompression so perhaps more
> realistically achievable. Still a tall order. I wonder if any general
> purpose languages (python, perl, etc) have developed their own built in
> LZMA decompression implementation? I doubt it, would have thought they'd
> just link against liblzma.so and not re-invent the wheel.
>
>
>
[Attachment #3 (text/html)]
<div dir="auto">A decompression implementation all in rust it would seem.</div><div \
dir="auto"><br></div><div dir="auto"><div><a \
href="https://github.com/gendx/lzma-rs">https://github.com/gendx/lzma-rs</a></div><br></div><div \
dir="auto"><br></div><div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Sat, Mar 30, 2024 at 12:36 PM Eddie Chapman <<a \
href="mailto:eddie@ehuk.net">eddie@ehuk.net</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">Stefan \
Schmiedl wrote:<br> > ------ Original Message ------<br>
><br>
>> From "Eddie Chapman" <<a href="mailto:eddie@ehuk.net" \
target="_blank">eddie@ehuk.net</a>><br> >><br>
> To <a href="mailto:gentoo-dev@lists.gentoo.org" \
target="_blank">gentoo-dev@lists.gentoo.org</a><br> > Date 30.03.2024 16:17:19<br>
> Subject Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo<br>
><br>
>> Michał Górny wrote:<br>
>><br>
>>> On Sat, 2024-03-30 at 14:57 +0000, Eddie Chapman wrote:<br>
>>><br>
>>><br>
>>>> Note, I'm not advocating ripping xz-utils out of tree, all \
I'm<br> >>>> saying is wouldn't it be nice if there were at least \
2 alternatives<br> >>>> to choose from? That doesn't have to be \
disruptive in any way,<br> >>>> people who wish to continue using and \
trusting xz-utils should be<br> >>>> able to continue to do so without \
any friction whatsoever.<br> >>><br>
>>> So, you're basically saying we should go out of our way, \
recompress<br> >>> all distfiles using two alternative compression formats, \
increase<br> >>> mirror load four times and add a lot of complexity to \
ebuilds, right?<br> >>><br>
>>> --<br>
>>> Best regards,<br>
>>> Michał Górny<br>
>><br>
>> Yes that's a very good point, that was something I was wondering in<br>
>> weighing up both sides, what the costs would be practically, as I \
don't<br> >> know the realities of running Gentoo infrastructure. And maybe \
the<br> >> costs is just too high of a price to pay.<br>
>><br>
>> I wonder if increased use of git repos rather than distributed tarballs<br>
>> could be part of a solution to those issues, although that could put<br>
>> quite a storage burden on every user. Unless they were all shallow git<br>
>> pulls and the user could optionally choose to tar up the git directory<br>
>> after clone with compression. But yes granted then there is even more<br>
>> ebuild complexity.<br>
>><br>
> Huh ... I read your original message as<br>
><br>
> "wouldn't it be nice to have at least 2 alternative [implementations \
of<br> > xz-utils] to choose from"<br>
><br>
> As long as the file format itself is not inherently messed up, the<br>
> archives could stay as .xz, only a "minimal" unxz (similar to unrar) \
would<br> > be required to access the contents.<br>
><br>
> Regards,<br>
> s.<br>
<br>
I see, no, I originally meant to have two compression/decompression<br>
formats; LZMA (xz) and another. But yes the way you understood it is<br>
interesting. Initially I dismissed this idea as would take too long for a<br>
new tool to reach stability. But yes what you suggest is it could be a<br>
very simple implementation that only does decompression so perhaps more<br>
realistically achievable. Still a tall order. I wonder if any general<br>
purpose languages (python, perl, etc) have developed their own built in<br>
LZMA decompression implementation? I doubt it, would have thought they'd<br>
just link against liblzma.so and not re-invent the wheel.<br>
<br>
<br>
</blockquote></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic