[prev in list] [next in list] [prev in thread] [next in thread]
List: beowulf
Subject: Re: [Beowulf] [EXTERNAL] Re: RIP CentOS 8
From: "Lux, Jim \(US 7140\) via Beowulf" <beowulf () beowulf ! org>
Date: 2020-12-12 16:15:48
Message-ID: EEFA4BB6-D5B3-43B8-AFF2-77B06F7531DA () jpl ! caltech ! edu
[Download RAW message or body]
[Attachment #2 (text/plain)]
I agree with Carlos here – what you're basically doing is going back to the \
"monolithic build" with carefully curated libraries. Is that not what APIs and \
shareable libraries were supposed to get away from. Just because we can now ship \
100GB images around conveniently doesn't mean it's a good idea.
Sure, I get it as a practical thing. The nirvana of infinitely multiversion \
intercompatible libraries is probably never to exist – and users "just want it to \
run" so the safest strategy is send "this image works on this model machine, buy that \
model, and we'll support it". And if you're a big enough dog, you can tell people \
that and they will buy your product – because they have to (especially if you've \
managed to do some "regulatory capture" a'la Ma Bell)
And this is not really a HPC thing – it's everywhere – Trying to set up a build \
environment for an embedded processor or FPGA is like this. In theory, you *could* \
download all the pieces, figure out the compile switches and symbols that need to be \
set (because, sure, the autoconfigure always works perfectly), recompile and build. \
Or, you can download the "known to work set of executables for Ubuntu 18.04 LTS" and \
get on with your life. (and set up multiple boot environments, because, yeah, you \
still need to support that system that was built with 16.04 LTS 4 years ago)
And I certainly don't want to go back to the early-mid 2000s where "which version of \
glibc" was the question of the day when doing builds.
From: Beowulf <beowulf-bounces@beowulf.org> on behalf of Carlos Bederián \
<carlos.bederian@unc.edu.ar>
Date: Saturday, December 12, 2020 at 12:36 AM
To: Douglas Eadline <deadline@eadline.org>
Cc: "beowulf@beowulf.org" <beowulf@beowulf.org>, Chris Samuel <chris@csamuel.org>
Subject: [EXTERNAL] Re: [Beowulf] RIP CentOS 8
I'm going off on a tangent here, but how is shipping an entire distribution for a \
single application something good? Many things have failed for us to get to this \
point where such a brute force approach makes sense, and nobody wants to tackle the \
underlying problems. On Fri, Dec 11, 2020, 3:57 PM Douglas Eadline \
<deadline@eadline.org<mailto:deadline@eadline.org>> wrote:
Now jump ahead to containers and HPCng \
(https://hpcng.org/<https://urldefense.us/v3/__https:/hpcng.org/__;!!PvBDto6Hs4WbVuu7!aUR5FTCW7oK-aG1cotqvcL7oXai5SsalvzOMh5rl-4Yi0-tY3-RkXWe3ebpg0t0-b-FFEl4$>)
An open source project will release a container that "contains"
everything thing it needs to run (along with the container recipe)
Using Singularity you can also sign the container to assure
provenance of the code. The scheduler runs containers. Simple.
Software Vendors will gladly do the same. Trying to support
multiple distribution goes away. Applications show up in
tested containers. The scheduler runs containers. Things just work,
less support issues for the vendor. Simple.
The need to maintain library version trees and Modules for
goes away, Of course if are developer writing your own application,
you need specific libraries, but not system wide. Build the
application in your working directly, include any specific libraries
you need in the local source tree and fold it all into a container.
Joe Landman also comments on this topic in his blog (does not seem
to be showing up for me today, however)
https://scalability.org/2020/12/the-future-of-linux-distributions-in-the-age-of-docker \
-and-k8s/<https://urldefense.us/v3/__https:/scalability.org/2020/12/the-future-of-linu \
x-distributions-in-the-age-of-docker-and-k8s/__;!!PvBDto6Hs4WbVuu7!aUR5FTCW7oK-aG1cotqvcL7oXai5SsalvzOMh5rl-4Yi0-tY3-RkXWe3ebpg0t0-K5VkGvo$>
Bottom line, it is all good, we are moving on.
--
Doug
[Attachment #3 (text/html)]
<html xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I agree with Carlos here – what you're basically doing is \
going back to the "monolithic build" with carefully curated libraries. Is that not \
what APIs and shareable libraries were supposed to get away from. Just because \
we can now ship 100GB images around conveniently doesn't mean it's a good idea.<br>
<br>
Sure, I get it as a practical thing. The nirvana of infinitely multiversion \
intercompatible libraries is probably never to exist – and users "just want it to \
run" so the safest strategy is send "this image works on this model machine, buy that \
model, and we'll support it". And if you're a big enough dog, you can tell \
people that and they will buy your product – because they have to (especially if \
you've managed to do some "regulatory capture" a'la Ma Bell)<o:p></o:p></p> <p \
class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">And this is not really a \
HPC thing – it's everywhere – Trying to set up a build environment for an \
embedded processor or FPGA is like this. In theory, you *<b>could</b>* download \
all the pieces, figure out the compile switches and symbols that need to be set \
(because, sure, the autoconfigure always works perfectly), recompile and build. Or, \
you can download the "known to work set of executables for Ubuntu 18.04 LTS" and get \
on with your life. (and set up multiple boot environments, because, \
yeah, you still need to support that system that was built with 16.04 LTS 4 years \
ago)<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And I certainly don't want to go back to the early-mid 2000s \
where "which version of glibc" was the question of the day when doing \
builds.<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: \
</span></b><span style="font-size:12.0pt;color:black">Beowulf \
<beowulf-bounces@beowulf.org> on behalf of Carlos Bederián \
<carlos.bederian@unc.edu.ar><br> <b>Date: </b>Saturday, December 12, 2020 at \
12:36 AM<br> <b>To: </b>Douglas Eadline <deadline@eadline.org><br>
<b>Cc: </b>"beowulf@beowulf.org" <beowulf@beowulf.org>, Chris Samuel \
<chris@csamuel.org><br> <b>Subject: </b>[EXTERNAL] Re: [Beowulf] RIP CentOS \
8<o:p></o:p></span></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I'm going off on a tangent here, \
but how is shipping an entire distribution for a single application something good? \
Many things have failed for us to get to this point where such a brute force approach \
makes sense, and nobody wants to tackle the underlying problems.<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">On Fri, Dec 11, 2020, 3:57 PM Douglas Eadline <<a \
href="mailto:deadline@eadline.org" target="_blank">deadline@eadline.org</a>> \
wrote:<o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <p class="MsoNormal" \
style="margin-bottom:12.0pt"><br> Now jump ahead to containers and HPCng (<a \
href="https://urldefense.us/v3/__https:/hpcng.org/__;!!PvBDto6Hs4WbVuu7!aUR5FTCW7oK-aG1cotqvcL7oXai5SsalvzOMh5rl-4Yi0-tY3-RkXWe3ebpg0t0-b-FFEl4$" \
target="_blank">https://hpcng.org/</a>)<br> <br>
An open source project will release a container that "contains"<br>
everything thing it needs to run (along with the container recipe)<br>
Using Singularity you can also sign the container to assure<br>
provenance of the code. The scheduler runs containers. Simple.<br>
<br>
Software Vendors will gladly do the same. Trying to support<br>
multiple distribution goes away. Applications show up in<br>
tested containers. The scheduler runs containers. Things just work,<br>
less support issues for the vendor. Simple.<br>
<br>
The need to maintain library version trees and Modules for<br>
goes away, Of course if are developer writing your own application,<br>
you need specific libraries, but not system wide. Build the<br>
application in your working directly, include any specific libraries<br>
you need in the local source tree and fold it all into a container.<br>
<br>
Joe Landman also comments on this topic in his blog (does not seem<br>
to be showing up for me today, however)<br>
<br>
<a href="https://urldefense.us/v3/__https:/scalability.org/2020/12/the-future-of-linux \
-distributions-in-the-age-of-docker-and-k8s/__;!!PvBDto6Hs4WbVuu7!aUR5FTCW7oK-aG1cotqvcL7oXai5SsalvzOMh5rl-4Yi0-tY3-RkXWe3ebpg0t0-K5VkGvo$" \
target="_blank">https://scalability.org/2020/12/the-future-of-linux-distributions-in-the-age-of-docker-and-k8s/</a><br>
<br>
Bottom line, it is all good, we are moving on.<br>
<br>
-- <br>
Doug<o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</body>
</html>
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit \
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
--===============9043849865508678847==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic