[prev in list] [next in list] [prev in thread] [next in thread]
List: wikitech-l
Subject: [Wikitech-l] [IMPORTANT] Announcing Toolforge Debian Stretch Grid Engine deprecation
From: Seyram Komla Sapaty <ssapaty () wikimedia ! org>
Date: 2022-02-15 16:42:15
Message-ID: CAJniVEn6ud1EGytEdXYtL6h3gJGmz5MxRmSGn3272evD51Ktrg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hello, all!
This email contains valuable information about the Toolforge service.
Starting today, we're initiating a process to migrate away from Debian
Stretch to Debian Buster for all of Toolforge servers, and the most
affected piece is the Grid Engine backend in particular.
Debian Stretch was released in June 2017, and long term support for it
(including security updates) will cease in June 2022. We need to shut
down all Stretch hosts before the end of support date to ensure that
Toolforge remains a secure platform. This migration will take several
months because many people still use the Stretch hosts and our users
are working on tools in their spare time.
You should be aware that our ultimate goal is to deprecate Grid Engine
entirely and replace it with Kubernetes. Read below for more information
on this.
== Initial timeline ==
Subject to change, see Wikitech[1] for living timeline.
* 2022-02-15: Availability of Debian Buster grid announced to community
* 2022-03-21: Weekly reminders via email to tool maintainers for tools
still running on Stretch
* Week of 2022-04-21:
** Daily reminders via email to tool maintainers for tools still running on
Stretch
** Switch login.toolforge.org to point to Buster bastion
* Week of 2022-05-02: Evaluate migration status and formulate plan for
final shutdown of Stretch grid
* Week of 2022-05-21: Shut down Stretch grid
== What is changing? ==
* New bastion hosts running Debian Buster with connectivity to the new job
grid
* New versions of PHP, Python3, and other language runtimes
* New versions of various support libraries
== What should I do? ==
You should migrate your Toolforge tool to a newer environment.
You have two options:
* migrate from Toolforge Stretch Grid Engine to Toolforge Kubernetes[3].
* migrate from Toolforge Stretch Grid Engine to Toolforge Buster Grid
Engine.
The Cloud Services team has created the Toolforge Stretch
deprecation[0] page on wikitech.wikimedia.org to document basic steps
needed to move web services, cron jobs, and continuous jobs from the
old Stretch grid to the new Buster grid. That page also provides more
details on the language runtime and library version changes and will
provide answers to common problems people encounter as we find them.
If the answer to your problem isn't on the wiki, ask for help using
any of our communication channels[2].
We encourage you to move to Kubernetes today if you can, see below for
more details.
For those who can't migrate to Kubernetes, the Debian Buster grid should
be adopted within the next three months.
== A note on the future of Toolforge, the Grid and Kubernetes ==
As of today, Toolforge is powered by both Grid Engine and Kubernetes.
For a number of reasons, we have decided to deprecate Grid Engine and
replace all of its functions with Kubernetes. We're not yet ready to
offer all grid-like features on Kubernetes, but we're working on it.
As soon as we are able, we will begin the process of migrating the
workloads and shutting down the grid. This is something we hope to do
between 2022 and 2023.
We share this information to encourage you to evaluate migrating your
tool away from Grid Engine to Kubernetes.
One of the most prominent missing features on Kubernetes was a friendly
command line interface to schedule jobs (like jsub). We've been working
on that, and have a beta-level interface that you can try today: the
Toolforge jobs framework [4].
[0]: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation
[1]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Timeline
[2]:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Communication_and_support
[3]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes
[4]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs-Framework
Thanks.
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
[Attachment #5 (text/html)]
<div dir="ltr"><div>Hello, all!</div><div><br></div>This email contains valuable \
information about the Toolforge service.<br><br>Starting today, we're initiating \
a process to migrate away from Debian<br>Stretch to Debian Buster for all of \
Toolforge servers, and the most<br>affected piece is the Grid Engine backend in \
particular.<br><br>Debian Stretch was released in June 2017, and long term support \
for it <br>(including security updates) will cease in June 2022. We need to \
shut<br>down all Stretch hosts before the end of support date to ensure \
that<br>Toolforge remains a secure platform. This migration will take \
several<br>months because many people still use the Stretch hosts and our \
users<br>are working on tools in their spare time.<br><br>You should be aware that \
our ultimate goal is to deprecate Grid Engine<br>entirely and replace it with \
Kubernetes. Read below for more information<br>on this.<br><br>== Initial timeline \
==<br>Subject to change, see Wikitech[1] for living timeline.<br><br>* 2022-02-15: \
Availability of Debian Buster grid announced to community<br>* 2022-03-21: Weekly \
reminders via email to tool maintainers for tools still running on Stretch<br>* Week \
of 2022-04-21:<br>** Daily reminders via email to tool maintainers for tools still \
running on Stretch<br>** Switch <a href="http://login.toolforge.org" \
target="_blank">login.toolforge.org</a> to point to Buster bastion<br>* Week of \
2022-05-02: Evaluate migration status and formulate plan for final shutdown of \
Stretch grid<br>* Week of 2022-05-21: Shut down Stretch grid<br><br>== What is \
changing? ==<br>* New bastion hosts running Debian Buster with connectivity to the \
new job grid<br>* New versions of PHP, Python3, and other language runtimes<br>* New \
versions of various support libraries<br><br>== What should I do? ==<br>You should \
migrate your Toolforge tool to a newer environment. <br>You have two options:<br>* \
migrate from Toolforge Stretch Grid Engine to Toolforge Kubernetes[3].<br>* migrate \
from Toolforge Stretch Grid Engine to Toolforge Buster Grid Engine.<br><br>The Cloud \
Services team has created the Toolforge Stretch<br>deprecation[0] page on <a \
href="http://wikitech.wikimedia.org" target="_blank">wikitech.wikimedia.org</a> to \
document basic steps<br>needed to move web services, cron jobs, and continuous jobs \
from the<br>old Stretch grid to the new Buster grid. That page also provides \
more<br>details on the language runtime and library version changes and \
will<br>provide answers to common problems people encounter as we find them.<br>If \
the answer to your problem isn't on the wiki, ask for help using<br>any of our \
communication channels[2].<br><br>We encourage you to move to Kubernetes today if you \
can, see below for<br>more details.<br><br>For those who can't migrate to \
Kubernetes, the Debian Buster grid should<br>be adopted within the next three \
months.<br><br>== A note on the future of Toolforge, the Grid and Kubernetes ==<br>As \
of today, Toolforge is powered by both Grid Engine and Kubernetes. <br>For a number \
of reasons, we have decided to deprecate Grid Engine and<br>replace all of its \
functions with Kubernetes. We're not yet ready to<br>offer all grid-like features \
on Kubernetes, but we're working on it.<br>As soon as we are able, we will begin \
the process of migrating the<br>workloads and shutting down the grid. This is \
something we hope to do<br>between 2022 and 2023.<br><br>We share this information to \
encourage you to evaluate migrating your<br>tool away from Grid Engine to \
Kubernetes.<br><br>One of the most prominent missing features on Kubernetes was a \
friendly<br>command line interface to schedule jobs (like jsub). We've been \
working<br>on that, and have a beta-level interface that you can try today: \
the<br>Toolforge jobs framework [4].<br><br>[0]: <a \
href="https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation" \
target="_blank">https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation</a><br>[1]: \
<a href="https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Timeline" \
target="_blank">https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Timeline</a><br>[2]: \
<a href="https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Communication_and_support" \
target="_blank">https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Communication_and_support</a><br>[3]: \
<a href="https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes" \
target="_blank">https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes</a><br>[4]: \
<a href="https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs-Framework" \
target="_blank">https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs-Framework</a> \
<br><div><br></div><div>Thanks.</div><div></div><div><br></div>-- <br><div dir="ltr" \
data-smartmail="gmail_signature"><div dir="ltr">Seyram Komla Sapaty<div>Developer \
Advocate</div><div>Wikimedia Cloud Services</div></div></div></div>
_______________________________________________
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic