[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gpfsug-discuss
Subject:    [gpfsug-discuss] GPFS diskless/stateless clients (was: Re: gpfs client and turning off swap)
From:       Ryan Novosielski <novosirj () rutgers ! edu>
Date:       2019-11-08 16:49:21
Message-ID: 229B3495-6D3D-40DF-8927-0960F751749C () rutgers ! edu
[Download RAW message or body]

We run stateless machines today with CentOS 7.x and have run GPFS 4.1.0.8, 4.2.3.x, \
and 5.0.x. We don't use swap either in most cases, and have never needed to do \
anything special for that. We've generally needed to rig ExecStartPre stuff in \
systemd in order to make sure that the machines could restore their cluster \
membership.

If anyone else can share how they do that, it might be interesting information. In \
our case, this works (our provisioning system writes these variables as the actual \
host names) — we used to put this in an override to gpfs.service; I believe \
mmautoload.service is relatively new (either in 4.2.3.x to 5.0.x or 4.1.x to 4.2.x).

[root@test mmautoload.service.d]# pwd
/etc/systemd/system/mmautoload.service.d

[root@test mmautoload.service.d]# cat override.conf 
[Unit]
Before=slurmd.service
After=sys-subsystem-net-devices-ib0.device

[Service]
ExecStartPre=/usr/lpp/mmfs/bin/mmsdrrestore -p $CLUSTERMGR -R /usr/bin/scp
ExecStartPre=/usr/lpp/mmfs/bin/mmauth genkey propagate -N $NODENAME

--
____
> > \\UTGERS,  	 |---------------------------*O*---------------------------
> > _// the State	 |         Ryan Novosielski - novosirj@rutgers.edu
> > \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
> > \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
     `'

> On Nov 8, 2019, at 11:37 AM, Damir Krstic <damir.krstic@gmail.com> wrote:
> 
> Thanks - that's what I thought. We also (many refreshes ago) ran diskless images \
> with RedHat 6 and GPFS 3.4 and ran into no issues with having swap off. I figured \
> to ask in case something has changed with the Spectrum Scale.  Damir
> 
> On Fri, Nov 8, 2019 at 10:31 AM <david_johnson@brown.edu> wrote:
> We have most of our clients network booted and diskless — no swap possible.  Gpfs \
> still works until someone runs the node out of memory....   
> -- ddj
> Dave Johnson
> 
> > On Nov 8, 2019, at 11:25 AM, Damir Krstic <damir.krstic@gmail.com> wrote:
> > 
> > 
> > I was wondering if it's safe to turn off swap on gpfs client machines? we have a \
> > case where checkpointing is swapping and we would like to prevent it from doing \
> > so by disabling swap. However, the gpfs manual admin. manual states to have swap \
> > enabled and sufficiently large, but it does not distinguish between clients and \
> > IO servers.  
> > Let me know when you get a chance.
> > 
> > Thank you.
> > Damir
> > _______________________________________________
> > gpfsug-discuss mailing list
> > gpfsug-discuss at spectrumscale.org
> > http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


[prev in list] [next in list] [prev in thread] [next in thread]