[prev in list] [next in list] [prev in thread] [next in thread] 

List:       npaci-rocks-discussion
Subject:    Re: [Rocks-Discuss] rocks 7 :: thoughts for the far future :)
From:       Adrian Sevcenco <Adrian.Sevcenco () cern ! ch>
Date:       2013-12-27 14:50:12
Message-ID: 52BD93A4.3030206 () cern ! ch
[Download RAW message or body]

On 12/27/2013 02:36 PM, Roy Dragseth wrote:
> I'm not sure I'm the right person to answer this, but it's christmas 
> holiday and I've got nothing better to do... (I'm a long term Rocks
Hi Roy! Well, to be honest i had the same reasons to send this email ..
just for a light talk of ideas in holidays (without the stress of
everyday work :) )

> user and the maintainer of the torque-roll.  Some of the details in
I am most grateful (and familiar) to your work and the work of guys from
Rocks clusters, my institute (and myself) using this for almost 10 years
already :)

> the answers might not be accurate as Rocks being a "it just works"
> solution has made me very lazy... )
:) yeap, good technology have this effect on people :D .. IMHO the best
technology is "from lazy people to even more lazy people" :D .. so why
not to try to be even more lazy? :)

> On Mon 23 Dec 2013 06:39:24 PM CET, Adrian Sevcenco wrote:
>> Hi! I was wondering if there are already plans for the far future
>> of rocks 7 ... related to this i would have a few of
>> ideas/questions: 1. Visualization/cloud : given that now openstack
>> is used now in scientific community (and is widely supported and
>> used at CERN) what are the developers and community thoughts on
>> using the openstack for the visualization/cloud part of rocks? (i
>> imagine that would be easier that to use in house developed scripts
>> and software)
> 
> Well, if you want to run a display wall then openstack (or any other 
> cloud solution) isn't going to solve the problem of driving the 
> graphics, you need a physical video connection to the screens.  This
> is where the viz-roll really shines, take a look at 
> http://youtu.be/8bHWuvzBtJo
err, no, i am not familiar with the "display wall" technology needs and
requirements.

> For simple remote desktop stuff cloud might be it, but for our part
> we do remote desktops because users cannot possible move TBs of data
> to their laptop or tablets so the vnc-servers need to sit on the
> high-speed internal network on our cluster to get a reasonable data
> throughput.
when i mentioned "cloud" i was thinking of IaaP clusters (like from the
main frontend to just instantiate a cluster (or more)) (like rocks
already can do it but based on internal developed software)

>> 2. I see that in Redhat 7 there is an interesting piece of
>> software named Open LMI (for me was the first time to hear about
>> it). From what i see here [1], [2] and [3] it is a solution for
>> host management similar with what Rocks has (and better than
>> ansible). So, in the interest of having rocks developers less work
>> to do, would this be a good replacement of the various rocks
>> system(s)? (maybe coupled with cobbler for the system distribution
>> part + puppet for configuration?)?
> 
> OpenLMI might be a solution for the future, but I would sit on the
> fence to see if it really takes off and gets adapted by multiple
> distros.  If RedHat abandons kickstart in favour of LMI I guess Rocks
> will have to follow suit.
well, if it is adopted by rh than will be supported for the entire life
cycle i imagine..
and kickstart is something completely different then openlmi as
kickstart just configure the machine at install time, but openlmi
manages the machine during the usage (post-install)

> One advantage of the xml-tree setup in Rocks is that it is very 
> scalable, it just hands out everything to the node and gives it the
> task to figure out for itself how the kickstart should look like.
i agree! i dont know what and how is better, i was curious about peoples
opinions.. (and plans :) )

>> 3. minor ideas : 3.1 wouldn't be useful to have for gmond
>> configuration rocks verbs for each setting?
> 
> Not quite sure I understand.  Do you have an idea how this should
> look from a command line perspective?
well, i was thinking about:

rocks set gmond globals {each acceptable gmond settings} "string"
similar for cluster and host

rocks add gmond channel tcp {bind, port, interface, family, timeout}
"string"
rocks add gmond channel udp send {mcast_join, mcast_if, host, port, ttl}
"string"
rocks add gmond channel udp recv {mcast_join, bind, port, mcast_if,
family} "string"

rocks set gmond channel tcp/udp acl default "string"
rocks add gmond channel tcp/udp acl ip="string" mask="string"
action="string"
rocks remove gmond channel tcp/udp acl ip="string"
rocks list gmond acl

rocks list gmond channel // list all channels

rocks add/remove gmond modules module="name_string"
rocks set gmond module="name_string" {language, enabled, path} = "string"
rocks set/remove gmond module="name_string" param="string" value="string"

rocks add/remove gmond include path="string"

well, at this point i was curious if this would need something useful to
everyone..

>> 3.2 similar for torque client configuration?
> 
> Ditto. The torque roll has a notion of node attributes, but that's 
> mostly for the server-side config of the nodelist.  I'm happy to
> take suggestions.
well, it would be nice to be able to set per node and membership the
settings of pbs_mom
http://docs.adaptivecomputing.com/torque/Content/topics/12-appendices/parameters.htm
imho it would be useful to control params like tmpdir,usecp,
$remote_reconfig, $rcpcmd but the others too ..
something like :
rocks set/remove host <host> torque_client param="string" value="string"
rocks set/remove membership <membership> torque_client param="string"
value="string"

(the list of params is fixed - only valid param names are accepted)

rocks list host/membership <name> torque_client config

with the settings of the host overriding and adding the ones from membership

and rocks sync config torque_mom

would something like this be useful for everyone? (for me it would :) )

also i imagine that equivalent for
http://docs.adaptivecomputing.com/torque/Content/topics/12-appendices/torque.cfgConfigFile.htm
could be of use too .. (i used torque.cfg only for submit filter path
customization)

>> 3.3 it is in my understanding that distributing the rpms for os 
>> installation over nfs is faster than over http. what pro and
>> contras would be for having the rpms distributed over ro nfs dir?
> 
> Rocks uses BitTorrent for distributing the rpms, it wraps anaconda
> which thinks it is using http.  This is really a strike of genius in
> Rocks. We have installed 700 compute nodes simultaneously from one
> puny little frontend with hardly any IO capability at all.  All nodes
> that install at the same time share the rpm-files over BitTorrent.
> Looking at the http-logs on the frontend we found that it handed out
> 9000 rpms and 500 000 torrents during the install process.  The
> torrent files are just a couple of hundred bytes and fit within one
> single tcp-packet, a normal gigabitcard can deliver more than 30 000
> packets a second.  We found that the real bottleneck was the delivery
then already this is as good as it can get!

> of the install kernel and initrd when booting from pxe, it easily
> maxed out the network interface on the frontend.  I'm not sure if it
> possible to use nfs instead of tftp as transport mechanism for
> handing out the install images, if it is it might be possible to
> shave off some overhead.
no, pxe is still tftpd only :(

>> 3.4 talking about nfs : would be possible (and useful) to have
>> rocks verbs that would modify and add nfs mount points for the
>> compute nodes?
> 
> Doesn't autofs already do that?  The autofs configs are distributed
> by 411 so you can have a very dynamic config, but it might not be
> very targeted as it will replicate on all nodes.
i was thinking about static mount points not the autofs ones ...
but you are right, per host and per membership configs could be useful too..

Thanks,
Merry Christmas everyone!
Adrian


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2272 bytes
Desc: S/MIME Cryptographic Signature
Url : https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20131227/c49e36d5/smime.p7s 

[prev in list] [next in list] [prev in thread] [next in thread]