[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensolaris-storage-discuss
Subject:    [storage-discuss] iscsi / CR6497777 : What should become a tunable?
From:       Nils Goroll <nils.goroll () hamburg ! de>
Date:       2008-08-23 10:09:45
Message-ID: 6592922.2761219511416727.JavaMail.Twebapp () sf-app1
[Download RAW message or body]

Hi,

quite some time ago in july I had exanged some emails with
Zhang.Yi@Sun.COM regarding

      CR6497777: iscsi initiator: Make default connection retry duration configurable

I had prepared a little patch to make those iscsi parameters tunable
which I though really should be adjustable. At the time, I was working
on a project where a customer actually needed to adjust some timers to
make the iscsi initiator more tolerant when the the target is
temporarily unavailable.

The patch I have prepared is available on
http://cr.opensolaris.org/~nigoroll/webrev_iscsi_tunables/ . This is
against the latest available nws tarball (nws-src-20080623). I am
aware that nws is to be integrated into onnv (or is it already? -
havn't checked), but I would expect the patch to work on onnv-gate as
well. The patch also includes some changes to get a more consistent
naming of the #defines and tunables used.

Because Zhang Yi and me did not find an agreement on what should be
tunable and what shouldn't, I would like to ask for the communities
opinions. ZhangYi agreed on doing so. Here are the relevant parts of
our discussion:

ZhangYi:

   Currently our situation is that we had prepared a prototype for
   demo propose, Here the main headache issue is not from the
   technology side, but some other issue such as easy management etc
   to make into consideration.

   In our internal discussion, some colleagues do not agree that we
   add this parameter tunable, cause this would make configure things
   much complex, and this tunable parameter need iscsi administrator
   have a deep knowledge of our iscsi initiator. also this issue could
   be avoid or there are always some workaround for it when this issue
   happens. So opposite side's opinion is not change anything for this
   CR. I am now under this dilemma situation.

Nils:

   I understand the dilemma and this is a typical question: Are you
   absolutely sure that you have chosen the best possible values so
   you don't want to give the end user (in this case rather: the
   administrator) a choice, or is there a chance that some parameters
   need adjustment for different use cases?

   My experience from 10yrs with Solaris (where I have come across the
   most annoying issues when working in HA environments with Sun
   Cluster) is that one should always provide runtime configuration
   options. Most likely, none of us will foresee the environments
   customers will use code in, so I am convinced that it is always a
   good idea to provide flexibility. To make this very transparent:
   Imagine an admin who needs to handle a downtime of an important
   enterprise system. He/she could solve the problem by adjusting some
   parameter, but if that one is not adjustable at all, his/her hands
   are bound.

   What we had so far in iscsi was exactly this: compile time defines,
   iow, hardcoded timing.

   The argument that tunables make configuration more complex is not
   sound, IMHO, as proper default values will give you exactly the
   same behavior as with hardcoded values for the default case.

   The goal, IMHO should be to provide flexible configuration options
   via iscsiadm, with scope per target, per connection, per session
   and/or per LUN, whatever is most appropriate.

   In the meantime, I am absolutely convinced that some flexibility is
   better than none at all.

   Please do also consider that kernel tunables were never meant to be
   used by inexperienced uses. Whoever tunes them, *always* needs at
   least some understanding of the internal design. This is precisely
   being stated in the Solaris Tunable Parameters Reference Guide:

    Tuning a Solaris System

    [...]

    Another thing to remember is that one system's /etc/system
    settings might not be applicable, either wholly or in part, to
    another environment. Carefully consider the values in the file
    with respect to the environment in which they will be
    applied. Make sure that you understand the behavior of a system
    before attempting to apply changes to the system variables
    described here.

    Caution – Caution –

    The variables described here and their meanings can and do change
    from release to release. A release is either a Solaris Update
    release or a new version such as Solaris 9. Publication of these
    variables and their description does not preclude changes to the
    variables and descriptions without notice.

   And, in this case, we are not even talking about "official"
   tunables, but rather undocumented ones. ;-)

ZhangYi:

   I am afraid that I can not fulfill your requirement. 

   If you want all those parameters tunable, I do not think it is
   prepared for end user, but more likely a prototype for test
   propose. Some of parameters in your patch maybe very useful for end
   users and administrator, but not all are useful. Some parameters
   are exposed to users to hack the system, and maybe lead security
   issues. I am sure even experts that familiar with iSCSI initiator
   do not know what the propose of those tunable parameters unless he
   or she had a deep knowledge of solaris and iscsi and thoroughly
   review the source code very carefully.

   Yes, I confessed that flexibility is better than none at all, but
   if you want me to make choice between every things tunnable and
   more clear and more manageable of iscsi. I prefer the latter.

Nils:

   Regarding your security concerns: Could you explain why you think
   that making parameters tunable poses additional risks?

   If no-one knows what these parameters mean, I believe they should
   be documented properly anyway, no matter if they are statically
   defined or tunable. Those people who have deep knowledge of the
   iscsi stack today might not even work for Sun anymore tomorrow.

ZhangYi:

   About the document, yes, it should be well formally documented.but
   not openly. As this internal parameters and interface are not open
   interface for end users, so they are meaningless to users and
   administrator.  that's reason "iscsiadm" operation manual do not
   formally documented those parameters.

   There might be security issues, but I just quote it as an
   example. concerning detailed issue. I still need a deep insight
   investigation.

Nils:


   I have got some doubts if they (timing parameters) are really
   meaningless to admins.

   Take the connection shutdown timeout, for example. Right now, this
   is 180 seconds fixed. This means: If your target is
   down/unreachable for 180 secs, you'll get hard I/O errors on your
   filesystem or application, whatever is using the respective iSCSI
   LUN.

   In HA environments, you will most likely want to tune this
   parameter to almost "never" yield hard errors, but rather only
   block I/O for extended periods.

   Or take connection login max: You might want to have a login fail
   very quickly and not wait for 180 secs. Why not allow admins to
   change it?

   Sorry to say, but aren't all of these parameters chosen quite
   arbitrarily? Why are 0, 5, 10, 60 and 180 seconds the best possible
   values for the various parameters? 1,6,11,47 and 130 could be much
   better, how can you know?

   (Regarding security issues)

   OK, but just because security concerns always need to be considered
   they are not a default argument for making things not tunable.

ZhangYi:

   As your requested tunable parameters, yes, I agree that we should
   make changes to current iSCSI initiator. and also I can understand
   your current situation that you want all parameters open to you so
   that you can make your test easily. but not so much.

   Two reasons:

   - Reason to make so many changes are not strong.

     We do need open some of parameters tunable , but not all. your
     requirement to make all those change are not so strong, you need
     an urgency or critical reason to persuade us to make so many
     parameters tunable.

   - Using other workaround avoid timeout issues.

     When encounter some iSCSI timeout issues, using tunable
     parameters maybe one method to solve it, but we can avoid it or
     find other workaround to solve the same problem.

   You ask me why those parameters chosen as fixed value. I had no
   answer for you. as I also do not know why it is designed as those
   value. maybe experimental value from experience.

   It is a good idea to discuss this question on opensolaris
   forum. Hopefully we can find some new ideas or get some comprised
   agreement after all guys joining this discuss thread.


So, now it's the community's turn: What do you think?

Nils
 
 
This message posted from opensolaris.org
_______________________________________________
storage-discuss mailing list
storage-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss

[prev in list] [next in list] [prev in thread] [next in thread]