[prev in list] [next in list] [prev in thread] [next in thread]
List: net-snmp-users
Subject: Re: Python 3 snmp set operations fail with octet strings with values >= 0x80
From: Bill Fenner <fenner () gmail ! com>
Date: 2023-02-03 15:54:19
Message-ID: CAF4SogazJGK-njY96ROQWYQoRQ=26WFJqg0UFoz9WterO+Ke-g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
The API has to deal with both DisplayString (underlying type OCTETSTR) and
actual octet strings. It makes sense for a DisplayString to be utf-8
encoded, but, it doesn't make sense for actual octet strings.
Python 3 does have two different types that can very usefully be used for
this: a string could be treated as being utf-8 encoded, assuming it's a
DisplayString, and a bytes value could be treated as a series of bytes.
Unfortunately, get() would have to return bytes only, as consulting the MIB
is the only way to tell whether a given incoming OCTETSTR is a
DisplayString or not.
I don't have cycles to work on this, unfortunately.
Bill
On Thu, Feb 2, 2023 at 4:10 PM Steve Hollywood <Steve.Hollywood@chorus.co.nz>
wrote:
> Net-SNMP v5.9.3 with python 3.6.3.
>
>
>
> I'm not sure if this has only become an issue since the move of strings
> from bytes to Unicode in Python 3 but I cannot set an OID value to an octet
> string containing octets >=128.
>
> I am trying to set a MAC address in an OID.
>
>
>
> macAddress = '\x02\0\0\0\x80\xff'
>
> varbind = netsnmp.Varbind(oidSvcMacNameAddr, iid, macAddress,
> 'OCTETSTR')
>
>
>
> The python bindings convert everything to strings in the Varbind object.
>
>
>
> class Varbind(object):
>
> def __init__(self, tag=None, iid=None, val=None, type_arg=None):
>
> self.tag = STR(tag)
>
> self.iid = STR(iid)
>
> self.val = STR(val)
>
> self.type = STR(type_arg)
>
>
>
> The result on the wire of the performing an SNMP set on using the values
> above is:
>
>
>
> variable-bindings: 2 items
>
>
> TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49
> (1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49):
> 02000000c280c3bf
>
> Object Name:
> 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49
> (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49)
>
> Value (OctetString): 02000000c280c3bf
>
>
> TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49
> (1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49):
> 020000007f55
>
> Object Name:
> 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49
> (TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49)
>
> Value (OctetString): 020000007f55
>
>
>
> As a result an error is returned because the octet string is 8 bytes
> rather than 6.
>
> Note: Setting values <0x80 works without issue. The second variable
> binding is using a macAddress = ‘\x02\0\0\0\x7f\x55' is correct and when
> used without the 1st binding sets the value.
>
>
>
> Within the library bindings, the val attribute is read using the function
> py_netsnmp_attr_string, which calls PyUnicode_AsUTF8AndSize function
> which returns a UTF8 encoded char buffer. The value sent on the wire is the
> UTF-8 encoded string.
>
>
>
> >>> macAddress = '\x02\0\0\0\x80\xff'
>
> >>> macAddress
>
> '\x02\x00\x00\x00\x80ÿ'
>
> >>> macAddress.encode('utf8')
>
> b'\x02\x00\x00\x00\xc2\x80\xc3\xbf'
>
>
>
> UTF8 encoding is probably a poor choice for encoding octet strings. I
> believe that if the string was encoded as latin-1 (which is the same as
> iso-8859-1) within the bindings the problem wouldn't occur.
>
>
>
> >>> macAddress.encode('latin-1')
>
> b'\x02\x00\x00\x00\x80\xff'
>
>
>
> Does anyone have any idea of how to cleverly work around this or failing
> that patch the bindings?
>
>
> The content of this email (including any attachments) is intended for the
> addressee only, is confidential and may be legally privileged. If you've
> received this email in error, you shouldn't read it - please contact me
> immediately, destroy it, and do not copy or use any of the content of this
> email . No confidentiality or privilege is waived or lost by any
> mis-transmission or error. This communication does not designate an
> information system for the purposes of Part 4 of the Contract and
> Commercial Law Act 2017. Although we have taken reasonable precautions to
> ensure no viruses are present in this email, we cannot accept
> responsibility for any loss or damage arising from the use of this email or
> its attachments.
> _______________________________________________
> Net-snmp-users mailing list
> Net-snmp-users@lists.sourceforge.net
> Please see the following page to unsubscribe or change other options:
> https://lists.sourceforge.net/lists/listinfo/net-snmp-users
>
[Attachment #5 (text/html)]
<div dir="ltr">The API has to deal with both DisplayString (underlying type OCTETSTR) \
and actual octet strings. It makes sense for a DisplayString to be utf-8 encoded, \
but, it doesn't make sense for actual octet strings.<div><br></div><div>Python 3 \
does have two different types that can very usefully be used for this: a string could \
be treated as being utf-8 encoded, assuming it's a DisplayString, and a bytes \
value could be treated as a series of bytes.</div><div><br></div><div>Unfortunately, \
get() would have to return bytes only, as consulting the MIB is the only way to tell \
whether a given incoming OCTETSTR is a DisplayString or \
not.</div><div><br></div><div>I don't have cycles to work on this, \
unfortunately.</div><div><br></div><div> Bill</div><div><br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 2, 2023 at 4:10 PM \
Steve Hollywood <<a \
href="mailto:Steve.Hollywood@chorus.co.nz">Steve.Hollywood@chorus.co.nz</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div \
class="msg-4573603972108979472">
<div lang="EN-NZ" style="overflow-wrap: break-word;">
<div class="m_-9042484721725263771WordSection1">
<p class="MsoNormal">Net-SNMP v5.9.3 with python 3.6.3.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I'm not sure if this has only become an issue since the move of \
strings from bytes to Unicode in Python 3 but I cannot set an OID value to an octet \
string containing octets >=128.<u></u><u></u></p> <p class="MsoNormal">I am trying \
to set a MAC address in an OID.<u></u><u></u></p> <p class="MsoNormal"><u></u> \
<u></u></p> <p class="MsoNormal"> macAddress = \
'\x02\0\0\0\x80\xff'<u></u><u></u></p> <p class="MsoNormal"> \
varbind = netsnmp.Varbind(oidSvcMacNameAddr, iid, macAddress, \
'OCTETSTR')<u></u><u></u></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">The python bindings convert everything to strings in the Varbind \
object. <u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:"Courier \
New"">class Varbind(object):<u></u><u></u></span></p> <p \
class="m_-9042484721725263771Code"> def __init__(self, tag=None, iid=None, \
val=None, type_arg=None):<span style="font-size:10pt"><u></u><u></u></span></p> <p \
class="MsoNormal"><span style="font-size:10pt;font-family:"Courier New""> \
self.tag = STR(tag)<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New""> self.iid = \
STR(iid)<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New""> self.val = \
STR(val)<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:10pt;font-family:"Courier New""> self.type = \
STR(type_arg)<u></u><u></u></span></p> <p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span>The result on the wire of the performing an SNMP set on \
using the values above is:<u></u><u></u></span></p> <p \
class="MsoNormal"><span><u></u> <u></u></span></p> <p \
class="MsoNormal"><span>variable-bindings: 2 items<u></u><u></u></span></p> <p \
class="MsoNormal"><span> \
TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 \
(1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49): \
02000000c280c3bf<u></u><u></u></span></p> <p class="MsoNormal"><span> \
Object Name: 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 \
(TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49)<u></u><u></u></span></p>
<p class="MsoNormal"><span> Value (OctetString): \
02000000c280c3bf<u></u><u></u></span></p> <p class="MsoNormal"><span> \
TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 \
(1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49): \
020000007f55<u></u><u></u></span></p> <p class="MsoNormal"><span> \
Object Name: 1.3.6.1.4.1.6527.3.1.2.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49 \
(TIMETRA-GLOBAL-MIB::tmnxSRObjs.4.2.59.1.4.10.83.84.69.45.80.79.76.84.48.49)<u></u><u></u></span></p>
<p class="MsoNormal"><span> Value (OctetString): \
020000007f55<u></u><u></u></span></p> <p class="MsoNormal"><span><u></u> \
<u></u></span></p> <p class="MsoNormal"><span>As a result an error is returned \
because the octet string is 8 bytes rather than 6. <u></u><u></u></span></p>
<p class="MsoNormal"><span>Note: Setting values <0x80 works without issue. The \
second variable binding is using a macAddress = ‘\x02\0\0\0\x7f\x55' is correct and \
when used without the 1<sup>st</sup> binding sets the value.<u></u><u></u></span></p> \
<p class="MsoNormal"><span><u></u> <u></u></span></p> <p \
class="MsoNormal"><span>Within the library bindings, the val attribute is read using \
the function </span>py_netsnmp_attr_string<span>, which calls PyUnicode_AsUTF8AndSize \
function which returns a UTF8 encoded char buffer. The value sent on the wire is the \
UTF-8 encoded string.<u></u><u></u></span></p> <p class="MsoNormal"><span><u></u> \
<u></u></span></p> <p class="MsoNormal"><span>>>> macAddress = \
'\x02\0\0\0\x80\xff'<u></u><u></u></span></p> <p \
class="MsoNormal"><span>>>> macAddress<u></u><u></u></span></p> <p \
class="MsoNormal"><span>'\x02\x00\x00\x00\x80ÿ'<u></u><u></u></span></p> <p \
class="MsoNormal"><span>>>> \
macAddress.encode('utf8')<u></u><u></u></span></p> <p \
class="MsoNormal"><span>b'\x02\x00\x00\x00\xc2\x80\xc3\xbf'<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>UTF8 encoding is probably a poor choice for encoding octet \
strings. I believe that if the string was encoded as latin-1 (which is the same as \
iso-8859-1) within the bindings the problem wouldn't occur.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>>>> \
macAddress.encode('latin-1')<u></u><u></u></span></p> <p \
class="MsoNormal"><span>b'\x02\x00\x00\x00\x80\xff'<u></u><u></u></span></p> \
<p class="MsoNormal"><span><u></u> <u></u></span></p> <p \
class="MsoNormal"><span>Does anyone have any idea of how to cleverly work around this \
or failing that patch the bindings?<u></u><u></u></span></p> <p \
class="MsoNormal"><span><u></u> <u></u></span></p> </div>
The content of this email (including any attachments) is intended for the addressee \
only, is confidential and may be legally privileged. If you've received this email in \
error, you shouldn't read it - please contact me immediately, destroy it, and do not \
copy or use any of the content of this email . No confidentiality or privilege is \
waived or lost by any mis-transmission or error. This communication does not \
designate an information system for the purposes of Part 4 of the Contract and \
Commercial Law Act 2017. Although we have taken reasonable precautions to ensure no \
viruses are present in this email, we cannot accept responsibility for any loss or \
damage arising from the use of this email or its attachments. </div>
_______________________________________________<br>
Net-snmp-users mailing list<br>
<a href="mailto:Net-snmp-users@lists.sourceforge.net" \
target="_blank">Net-snmp-users@lists.sourceforge.net</a><br> Please see the following \
page to unsubscribe or change other options:<br> <a \
href="https://lists.sourceforge.net/lists/listinfo/net-snmp-users" rel="noreferrer" \
target="_blank">https://lists.sourceforge.net/lists/listinfo/net-snmp-users</a><br> \
</div></blockquote></div>
_______________________________________________
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic