[prev in list] [next in list] [prev in thread] [next in thread] 

List:       postgresql-general
Subject:    Re: [HACKERS] Bug in UTF8-Validation Code?
From:       Tom Lane <tgl () sss ! pgh ! pa ! us>
Date:       2007-04-03 17:06:38
Message-ID: 9805.1175619998 () sss ! pgh ! pa ! us
[Download RAW message or body]

Mark Dilger <pgsql@markdilger.com> writes:
> Martijn van Oosterhout wrote:
>> Just about every multibyte encoding other than Unicode has the problem
>> of not distinguishing between the code point and the encoding of it.

> Thanks for the feedback.  Would you say that the way I implemented things in the 
> example code would be correct for multibyte non Unicode encodings?

I think it's probably defensible for non-Unicode encodings.  To do
otherwise would require (a) figuring out what the equivalent concept to
"code point" is for each encoding, and (b) having a separate code path
for each encoding to perform the mapping.  It's not clear that there
even is an answer to (a), and (b) seems like more work than chr() is
worth.  But we know what the right way is for Unicode, so we should
special case that one.

Note the points made that in all cases ascii() and chr() should be
inverses, and that you shouldn't just fall back to the old behavior
in SQL_ASCII encoding.  (My vote for SQL_ASCII would be to reject
values > 255.)

			regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings
[prev in list] [next in list] [prev in thread] [next in thread]