[prev in list] [next in list] [prev in thread] [next in thread]
List: slide-dev
Subject: Re: AW: 2.1 Release, serious issue with encodings, show stopper?
From: Eirikur Hrafnsson <eiki () idega ! is>
Date: 2004-12-30 14:29:27
Message-ID: 360FA512-5A6F-11D9-B394-000A95971DFC () idega ! is
[Download RAW message or body]
Thanks I will try it : )
On 30.12.2004, at 12:05, Thomas Draier wrote:
> hi,
> i've made an update to the patch so it can apply on the current cvs
> head.
> eirikur, can you try the patch and tell me if it works for you ? just
> download it from
> http://issues.eu.apache.org/bugzilla/attachment.cgi?id=13862 and apply
> it on head. in the slide.properties, the useUtf8 parameter tells if
> your database supports utf8 or not - and urlEncoding is the encoding
> that should be used by your clients if they are not talking utf8, and
> the encoding that will be used internally if useUtf8 is set to false.
> if it's ok for you, i'll commit it, but i don't really know yet where
> i should put it - bugfix in 2.1 or 2.2 ? i'd like the patch to be
> tested by some people before going to a release, cause it might cause
> some problems for people using non latin encodings (i don't really
> know how, but since i haven't tested this, and changed strange stuff,
> can't be sure..)
> thomas
>
> Le 29 déc. 04, à 15:22, Eirikur Hrafnsson a écrit :
>
>>
>> On 28.12.2004, at 19:50, Oliver Zeigermann wrote:
>>
>>> On Tue, 28 Dec 2004 19:23:15 +0000, Eirikur Hrafnsson
>>> <eiki@idega.is> wrote:
>>>> On 28.12.2004, at 18:53, Oliver Zeigermann wrote:
>>>>> On Tue, 28 Dec 2004 15:01:08 +0000, Eirikur Hrafnsson
>>>>> <eiki@idega.is>
>>>>> wrote:
>>>>>> Fortunately, at least with Oracle and the thin driver, that works
>>>>>> with
>>>>>> an ISO-8859-1 database since that encoding is a "subset" of UTF-8
>>>>>> and
>>>>>> has a 1-1 mapping in UTF-8.
>>>>>> However this makes Slide unusable with other "stranger" encodings
>>>>>> such
>>>>>> as Russian UNLESS you have a unicode database.
>>>>>
>>>>> Yes, if you want to store unicode into you database the database
>>>>> will
>>>>> have to support it, right?
>>>> Yes of course but I don't want to store unicode in our case because
>>>> we
>>>> are trying to add Slide to already running webapps that were
>>>> created on
>>>> non-unicode databases.
>>>> I don't want for this project to have both Russian and Icelandic in
>>>> the
>>>> same page, sorry for that misunderstanding. I know you need an
>>>> unicode
>>>> database to do that and we have done that already.
>>>> Our problem only arose because the existing database was ISO-8859
>>>> encoded.
>>>
>>> No idea how to solve this. If your database has ISO-8859-1 (is it 1?)
>>> encoding you simply can not store Russian into it. A solution would
>>> be
>>> not to encode anything in the database, but leave it as bytes. Of
>>> course you can not search meaningful then...
>>
>> I'm not trying to store Russian in an ISO-8859-1 database but Swedish
>> and Iceland.
>> When I need to store Russian I always use an UTF-8 database.
>>
>>>
>>>>>
>>>>>> I looked at the patch that Thomas put in Bugzilla and it looks
>>>>>> like it
>>>>>> could work but I think the FAQ or the Wiki should contain clear
>>>>>> instructions on how to use Slide in a non UTF-8 environment.
>>>>>
>>>>> Still unclear to me what is a non UTF-8 environment.
>>>> : ) hehe ok sorry,
>>>> Non UTF-8 environment would be e.g. a Tomcat server that for some
>>>> reason has to be run in an ISO encoding. One reason is like in our
>>>> case
>>>> that the existing DB is in ISO format and the JDBC driver might not
>>>> be
>>>> able to convert UTF-8 to the encoding being used in the DB when we
>>>> want
>>>> to save something (other than Slide stuff).
>>>
>>> Are you talking about content or URIs now? If things work alright,
>>> the
>>> URI should be a string and then there is no decoding/encoding issue
>>> any more. Content should be BLOBs anyhow...
>>
>> I was always talking about URI's.
>>
>>>
>>>>>
>>>>>> Thanks for all the help guys, hope this will be resolved in the
>>>>>> next
>>>>>> minor version.
>>>>>
>>>>> Could you clarify what "this" means in this context.
>>>> This meaning:
>>>> That Slide could run without having to use UTF-8 for the encoding.
>>>> The
>>>> putMethod today will assume we are using utf-8 or the system
>>>> default.
>>>
>>> PutMethod of the server?
>>
>> Yes on the server, see my previous looong post with code examples ; )
>>
>>>
>>>> However the WebDavResource will always use the clients system
>>>> encoding
>>>> and there is no detection of the URLEncoding in the putMethod and
>>>> therefor the URI_STRING saved to the database is gibberish. (I
>>>> explained this in much more detail in a previous email).
>>>
>>> So it's a client issue? Can you propose a patch that allows to set
>>> how
>>> to encode the request URI?
>>
>> It's both a client and server issue, the client (WebdavResource)
>> should at least have putMethods (and others xxxMethods probably or a
>> constructor) that take into them the desired encoding.
>> like putMethod(filename,inputstream,encoding). Should be simple to
>> add.
>>
>> On the server side we need for Slide to figure out in what encoding
>> the client is sending. For example Web Folders on WIndows does not
>> seem to work correcly for Icelandic NOR Swedish letters when running
>> Tomcat in UTF-8 with an Oracle store. I don't know what encoding they
>> use but it is definately not UTF-8!
>>
>> However Novel NetDrive works great once you check the "Use UTF-8 for
>> paths" setup option.
>>
>> I'm most interested in seeing if Thomas' patch can guess the correct
>> URI encoding and make sure to convert the file/folder names that you
>> want to upload to the correct encoding before saving them?
>>
>> -Eiki, Idega.
>>
>>
>>>
>>> Maybe I am just no smart enough to help, sorry...
>>>
>>> Oliver
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>>
>> Best Regards
>>
>> Eirikur S. Hrafnsson, eiki@idega.is
>> Chief Software Engineer
>> Idega Software
>> http://www.idega.com
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
Best Regards
Eirikur S. Hrafnsson, eiki@idega.is
Chief Software Engineer
Idega Software
http://www.idega.com
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic