[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lucene-dev
Subject:    [jira] Issue Comment Edited: (LUCENE-2731)
From:       "Uwe Schindler (JIRA)" <jira () apache ! org>
Date:       2010-10-31 10:07:24
Message-ID: 32692310.161321288519644817.JavaMail.jira () thor
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/LUCENE-2731?page=com.atlassian.jira.plugin \
.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926689#action_12926689 ] 

Uwe Schindler edited comment on LUCENE-2731 at 10/31/10 6:05 AM:
-----------------------------------------------------------------

By the way, the whole loading in the TokenFilter is broken! XML files should never be \
load by a Reader, always by InputStream. The charset detection is part of the XML \
spec and the default is UTF-8 if not overridden in the XML file. The \
HyphenationCompoundWordTokenFilter supplies ISO-8859-1 as fixed for the reader, so it \
is never possible to load other xml files with different charsets. This is another \
issue, I will open for 3.x and trunk (as it needs API change).

      was (Author: thetaphi):
    By the way, the whole loading in the TokenFilter is broken! XML files should \
never be load by a Reaer, always by InputStream. The charset detection is part of the \
XML spec and the default is UTF-8 if not overridden in the XML file. The \
HyphenationCompoundWordTokenFilter supplies ISO-8859-1 as fixed for the reader, so it \
is never possible to load other xml files with different charsets. This is another \
issue, I will open for 3.x and trunk (as it needs API change).  
> HyphenationCompoundWordTokenFilter fails to load DTD in Crimson parser (JDK 1.4)
> --------------------------------------------------------------------------------
> 
> Key: LUCENE-2731
> URL: https://issues.apache.org/jira/browse/LUCENE-2731
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/analyzers
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9.4
> 
> Attachments: LUCENE-2731.patch
> 
> 
> HyphenationCompoundWordTokenFilter loads the DTD in its XML parser from memory by \
> supplying EntityResolver. In Java 1.4 (affects Lucene 2.9, but also later versions \
> if not Apache Xerces is used as XML parser) this does not work, because Cromson \
> does not even ask the entity resolver, if no base URI is known. As the hyphenation \
> file is loaded from Reader/InputStream no base URI is known. Crimson needs at least \
> a non-null systemId to proceed. This patch (Lucene 2.9 only)  fakes this by \
> supplying a fake systemId to the InputSource.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]