[prev in list] [next in list] [prev in thread] [next in thread]
List: xml-dev
Subject: Re: [xml-dev] XHTML 5 and validation
From: Michael Glavassevich <mrglavas () ca ! ibm ! com>
Date: 2011-05-21 4:13:18
Message-ID: OF1E6AC361.AEBDC66D-ON85257897.0015ED36-85257897.00173432 () ca ! ibm ! com
[Download RAW message or body]
Michael Sokolov <sokolov@ifactory.com> wrote on 05/20/2011 10:41:47 PM:
> Thanks for clearing that up - I should have asked around when I had the
> problem originally, I guess! You have correctly inferred the source of
> our problem - using a JDK InputStreamReader in front of the parser.
When you pass in a java.io.InputStreamReader (or any other Reader that
pulls from an InputStream) as input your application assumes full
responsibility for decoding the byte stream into chars. The parser then has
no role in the decoding process. It only sees the chars.
> cheers
>
> -Mike
>
> On 5/20/2011 9:28 PM, Michael Glavassevich wrote:
> >
> > John Cowan <cowan@ccil.org> wrote on 05/20/2011 06:59:04 PM:
> >
> > > Mike Sokolov scripsit:
> > >
> > > > BOM in UTF-8 seems to cause problems with some XML parsers
> > > > (incl. Xerces 2.9.1). They seem to believe it is white space in
the
> > > > prolog. To deal with this, we have had to insert a processor prior
to
> > > > our parser which checks for BOM and strips it out.
> > >
> > > Support for the 8-BOM was not explicitly required until the XML 1.0
> > > Third Edition of 2004. Xerces 2.9.1 may be out of date.
> >
> > What doesn't work? Xerces has known how to handle the UTF-8 BOM for
> > much longer than that. All releases since 2003 [1] have supported it.
> >
> > Note that you need to the let parser use its own encoding support for
> > the InputStream.
> >
> > Don't pass in a UTF-8 Reader from the JDK. The JDK UTF-8
> > InputStreamReader [2] apparently doesn't recognize the BOM and perhaps
> > never will.
> >
> > > --
> > > XQuery Blueberry DOM John Cowan
> > > Entity parser dot-com cowan@ccil.org
> > > Abstract schemata http://www.ccil.org/~cowan
> > <http://www.ccil.org/%7Ecowan>
> > > XPointer errata
> > > Infoset Unicode BOM --Richard Tobin
> > >
> > >
_______________________________________________________________________
> > >
> > > XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> > > to support XML implementation and development. To minimize
> > > spam in the archives, you must subscribe before posting.
> > >
> > > [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> > > Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> > > subscribe: xml-dev-subscribe@lists.xml.org
> > > List archive: http://lists.xml.org/archives/xml-dev/
> > > List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> >
> > [1]
> > http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/
> xerces/impl/XMLEntityManager.java?r1=318934&r2=318940&diff_format=h
> > <http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/
> xerces/impl/XMLEntityManager.java?r1=318934&r2=318940&diff_format=h>
> > [2] http://bugs.sun.com/view_bug.do?bug_id=4508058
> >
> > Michael Glavassevich
> > XML Parser Development
> > IBM Toronto Lab
> > E-mail: mrglavas@ca.ibm.com
> > E-mail: mrglavas@apache.org
> >
>
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
Michael Glavassevich
XML Parser Development
IBM Toronto Lab
E-mail: mrglavas@ca.ibm.com
E-mail: mrglavas@apache.org
[Attachment #3 (text/html)]
<html><body>
<p><tt>Michael Sokolov <sokolov@ifactory.com> wrote on 05/20/2011 10:41:47 \
PM:<br> <br>
> Thanks for clearing that up - I should have asked around when I had the <br>
> problem originally, I guess! You have correctly inferred the source of \
<br> > our problem - using a JDK InputStreamReader in front of the parser.<br>
</tt><br>
<tt>When you pass in a java.io.InputStreamReader (or any other Reader that pulls from \
an InputStream) as input your application assumes full responsibility for decoding \
the byte stream into chars. The parser then has no role in the decoding process. It \
only sees the chars.</tt><br> <tt><br>
> cheers<br>
> <br>
> -Mike<br>
> <br>
> On 5/20/2011 9:28 PM, Michael Glavassevich wrote:<br>
> ><br>
> > John Cowan <cowan@ccil.org> wrote on 05/20/2011 06:59:04 PM:<br>
> ><br>
> > > Mike Sokolov scripsit:<br>
> > ><br>
> > > > BOM in UTF-8 seems to cause problems with some XML parsers<br>
> > > > (incl. Xerces 2.9.1). They seem to believe it is white \
space in the<br> > > > > prolog. To deal with this, we have had to \
insert a processor prior to<br> > > > > our parser which checks for BOM \
and strips it out.<br> > > ><br>
> > > Support for the 8-BOM was not explicitly required until the XML \
1.0<br> > > > Third Edition of 2004. Xerces 2.9.1 may be out of \
date.<br> > ><br>
> > What doesn't work? Xerces has known how to handle the UTF-8 BOM for <br>
> > much longer than that. All releases since 2003 [1] have supported it.<br>
> ><br>
> > Note that you need to the let parser use its own encoding support for <br>
> > the InputStream.<br>
> ><br>
> > Don't pass in a UTF-8 Reader from the JDK. The JDK UTF-8 <br>
> > InputStreamReader [2] apparently doesn't recognize the BOM and perhaps <br>
> > never will.<br>
> ><br>
> > > --<br>
> > > XQuery Blueberry DOM \
John Cowan<br> > > > Entity \
parser dot-com \
cowan@ccil.org<br> > > > Abstract \
schemata <a href="http://www.ccil.org/~cowan">http://www.ccil.org/~cowan</a> <br> \
> > <<a href="http://www.ccil.org/%7Ecowan">http://www.ccil.org/%7Ecowan</a>><br>
> > > XPointer errata<br>
> > > Infoset Unicode BOM \
--Richard Tobin<br> \
> > ><br> > > > \
_______________________________________________________________________<br> > > \
><br> > > > XML-DEV is a publicly archived, unmoderated list hosted by \
OASIS<br> > > > to support XML implementation and development. To \
minimize<br> > > > spam in the archives, you must subscribe before \
posting.<br> > > ><br>
> > > [Un]Subscribe/change address: <a \
href="http://www.oasis-open.org/mlmanage/">http://www.oasis-open.org/mlmanage/</a><br>
> > > Or unsubscribe: xml-dev-unsubscribe@lists.xml.org<br>
> > > subscribe: xml-dev-subscribe@lists.xml.org<br>
> > > List archive: <a \
href="http://lists.xml.org/archives/xml-dev/">http://lists.xml.org/archives/xml-dev/</a><br>
> > > List Guidelines: <a \
href="http://www.oasis-open.org/maillists/guidelines.php">http://www.oasis-open.org/maillists/guidelines.php</a><br>
> ><br>
> > [1] <br>
> > <a href="http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/">http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/</a><br>
> xerces/impl/XMLEntityManager.java?r1=318934&r2=318940&diff_format=h \
<br> > > <<a \
href="http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/">http://svn.apache.org/viewvc/xerces/java/trunk/src/org/apache/</a><br>
> xerces/impl/XMLEntityManager.java?r1=318934&r2=318940&diff_format=h><br>
> > [2] <a href="http://bugs.sun.com/view_bug.do?bug_id=4508058">http://bugs.sun.com/view_bug.do?bug_id=4508058</a><br>
> ><br>
> > Michael Glavassevich<br>
> > XML Parser Development<br>
> > IBM Toronto Lab<br>
> > E-mail: mrglavas@ca.ibm.com<br>
> > E-mail: mrglavas@apache.org<br>
> ><br>
> <br>
> <br>
> _______________________________________________________________________<br>
> <br>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS<br>
> to support XML implementation and development. To minimize<br>
> spam in the archives, you must subscribe before posting.<br>
> <br>
> [Un]Subscribe/change address: <a \
href="http://www.oasis-open.org/mlmanage/">http://www.oasis-open.org/mlmanage/</a><br>
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org<br>
> subscribe: xml-dev-subscribe@lists.xml.org<br>
> List archive: <a \
href="http://lists.xml.org/archives/xml-dev/">http://lists.xml.org/archives/xml-dev/</a><br>
> List Guidelines: <a \
href="http://www.oasis-open.org/maillists/guidelines.php">http://www.oasis-open.org/maillists/guidelines.php</a><br>
</tt><br>
<tt>Michael Glavassevich<br>
XML Parser Development<br>
IBM Toronto Lab<br>
E-mail: mrglavas@ca.ibm.com</tt><br>
<tt>E-mail: mrglavas@apache.org</tt></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic