[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-kafka
Subject:    Re: Tokenizer
From:       Stephan Heigl <stephan () eisscholle ! de>
Date:       2000-10-26 13:09:12
[Download RAW message or body]

On Thursday 26 October 2000 10:08, Jono Bacon wrote:
> On Wednesday 25 October 2000 20:53, Stefan Schimanski wrote:
> > > >From my understanding of the conversation, it sounds like Schimmi is
> > > > talking
> > >
> > > about building a layer that first of all contains the output of the
> > > DOM, and secondly contains further unknown DOM elements.
> > >
> > > It seems that the main issue at hand is that there is no way of knowing
> > > to refresh normal DOM tree when the token buffer DOM tree is updated.
> > > Am I right?
> >
> > right. The tokenbuffer btw is no tree, but a linear list of tokens.
> >
> > > I have a few questions about this:
> > >
> > >   - How will the normal DOM tree and the tokenizer DOM fit together to
> > > form the same tree and representation of the document.
> >
> > That would be the main issue to solve. One way to do this is that all
> > changes are done on the token stream. The idea of the token stream is,
> > that every token gets a pointer to the DOM element. So the DOM element
> > can be found whose children has to be recreated on manipulation.
> > If possible I would prefer to drop the idea of another layer because it
> > add another level of complexity. If we get the DOM to store _all_
> > information of the html file that would make our life much easier.
>
> Another option is if we are using a QObject event layer - when the user
> triggers an event on the layer, the coordinates are passed in the method
> and we can determine where in the DOM tree the action should affect. We can
> then identify if we need to update the tokenizer or not.
>
> > >   - when you say there are unknown tags to be used by the tokenizer, do
> > > you mean that it will parse each line of a <SCRIPT> block for example
> > > putting each scriping line into the enhanced tokenizer DOM? If this
> > > were the case we could have a WYSIWYG interface to building scripts
> > > also. :-)
> >
> > I would put the whole script in a single DOM element. But you're right
> > that this would be another special DOM element. A script editor of course
> > could get the needed information from that.
>
> Maybe we should ensure the plugin interface can have a connection with the
> DOM, and maybe there is a specific plugin to visually edit scripts and load
> scripts into a custom DOM. We could theoretically build a DOM for each type
> of content; e.g - PHP, JavaScript etc.
>
> > >   - when you say that the tokenizer will generate tokens, how will this
> > > work? I have no idea of how the DOM implementation in KHTML works, and
> > > I assume it tokenizes things also, but could you expand on how this
> > > works.
> > >
> > > :-)
> >
> > Of course. The KHTMLPart passes the text streams of the webpage through a
> > tokenizer. This checks character for character divides the streams into
> > tags, text, comment and so on. These elementary units are called tokens.
> > For example a tag token gets number 0, a text 1, ...
> > The token stream is passed to the parser. It interpretes the tokens and
> > creates the DOM elements for them. It create tree like data structure out
> > of the still sequencial token stream.
> > After a close tag is found the corresponding DOM node is told to create
> > render objects. Consequently you have a DOM tree and rendering tree that
> > is cross linked with the DOM. The layouting is done by the rendering
> > elements while the tag attributes are stored in the DOM elements.
>
> What do you mean by a 'close tag'. Is this the EOF marker at the end of the
> document?
>
> 	Jono

I assume a close tag is </tag>

-- 

regards,

stephan

___________________________________________________
Website: www.eisscholle.de 
ICQ: 12707332 
Jabber: stephan@jabber.com
_______________________________________________
Kde-kafka mailing list
Kde-kafka@master.kde.org
http://master.kde.org/mailman/listinfo/kde-kafka

[prev in list] [next in list] [prev in thread] [next in thread]