'Re: Review Request 111754: syntax highlighter must deal more robustly with infinite loops'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kwrite-devel
Subject:    Re: Review Request 111754: syntax highlighter must deal more robustly with infinite loops
From:       "Milian Wolff" <mail () milianw ! de>
Date:       2013-12-22 18:14:50
Message-ID: 20131222181450.4579.49076 () probe ! kde ! org
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/111754/#review46055
-----------------------------------------------------------

ugh the huge whitespace changes make it hard to review this patch. I think some of \
the stuff below actually is old stuff just moved around or so...

generally, there are some issues though and I agree with Christoph and would like to \
see a simplified approach - or is that not possible?

part/syntax/katehighlight.cpp
<https://git.reviewboard.kde.org/r/111754/#comment32854>

    this is quite confusing code layout.

    please invert the contents and drop the if, only use the else branch by inverting \
the conditional:  
    if (!visited.contains(context)) { ... }

part/syntax/katehighlight.cpp
<https://git.reviewboard.kde.org/r/111754/#comment32856>

    you assign a bool to a QHash<..., int> - thats wrong. Do you maybe want to use a \
simple QSet?

part/syntax/katehighlight.cpp
<https://git.reviewboard.kde.org/r/111754/#comment32855>

    where's the type of item declared? or is this supposed to shadow something? clean \
this up please

part/syntax/katehighlight.cpp
<https://git.reviewboard.kde.org/r/111754/#comment32857>

    again empty conditional, please restructure this

part/syntax/katehighlight.cpp
<https://git.reviewboard.kde.org/r/111754/#comment32858>

    wht is changed here and below? the diff is really hard to figure out

tests/data/syntax/infinite.xml
<https://git.reviewboard.kde.org/r/111754/#comment32859>

    remove trailing spaces

- Milian Wolff

On Dec. 22, 2013, 6:08 p.m., Bruno Vasselle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/111754/
> -----------------------------------------------------------
> 
> (Updated Dec. 22, 2013, 6:08 p.m.)
> 
> 
> Review request for Kate, Christoph Cullmann, Dominik Haumann, and Milian Wolff.
> 
> 
> Repository: kate
> 
> 
> Description
> -------
> 
> The syntax highlighter currently has some kind of protection against infinite loops \
> induced by buggy highlighting XML. Resolution of bug 144599 is an example of such a \
> protection. 
> However, with such concepts as "lookAhead", which basically matches null strings, \
> the risk of infinite loops cannot be sanely guaranteed by ad-hoc patches produced \
> on an on-bug basis, but rather requires a framework that per se offers such a \
> guaranty. 
> The patch proposes an implementation for this. It registers the current state, in \
> terms of offset within the line coupled with context so that it is able to branch \
> to "no match" when it encounters the same situation twice. The "no match" branch in \
> turn does not loop on an already encountered context, but rather leads to the \
> default path, which increments the offset. 
> The patch is a few lines long, but requires indentation, which leads to white space \
> noise. 
> Test XML and text are included for the specific case of some infinite loops not \
> dealt until now. 
> 
> Diffs
> -----
> 
> part/syntax/katehighlight.cpp f44f74e 
> tests/data/syntax/infinite.txt PRE-CREATION 
> tests/data/syntax/infinite.xml PRE-CREATION 
> 
> Diff: https://git.reviewboard.kde.org/r/111754/diff/
> 
> 
> Testing
> -------
> 
> Opened randomly files with different syntaxes. Opened specifically infinite.xml \
> with XML highlighter, and infinite.txt with infinite.xml highlighter.  
> infinite.txt with infinite.xml highlighter falls into infinite loop without the \
> patch. 
> 
> Thanks,
> 
> Bruno Vasselle
> 
> 

[Attachment #5 (text/html)]

<html>
 <body>
  <div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
   <table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 \
solid;">  <tr>
     <td>
      This is an automatically generated e-mail. To reply, visit:
      <a href="https://git.reviewboard.kde.org/r/111754/">https://git.reviewboard.kde.org/r/111754/</a>
  </td>
    </tr>
   </table>
   <br />

 <pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">ugh the huge whitespace \
changes make it hard to review this patch. I think some of the stuff below actually \
is old stuff just moved around or so...

generally, there are some issues though and I agree with Christoph and would like to \
see a simplified approach - or is that not possible?</pre>  <br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174022#file174022line356" \
style="color: black; font-weight: bold; text-decoration: \
underline;">part/syntax/katehighlight.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
  <tr>

   <td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">void \
KateHighlighting::doHighlight ( const Kate::TextLineData *_prevLine,</pre></td>

  </tr>
 </tbody>

 <tbody>

  <tr>
    <th bgcolor="#e9eaa8" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">355</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">      <span \
class="n">foreach</span> <span class="p">(</span><span class="n">item</span><span \
class="p">,</span> <span class="n">context</span><span class="o">-&gt;</span><span \
class="n">items</span><span class="p">)</span></pre></td>  <th bgcolor="#e9eaa8" \
style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">356</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">      <span \
class="k">if</span> <span class="p">(</span><span class="n">visited</span><span \
class="p">.</span><span class="n">contains</span><span class="p">(</span><span \
class="n">context</span><span class="p">))</span> <span class="cm">/* already done at \
this position with this context: infinite loop */</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">this is \
quite confusing code layout.

please invert the contents and drop the if, only use the else branch by inverting the \
conditional:

if (!visited.contains(context)) { ... }</pre>
</div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174022#file174022line363" \
style="color: black; font-weight: bold; text-decoration: \
underline;">part/syntax/katehighlight.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
  <tr>

   <td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">void \
KateHighlighting::doHighlight ( const Kate::TextLineData *_prevLine,</pre></td>

  </tr>
 </tbody>

 <tbody>

  <tr>
    <th bgcolor="#e9eaa8" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">362</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span \
class="k">if</span> <span class="p">((</span><span class="n">item</span><span \
class="o">-&gt;</span><span class="n">column</span> <span class="o">!=</span> <span \
class="o">-</span><span class="mi">1</span><span class="p">)</span> <span \
class="o">&amp;&amp;</span> <span class="p">(</span><span class="n">item</span><span \
class="o">-&gt;</span><span class="n">column</span> <span class="o">!=</span> <span \
class="n">offset</span><span class="p">))</span></pre></td>  <th bgcolor="#e9eaa8" \
style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">360</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span \
class="n">visited</span><span class="p">[</span><span class="n">context</span><span \
class="p">]</span> <span class="o">=</span> <span class="nb">true</span><span \
class="p">;</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">you assign \
a bool to a QHash&lt;..., int&gt; - thats wrong. Do you maybe want to use a simple \
QSet?</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174022#file174022line366" \
style="color: black; font-weight: bold; text-decoration: \
underline;">part/syntax/katehighlight.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
  <tr>

   <td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">void \
KateHighlighting::doHighlight ( const Kate::TextLineData *_prevLine,</pre></td>

  </tr>
 </tbody>

 <tbody>

  <tr>
    <th bgcolor="#e9eaa8" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">365</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span \
class="k">if</span> <span class="p">(</span><span class="o">!</span><span \
class="n">item</span><span class="o">-&gt;</span><span \
class="n">alwaysStartEnable</span><span class="p">)</span></pre></td>  <th \
bgcolor="#e9eaa8" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">362</font></th>  <td bgcolor="#fdfebc" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span \
class="n">foreach</span> <span class="p">(</span><span class="n">item</span><span \
class="p">,</span> <span class="n">context</span><span class="o">-&gt;</span><span \
class="n">items</span><span class="p">)</span> <span class="p">{</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">where&#39;s \
the type of item declared? or is this supposed to shadow something? clean this up \
please</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174022#file174022line398" \
style="color: black; font-weight: bold; text-decoration: \
underline;">part/syntax/katehighlight.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
  <tr>

   <td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">void \
KateHighlighting::doHighlight ( const Kate::TextLineData *_prevLine,</pre></td>

  </tr>
 </tbody>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">386</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">              \
<span class="k">if</span> <span class="p">(</span><span \
class="n">lastDelimChar</span> <span class="o">==</span> <span \
class="n">lastChar</span><span class="p">)</span> <span class="p">{</span></pre></td> \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">again empty \
conditional, please restructure this</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174022#file174022line463" \
style="color: black; font-weight: bold; text-decoration: \
underline;">part/syntax/katehighlight.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody style="background-color: #e4d9cb; padding: 4px 8px; text-align: center;">
  <tr>

   <td colspan="4"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">void \
KateHighlighting::doHighlight ( const Kate::TextLineData *_prevLine,</pre></td>

  </tr>
 </tbody>

 <tbody>

  <tr>
    <th bgcolor="#e9eaa8" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">432</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span class="c1">// \
handle folding end or begin</span></pre></td>  <th bgcolor="#e9eaa8" \
style="border-left: 1px solid #C0C0C0; border-right: 1px solid #C0C0C0;" \
align="right"><font size="2">436</font></th>  <td bgcolor="#fdfebc" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; ">          <span class="c1">// \
handle folding end or begin</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">wht is \
changed here and below? the diff is really hard to figure out</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/111754/diff/1/?file=174024#file174024line4" \
style="color: black; font-weight: bold; text-decoration: \
underline;">tests/data/syntax/infinite.xml</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">4</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span \
class="cp"><span class="ew">  </span></span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">remove \
trailing spaces</pre> </div>
<br />

<p>- Milian Wolff</p>

<br />
<p>On December 22nd, 2013, 6:08 p.m. UTC, Bruno Vasselle wrote:</p>

<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" \
style="background-image: \
url('https://git.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); \
background-position: left top; background-repeat: repeat-x; border: 1px black \
solid;">  <tr>
  <td>

<div>Review request for Kate, Christoph Cullmann, Dominik Haumann, and Milian \
Wolff.</div> <div>By Bruno Vasselle.</div>

<p style="color: grey;"><i>Updated Dec. 22, 2013, 6:08 p.m.</i></p>

<div style="margin-top: 1.5em;">
 <b style="color: #575012; font-size: 10pt;">Repository: </b>
kate
</div>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
 <table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" \
style="border: 1px solid #b8b5a0">  <tr>
  <td>
   <pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">The syntax highlighter currently has some kind of protection against \
infinite loops induced by buggy highlighting XML. Resolution of bug 144599 is an \
example of such a protection.

However, with such concepts as &quot;lookAhead&quot;, which basically matches null \
strings, the risk of infinite loops cannot be sanely guaranteed by ad-hoc patches \
produced on an on-bug basis, but rather requires a framework that per se offers such \
a guaranty.

The patch proposes an implementation for this. It registers the current state, in \
terms of offset within the line coupled with context so that it is able to branch to \
&quot;no match&quot; when it encounters the same situation twice. The &quot;no \
match&quot; branch in turn does not loop on an already encountered context, but \
rather leads to the default path, which increments the offset.

The patch is a few lines long, but requires indentation, which leads to white space \
noise.

Test XML and text are included for the specific case of some infinite loops not dealt \
until now. </pre>
  </td>
 </tr>
</table>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0">  <tr>
  <td>
   <pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">Opened randomly files with different syntaxes. Opened specifically \
infinite.xml with XML highlighter, and infinite.txt with infinite.xml highlighter. 

infinite.txt with infinite.xml highlighter falls into infinite loop without the \
patch.</pre>  </td>
 </tr>
</table>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">

 <li>part/syntax/katehighlight.cpp <span style="color: grey">(f44f74e)</span></li>

 <li>tests/data/syntax/infinite.txt <span style="color: \
grey">(PRE-CREATION)</span></li>

 <li>tests/data/syntax/infinite.xml <span style="color: \
grey">(PRE-CREATION)</span></li>

</ul>

<p><a href="https://git.reviewboard.kde.org/r/111754/diff/" style="margin-left: \
3em;">View Diff</a></p>

  </td>
 </tr>
</table>

  </div>
 </body>
</html>

_______________________________________________
KWrite-Devel mailing list
KWrite-Devel@kde.org
https://mail.kde.org/mailman/listinfo/kwrite-devel

[prev in list] [next in list] [prev in thread] [next in thread] 