'Re: Review Request 118231: Scheduler for the akonadi indexer.'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: Review Request 118231: Scheduler for the akonadi indexer.
From:       "Vishesh Handa" <me () vhanda ! in>
Date:       2014-05-22 16:00:48
Message-ID: 20140522160048.1195.68666 () probe ! kde ! org
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/118231/#review58317
-----------------------------------------------------------

Overall, the code is getting quite complex, and it's at a state where I would not be \
comfortable modifying stuff without unit tests.

src/pim/agent/collectionindexingjob.h
<https://git.reviewboard.kde.org/r/118231/#comment40571>

    The class name is different than the header name.

src/pim/agent/collectionindexingjob.h
<https://git.reviewboard.kde.org/r/118231/#comment40574>

    Whoa. This variable name is very very confusing. It's true, if there are indexed \
items which are unindexed? :O

src/pim/agent/collectionindexingjob.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40576>

    typo

src/pim/agent/collectionindexingjob.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40572>

    You can just directly remove the item, and then check if it was successfully \
removed.

src/pim/agent/collectionindexingjob.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40573>

    I'm sorry. I'm a little confused as to how this could ever happen unless Akonadi \
is messed up and does not reliably send item removed notifications?

src/pim/agent/index.h
<https://git.reviewboard.kde.org/r/118231/#comment40567>

    I'm not too happy with this classes name. How about "Indexer" instead? But then \
it would clash with "AbstractIndexer".

src/pim/agent/index.h
<https://git.reviewboard.kde.org/r/118231/#comment40569>

    Perhaps this can be made private? Ditto for some other functions.

src/pim/agent/index.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40568>

    Is this required? From a Xapian point of view you're spending an extra amount of \
time first removing the data and then adding it back again.  
    If you just index the item, Xapian will internally do a diff on the terms that \
have changed, and then just update those.

src/pim/agent/scheduler.h
<https://git.reviewboard.kde.org/r/118231/#comment40570>

    This is confusing. Perhaps some more documentation?

    Also, couldn't you just directly do a

    QMap<Akonadi::Collection::ID, QQueue<Akonadi::Item::ID>>

    I'm not sure what the shared pointer is doing.

src/pim/agent/scheduler.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40577>

    Why 100?

src/pim/agent/scheduler.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40579>

    This is quite dangerous. If an email is not indexed it results in the entire \
collection being sync and all a full collection fetch job going on.  
    I'm very much against this. I get enough angry emails about how the baloo indexer \
is sucking all their cpu.

src/pim/agent/scheduler.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40578>

    So, even if someone switched off their system before the initial indexing was \
done, we mark the initial indexing as completed?

src/pim/agent/scheduler.cpp
<https://git.reviewboard.kde.org/r/118231/#comment40575>

    So each time an item is added, you pass the parent collection to the \
CollectionIndexingJob and there you fetch the entire collection (+statistics) and do \
a query on Xapian to check how many items are already indexed?  
    Arguably, the xapian query would be quite fast, but still.

- Vishesh Handa

On May 20, 2014, 10:22 p.m., Christian Mollekopf wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/118231/
> -----------------------------------------------------------
> 
> (Updated May 20, 2014, 10:22 p.m.)
> 
> 
> Review request for Baloo and Vishesh Handa.
> 
> 
> Repository: baloo
> 
> 
> Description
> -------
> 
> A scheduler for baloo: 
> * delays the indexing until no new item has been added for at least 5 seconds to \
>                 avoid indexing during a collection sync.
> * remembers if it failed to index something and triggers recovery path on next \
>                 start.
> * supports manual triggering of recovery path if required.
> 
> 
> Diffs
> -----
> 
> src/pim/agent/CMakeLists.txt e917915a3414738595caea5497859ef4810ec44c 
> src/pim/agent/agent.h 1dbf0fc0a16d0615dbfa4878706359bb687facd0 
> src/pim/agent/agent.cpp 8904d49d3579b58b634d2570fbcc8007e5ee41ed 
> src/pim/agent/collectionindexingjob.h PRE-CREATION 
> src/pim/agent/collectionindexingjob.cpp PRE-CREATION 
> src/pim/agent/index.h PRE-CREATION 
> src/pim/agent/index.cpp PRE-CREATION 
> src/pim/agent/scheduler.h PRE-CREATION 
> src/pim/agent/scheduler.cpp PRE-CREATION 
> 
> Diff: https://git.reviewboard.kde.org/r/118231/diff/
> 
> 
> Testing
> -------
> 
> I'm running it for a while, and it reduced the stress that baloo imposed on my \
> system and all my mails are indexed since I'm using it (wasn't the case before). 
> 
> Thanks,
> 
> Christian Mollekopf
> 
> 

[Attachment #5 (text/html)]

<html>
 <body>
  <div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
   <table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 \
solid;">  <tr>
     <td>
      This is an automatically generated e-mail. To reply, visit:
      <a href="https://git.reviewboard.kde.org/r/118231/">https://git.reviewboard.kde.org/r/118231/</a>
  </td>
    </tr>
   </table>
   <br />

 <pre style="white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: \
-pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Overall, the code is \
getting quite complex, and it&#39;s at a state where I would not be comfortable \
modifying stuff without unit tests.</pre>  <br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273888#file273888line37" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/collectionindexingjob.h</a>  <span style="font-weight: \
normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">37</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span \
class="n">class</span> <span class="n">IndexingJob</span> <span class="o">:</span> \
<span class="n">public</span> <span class="n">KJob</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">The class \
name is different than the header name.</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273888#file273888line66" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/collectionindexingjob.h</a>  <span style="font-weight: \
normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">66</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="kt">bool</span> <span class="n">m_indexedUnindexed</span><span \
class="p">;</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Whoa. This \
variable name is very very confusing. It&#39;s true, if there are indexed items which \
are unindexed? :O</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273889#file273889line56" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/collectionindexingjob.cpp</a>  <span style="font-weight: \
normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">56</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="c1">//Fetch collection for statisticst</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">typo </pre>
</div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273889#file273889line184" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/collectionindexingjob.cpp</a>  <span style="font-weight: \
normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">184</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">            \
<span class="n">m_indexedItems</span><span class="p">.</span><span \
class="n">remove</span><span class="p">(</span><span class="n">item</span><span \
class="p">.</span><span class="n">id</span><span class="p">());</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">You can \
just directly remove the item, and then check if it was successfully removed.</pre> \
</div> <br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273889#file273889line200" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/collectionindexingjob.cpp</a>  <span style="font-weight: \
normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">200</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="k">if</span> <span class="p">(</span><span class="o">!</span><span \
class="n">m_indexedItems</span><span class="p">.</span><span \
class="n">isEmpty</span><span class="p">())</span> <span \
class="p">{</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I&#39;m \
sorry. I&#39;m a little confused as to how this could ever happen unless Akonadi is \
messed up and does not reliably send item removed notifications?</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273890#file273890line33" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/index.h</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">33</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span \
class="n">class</span> <span class="n">Index</span> <span class="o">:</span> <span \
class="n">public</span> <span class="n">QObject</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">I&#39;m not \
too happy with this classes name. How about &quot;Indexer&quot; instead? But then it \
would clash with &quot;AbstractIndexer&quot;.</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273890#file273890line52" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/index.h</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">52</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="n">AbstractIndexer</span><span class="o">*</span> <span \
class="n">indexerForItem</span><span class="p">(</span><span class="k">const</span> \
<span class="n">Akonadi</span><span class="o">::</span><span \
class="n">Item</span><span class="o">&amp;</span> <span class="n">item</span><span \
class="p">)</span> <span class="k">const</span><span class="p">;</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Perhaps \
this can be made private? Ditto for some other functions.</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273891#file273891line128" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/index.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">128</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span \
class="kt">void</span> <span class="n">Index</span><span class="o">::</span><span \
class="n">reindex</span><span class="p">(</span><span class="k">const</span> <span \
class="n">Akonadi</span><span class="o">::</span><span class="n">Item</span><span \
class="o">&amp;</span> <span class="n">item</span><span class="p">)</span></pre></td> \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Is this \
required? From a Xapian point of view you&#39;re spending an extra amount of time \
first removing the data and then adding it back again.

If you just index the item, Xapian will internally do a diff on the terms that have \
changed, and then just update those.</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273892#file273892line66" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/scheduler.h</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">66</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="n">QMap</span><span class="o">&lt;</span><span class="n">Akonadi</span><span \
class="o">::</span><span class="n">Collection</span><span class="o">::</span><span \
class="n">Id</span><span class="p">,</span> <span \
class="n">QSharedPointer</span><span class="o">&lt;</span><span \
class="n">QQueue</span><span class="o">&lt;</span><span class="n">Akonadi</span><span \
class="o">::</span><span class="n">Item</span><span class="o">::</span><span \
class="n">Id</span><span class="o">&gt;</span> <span class="o">&gt;</span> <span \
class="o">&gt;</span> <span class="n">m_queues</span><span \
class="p">;</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is \
confusing. Perhaps some more documentation?

Also, couldn&#39;t you just directly do a

QMap&lt;Akonadi::Collection::ID, QQueue&lt;Akonadi::Item::ID&gt;&gt;

I&#39;m not sure what the shared pointer is doing.</pre>
</div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273893#file273893line36" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/scheduler.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">36</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="n">m_processTimer</span><span class="p">.</span><span \
class="n">setInterval</span><span class="p">(</span><span class="mi">100</span><span \
class="p">);</span></pre></td>  </tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">Why \
100?</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273893#file273893line46" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/scheduler.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">46</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">        <span \
class="n">scheduleCollection</span><span class="p">(</span><span \
class="n">Akonadi</span><span class="o">::</span><span \
class="n">Collection</span><span class="p">(</span><span class="n">col</span><span \
class="p">),</span> <span class="nb">true</span><span class="p">);</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">This is \
quite dangerous. If an email is not indexed it results in the entire collection being \
sync and all a full collection fetch job going on.

I&#39;m very much against this. I get enough angry emails about how the baloo indexer \
is sucking all their cpu.</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273893#file273893line53" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/scheduler.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">53</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; ">    <span \
class="n">group</span><span class="p">.</span><span class="n">writeEntry</span><span \
class="p">(</span><span class="s">&quot;initialIndexing&quot;</span><span \
class="p">,</span> <span class="nb">true</span><span class="p">);</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">So, even if \
someone switched off their system before the initial indexing was done, we mark the \
initial indexing as completed?</pre> </div>
<br />

<div>

<table width="100%" border="0" bgcolor="white" style="border: 1px solid #C0C0C0; \
border-collapse: collapse; margin: 2px padding: 2px;">  <thead>
  <tr>
   <th colspan="4" bgcolor="#F0F0F0" style="border-bottom: 1px solid #C0C0C0; \
font-size: 9pt; padding: 4px 8px; text-align: left;">  <a \
href="https://git.reviewboard.kde.org/r/118231/diff/1/?file=273893#file273893line97" \
style="color: black; font-weight: bold; text-decoration: \
underline;">src/pim/agent/scheduler.cpp</a>  <span style="font-weight: normal;">

     (Diff revision 1)

    </span>
   </th>
  </tr>
 </thead>

 <tbody>

  <tr>
    <th bgcolor="#b1ebb0" style="border-right: 1px solid #C0C0C0;" \
align="right"><font size="2"></font></th>  <td bgcolor="#c5ffc4" width="50%"><pre \
style="font-size: 8pt; line-height: 140%; margin: 0; "></pre></td>  <th \
bgcolor="#b1ebb0" style="border-left: 1px solid #C0C0C0; border-right: 1px solid \
#C0C0C0;" align="right"><font size="2">97</font></th>  <td bgcolor="#c5ffc4" \
width="50%"><pre style="font-size: 8pt; line-height: 140%; margin: 0; "><span \
class="kt">void</span> <span class="n">Scheduler</span><span class="o">::</span><span \
class="n">addItem</span><span class="p">(</span><span class="k">const</span> <span \
class="n">Akonadi</span><span class="o">::</span><span class="n">Item</span> <span \
class="o">&amp;</span><span class="n">item</span><span class="p">)</span></pre></td>  \
</tr>

 </tbody>

</table>

<pre style="margin-left: 2em; white-space: pre-wrap; white-space: -moz-pre-wrap; \
white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">So each \
time an item is added, you pass the parent collection to the CollectionIndexingJob \
and there you fetch the entire collection (+statistics) and do a query on Xapian to \
check how many items are already indexed?

Arguably, the xapian query would be quite fast, but still.</pre>
</div>
<br />

<p>- Vishesh Handa</p>

<br />
<p>On May 20th, 2014, 10:22 p.m. UTC, Christian Mollekopf wrote:</p>

<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" \
style="background-image: \
url('https://git.reviewboard.kde.org/static/rb/images/review_request_box_top_bg.ab6f3b1072c9.png'); \
background-position: left top; background-repeat: repeat-x; border: 1px black \
solid;">  <tr>
  <td>

<div>Review request for Baloo and Vishesh Handa.</div>
<div>By Christian Mollekopf.</div>

<p style="color: grey;"><i>Updated May 20, 2014, 10:22 p.m.</i></p>

<div style="margin-top: 1.5em;">
 <b style="color: #575012; font-size: 10pt;">Repository: </b>
baloo
</div>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
 <table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" \
style="border: 1px solid #b8b5a0">  <tr>
  <td>
   <pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
                break-word;">A scheduler for baloo: 
* delays the indexing until no new item has been added for at least 5 seconds to \
                avoid indexing during a collection sync.
* remembers if it failed to index something and triggers recovery path on next start.
* supports manual triggering of recovery path if required.</pre>
  </td>
 </tr>
</table>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0">  <tr>
  <td>
   <pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">I&#39;m running it for a while, and it reduced the stress that baloo \
imposed on my system and all my mails are indexed since I&#39;m using it (wasn&#39;t \
the case before).</pre>  </td>
 </tr>
</table>

<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">

 <li>src/pim/agent/CMakeLists.txt <span style="color: \
grey">(e917915a3414738595caea5497859ef4810ec44c)</span></li>

 <li>src/pim/agent/agent.h <span style="color: \
grey">(1dbf0fc0a16d0615dbfa4878706359bb687facd0)</span></li>

 <li>src/pim/agent/agent.cpp <span style="color: \
grey">(8904d49d3579b58b634d2570fbcc8007e5ee41ed)</span></li>

 <li>src/pim/agent/collectionindexingjob.h <span style="color: \
grey">(PRE-CREATION)</span></li>

 <li>src/pim/agent/collectionindexingjob.cpp <span style="color: \
grey">(PRE-CREATION)</span></li>

 <li>src/pim/agent/index.h <span style="color: grey">(PRE-CREATION)</span></li>

 <li>src/pim/agent/index.cpp <span style="color: grey">(PRE-CREATION)</span></li>

 <li>src/pim/agent/scheduler.h <span style="color: grey">(PRE-CREATION)</span></li>

 <li>src/pim/agent/scheduler.cpp <span style="color: grey">(PRE-CREATION)</span></li>

</ul>

<p><a href="https://git.reviewboard.kde.org/r/118231/diff/" style="margin-left: \
3em;">View Diff</a></p>

  </td>
 </tr>
</table>

  </div>
 </body>
</html>

>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread] 