[prev in list] [next in list] [prev in thread] [next in thread]
List: kdepim-users
Subject: Re: [kdepim-users] Tbird versus Kmail, performance
From: James Cain <james.cain.25 () gmail ! com>
Date: 2014-11-15 23:14:29
Message-ID: CA+6ejqBwxAj1L1090jQ__Vp8zrj0Oy23RH3Yj18qdK0i8EiBUw () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
It might be beneficial also to take this conversation to IRC, since that's
where you'll find the most ears for this seemingly important discussion :)
On Sat, Nov 15, 2014 at 5:17 PM, Pablo Sanchez <pablo@blueoakdb.com> wrote:
> [ Comments below, in-line ]
>
> On 11/15/2014 05:05 PM, Ren=C3=A9 J.V. Bertin wrote:
> > On Saturday November 15 2014 16:38:29 Pablo Sanchez wrote:
> >
> > Hi,
>
> Hi Rene,
>
> >> When I engage in performance tuning efforts, unless a previous
> >> version's changes are minimal, I only look at the present. It's a
> >> bit easier when a problem is glaring.
> >
> > I'm not exactly sure what you're saying here.
>
> Oh, sorry for being unclear. What I was trying to say is given the
> set of changes from one version to others, it's usually most fruitful
> to look at the current s/w and fix it.
>
> > I know one has to be very careful with subjective impressions like
> > "everything was better before", but in this case I was pretty sure
> > that the real issues started after updating to 4.14
>
> I'm not denying the above. I suspect your impressions are right.
> Ultimately the goal is to fix the existing s/w so that's why I tend to
> focus on it (when I'm in charge of fixing P&T issues. :)
>
> > after Daniel's explanations from yesterday which suggest that the
> > database is not the bottleneck.
>
> The data doesn't indicate the above at all. What I saw is we're
> hitting the database very hard. Doing some crude sampling against the
> database I showed the queries. These queries, as I mentioned, are
> nearly unbounded: no timestamp is used to reduce their database
> bandwidth demand.
>
> I also indicated how it's possible to add a timestamp and set it when
> the row is added to the table. This timestamp is used to reduce the
> demand against the database to improve scalability.
>
> Let me give you an example. If you run a query against a table and
> crunch through 100,000 rows (let's just ignore how much space they
> require) and if all the data is in cache, from the O/S perspective,
> the DBMS will appear as it's only consuming CPU.
>
> If you repeatedly run the query many times, the DBMS will be CPU bound
> as it services the query, over and over again.
>
> From what I saw, we have /several/ of these queries running over and
> over again.
>
> Now, if you take the same query but add a timestamp filter to only
> return the rows between the last time it ran and /now/, it may be that
> we only loaded, say, 100 mail messages between queries. Crunching 100
> rows rather than 100,000 rows ... well .. it's obvious right? Less
> demand and thus we improve scalability.
>
> I hope the above helps explain it.
>
> Daniel didn't respond to me personally so I assume either a) he's away
> or b) not interested in my help. :)
>
> > Anyway, I kept akonadi at the latest version (a recent git
> > clone). Talking about database options: I came across a suggestion
> > on one of the Arch wikipages to unset an innodb setting related to
> > aio_write on ZFS. Not that I ever saw the error in question, but I
> > guess I should figure out what that setting does (I just suspended
> > my Linux rig so I don't have the exact name handy right now).
>
> The database is not I/O bound. It's CPU bound so unfortunately no
> amount of I/O tuning is going to help.
>
> Cheers,
> --
> Pablo Sanchez - Blueoak Database Engineering, Inc
> Ph: 819.459.1926 Blog: http://pablo-blog.blueoakdb.com
> iNum: 883.5100.0990.1054
>
> _______________________________________________
> KDE PIM users mailing list
> Subscription management:
> https://mail.kde.org/mailman/listinfo/kdepim-users
>
[Attachment #5 (text/html)]
<div dir="ltr"><div class="gmail_default" style="font-family:trebuchet \
ms,sans-serif">It might be beneficial also to take this conversation to IRC, since \
that's where you'll find the most ears for this seemingly important \
discussion :)</div></div><div class="gmail_extra"><br><div class="gmail_quote">On \
Sat, Nov 15, 2014 at 5:17 PM, Pablo Sanchez <span dir="ltr"><<a \
href="mailto:pablo@blueoakdb.com" target="_blank">pablo@blueoakdb.com</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex">[ Comments below, in-line ]<br> <span class=""><br>
On 11/15/2014 05:05 PM, René J.V. Bertin wrote:<br>
> On Saturday November 15 2014 16:38:29 Pablo Sanchez wrote:<br>
><br>
> Hi,<br>
<br>
</span>Hi Rene,<br>
<span class=""><br>
>> When I engage in performance tuning efforts, unless a previous<br>
>> version's changes are minimal, I only look at the present. It's \
a<br> >> bit easier when a problem is glaring.<br>
><br>
> I'm not exactly sure what you're saying here.<br>
<br>
</span>Oh, sorry for being unclear. What I was trying to say is given the<br>
set of changes from one version to others, it's usually most fruitful<br>
to look at the current s/w and fix it.<br>
<span class=""><br>
> I know one has to be very careful with subjective impressions like<br>
> "everything was better before", but in this case I was pretty sure<br>
> that the real issues started after updating to 4.14<br>
<br>
</span>I'm not denying the above. I suspect your impressions are right.<br>
Ultimately the goal is to fix the existing s/w so that's why I tend to<br>
focus on it (when I'm in charge of fixing P&T issues. :)<br>
<span class=""><br>
> after Daniel's explanations from yesterday which suggest that the<br>
> database is not the bottleneck.<br>
<br>
</span>The data doesn't indicate the above at all. What I saw is we're<br>
hitting the database very hard. Doing some crude sampling against the<br>
database I showed the queries. These queries, as I mentioned, are<br>
nearly unbounded: no timestamp is used to reduce their database<br>
bandwidth demand.<br>
<br>
I also indicated how it's possible to add a timestamp and set it when<br>
the row is added to the table. This timestamp is used to reduce the<br>
demand against the database to improve scalability.<br>
<br>
Let me give you an example. If you run a query against a table and<br>
crunch through 100,000 rows (let's just ignore how much space they<br>
require) and if all the data is in cache, from the O/S perspective,<br>
the DBMS will appear as it's only consuming CPU.<br>
<br>
If you repeatedly run the query many times, the DBMS will be CPU bound<br>
as it services the query, over and over again.<br>
<br>
From what I saw, we have /several/ of these queries running over and<br>
over again.<br>
<br>
Now, if you take the same query but add a timestamp filter to only<br>
return the rows between the last time it ran and /now/, it may be that<br>
we only loaded, say, 100 mail messages between queries. Crunching 100<br>
rows rather than 100,000 rows ... well .. it's obvious right? Less<br>
demand and thus we improve scalability.<br>
<br>
I hope the above helps explain it.<br>
<br>
Daniel didn't respond to me personally so I assume either a) he's away<br>
or b) not interested in my help. :)<br>
<span class=""><br>
> Anyway, I kept akonadi at the latest version (a recent git<br>
> clone). Talking about database options: I came across a suggestion<br>
> on one of the Arch wikipages to unset an innodb setting related to<br>
> aio_write on ZFS. Not that I ever saw the error in question, but I<br>
> guess I should figure out what that setting does (I just suspended<br>
> my Linux rig so I don't have the exact name handy right now).<br>
<br>
</span>The database is not I/O bound. It's CPU bound so unfortunately no<br>
amount of I/O tuning is going to help.<br>
<span class="im HOEnZb"><br>
Cheers,<br>
--<br>
Pablo Sanchez - Blueoak Database Engineering, Inc<br>
Ph: <a href="tel:819.459.1926" value="+18194591926">819.459.1926</a> \
Blog: <a href="http://pablo-blog.blueoakdb.com" \
target="_blank">http://pablo-blog.blueoakdb.com</a><br>
iNum: 883.5100.0990.1054<br>
<br>
</span><div class="HOEnZb"><div \
class="h5">_______________________________________________<br> KDE PIM users mailing \
list<br> Subscription management: <a \
href="https://mail.kde.org/mailman/listinfo/kdepim-users" \
target="_blank">https://mail.kde.org/mailman/listinfo/kdepim-users</a><br> \
</div></div></blockquote></div><br></div>
_______________________________________________
KDE PIM users mailing list
Subscription management: https://mail.kde.org/mailman/listinfo/kdepim-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic