[prev in list] [next in list] [prev in thread] [next in thread]
List: rrd-users
Subject: Re: [rrd-users] Sanity checks on Buffer Cache and I/O at
From: Steve Shipway <s.shipway () auckland ! ac ! nz>
Date: 2011-09-27 22:46:23
Message-ID: 7294716191A1E142B80615ED2C633BCAF54C88 () uxcn10-2 ! UoA ! auckland ! ac ! nz
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
>From: rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch
[mailto:rrd-users->bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch] On
Behalf Of Ryan Kubica
>
>2) don't bother with rrdcached ( it's slow, adds complication, so just
adds an intermediate
>buffer (with it's own IO) which doesn't serve a useful purpose )
I would, in general, disagree with this ("rrdcached is slow"); it really
comes down to how you configure it.
I've done a number of tests for performance of MRTG + RRDTool + rrdcached to
see the relative performance of different versions and options. The benefit
of rrdcached is heavily dependent on how aggressively it is configured; if
you set it to cache minimally then you have little benefit, if any.
However, a more aggressive configuration (>5 steps cache time) will give you
gains as it updates the files less frequently. Of course, it depends again
on how often you are reading the files now, as reading via rrdcached forces
a flush and you lose any additional benefits that would be accrued over the
remainder of the caching window.
Another benefit of rrdcached (at least for MRTG users) is that it allows you
to decouple your collector, frontend and backend so that you can have your
RRD on one server (with a fast disk IO) but have multiple other servers
collecting data and feeding it back via rrdcached over TCP. Similarly,
frontend web servers displaying the data can pull it via rrdcached over the
network.
Also, since the rrd client just has to pass over the data to rrdcached, and
not wait for the actual write, return is much faster.
However, on the downside, rrdcached is relatively immature - you need to use
the trunk version to get all the functionality, the stable version is
limited to update, fetch and graph. The security side can sometimes bite
you when running over TCP. And, of course, you have to tune it and
configure it correctly in order to get the maximum benefit, and there is as
yet not so much documentation available on how to best achieve this.
Steve
_____
Steve Shipway
ITS Unix Services Design Lead
University of Auckland, New Zealand
Floor 1, 58 Symonds Street, Auckland
Phone: +64 (0)9 3737599 ext 86487
DDI: +64 (0)9 924 6487
Mobile: +64 (0)21 753 189
Email: <mailto:s.shipway@auckland.ac.nz> s.shipway@auckland.ac.nz
P Please consider the environment before printing this e-mail
[Attachment #5 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" \
CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 \
(filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);} o\:* \
{behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Webdings;
panose-1:5 3 1 2 1 5 9 6 7 3;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Arial Narrow";
panose-1:2 11 6 6 2 2 2 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-tab-span
{mso-style-name:apple-tab-span;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-NZ link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal style='margin-left:5.25pt'><b><span lang=EN-US \
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>>From:</span></b><span \
lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> \
rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch \
[mailto:rrd-users->bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch] <b>On Behalf \
Of </b>Ryan Kubica<br></span><span \
style='font-size:10.0pt;font-family:"Arial","sans-serif";color:black'>><o:p></o:p></span></p><p \
class=MsoNormal style='background:white'><span \
style='font-size:10.0pt;font-family:"Arial","sans-serif";color:black'> >2) \
don't bother with rrdcached ( it's slow, adds complication, so just adds an \
intermediate <o:p></o:p></span></p><p class=MsoNormal style='background:white'><span \
style='font-size:10.0pt;font-family:"Arial","sans-serif";color:black'> >buffer \
(with it's own IO) which doesn't serve a useful purpose )<o:p></o:p></span></p><p \
class=MsoNormal style='margin-bottom:12.0pt;background:white'><span \
style='color:black'><o:p> </o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I would, in \
general, disagree with this (“rrdcached is slow”); it really comes down \
to how you configure it.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I’ve \
done a number of tests for performance of MRTG + RRDTool + rrdcached to see the \
relative performance of different versions and options. The benefit of \
rrdcached is heavily dependent on how aggressively it is configured; if you set it to \
cache minimally then you have little benefit, if any. However, a more \
aggressive configuration (>5 steps cache time) will give you gains as it updates \
the files less frequently. Of course, it depends again on how often you are \
reading the files now, as reading via rrdcached forces a flush and you lose any \
additional benefits that would be accrued over the remainder of the caching \
window.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Another \
benefit of rrdcached (at least for MRTG users) is that it allows you to decouple your \
collector, frontend and backend so that you can have your RRD on one server (with a \
fast disk IO) but have multiple other servers collecting data and feeding it back via \
rrdcached over TCP. Similarly, frontend web servers displaying the data can \
pull it via rrdcached over the network.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Also, since \
the rrd client just has to pass over the data to rrdcached, and not wait for the \
actual write, return is much faster.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>However, on \
the downside, rrdcached is relatively immature – you need to use the trunk \
version to get all the functionality, the stable version is limited to update, fetch \
and graph. The security side can sometimes bite you when running over \
TCP. And, of course, you have to tune it and configure it correctly in order to \
get the maximum benefit, and there is as yet not so much documentation available on \
how to best achieve this.<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve<o:p></o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div \
class=MsoNormal align=center style='text-align:center'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><hr size=2 \
width="100%" align=center></span></div><p class=MsoNormal><b><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve \
Shipway<o:p></o:p></span></b></p><p class=MsoNormal><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>ITS Unix \
Services Design Lead<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>University \
of Auckland, New Zealand<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Floor 1, 58 \
Symonds Street, Auckland<o:p></o:p></span></p><p class=MsoNormal><i><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Phone: +64 \
(0)9 3737599 ext 86487<o:p></o:p></span></i></p><p class=MsoNormal><i><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>DDI: +64 \
(0)9 924 6487<o:p></o:p></span></i></p><p class=MsoNormal><i><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Mobile: +64 \
(0)21 753 189<o:p></o:p></span></i></p><p class=MsoNormal><i><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Email: <a \
href="mailto:s.shipway@auckland.ac.nz"><span \
style='color:#595959'>s.shipway@auckland.ac.nz</span></a><o:p></o:p></span></i></p><p \
class=MsoNormal><span lang=EN-GB \
style='font-size:18.0pt;font-family:Webdings;color:green'>P</span><span lang=EN-GB \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> </span><span \
lang=EN-GB style='font-size:10.0pt;font-family:"Arial \
Narrow","sans-serif";color:green'>Please consider the environment before printing \
this e-mail</span><span lang=EN-GB \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> </span><span \
lang=EN-GB style='font-size:7.5pt;font-family:"Verdana","sans-serif";color:navy'><o:p></o:p></span></p><p \
class=MsoNormal><i><span \
style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></i></p></div><div \
style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm \
4.0pt'><div><div><div><p class=MsoNormal \
style='margin-bottom:12.0pt;background:white'><span \
style='color:black'><o:p> </o:p></span></p></div></div></div></div></div></body></html>
["smime.p7s" (application/x-pkcs7-signature)]
_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
--===============1699188270==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic