[prev in list] [next in list] [prev in thread] [next in thread]
List: bacula-users
Subject: Re: [Bacula-users] Help with monitoring/tunning Bacula v5.2.5
From: Bryn Hughes <linux () nashira ! ca>
Date: 2015-02-20 2:06:02
Message-ID: 54E6968A.603 () nashira ! ca
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Guessing those 3.4M files are mostly small files? They are likely
scattered across the disk.
Are you backing up to a disk destination, or to tape? From your verbiage
I'm guessing to disk...
Have you tried looking at iostat on the source server when a backup is
running? I would not at all be surprised to see that your source disks
are basically 100% busy if you have a very large number of small files.
What kind of data are you backing up? Is this something like a mail
store filled with maildirs or a subversion repository filled with small
files?
Bryn
On 2015-01-28 03:43 PM, Sergio Nemirovsky wrote:
>
> We are using Bacula 5.2.5 to backup a server with a huge amount of files.
>
> After a full backup was done (It took 3+ days), the incremental took
> 10+ hours.
>
> Notice the lines in the log:
>
> 27-Jan 23:10 server-xxx-fd JobId 1157: /run is a different filesystem.
> Will not descend from / into it.
>
> 28-Jan 09:16 server-xxx-fd JobId 1157: /boot is a different
> filesystem. Will not descend from / into it.
>
> We are not defining the size of the volumes, so all the data from the
> job goes into just one volume.
>
> We have 3.4M files in this incremental.
>
> Could this be the cause of the delay?
>
> What would be the best approach to tune this beast?
>
> Thanks!
>
> Log:
>
> ------------------------------------------------
>
> 27-Jan 23:10 server-xxx-fd JobId 1157: /dev is a different filesystem.
> Will not descend from / into it.
>
> 27-Jan 23:10 server-xxx-fd JobId 1157: /run is a different filesystem.
> Will not descend from / into it.
>
> 28-Jan 09:16 server-xxx-fd JobId 1157: /boot is a different
> filesystem. Will not descend from / into it.
>
> 28-Jan 09:16 server-xxx-sd JobId 1157: Job write elapsed time =
> 10:06:52, Transfer rate = 253.8 K Bytes/second
>
> .
>
> .
>
> .Non-fatal FD errors: 0
>
> SD Errors: 0
>
> FD termination status: OK
>
> SD termination status: OK
>
> Termination: Backup OK
>
> ------------------------------------------------
>
>
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming. The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net/
>
>
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
[Attachment #5 (text/html)]
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Guessing those 3.4M files are mostly
small files? They are likely scattered across the disk.<br>
<br>
Are you backing up to a disk destination, or to tape? From your
verbiage I'm guessing to disk...<br>
<br>
Have you tried looking at iostat on the source server when a
backup is running? I would not at all be surprised to see that
your source disks are basically 100% busy if you have a very large
number of small files. What kind of data are you backing up? Is
this something like a mail store filled with maildirs or a
subversion repository filled with small files?<br>
<br>
Bryn<br>
<br>
On 2015-01-28 03:43 PM, Sergio Nemirovsky wrote:<br>
</div>
<blockquote
cite="mid:639E2A0778F9CC439E2AFC78A9928982790C21@dwmail2.dwavesys.local"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">We are using Bacula 5.2.5 to backup a
server with a huge amount of files.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">After a full backup was done (It took 3+
days), the incremental took 10+ hours.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Notice the lines in the log:<o:p></o:p></p>
<p class="MsoNormal">27-Jan 23:10 server-xxx-fd JobId 1157:
/run is a different filesystem. Will not descend from / into
it.<o:p></o:p></p>
<p class="MsoNormal">28-Jan 09:16 server-xxx-fd JobId 1157:
/boot is a different filesystem. Will not descend from / into
it.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are not defining the size of the
volumes, so all the data from the job goes into just one
volume.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have 3.4M files in this incremental. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Could this be the cause of the delay?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">What would be the best approach to tune
this beast?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Log:<o:p></o:p></p>
<p class="MsoNormal">------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal">27-Jan 23:10 server-xxx-fd JobId 1157:
/dev is a different filesystem. Will not descend from / into
it.<o:p></o:p></p>
<p class="MsoNormal">27-Jan 23:10 server-xxx-fd JobId 1157:
/run is a different filesystem. Will not descend from / into
it.<o:p></o:p></p>
<p class="MsoNormal">28-Jan 09:16 server-xxx-fd JobId 1157:
/boot is a different filesystem. Will not descend from / into
it.<o:p></o:p></p>
<p class="MsoNormal">28-Jan 09:16 server-xxx-sd JobId 1157: Job
write elapsed time = 10:06:52, Transfer rate = 253.8 K
Bytes/second<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">.<o:p></o:p></p>
<p class="MsoNormal">.<o:p></o:p></p>
<p class="MsoNormal">.Non-fatal FD errors: 0<o:p></o:p></p>
<p class="MsoNormal"> SD Errors: 0<o:p></o:p></p>
<p class="MsoNormal"> FD termination status: OK<o:p></o:p></p>
<p class="MsoNormal"> SD termination status: OK<o:p></o:p></p>
<p class="MsoNormal"> Termination: Backup OK<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">------------------------------------------------<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. <a class="moz-txt-link-freetext" \
href="http://goparallel.sourceforge.net/">http://goparallel.sourceforge.net/</a></pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Bacula-users mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:Bacula-users@lists.sourceforge.net">Bacula-users@lists.sourceforge.net</a>
<a class="moz-txt-link-freetext" \
href="https://lists.sourceforge.net/lists/listinfo/bacula-users">https://lists.sourceforge.net/lists/listinfo/bacula-users</a>
</pre>
</blockquote>
<br>
</body>
</html>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic