[prev in list] [next in list] [prev in thread] [next in thread]
List: bacula-users
Subject: Re: [Bacula-users] Issue with config setting "Max Virtual Full Interval"
From: Kern Sibbald <kern () sibbald ! com>
Date: 2020-08-16 17:17:58
Message-ID: 7ff30d29-e1f4-1804-a5c2-ffbfe101c4c4 () sibbald ! com
[Download RAW message or body]
[Attachment #2 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello,</p>
<p>This means that you do not have both a read device and a write
device defined for the job that is running. Unfortunately the
error message does not tell you which device (read or write) is
not defined in the job definition.</p>
<p>Perhaps if you turn on a debug level of about 150 or possibly
200, you might see more clearly what is wrong. In the next
version, I will improve the error message to be specific about
which of the two devices (read and write) is not defined.<br>
</p>
<p>Best regards,</p>
<p>Kern<br>
</p>
<div class="moz-cite-prefix">On 8/15/20 12:40 PM, Shaligram Bhagat,
Yateen (Nokia - IN/Bangalore) wrote:<br>
</div>
<blockquote type="cite"
cite="mid:DB6PR0701MB25825E4BBB5BA676BA0F1F99DB410@DB6PR0701MB2582.eurprd07.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: \
#800000 2px solid; } --></style> <font size="2" face="Calibri"><span \
style="font-size:11pt;"> <div>Hi Martin & Radoslaw,</div>
<div> </div>
<div>This is an old issue raised by me in Jan 2020; which both
of you had responded to earlier.</div>
<div>I am seeing it again as the particular scenario causing
the issue did not appear till now since my last reporting.</div>
<div> </div>
<div>I am using bacula v 9.4.4 on Centos 6.4. for disk file
based backup.</div>
<div> </div>
<div>The backup scheme is weekdays incremental and virtual
full back-up on Saturday.</div>
<div> </div>
<div>Things were running fine till the interval between the
last virtual full back up job and the current incremental
job was less than <b>Max Virtual Full Interval = 7 days</b></div>
<div> </div>
<div> When this interval exceeded Max Virtual Full Interval =
7 days ; for an incremental backup, bacula tried to create
a new virtual full back-up by consolidating the latest
virtual full backup and all subsequent incremental backups
(...fair enough this
is as expected) But while consolidating the jobs ; it gave
error :</div>
<div> </div>
<div> </div>
<table style="width:408.2pt;margin-left:70.6pt;" width="680">
<colgroup><col style="width:408.2pt;" width="680">
</colgroup><tbody>
<tr>
<td>J<b>ob Log</b>
<div>bacula-server-dir </div>
<div> No prior or suitable Full backup found in
catalog. Doing Virtual FULL backup.</div>
<div>bacula-server-dir shell command: run BeforeJob
"/opt/bacula/srpg/scripts/validate_testbed.sh
blhwsync11"</div>
<div>bacula-server-dir Consolidating
JobIds=49733,49985,50189,50390,50575,50769,50964</div>
<div> Start Virtual Backup JobId 51162,
Job=blhwsync11.2020-08-15_15.40.22_18</div>
<div> BeforeJob: blhwsync11 is available in
regress.params and reachable</div>
<div>bacula-server-dir shell command: run AfterJob
"/opt/bacula/srpg/scripts/send_fail_mail.sh 51162
blhwsync11.2020-08-15_15.40.22_18
<a class="moz-txt-link-abbreviated" \
href="mailto:sas-backup-admin@list.nokia.com">sas-backup-admin@list.nokia.com</a>"</div>
<div>bacula-server-sd Elapsed time=443746:10:30,
Transfer rate=0 Bytes/second</div>
<div>bacula-server-dir </div>
<div>Error: Bacula bacula-server-dir 9.4.4 (28May19):</div>
<div> Build OS:
x86_64-redhat-linux-gnu-bacula redhat </div>
<div> JobId: 51162</div>
<div> Job:
blhwsync11.2020-08-15_15.40.22_18</div>
<div> Backup Level: Virtual Full</div>
<div> Client: "blhwsync11-fd" 9.4.4
(28May19) x86_64-redhat-linux-gnu-bacula,redhat,</div>
<div> FileSet: "blhwsync11" 2020-07-29
13:30:02</div>
<div> Pool: "TestbedFull-F" (From
Job VFullPool override)</div>
<div> Catalog: "MyCatalog" (From
Client resource)</div>
<div> Storage: "StorageF" (From Pool
resource)</div>
<div> Scheduled time: 15-Aug-2020 15:40:22</div>
<div> Start time: 13-Aug-2020 22:01:41</div>
<div> End time: 13-Aug-2020 22:11:43</div>
<div> Elapsed time: 10 mins 2 secs</div>
<div> Priority: 10</div>
<div> SD Files Written: 0</div>
<div> SD Bytes Written: 0 (0 B)</div>
<div> Rate: 0.0 KB/s</div>
<div> Volume name(s): </div>
<div> Volume Session Id: 28</div>
<div> Volume Session Time: 1597408469</div>
<div> Last Volume Bytes: 1 (1 B)</div>
<div> SD Errors: 1</div>
<div> SD termination status: Error</div>
<div> Termination: *** Backup Error ***</div>
<div><font color="red"><b>bacula-server-sd Fatal
error: Read and write devices not properly
initialized.</b></font></div>
<div>bacula-server-dir Using Device "DeviceF1" to
write.</div>
<div> Found 40845 files to consolidate into Virtual
Full.</div>
</td>
</tr>
</tbody>
</table>
<div> </div>
<div>Later I tried running the virtual full backup
"explicitly" to consolidate the last virtual full and
subsequent incremental jobs. This went through fine without
any errors.</div>
<div>That means there are enough devices available for read
and write for the virtual full job. </div>
<div> </div>
<table style="width:407.35pt;margin-left:71.45pt;" width="678">
<colgroup><col style="width:407.35pt;" width="678">
</colgroup><tbody>
<tr>
<td><b>Job </b><b>Config </b>
<div>Job {</div>
<div> Name = "blhwsync11"</div>
<div> Max Virtual Full Interval = 7 days</div>
<div> Accurate = no # With Accurate = yes,
even file deletions, move etc are covered in
differential/incremental backup</div>
<div> DeleteConsolidatedJobs = yes</div>
<div> JobDefs = "blhwsync11"</div>
<div> RunBeforeJob =
"/opt/bacula/srpg/scripts/validate_testbed.sh
blhwsync11"</div>
<div> }</div>
<div> </div>
</td>
</tr>
<tr>
<td><b>Job Definition Config</b>
<div>JobDefs {</div>
<div> Name = "blhwsync11"</div>
<div> Type = Backup</div>
<div> Level = Incremental</div>
<div> Client = blhwsync11-fd</div>
<div> FileSet = "blhwsync11"</div>
<div> Schedule = TestbedCycle1</div>
<div> Messages = Standard</div>
<div> Pool = Default</div>
<div> Next Pool = TestbedFull-F</div>
<div> Full Backup Pool = TestbedFull-F</div>
<div> VirtualFull Backup Pool = TestbedFull-F</div>
<div> Differential Backup Pool = TestbedIncr-F</div>
<div> Incremental Backup Pool = TestbedIncr-F</div>
<div> SpoolAttributes = yes</div>
<div> SpoolData = no</div>
<div> Priority = 10</div>
<div> Write Bootstrap = "/opt/bacula/working/%c.bsr"</div>
<div>}</div>
<div> </div>
</td>
</tr>
<tr>
<td><b>Pool Config</b>
<div>Pool {</div>
<div> Name = TestbedFull-C # This is a
pool for weekly full back of user's data</div>
<div> # each testbed to have one single pool for full
backup of max size 10 GB, retained for 10 days</div>
<div> Pool Type = Backup</div>
<div> Next Pool = TestbedFull-C</div>
<div> Storage = StorageC # <b>this
storage maps to disk through virtual autochange</b><b>r
</b></div>
<div> Recycle = yes # Bacula
can automatically recycle Volumes</div>
<div> AutoPrune = no # Prune
expired volumes</div>
<div> Volume Retention = 90 days # </div>
<div> Maximum Volume Bytes = 10G # Max full
backup size per Testbed = 10G</div>
<div> Maximum Volumes = 1000</div>
<div> Label Format = "TestbedFull-C-" # Auto
label</div>
<div> Maximum Volume Jobs = 1 # one
volume per user's job </div>
<div> Action On Purge = Truncate</div>
<div>}</div>
<div>Pool {</div>
<div> Name = TestbedIncr-C # This is a
pool for incre or differential back of user's data</div>
<div> # each testbed to have one single volume for
incr or diff backup of max size 1GB, retained for
14 days</div>
<div> Pool Type = Backup</div>
<div> Next Pool = TestbedIncr-C</div>
<div> Storage = StorageC</div>
<div> Recycle = yes # Bacula
can automatically recycle Volumes</div>
<div> AutoPrune = no # Prune
expired volumes</div>
<div> Volume Retention = 90 days # </div>
<div> Maximum Volume Bytes = 1G # Max full
backup size per Testbed = 10G</div>
<div> Maximum Volumes = 1000</div>
<div> Label Format = "TestbedIncr-C-" # Auto
label</div>
<div> Maximum Volume Jobs = 1 # one
volume per testbed backup </div>
<div> Action On Purge = Truncate</div>
<div>}</div>
<div> </div>
</td>
</tr>
</tbody>
</table>
<div> </div>
<div>Thanks</div>
<div>-Yateen</div>
<div> </div>
<div> </div>
<div>-----Original Message-----</div>
<div>From: Martin Simmons <a class="moz-txt-link-rfc2396E" \
href="mailto:martin@lispworks.com"><martin@lispworks.com></a> </div> \
<div>Sent: Monday, January 20, 2020 8:02 PM</div> <div>To: Shaligram Bhagat, Yateen \
(Nokia - IN/Bangalore)
<a class="moz-txt-link-rfc2396E" \
href="mailto:yateen.shaligram_bhagat@nokia.com"><yateen.shaligram_bhagat@nokia.com></a></div>
<div>Cc: <a class="moz-txt-link-abbreviated" \
href="mailto:Bacula-users@lists.sourceforge.net">Bacula-users@lists.sourceforge.net</a></div>
<div>Subject: Re: [Bacula-users] Issue with config setting
"Max Virtual Full Interval"</div>
<div> </div>
<div>Can you post the JobDefs named "blhwsync11" and also the
Pool resource definitions?</div>
<div> </div>
<div>__Martin</div>
<div> </div>
<div> </div>
<div>> On Mon, 20 Jan 2020 11:57:45 +0000, "Shaligram
Bhagat, Yateen (Nokia said:</div>
<div>> </div>
<div>> Thanks Martin,</div>
<div>> </div>
<div>> Yes, other backups were running at the time of this
jobid 5738, I </div>
<div>> have 100 devices to handle ~200 concurrent backup
jobs. Looks like there was shortage of free device for
reading the previous full backup, although a device was
reserved for writing.</div>
<div>> </div>
<div>> Regards</div>
<div>> Yateen</div>
<div>> </div>
<div>> </div>
<div>> -----Original Message-----</div>
<div>> From: Martin Simmons <a class="moz-txt-link-rfc2396E" \
href="mailto:martin@lispworks.com"><martin@lispworks.com></a></div> <div>> \
Sent: Friday, January 17, 2020 8:06 PM</div> <div>> To: Shaligram Bhagat, Yateen \
(Nokia - IN/Bangalore) </div>
<div>> <a class="moz-txt-link-rfc2396E" \
href="mailto:yateen.shaligram_bhagat@nokia.com"><yateen.shaligram_bhagat@nokia.com></a></div>
<div>> Cc: <a class="moz-txt-link-abbreviated" \
href="mailto:Bacula-users@lists.sourceforge.net">Bacula-users@lists.sourceforge.net</a></div>
<div>> Subject: Re: [Bacula-users] Issue with config
setting "Max Virtual Full Interval"</div>
<div>> </div>
<div>> Did you have any other backups running at the same
time as jobid 5738? It looks like it didn't find a device
to read the previous backups.</div>
<div>> </div>
<div>> __Martin</div>
<div>> </div>
<div>> </div>
<div>>>> On Fri, 10 Jan 2020 09:22:04 +0000,
"Shaligram Bhagat, Yateen (Nokia said:</div>
<div>> > </div>
<div>> > Hi all,</div>
<div>> > </div>
<div>> > I am using bacula v 9.4.4 on Centos 6.4. for
disk file based backup.</div>
<div>> > </div>
<div>> > The backup scheme is weekdays incremental and
virtual full back-up on Sunday.</div>
<div>> > </div>
<div>> > Job {</div>
<div>> > Name = "blhwsync11"</div>
<div>> > Max Virtual Full Interval = 7 days</div>
<div>> > Accurate = no # With Accurate = yes,
even file deletions, move etc are covered in
differential/incremental backup</div>
<div>> > ##Backups To Keep = 3 # default = 0, means
all incremental backups till the VirtualFull are
consolidated</div>
<div>> > DeleteConsolidatedJobs = yes</div>
<div>> > JobDefs = "blhwsync11"</div>
<div>> > RunBeforeJob =
"/opt/bacula/srpg/scripts/validate_testbed.sh blhwsync11"</div>
<div>> > }</div>
<div>> > </div>
<div>> > Things were running fine till the interval
between the last virtual </div>
<div>> > full back up job and the current incremental
job was less than Max </div>
<div>> > Virtual Full Interval = 7 days</div>
<div>> > </div>
<div>> > When this interval exceeded Max Virtual Full
Interval = 7 days ; </div>
<div>> > for an incremental backup, bacula tried to
create a new virtual full backup by consolidating the latest
virtual full backup and all subsequent incremental backups
(...fair enough this is as expected) But while consolidating
the jobs ; it gave error
:</div>
<div>> > </div>
<div>> > 10-Jan 13:53 bacula-server-dir JobId 5737:
shell command: run AfterJob
"/opt/bacula/srpg/scripts/send_fail_mail.sh 5737
blhwsync11.2020-01-10_13.53.18_22
<a class="moz-txt-link-abbreviated" \
href="mailto:sas-backup-admin@list.nokia.com">sas-backup-admin@list.nokia.com</a>"</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
10-Jan 13:54 bacula-server-dir JobId 5738: No prior or
suitable Full backup found in catalog. Doing Virtual FULL
backup.</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
shell command: run BeforeJob
"/opt/bacula/srpg/scripts/validate_testbed.sh blhwsync11"</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
Start Virtual Backup </div>
<div>> > JobId 5738,
Job=blhwsync11.2020-01-10_13.54.16_04</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
Warning: This Job is not an Accurate backup so is not
equivalent to a Full backup.</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
Consolidating</div>
<div>> > JobIds=4739,5004,5216,5389 10-Jan 13:54
bacula-server-dir JobId 5738: Found 43700 files to
consolidate into Virtual Full.</div>
<div>> > 10-Jan 13:54 bacula-server-dir JobId 5738:
Using Device "DeviceF1" to write.</div>
<div>> > 10-Jan 13:54 bacula-server-sd JobId 5738: Fatal
error: Read and write devices not properly initialized.</div>
<div>> > 10-Jan 13:54 bacula-server-sd JobId 5738:
Elapsed time=438512:24:21, </div>
<div>> > Transfer rate=0 Bytes/second 10-Jan 13:54
bacula-server-dir JobId 5738: Error: Bacula
bacula-server-dir 9.4.4 (28May19):</div>
<div>> > Build OS:
x86_64-redhat-linux-gnu-bacula redhat</div>
<div>> > JobId: 5738</div>
<div>> > Job:
blhwsync11.2020-01-10_13.54.16_04</div>
<div>> > Backup Level: Virtual Full</div>
<div>> > Client: "blhwsync11-fd" 9.4.4
(28May19) x86_64-redhat-linux-gnu-bacula,redhat,</div>
<div>> > FileSet: "blhwsync11"
2019-12-10 22:00:01</div>
<div>> > Pool: "TestbedFull-F" (From
Job VFullPool override)</div>
<div>> > Catalog: "MyCatalog" (From
Client resource)</div>
<div>> > Storage: "StorageF" (From Pool
resource)</div>
<div>> > Scheduled time: 10-Jan-2020 13:54:14</div>
<div>> > Start time: 08-Jan-2020 22:00:02</div>
<div>> > End time: 08-Jan-2020 22:00:14</div>
<div>> > Elapsed time: 12 secs</div>
<div>> > Priority: 10</div>
<div>> > SD Files Written: 0</div>
<div>> > SD Bytes Written: 0 (0 B)</div>
<div>> > Rate: 0.0 KB/s</div>
<div>> > Volume name(s):</div>
<div>> > Volume Session Id: 1</div>
<div>> > Volume Session Time: 1578644639</div>
<div>> > Last Volume Bytes: 1 (1 B)</div>
<div>> > SD Errors: 1</div>
<div>> > SD termination status: Error</div>
<div>> > Termination: *** Backup Error ***</div>
<div>> > </div>
<div>> > Later I tried running the virtual full backup
"explicitly" to consolidate the last virtual full and
subsequent incremental jobs. This went through fine without
any errors.</div>
<div>> > </div>
<div>> > So the issue seems to be in the Bacula's
handling of the configuration setting : "Max Virtual Full
Interval " ?</div>
<div>> > Any advice ?</div>
<div>> > </div>
<div>> > Thanks,</div>
<div>> > Yateen</div>
<div>> > </div>
<div>> > </div>
<div>> </div>
<div> </div>
</span></font>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" \
wrap="">_______________________________________________ Bacula-users mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:Bacula-users@lists.sourceforge.net">Bacula-users@lists.sourceforge.net</a>
<a class="moz-txt-link-freetext" \
href="https://lists.sourceforge.net/lists/listinfo/bacula-users">https://lists.sourceforge.net/lists/listinfo/bacula-users</a>
</pre>
</blockquote>
</body>
</html>
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic