[prev in list] [next in list] [prev in thread] [next in thread]
List: amanda-users
Subject: Request for help: 'amdump' creates empty backup
From: Thomas Fischer <fischer () unix-ag ! uni-kl ! de>
Date: 2016-12-11 20:13:04
Message-ID: 20161211201304.GA28001 () unix-ag ! uni-kl ! de
[Download RAW message or body]
Hello,
I am trying to setup Amanda 3.3.9 on two Linux (ArchLinux) systems. I have succeeded \
to get a basic backup-recovery cycle running on one machine, but failed to reproduce \
the setup on the other machine. After staring on my screen for two days, I have to \
admit that I cannot see which mistake I have made so that Amanda fails on one machine \
but works on the other one. But first the details:
On my desktop machine running a current ArchLinux where I succeeded to run Amanda, I \
have compile Amanda 3.3.9 from source using the following invocations:
./autogen
./configure --without-force-uid \
--with-security-file="${DESTDIR}/etc/amanda/amanda-security.conf" \
--with-single-userid --prefix=${DESTDIR}${INSTALLDIRECTORY} --disable-rpath \
--disable-nls --disable-s3-device --disable-dvdrw-device --disable-ndmp-device \
--with-user=${AMANDA_USER} --with-group=${AMANDA_GROUP} --with-index-server=localhost \
--with-readline --without-ipv6 --sysconfdir=${DESTDIR}/etc \
--with-configdir=${DESTDIR}/etc/amanda --localstatedir=${DESTDIR}/var \
--with-amperldir=${DESTDIR}${INSTALLDIRECTORY}/share/perl5/site_perl make && make \
install
As this is supposed to be a test installation, I set various variables used in above \
invocations as follows:
AMANDA_USER=$(id -u -n)
AMANDA_GROUP=$(id -g -n)
DESTDIR="/tmp/amanda-destdir"
INSTALLDIRECTORY="/usr"
To be able to run Amanda from command line, I set variables PATH, PERL5LIB, and \
LD_LIBRARY_PATH to cover the locations where Amanda got installed to. You may have \
also noticed that I am not using a special user (just my regular account) as well as \
using the configure flag '--with-single-userid'.
I created the following directories:
/tmp/backup/vtapes/slot{1,2,3,4,5}
/tmp/backup/holding
/tmp/backup/state/{curinfo,log,index}
/tmp/amanda-destdir/var/amanda/gnutar-lists
.. and used the following configuration files, inspired by the GSWA Wiki \
documentation and examples ('f15h' is my regular username):
---8<--- /tmp/amanda-destdir/etc/amanda/mybackup/amanda.conf ---8<---
org "LIM" # your organization name for reports
displayunit "k" # k=kilo, m=mega, g=giga, t=tera
infofile "/tmp/backup/state/curinfo"
logdir "/tmp/backup/state/log"
indexdir "/tmp/backup/state/index"
dumpuser "f15h"
runtapes 1 # number of tapes to be used in a single run of amdump
# tpchanger ('tape changer' can alternate between multiple tapes (like a robot)
# specifically 'chg-disk' is able to switch between multiple directories representing \
virtual tapes tpchanger "chg-disk:/tmp/backup/vtapes"
labelstr "^User_f15h_Backup_[0-9][0-9]$" # regular expression, to recognize labels
# No auto-labeling, should have been done using 'amlabel' before running 'amdump'
### autolabel "User_f15h_Backup_%%" EMPTY VOLUME_ERROR
# The maximum time between full backups (without specifying week day):
dumpcycle 3 days
# How often is 'amdump' run within the time period of 'dumpcycle'?
# For example, if dumpcycle is '4 weeks' and amdump is run on every weekday (mo-fr),
# 'runspercycle' should be 20 (4*5)
runspercycle 1
# Number of tapes in use, here the number of vtapes/slot* directories
# Must be at least 'runspercycle' (assuming one tape is sufficient for a full backup)
tapecycle 3
amrecover_changer "changer" # 'amrecover' will use the changer if you restore from \
this device
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
# By default, Amanda can only track at most one run per calendar day. When
# the usetimestamps option is enabled, however, Amanda can track as many
# runs as you care to make.
usetimestamps yes
tapetype "TEST-TAPE"
define tapetype TEST-TAPE {
length 1000 mbytes
filemark 4 kbytes
}
define dumptype simple-gnutar-local {
auth "local"
index yes
record no
compress none
program "GNUTAR"
}
# Holding disk is buffer between original data and 'tape', thus a staging area for \
backups holdingdisk hd1 {
comment "Holding disk"
directory "/tmp/backup/holding"
use -500 Mb # how much space can we use on it; a non-positive value means: use \
all space but that value chunksize 1 mbyte
}
---8<---8<---8<---
---8<--- /tmp/amanda-destdir/etc/amanda/mybackup/disklist ---8<---
# HOST DIRECTORY-ON-HOST-TO-BACKUP DUMPTYPE
localhost /tmp/example-data simple-gnutar-local
---8<---8<---8<---
Directory /tmp/example-data is filled with some files to see if backup and restore is \
working.
---8<--- /tmp/amanda-destdir/etc/amanda/amanda-client.conf ---8<---
index_server "localhost"
tapedev "changer"
auth "local" # no special authentication, rely on operating system
---8<---8<---8<---
Labels are set on the virtual tapes as follow:
cd /tmp/backup/vtapes && for slotdir in slot[1-9]* ; do export \
slotnumber=${slotdir/slot} ; amlabel mybackup User_f15h_Backup_$(printf "%02d" \
${slotnumber}) slot ${slotnumber} ; done
The first test is to run 'amcheck':
---8<---8<---8<---
$ amcheck mybackup
Amanda Tape Server Host Check
-----------------------------
Holding disk /tmp/backup/holding: 7258112 KB disk space available, using 6746112 KB
slot 1: volume 'User_f15h_Backup_01'
Will write to volume 'User_f15h_Backup_01' in slot 1.
NOTE: skipping tape-writable test
NOTE: host info dir /tmp/backup/state/curinfo/localhost does not exist
NOTE: it will be created on the next run.
NOTE: index dir /tmp/backup/state/index/localhost does not exist
NOTE: it will be created on the next run.
Server check took 0.116 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 1.030 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
.. and 'amreport':
---8<---8<---8<---
$ amreport mybackup
nothing to report on!
---8<---8<---8<---
(well, nothing had been backup'ed)
Then, I try to trigger a backup, running 'amdump'. No output, so I run 'amcheck' and \
'amreport' again:
---8<---8<---8<---
$ amdump mybackup
$ amcheck mybackup
Amanda Tape Server Host Check
-----------------------------
Holding disk /tmp/backup/holding: 6639616 KB disk space available, using 6127616 KB
slot 1: volume 'User_f15h_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_f15h_Backup_02'
Will write to volume 'User_f15h_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.114 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 1.021 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
$ amreport mybackup
Hostname: desktop
Org : LIM
Config : mybackup
Date : december 11, 2016
These dumps were to tape User_f15h_Backup_01.
The next tape Amanda expects to use is: 1 new tape.
The next new tape already labelled is: User_f15h_Backup_02.
STATISTICS:
Total Full Incr. Level:#
-------- -------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 603.3 603.3 0.0
Original Size (meg) 603.3 603.3 0.0
Avg Compressed Size (%) 100.0 100.0 --
DLEs Dumped 1 1 0
Avg Dump Rate (k/s) 747857 747857 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 603.3 603.3 0.0
Tape Used (%) 60.3 60.3 0.0
DLEs Taped 1 1 0
Parts Taped 1 1 0
Avg Tp Write Rate (k/s) 6177300 6177300 --
USAGE BY TAPE:
Label Time Size % DLEs Parts
User_f15h_Backup_01 0:00 617730K 60.3 1 1
NOTES:
planner: Adding new disk localhost:/tmp/example-data.
taper: Slot 1 with label User_f15h_Backup_01 is usable
taper: tape User_f15h_Backup_01 kb 617730 fm 1 [OK]
DUMP SUMMARY:
DUMPER STATS TAPER \
STATS HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS \
KB/s
--------------------------------8<----------------------8<----------------8<----------------
localhost /tmp/example-data 0 617730 617730 -- 0:01 747807.3 0:00 \
6177300.0
(brought to you by Amanda version 3.3.9)
---8<---8<---8<---
As seen in the STATISTICS section in above output, about 600 MB have been backup'ed, \
which is about right. Indeed, I can recover those files with 'amrecover', although I \
omit the output here.
Now, my problem starts when I try to configure Amanda in a virtual machine in a setup \
closer to the intended deployment target, but still a testing environment. The guest \
in the virtual machine is running a current ArchLinux as well. This ArchLinux \
instance is my custom-built USB/ISO live system, built and configured as described \
here: https://wiki.archlinux.org/index.php/Archiso Being a read-only live system, I \
have attached a virtual harddrive to this virtual machine; this harddrive is mounted \
under /backup. The data to be backup'ed is read from /windows/documents.
Again, I start by building Amanda from source on the desktop system, where the live \
system's ISO file will be assembled. The instructions look as follows (should be \
identical to above, except for different paths and users):
./autogen
./configure --without-force-uid \
--with-security-file="/etc/amanda/amanda-security.conf" --with-single-userid \
--prefix=${INSTALLDIRECTORY} --disable-rpath --disable-nls --disable-s3-device \
--disable-dvdrw-device --disable-ndmp-device --with-user=${AMANDA_USER} \
--with-group=${AMANDA_GROUP} --with-index-server=localhost --with-readline \
--without-ipv6 --sysconfdir=/etc --with-configdir=/etc/amanda --localstatedir=/var \
--with-amperldir=${INSTALLDIRECTORY}/share/perl5/site_perl
.. using the following variables:
AMANDA_USER=amanda (regular user account, UID >= 1000, auto-loggedin on the live \
system) AMANDA_GROUP=amanda (corresponding group)
DESTDIR="/path/to/airootfs" ('airootfs' is the name of the directory during live \
system customization that will later become '/' on the running live system) \
INSTALLDIRECTORY="/usr" (not '/usr/local')
The configuration files look as follows ('usb' is the name of the configuration, \
paths are as seen on a running live system):
---8<--- /etc/amanda/usb/amanda.conf ---8<---
org "LIM" # your organization name for reports
displayunit "k" # k=kilo, m=mega, g=giga, t=tera
infofile "/backup/state/curinfo"
logdir "/backup/state/log"
indexdir "/backup/state/index"
dumpuser "amanda"
runtapes 1 # number of tapes to be used in a single run of amdump
# tpchanger ('tape changer' can alternate between multiple tapes (like a robot)
# specifically 'chg-disk' is able to switch between multiple directories representing \
virtual tapes tpchanger "chg-disk:/backup/vtapes"
labelstr "^User_amanda_Backup_[0-9][0-9]$" # regular expression, to recognize labels
# No auto-labeling, should have been done using 'amlabel' before running 'amdump'
### autolabel "User_amanda_Backup_%%" EMPTY VOLUME_ERROR
# The maximum time between full backups (without specifying week day):
dumpcycle 3 days
# How often is 'amdump' run within the time period of 'dumpcycle'?
# For example, if dumpcycle is '4 weeks' and amdump is run on every weekday (mo-fr),
# 'runspercycle' should be 20 (4*5)
runspercycle 1
# Number of tapes in use, here the number of vtapes/slot* directories
# Must be at least 'runspercycle' (assuming one tape is sufficient for a full backup)
tapecycle 3
amrecover_changer "changer" # 'amrecover' will use the changer if you restore from \
this device
bumpsize 20 Mb # minimum savings (threshold) to bump level 1 -> 2
bumppercent 20 # minimum savings (threshold) to bump level 1 -> 2
bumpdays 1 # minimum days at each level
bumpmult 4 # threshold = bumpsize * bumpmult^(level-1)
# By default, Amanda can only track at most one run per calendar day. When
# the usetimestamps option is enabled, however, Amanda can track as many
# runs as you care to make.
usetimestamps yes
tapetype "TEST-TAPE"
define tapetype TEST-TAPE {
length 1000 mbytes
filemark 4 kbytes
}
define dumptype simple-gnutar-local {
auth "local"
index yes
record no
compress none
program "GNUTAR"
}
# Holding disk is buffer between original data and 'tape', thus a staging area for \
backups holdingdisk hd1 {
comment "Holding disk"
directory "/backup/holding"
use -500 Mb # how much space can we use on it; a non-positive value means: use \
all space but that value chunksize 1 mbyte
}
---8<---8<---8<---
---8<--- /etc/amanda/usb/disklist ---8<---
# HOST DIRECTORY-ON-HOST-TO-BACKUP DUMPTYPE
localhost /windows/documents simple-gnutar-local
---8<---8<---8<---
---8<--- /etc/amanda/amanda-client.conf ---8<---
index_server "localhost"
tapedev "changer"
auth "local" # no special authentication, rely on operating system
---8<---8<---8<---
During boot time, a special script is run as root to initialize all directories for \
Amanda:
---8<---8<---8<---
AMANDA_USER=amanda
WRITEABLEETCAMANDA=/backup/etc-amanda
READONLYETCAMANDA=/etc/amanda
WRITEABLEVARAMANDA=/backup/var-amanda
READONLYVARAMANDA=/var/amanda
# Create, mount, and populate a writable /etc/amanda directory
# Assumption: /backup/etc-amanda is a writable directory and
# /etc/amanda is read-only but contains valid data
mkdir -p ${WRITEABLEETCAMANDA} ${READONLYETCAMANDA} # in case it does not yet exist
umount ${READONLYETCAMANDA} 2>/dev/null # in case it is mounted
# If there is no valid content in /backup/etc-amanda/, copy valid, read-only data \
from /etc/amanda there test ! -f ${WRITEABLEETCAMANDA}/amanda-client.conf && test -d \
${READONLYETCAMANDA} && cp -r ${READONLYETCAMANDA}/* ${WRITEABLEETCAMANDA}/ # Always \
use latest, read-only copy of certain configuration files cp \
${READONLYETCAMANDA}/usb/disklist ${READONLYETCAMANDA}/usb/amanda.conf \
${WRITEABLEETCAMANDA}/usb/ cp ${READONLYETCAMANDA}/amanda-client.conf \
${WRITEABLEETCAMANDA}/ chown -R amanda:amanda ${WRITEABLEETCAMANDA} # fix ownership
chown root:root /etc/amanda/amanda-security.conf
mount -o bind ${WRITEABLEETCAMANDA} ${READONLYETCAMANDA} # mount writable location \
/backup/etc-amanda to /etc/amanda, shadowing read-only content
# Create, mount, and populate a writable /var/amanda directory
mkdir -p ${WRITEABLEVARAMANDA} ${READONLYVARAMANDA} # in case it does not yet exist
mkdir -p ${WRITEABLEVARAMANDA}/gnutar-lists # Create directory that may be missing
umount ${READONLYVARAMANDA} 2>/dev/null # in case it is mounted
chown -R amanda:amanda ${WRITEABLEVARAMANDA} # fix ownership
mount -o bind ${WRITEABLEVARAMANDA} ${READONLYVARAMANDA} # mount writable location \
/backup/var-amanda to /var/amanda, shadowing read-only content
# Create directories that may be missing
mkdir -p /backup/vtapes/slot{1,2,3,4,5} /backup/holding \
/backup/state/{curinfo,log,index} chown -R amanda:amanda /backup/vtapes \
/backup/holding /backup/state
( cd /backup/vtapes && for slotdir in slot[1-9]* ; do export \
slotnumber=${slotdir/slot} ; su -c "amlabel usb User_amanda_Backup_$(printf "%02d" \
${slotnumber}) slot ${slotnumber}" amanda ; done )
su -c "amtape usb reset" amanda
---8<---8<---8<---
Some explanation: To allow for some persistence across reboots, /var/amanda and \
/etc/amanda will be bind-mounted from /backup/var-amanda and /backup/etc-amanda, \
respectively. Before mounting /backup/etc-amanda into /etc/amanda, some/all \
configuration files are copied from /etc/amanda to /backup/etc-amanda (to be precise, \
files 'disklist' and 'amanda.conf' as shown above. Also, under /backup, the \
directories vtapes/slot{1,2,3,4,5}, holding, and state/{curinfo,log,index} are \
created. All ownerships are set to amanda:amanda.
'amlabel' is used to set the labels for the virtual tapes. Eventually, 'amtape' is \
run.
The output of above script, as extract from systemd's logs looks as follows:
---8<---8<---8<---
Dec 11 15:17:37 amandausb systemd[1]: Starting Initialize directories and \
configuration for Amanda...
Dec 11 15:17:38 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:38 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:38 amandausb init-amanda[314]: Writing label 'User_amanda_Backup_01'...
Dec 11 15:17:38 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:38 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label 'User_amanda_Backup_02'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label 'User_amanda_Backup_03'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label 'User_amanda_Backup_04'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: Reading label...
Dec 11 15:17:39 amandausb init-amanda[314]: Found an empty tape.
Dec 11 15:17:39 amandausb init-amanda[314]: Writing label 'User_amanda_Backup_05'...
Dec 11 15:17:39 amandausb init-amanda[314]: Checking label...
Dec 11 15:17:39 amandausb init-amanda[314]: Success!
Dec 11 15:17:39 amandausb init-amanda[314]: changer is reset
Dec 11 15:17:39 amandausb systemd[1]: Started Initialize directories and \
configuration for Amanda.
---8<---8<---8<---
Now I am trying to run 'amcheck usb' and 'amreport usb' as regular user 'amanda' \
inside the running live system on a fresh /backup (i.e. no data was ever dumped):
---8<---8<---8<---
$ amreport usb
nothing to report on!
$ amcheck usb
Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01'
Will write to volume 'User_amanda_Backup_01' in slot 1.
NOTE: skipping tape-writable test
NOTE: host info dir /backup/state/curinfo/localhost does not exist
NOTE: it will be created on the next run.
NOTE: index dir /backup/state/index/localhost does not exist
NOTE: it will be created on the next run.
Server check took 0.141 seconds
Amanda Backup Client Hosts Check
--------------------------------
ERROR: localhost: [/etc/amanda/amanda-security.conf \
(/etc/amanda/amanda-security.conf) is not owned by root] Client check: 1 host checked \
in 1.026 seconds. 1 problem found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
Running 'amdump usb' now, followed by 'amcheck usb' and 'amreport usb', all as \
regular user 'amanda':
---8<---8<---8<---
$ amdump usb
$ amreport usb
Hostname: amandausb
Org : LIM
Config : usb
Date : December 11, 2016
These dumps were to tape User_amanda_Backup_01.
The next tape Amanda expects to use is: 1 new tape.
The next new tape already labelled is: User_amanda_Backup_02.
STATISTICS:
Total Full Incr. Level:#
-------- -------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:00
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) 100.0 100.0 --
DLEs Dumped 1 1 0
Avg Dump Rate (k/s) 714.3 714.3 --
Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_amanda_Backup_02'
Will write to volume 'User_amanda_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.158 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 2.029 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
0.0 0.0 0.0
DLEs Taped 1 1 0
Parts Taped 1 1 0
Avg Tp Write Rate (k/s) 100.0 100.0 --
USAGE BY TAPE:
Label Time Size % DLEs Parts
User_amanda_Backup_01 0:00 10K 0.0 1 1
NOTES:
planner: Adding new disk localhost:/windows/documents.
taper: Slot 1 with label User_amanda_Backup_01 is usable
taper: tape User_amanda_Backup_01 kb 10 fm 1 [OK]
DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-KB OUT-KB COMP% MMM:SS KB/s MMM:SS KB/s
--------------------------------- ---------------------- -------------- -------------
localhost /windows/documents 0 10 10 -- 0:00 682.7 0:00 100.0
(brought to you by Amanda version 3.3.9)
$ amcheck usb
Amanda Tape Server Host Check
-----------------------------
Holding disk /backup/holding: 1007616 KB disk space available, using 495616 KB
slot 1: volume 'User_amanda_Backup_01' is still active and cannot be overwritten
slot 2: volume 'User_amanda_Backup_02'
Will write to volume 'User_amanda_Backup_02' in slot 2.
NOTE: skipping tape-writable test
Server check took 0.158 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 2.029 seconds. 0 problems found.
(brought to you by Amanda 3.3.9)
---8<---8<---8<---
As can be seen in the output, 'Output Size (meg)' and 'Original Size (meg)', despite \
having data in the directory to be backup'ed:
---8<---8<---8<---
$ ls -l /windows/documents
total 8256
-rw-r--r-- 1 root root 30 Dec 11 18:34 date.txt
-rw-r--r-- 1 root root 53103 Dec 11 18:34 dmesg.txt
-rw-r--r-- 1 root root 11487 Dec 11 18:34 ps-ax.txt
-rw-r--r-- 1 root root 8388608 Dec 11 18:34 rand.bin
---8<---8<---8<---
So, the question is: what am I doing wrong in my second, live CD/USB setup? I cannot \
see any obvious problems, so I am open for suggestions!
Looking forward to your help!
Bye,
Thomas
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic