[prev in list] [next in list] [prev in thread] [next in thread] 

List:       bacula-devel
Subject:    [Bacula-devel] [bug?] bacula-fd: strippath vs. accurate (checksums)
From:       Markus Jung <maju.jufo () web ! de>
Date:       2016-03-13 19:12:11
Message-ID: 56E5BB8B.9010901 () web ! de
[Download RAW message or body]

Hi devs,

i think bacula-fd has a slight problem with the combination of strippath
and accurate backup using checksums.

Short description:
bacula-fd (tries) to open files using the stripped path instead of using
the location at which the file really resides. This happens most likely
while performing the accurate backup checks.

OS: Ubuntu 15.10
bacula-fd: 5.2.6 (stock ubuntu)
(i think the problem is present in recent bacula versions, too)

Symptom: log messages like
> Error:      Cannot open /etc/lvm/archive/vg_hdd_00039-1823321258.vg: ERR=Datei oder \
> Verzeichnis nicht gefunden.
(file or directory no found)

while running a backup on an snapshot volume. The file existed within
the snapshot, but not outside. The backup itself runs on something like
> /mnt/snapshot/etc/...

strippath = 2 takes the snapshot prefix away.

And in fact, while bacula-fd is digging into the snapshot directory
tree, it opens original files and not the ones within the snapshot.
> lsof -p $(pidof bacula-fd) | tail
> bacula-fd 5778 root   10r   DIR 252,21     4096 3670018 \
> /mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014 bacula-fd 5778 root   11r  \
> DIR 252,21     4096 3670311 \
> /mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 Wald bacula-fd 5778 \
> root   12r   DIR 252,21     4096 3670313 \
> /mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 Wald/NEF bacula-fd \
> 5778 root   13r   REG  252,7 14530555 3670387 \
> /mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 Wald/NEF/DSC_0007.NEF

The last line should not be like this.

I suppose the problem is related to src/filed/accurate.c. Below is an
excerpt of bacula-fd running at a debug-level of 600
> markusnb-fd: find.c:291-0 enter accept_file: \
> fname=/mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
>                 Wald/NEF/DSC_0026.NEF.xmp
> markusnb-fd: find_one.c:374-0 File ----: \
>                 /mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
>                 Wald/NEF/DSC_0007.NEF
> markusnb-fd: backup.c:1453-0 stripped=2 count=2 numsep=12 sep>count=1
> markusnb-fd: backup.c:1503-0 \
> fname=/mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
> Wald/NEF/DSC_0007.NEF stripped=/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
> Wald/NEF/DSC_0007.NEF link=/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
>                 Wald/NEF/DSC_0007.NEF
> markusnb-fd: htable.c:132-0 Leave hash_index hash=0x9e23d3a7a5e4fc78 index=61248
> markusnb-fd: htable.c:404-0 lookup return 7f5c1b3d53d8
> markusnb-fd: accurate.c:82-0 lookup \
>                 </mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
>                 Wald/NEF/DSC_0007.NEF> ok
> markusnb-fd: crypto.c:607-0 crypto_digest_new jcr=7f5c1c0008e8
> markusnb-fd: verify.c:297-0 === digest_file
> markusnb-fd: bfile.c:963-0 open file \
>                 /mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 \
>                 Wald/NEF/DSC_0007.NEF
> markusnb-fd: bfile.c:987-0 Open file 13
> markusnb-fd: verify.c:354-0 === read_digest
> markusnb-fd: bfile.c:1028-0 Close file 13
> markusnb-fd: htable.c:132-0 Leave hash_index hash=0x9e23d3a7a5e4fc78 index=61248
> markusnb-fd: htable.c:404-0 lookup return 7f5c1b3d53d8
> markusnb-fd: find_one.c:436-0 Non-directory incremental: \
> /mnt/snapshot/mnt/vg_hdd/static/markus/Bilder/Fotos/2014/0131 Wald/NEF/DSC_0007.NEF

I think accurate.c:accurate_check_file() does not handle the stripped
prefix properly. It calls strip_path() and performs all computations
until the very end using the stripped path, including the checksum
computation which bopens the stripped path, if i understood everything
right. unstrip_path() is called just before leaving the function.

By roughly comparing the ubuntu source package to the git branches of
7.0, 7.2 and 7.4 i think they are affected by this problem as well.

One final remark: If strippath is not used to restore the real file
hierarchy, this issue might be much more problematic, because it will
trigger for every file (and not just for every changed one).
However, i am unsure if it has a real influence on the backup archive or
just spams the backup log.

Regards,
Markus

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel


[prev in list] [next in list] [prev in thread] [next in thread]