[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gentoo-user
Subject:    Re: [gentoo-user] Suggestions for backup scheme?
From:       Frank Steinmetzger <Warp_7 () gmx ! de>
Date:       2024-02-07 22:36:36
Message-ID: ZcQF9FiRdaCKkj8n () schatulle
[Download RAW message or body]


Am Tue, Jan 30, 2024 at 06:15:09PM -0000 schrieb Grant Edwards:
> I need to set up some sort of automated backup on a couple Gentoo
> machines (typical desktop software development and home use). One of
> them used rsnapshot in the past but the crontab entries that drove
> that have vanished :/ (presumably during a reinstall or upgrade --
> IIRC, it took a fair bit of trial and error to get the crontab entries
> figured out).
> 
> I believe rsnapshot ran nightly and kept daily snapshots for a week,
> weekly snapshots for a month, and monthly snapshots for a couple
> years.
> 
> Are there other backup solutions that people would like to suggest I
> look at to replace rsnapshot?  I was happy enough with rsnapshot (when
> it was running), but perhaps there's something else I should consider?

In my early backup times I, too, used rsnapshot to back up my ~ and rsync 
for my big media files. But that only included my PC. My laptop was wholly 
un-backed-up. I only syncronised much of my home and my audio collection 
between the two with unison. At some point my external 3 TB drive became 
free and then I started using borg to finally do proper backups.

Borg is very similar to restic, I actually used the two in parallel for a 
while to compare them, but stayed with borg. One pain point was that I 
couln't switch off restic's own password protection. Since all my backup 
disks are LUKSed anyway, I don't need that.

Since borg works block-based, it does deduplication without extra cost and 
it is suitable for big image files which don't change much. I do full 
filesystem backups of /, ~ and my media partition of my main PC and my 
laptop. I have one repository for each of those three filesystems, and each 
repo receives the data from both machines, so they are deduped. Since both 
machines run Arch, their roots are binary identical. The same goes for my 
unison-synced homes.

Borg has retention logic built-in. You can say I want to keep the latest 
archive of each of the last 6 days/weeks/months/years, and it even goes down 
to seconds. And of course you can combine those rules. The only thing is 
they don't overlap, meaning if you want to keep the last 14 days and the 
last four weeks, those weekly retentions start after the last daily 
snapshots.

In summary, advantages:
+ fast dedup, built-in compression (different algos and levels configurable)
+ big data files allow for quick mirroring of repositories.
  I simply rsync my primary backup disk to two other external HDDs.
+ Incremental backups are quite fast because borg uses a cache to detect
  changed files quickly.
Disadvantages:
- you need borg to mount the backups it
- it is not as fast as native disk access, especially during restore and 
  when getting a total file listing due to lots of random I/O on the HDD.


As example, I currently have 63 snapshots in my data partition repository:

# borg list data/
tp_2021-06-07           Mon, 2021-06-07 16:27:44 \
[5f9ebd9f24353c340691b2a71f5228985a41699d2e23473ae4e9e795669c8440] kern_2021-06-07    \
Mon, 2021-06-07 23:58:56 \
[19c76211a9c35432e6a66ac1892ee19a08368af28d2d621f509af3d45f203d43] [... 55 more lines \
...] kern_2024-01-14         Sun, 2024-01-14 20:53:23 \
[499ce7629e64cffb7ec6ec9ffbf0c595e4ede3d93f131a9a4b424b165647f645] tp_2024-01-14      \
Sun, 2024-01-14 20:57:42 \
[ea2baef3e4bb49c5aec7cf8536f7b00b55fb27ecae3a80ef9f5a5686a1da30d5] kern_2024-01-21    \
Sun, 2024-01-21 23:42:46 \
[71aa2ce6cf4021712f949af068498bfda7797b5d1c5ddc0f0ce8862b89e48961] tp_2024-01-21      \
Sun, 2024-01-21 23:48:24 \
[45e35ed9206078667fa62d0e4a1ac213e77f52415f196101d14ee21e79fc393d] kern_2024-02-04    \
Sun, 2024-02-04 23:16:43 \
[e1b015117143fad6b89cea66329faa888cffc990644e157b1d25846220c62448] tp_2024-02-04      \
Sun, 2024-02-04 23:23:15 \
[e9b167ceec1ab9a80cbdb1acf4ff31cd3935fc23e81674cad1b8694d98547aeb]

The last "tp" (Thinkpad) snapshot contains 1 TB, "kern" (my PC) 809 GB.
And here you see how much space this actually takes on disk:

# borg info data/
[ ... ]
                 Original size   Compressed size    Deduplicated size
All archives:         56.16 TB          54.69 TB              1.35 TB

Obviously, compression doesn't do much for media files. But it is very 
effective in the repository for the root partitions:

# borg info arch-root/
[ ... ]
                 Original size   Compressed size    Deduplicated size
All archives:          1.38 TB         577.58 GB             79.41 GB

-- 
Grüße | Greetings | Salut | Qapla'
Please do not share anything from, with or about me on any social network.

"She understands.  She doesn't comprehend." – River Tam, Firefly


["signature.asc" (application/pgp-signature)]

[prev in list] [next in list] [prev in thread] [next in thread]