[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-btrfs
Subject:    Rebuilding 24TB Raid5 array (was btrfs corruption: parent transid verify failed + open_ctree failed)
From:       Marc MERLIN <marc () merlins ! org>
Date:       2022-03-31 17:19:27
Message-ID: 20220331171927.GL1314726 () merlins ! org
[Download RAW message or body]

I'm going to wait a few more days before I give up and restore from
backup and will consider whether if I should go back to ext4 which for
me has clearly been more resilient of disk misbehaviours like I get
sometimes (of course, nothing will recover from a real double risk
failure in raid5, but over the last 20 years, it's not been uncommon for
me to see more than one drive kicked out of an array due to SCSI or sata
issues, or a drive having a weird shutdown and being able to come back
after a power cycle with all data except the last blocks written to it).

If I go back with btrfs (despite it being non resilient to any problem
described above, btrfs send is great for backups of course), what are
today's best recommended practices?

Kernel will be 5.16. Filesystem will be 24TB and contain mostly bigger
files (100MB to 10GB).

1) mdadm --create /dev/md7 --level=5 --consistency-policy=ppl --raid-devices=5 \
/dev/sd[abdef]1 --chunk=256 --bitmap=internal 2) echo \
0fb96f02-d8da-45ce-aba7-070a1a8420e3 >  /sys/block/bcache64/bcache/attach   \
gargamel:/dev# cat /sys/block/md7/bcache/cache_mode  [writethrough] writeback \
writearound none 3) cryptsetup luksFormat --align-payload=2048 -s 256 -c \
aes-xts-plain64  /dev/bcache64 4) cryptsetup luksOpen /dev/bcache64 dshelf1
5) mkfs.btrfs -m dup -L dshelf1 /dev/mapper/dshelf1

Any other btrfs options I should set for format to improve reliability
first and performance second?
I'm told I should use space_cache=v2, is it default now with btrfs-progs 5.10.1-2 ?

Anything else you recommend given that I indeed I have layers in the
middle. I'm actually considering dropping bcache for this filesystem
given that it could be another cause for corruption that btrfs can't
deal with.

Thanks
Marc

> > > mdadm --assemble --run --force /dev/md7 /dev/sd[gijk]1
> > > cryptsetup luksOpen /dev/bcache3 dshelf1a
> > > btrfs device scan --all-devices
> > > mount /dev/mapper/dshelf1a /mnt/btrfs_pool1/
> > > 
> > > BTRFS error (device dm-17): parent transid verify failed on 22216704 wanted \
> > > 1600938 found 1602177 BTRFS error (device dm-17): parent transid verify failed \
> > > on 22216704 wanted 1600938 found 1602177 BTRFS error (device dm-17): failed to \
> > > read chunk tree: -5 BTRFS error (device dm-17): open_ctree failed
> 
> But clearly the drive that went offline during shutdown, must have
> caused some corruption, even if it all it did was just refuse all
> writes.
> That said, I was somehow hoping that btrfs could unwind the last writes
> that failed/were incomplete and get back to a proper state. I'm still
> saddened by how fragile btrfs seems compared to ext4 in those cases
> (I've had similar issues happen with ext4 in the past, and was always
> able to repair the filesystem even if I lost a few files in the process)

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08


[prev in list] [next in list] [prev in thread] [next in thread]