[prev in list] [next in list] [prev in thread] [next in thread] 

List:       debian-user
Subject:    Re: file systems
From:       Stan Hoeppner <stan () hardwarefreak ! com>
Date:       2011-05-04 23:23:53
Message-ID: 4DC1E009.30209 () hardwarefreak ! com
[Download RAW message or body]

On 5/2/2011 4:02 PM, Boyd Stephen Smith Jr. wrote:

> They are also essential for any journaled filesystem to have correct behavior
> in the face of sudden pwoer loss.

This is true only if you don't have BBWC.

> Barriers ensure, (e.g.) that the journal
> entry creating a file is flushed to the backing store before the journal
> entry.

XFS only journals metadata not file data.  Barriers are not used by XFS 
to guarantee write order.  It uses another mechanism for this.  The 
reason is that XFS allows the log journal to be on a different physical 
device/path than the filesystem device in order to increase performance.

> Even with a a battery-packed RAID cache, like I have in my desktop, executing
> without barrier can result in extra data loss that executing with a barrier
> prevents.

Then I'd say you have a problem with your BBWC RAID controller in your 
desktop.  Which BBWC RAID card do you have?

> Never run without barriers if you value your data.

Again, with a quality BBWC implementation, using barriers simply 
decreases performance.  Disabling barriers with BBWC does not expose you 
to any additional filesystemm inconsistency risk due to cached data.

> Of course, even with out barriers a properly journaled or log-structed
> filesystem should be able to immediately and silently recover.

This contradicts what you stated above.

> Barriers do have a cost, however, on modern file systems it is not anwhere
> near 50% even in the random write case, since a barrier doesn't have to occur
> between every journal entry.
> <http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-
> rc5_Large_file_random_writes._num_threads=128.html>  shows less than a 10%
> difference between XFS performance with barriers and without.

This is but one synthetic test case.  Do an 'rm -rf' of a subdirectory 
with 1,000,000 entries on a shared filesystem where other multiple users 
are performing medium to heavy metadata operations, with and without 
barriers, and you'll see what I'm talking about.  Such a million file 
delete is actually pretty common with many research applications.

>  From this LWN article<http://lwn.net/Articles/283161/>: "The filesystem code
> must, before writing the [journaling] commit record, be absolutely sure that
> all of the transaction's information has made it to the journal. Just doing
> the writes in the proper order is insufficient; contemporary drives maintain
> large internal caches and will reorder operations for better performance.

This is why (good) BBWC enabled RAID cards automatically disable the 
caches on all the drives, and thus why it is recommended to disable 
barriers for filesystems on BBWC RAID cards.  With some BBWC RAID cards 
you have to manually disable the drive's caches in the controller BIOS 
or driver.

>> The nobarrier results are far more relevant than the barrier results,
>> especially the 16 and 128 thread results, for those SAs with high
>> performance persistent storage.
>
> I disagree entirely.  You should be looking at the threaded results, probably
> 128 threads (depending on what the server does), but you should also be using
> barriers.

You just said you "disagree entirely" and then say 128 threads, same 
thing I said.  But then you recommend barriers, which is the disagreement.

>> In those test where XFS has the best 'score' it typically beats most
>> others by a wide margin.
>
> No, it doesn't.  On a few tests, it does amazingly outpace other file systems.
> On most of the others, even when it is first, one of the other filesystems is
> a close second.

Considering the fact that JFS was abandoned by IBM over 2 years ago, is 
no longer developed, is thus deprecated, and is used by no one, my 
statement is correct.  Across the board JFS is the only FS close to XFS 
in the more than 1 thread tests.

> XFS without barriers is first -- but I prefaced my statements with the fact
> that I was ignoring -nobarrier variants and the -nocow variant of btrfs.

This is because you don't yet understand the significance of barriers in 
relation to write cache WRT enterprise hardware with BBWC.  The whole 
point of BBWC is avoiding the use of barriers.  Point of fact:  most of 
the very high end SAN array controllers, and even some of the midrange 
controllers, discard all barrier commands, period, to prevent flushing 
the mammoth (4-64GB) caches on such controllers.  With some it's a 
config option.  Many low end units don't offer it, so you have to 
disable barriers at the filesystem level.

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org 
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/4DC1E009.30209@hardwarefreak.com

[prev in list] [next in list] [prev in thread] [next in thread]