[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it ex
From:       Xiaoqiao He <xq.he2009 () gmail ! com>
Date:       2020-12-30 10:40:59
Message-ID: CAF3Ajaym+L=MUqR-5imG-4hz+2Lsjt0miatLaH9Bv79Ce2Q7Lw () mail ! gmail ! com
[Download RAW message or body]


Hi Steve and Chao,

Konstatin has updated HDFS-15751 and attached patch for documentation msync
API, Would you mind to TAL. I want to involve this update to 3.2.2 release,
but for now it seems there are not enough committers to review. Please
check when you have bandwidth. Thanks all.

- He Xiaoqiao

On Fri, Dec 25, 2020 at 5:40 AM Konstantin Shvachko <shv.hadoop@gmail.com>
wrote:

> Hi Steve,
>
> I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751>
> for
> documenting msync API.
> Would appreciate your suggestions.
>
> Stay safe,
> --Konstantin
>
> On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <stevel@cloudera.com>
> wrote:
>
> >
> >
> > On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.hadoop@gmail.com>
> > wrote:
> >
> >> Hey Steve,
> >>
> >> Thanks for the references. I was reading but still need to understand
> how
> >> exactly this applies to msync.
> >>
> >
> > mainly: pull it up and it becomes part of the broader API, so needs to be
> > specified in a way which can be understood by users and for implementors
> of
> > others stores: to give their own stores the same semantics.
> >
> > What does the HDFS one do?
> >
> >
> >
> >> Will come up with a plan and post it on a new jira.
> >> Will make sure to create it under HADOOP and ping Hadoop Common list for
> >> visibility.
> >>
> >>
> > thanks
> >
> >
> >> You are right about ViewFS. The impl should make sure it calls msync()
> on
> >> all mount points that enabled observer reads.
> >>
> >>
> > That's the kind of issue this process aims to resolve. Another is to
> > identify where we have HDFS-layer "quirks" and at least document them
> (e.g.
> > how hdfs streams are thread safe, rename isn't Posix, ...) and list what
> we
> > know breaks if you don't re-implement
> >
>


[prev in list] [next in list] [prev in thread] [next in thread]