'Re: [Notice] Merge HDFS-13891(RBF) to trunk'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    Re: [Notice] Merge HDFS-13891(RBF) to trunk
From:       Brahma Reddy Battula <brahmareddy.battula () hotmail ! com>
Date:       2019-06-24 18:13:16
Message-ID: SN1PR18MB2173ED96343FD23836576555E1E00 () SN1PR18MB2173 ! namprd18 ! prod ! outlook ! com
[Download RAW message or body]


Merged. thanks to Arpit for quoting the merge process.
________________________________
From: Brahma Reddy Battula <brahmareddy.battula@hotmail.com>
Sent: Tuesday, June 25, 2019 12:39 AM
To: hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; \
                Hdfs-dev; Iñigo Goiri; Hadoop Common
Subject: [Notice] Merge HDFS-13891(RBF) to trunk

Rebased HDFS-13891 against trunk and ran the jenkins (1).Only two four testcases are \
failing which flaky and there is already jira to track this. (HDFS-14461)


Going to merge the branch soon which contains 77 commits,so please bear with us.



1.https://builds.apache.org/job/PreCommit-HDFS-Build/27044/console


________________________________
From: Brahma Reddy Battula <brahma@apache.org>
Sent: Sunday, June 9, 2019 9:56 PM
To: Xiaoqiao He
Cc: Akira Ajisaka; Chittaranjan Hota; Giovanni Matteo Fumarola; Hadoop Common; \
                Hdfs-dev; Iñigo Goiri
Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk

Hi All,

    Given the positive response to the discussion thread [1], here is the
formal vote thread to merge HDFS-13891 in to trunk.

    Summary of code changes:
    1. Code changes for this branch are done in the hadoop-hdfs-rbf
subproject, there is no impact to hadoop-hdfs and hadoop-common.
    2. Added Security support for RBF
    3. Added Missing Client Protocol API's
    4. Bug Fixes/ Improvments


     The vote will run for 7 days, ending Sat June 15th. I will start this
vote with my +1.

Regards,
Brahma Reddy Battula

1).
https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E



On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq.he2009@gmail.com> wrote:

> Thanks Brahma for starting the thread.
> +1 for merging.
> 
> He Xiaoqiao
> 
> On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <chitts.hota@gmail.com>
> wrote:
> 
> > Thanks Brahma initiating this.
> > +1(non-binding) for merge.
> > 
> > @Uber we have almost all changes specially rbf security in production for
> > a
> > while now without issues.
> > 
> > On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> > giovanni.fumarola@gmail.com> wrote:
> > 
> > > +1 on merging.
> > > 
> > > Thanks Brahma for starting the thread.
> > > 
> > > On Mon, Jun 3, 2019 at 10:00 AM Iñigo Goiri <elgoiri@gmail.com> wrote:
> > > 
> > > > Thank you Brahma for pushing this.
> > > > 
> > > > As you mentioned, we have already taken most of the changes into
> > > > production.
> > > > I want to highlight that the main contribution is the addition of
> > > security.
> > > > We have been able to test this at a smaller scale (~500 servers and 4
> > > > subclusters) and the performance is great with our current ZooKeeper
> > > > deployment.
> > > > I would also like to highlight that all the changes are constrained to
> > > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> > > > 
> > > > +1 on merging
> > > > 
> > > > Inigo
> > > > 
> > > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aajisaka@gmail.com>
> > > wrote:
> > > > 
> > > > > Thanks Brahma for starting the discussion.
> > > > > I'm +1 for merging this.
> > > > > 
> > > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
> > > > > with 2 routers (not in production) and running several tests.
> > > > > 
> > > > > Regards,
> > > > > Akira
> > > > > 
> > > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> > > brahma@apache.org>
> > > > > wrote:
> > > > > > 
> > > > > > Dear Hadoop Developers
> > > > > > 
> > > > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk.
> > We
> > > > have
> > > > > > been working on this feature from last several months.
> > > > > > This feature work received the contributions from different
> > > companies.
> > > > > All
> > > > > > of the feature development happened smoothly and collaboratively
> > in
> > > > > JIRAs.
> > > > > > 
> > > > > > Kindly do take a look at the branch and raise issues/concerns that
> > > need
> > > > > to
> > > > > > be addressed before the merge.
> > > > > > 
> > > > > > *Highlights of HDFS-13891 Branch:*
> > > > > > =============================
> > > > > > 
> > > > > > Adding Security to RBF(1)
> > > > > > Adding Missing Client API's(2)
> > > > > > Improvements/Bug Fixing
> > > > > > Critical - HDFS-13637, HDFS-13834
> > > > > > 
> > > > > > *Commits:*
> > > > > > ========
> > > > > > 
> > > > > > No of JIRAs Resolved: 72
> > > > > > 
> > > > > > All this commits are in RBF Module. No changes in hdfs/common.
> > > > > > 
> > > > > > *Tested Cluster:*
> > > > > > =============
> > > > > > 
> > > > > > Most of these changes verified at Uber,Microsoft,Huawei and some
> > > other
> > > > > > compaines.
> > > > > > 
> > > > > > *Uber*: Most changes are running in production @Uber including the
> > > > > critical
> > > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> > Routers.
> > > > > > Zookeeper as a state store to hold delegation tokens were also
> > stress
> > > > > > tested to hold more than 2 Million tokens. --CR Hota
> > > > > > 
> > > > > > *MicroSoft*: Most of these changes are currently running in
> > > production
> > > > at
> > > > > > Microsoft.The security has also been tested in a 500 server
> > cluster
> > > > with
> > > > > 4
> > > > > > subclsuters. --Inigo Goiri
> > > > > > 
> > > > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
> > > > > > routers.Planning deploy 10K production cluster.
> > > > > > 
> > > > > > *Contributors:*
> > > > > > ===========
> > > > > > 
> > > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> > Asanuma,Shubham
> > > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
> > > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> > > > Xiaoqiao,Sherwood
> > > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> > > discussions
> > > > > and
> > > > > > contributing to this.
> > > > > > 
> > > > > > *Future Tasks:*
> > > > > > ============
> > > > > > 
> > > > > > will cleanup the jira's under this umbrella and contiue to work.
> > > > > > 
> > > > > > Reference:
> > > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> > > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > --Brahma Reddy Battula
> > > > > 
> > > > > 
> > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > > > 
> > > > > 
> > > > 
> > > 
> > 
> > 
> > --
> > Regards,
> > Chitta
> > 
> 

--



--Brahma Reddy Battula
--



--Brahma Reddy Battula



[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic