[prev in list] [next in list] [prev in thread] [next in thread]
List: ceph-devel
Subject: Re: wip-auth
From: Sage Weil <sweil () redhat ! com>
Date: 2015-01-30 21:08:10
Message-ID: alpine.DEB.2.00.1501300831300.25803 () cobra ! newdream ! net
[Download RAW message or body]
Hi Andreas,
It looks like that was a stale sha1, but the newer one was also
broken. I've retested and it's working for me now. See latest wip-auth,
sha1 0c21a7875059bef80842756dfb003f47cc2d66a6.
Thanks!
sage
On Fri, 30 Jan 2015, Andreas Bluemle wrote:
> Hi Sage,
>
> I tried to integrate wip-auth into my 0.91 build environment.
>
> I had not been able to start the cluster successfully: ceph-mon
> crashes with a segmentation fault in CryptoKey::encrypt()
> (see attachment).
>
> This happens when linking with libnss or libcryptopp (version 5.6.2).
>
> I created the patch to add wip-auth based on github
> pull request 3523 and was able to use this patch directly
> with 0.91 with only a minor adaptation for common/ceph_context.h:
> the 0.91 version of ceph_context.h did not know anything about
> the experimental "class CephContextObs".
>
> wip-auth commit ID is 1a0507a2940f6edcc2bf9533cfa6c210b0b41933.
>
> As my build environment is rpm, I had to modify the invocation
> of the configure script in the spec file instead of the do_autogen.sh
> script.
>
>
> Best Regards
>
> Andreas Bluemle
>
>
> On Tue, 27 Jan 2015 09:18:45 -0800 (PST)
> Sage Weil <sweil@redhat.com> wrote:
>
> > I spent some time focusing on just CryptoKey::encrypt(). I
> > benchmarked 100,000 encrypts of 128 bytes and got, at baseline,
> >
> > cryptopp: 100000 encoded in 0.655651
> > libnss : 100000 encoded in 1.288786
> >
> > Ouch! With a (fixed) version of my earlier patch that avoids
> > uselessly copying the input buffer:
> >
> > 100000 encoded in 1.231977
> >
> > With a patch that puts the key structures in CryptoKey instead of
> > recreating them each time:
> >
> > 100000 encoded in 0.396208 -- ~70% improvement over original
> >
> > This is pushed to wip-auth. There's also a patch that caches key
> > structs for crypopp.. it now takes
> >
> > 100000 encoded in 0.440758 -- ~33% improvement over original
> >
> > (Not that almost anybody will ever care; we use libnss by default for
> > both rpm and deb distros.)
> >
> > So, yay, nss is now a bit faster. What I'm not completely certain
> > about is whether the structures I've preserved are truly stateless
> > (and can be shared across threads, etc.). They encrypt/decrypt
> > methods are const so, if the libraries are const-correct, it should
> > be fine... but perhaps someone familiar with nss and/or crypto++ can
> > review this?
> >
> > This is pushed to the latest wip-auth branch:
> >
> > https://github.com/ceph/ceph/commits/wip-auth
> >
> > Andreas and Stephen, what effect does this have on your numbers?
> >
> > Thanks!
> > sage
> >
> >
> > On Mon, 26 Jan 2015, Blinick, Stephen L wrote:
> >
> > > Good to know, I was wondering why the spec file defaulted to
> > > lib-nss.. the dpkg-build for debian packages just uses whatever
> > > configuration you had built, and I believe that will use
> > > libcryptopp if the dependency is installed on the build machine
> > > (last I looked).
> > >
> > > I forgot to mention the numbers below were based on v.91.
> > >
> > > Thanks,
> > >
> > > Stephen
> > >
> > > -----Original Message-----
> > > From: ceph-devel-owner@vger.kernel.org
> > > [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Sage Weil
> > > Sent: Monday, January 26, 2015 10:24 AM To: Blinick, Stephen L
> > > Cc: andreas.bluemle@itxperts.de; ceph-devel@vger.kernel.org
> > > Subject: RE: wip-auth
> > >
> > > On Mon, 26 Jan 2015, Blinick, Stephen L wrote:
> > > > I noticed that the spec file for building RPM's defaults to
> > > > building with libnss, instead of libcrypto++. Since the
> > > > measurements I'd done so far were from those RPM's I rebuilt with
> > > > libcrypto++.. so FWIW here is the difference between those two on
> > > > my system, memstore backend with a single OSD, and single
> > > > client.
> > > >
> > > > Dual socket Xeon E5 2620v3, 64GB Memory, RHEL7
> > > > Kernel: 3.10.0-123.13.2.el7
> > > >
> > > > 100% 4K Writes, 1xOSD w/ Rados Bench
> > > > libnss |
> > > > Cryptopp # QD IOPS Latency(ms) |
> > > > IOPS Latency(ms) IOPS Improvement % 16
> > > > 14432.57 1.11 | 18896.60 0.85
> > > > 30.93% 100% 4K Reads, 1xOSD w/ Rados
> > > > Bench libnss | Cryptopp # QD IOPS Latency(ms) | IOPS Latency(ms)
> > > > IOPS Improvement % 16 19532.53 0.82 | 25708.70 0.62 31.62%
> > >
> > > Yikes, 30%! I think this definitely worth some effort. We
> > > switched to libnss because it has the weird government
> > > certfiications that everyone wants and is more prevalent. crypto++
> > > is also not packaged for Red Hat distros at all (presumably for
> > > that reason).
> > >
> > > I suspect that most of the overhead is in the encryption context
> > > setup and can be avoided with a bit of effort..
> > >
> > > sage
> > >
> > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Stephen
> > > >
> > > > -----Original Message-----
> > > > From: ceph-devel-owner@vger.kernel.org
> > > > [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Sage Weil
> > > > Sent: Thursday, January 22, 2015 4:56 PM
> > > > To: andreas.bluemle@itxperts.de
> > > > Cc: ceph-devel@vger.kernel.org
> > > > Subject: wip-auth
> > > >
> > > > Hi Andreas,
> > > >
> > > > I took a look at the wip-auth I mentioned in the security call
> > > > last week... and the patch didn't work at all. Sorry if you
> > > > wasted any time trying it.
> > > >
> > > > Anyway, I fixed it up so that it actually worked and made one
> > > > other optimization. It would be great to hear what latencies you
> > > > measure with the changes in place.
> > > >
> > > > Also, it might be worth trying --with-cryptopp (or --with-nss if
> > > > you built cryptopp by default) to see if there is a difference.
> > > > There is a ton of boilerplate setting up encryption contexts and
> > > > key structures and so on that I suspect could be cached (perhaps
> > > > stashed in the CryptoKey struct?) with a bit of effort. See
> > > >
> > > > https://github.com/ceph/ceph/blob/master/src/auth/Crypto.cc#L99-L213
> > > >
> > > > sage
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe
> > > > ceph-devel" in the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > >
> > > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe
> > > ceph-devel" in the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > > -- To unsubscribe from this list: send the line "unsubscribe
> > > ceph-devel" in the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > >
> > >
> >
> >
>
>
>
> --
> Andreas Bluemle mailto:Andreas.Bluemle@itxperts.de
> ITXperts GmbH http://www.itxperts.de
> Balanstrasse 73, Geb. 08 Phone: (+49) 89 89044917
> D-81541 Muenchen (Germany) Fax: (+49) 89 89044910
>
> Company details: http://www.itxperts.de/imprint.htm
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic