[prev in list] [next in list] [prev in thread] [next in thread] 

List:       subversion-users
Subject:    Fwd: Re: Subversion svn+ssh, sshd 100% CPU
From:       Johan Corveleyn <jcorvel () gmail ! com>
Date:       2017-09-21 21:06:13
Message-ID: CAB84uBUNGC57WamY3oqn2EQ5r8a=19z-LtSqLzTRvFaYvBw2uQ () mail ! gmail ! com
[Download RAW message or body]

Forwarding to the list (Zoran, please use reply all to keep the list in cc).


---------- Doorgestuurd bericht ----------
Van: "Zoran Petkovic" <zoran@ietcombustion.com>
Datum: 21 sep. 2017 19:01
Onderwerp: Re: Subversion svn+ssh, sshd 100% CPU
Aan: "Johan Corveleyn" <jcorvel@gmail.com>
Cc:

Correction, I meant to say KBytes/s.

--
Zoran Petkovic
Industrial Engineering Technology Pty Ltd
+61 413 254 315 <tel:0413%20254%20315> | zoran@ietcombustion.com


On 21/09/2017, 4:17 PM, "Zoran Petkovic" <zoran@ietcombustion.com> wrote:

    My results show that Linux clients stress the server less and complete
the checkout quicker (5-10% on sshd and svnserve). Windows clients
connecting remotely start off fast 2000-3000kbit/s then slow down to
maximum 800kbit/s, the sshd process running at 100%, while the svnserve at
5%. So, I believe it is an issue with ssh rather than svn. Skelta mode
seems to refer to apache, however these clients connect over ssh.

    --
    Zoran Petkovic
    Industrial Engineering Technology Pty Ltd
    +61 413 254 315 <tel:0413%20254%20315> | zoran@ietcombustion.com


    On 14/09/2017, 10:10 PM, "Johan Corveleyn" <jcorvel@gmail.com> wrote:

        On Wed, Sep 13, 2017 at 7:20 AM, Matt Simmons <bandman@gmail.com>
wrote:
        > On Tue, Sep 12, 2017 at 12:14 PM Zoran Petkovic <
zoran@ietcombustion.com>
        > wrote:
        >>
        >> In the past few days I have been doing extensive testing of
Subversion
        >> with different clients, operating systems, client and server
versions and
        >> have noticed very strange behaviour with windows clients
connecting to Linux
        >> servers, hitting them with excessive CPU usage on the sshd
process, where
        >> the Linux clients do not exhibit this behaviour.
        >>
        >>
        >>
        >> A sample test setup is as follows:
        >>
        >> Server Linux Ubuntu 16.04.3 LTS, OpenSSH_7.2p2
Ubuntu-4ubuntu2.2, OpenSSL
        >> 1.0.2g  1 Mar 2016, Subversion version 1.9.3 (and 1.9.7).
        >>
        >> Client TortoiseSVN 1.9.7
        >>
        >>
        >>
        >> When checking out large repositories the linux server is hit on
the sshd
        >> process, the process running with 100% cpu usage. This in effect
slows down
        >> the performance and ultimately the speed at which the checkout
runs. Linux
        >> clients connecting to the same server do not cause this load on
the server.
        >>
        >>
        >>
        >> This happens even when compressions is turned off and when
encryption
        >> Cyphers are changed, as well as different versions of
subversion. The
        >> behaviour is identical. I'm not sure who to address for this
issue as this
        >> not only happens with TortoiseSVN but with SlikSVN as well. Any
direction
        >> would be appreciated.
        >
        > Why does iostat show? Could it be that your underlying disk is
io-saturated
        > and your CPU spike is due to iowait?

        One possible explanation would be that the windows clients perform
the
        checkout faster than the linux clients for some reason, thus being
        able to put higher load on the server, saturating its cpu or disk
I/O.

        Other than that I have no idea.

        The Linux client you're testing is also 1.9.7?

        Perhaps the windows client and the linux client are performing the
        checkout in a different way for some reason (I'm thinking of the
        so-called "skelta mode" vs. "bulk mode", see
        http://subversion.apache.org/docs/release-notes/1.8.html#
serf-skelta-default).

        --
        Johan

[Attachment #3 (text/html)]

<div dir="auto"><div>Forwarding to the list (Zoran, please use reply all to keep the \
list in cc).</div><div dir="auto"><br></div><div dir="auto"><br><div \
class="gmail_quote" dir="auto">---------- Doorgestuurd bericht ----------<br>Van: \
&quot;Zoran Petkovic&quot; &lt;<a \
href="mailto:zoran@ietcombustion.com">zoran@ietcombustion.com</a>&gt;<br>Datum: 21 \
sep. 2017 19:01<br>Onderwerp: Re: Subversion svn+ssh, sshd 100% CPU<br>Aan: \
&quot;Johan Corveleyn&quot; &lt;<a \
href="mailto:jcorvel@gmail.com">jcorvel@gmail.com</a>&gt;<br>Cc: <br><br \
type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex">Correction, I meant to say KBytes/s.<br> <div \
                class="quoted-text"><br>
--<br>
Zoran Petkovic<br>
Industrial Engineering Technology Pty Ltd<br>
<a href="tel:%2B61%20413%20254%20315" value="+61413254315">+61 413 254 315</a>  \
&lt;tel:0413%20254%20315&gt; | <a \
href="mailto:zoran@ietcombustion.com">zoran@ietcombustion.com</a><br> <br>
<br>
</div><div class="elided-text">On 21/09/2017, 4:17 PM, &quot;Zoran Petkovic&quot; \
&lt;<a href="mailto:zoran@ietcombustion.com">zoran@ietcombustion.com</a>&gt; \
wrote:<br> <br>
      My results show that Linux clients stress the server less and complete the \
checkout quicker (5-10% on sshd and svnserve). Windows clients connecting remotely \
start off fast 2000-3000kbit/s then slow down to maximum 800kbit/s, the sshd process \
running at 100%, while the svnserve at 5%. So, I believe it is an issue with ssh \
rather than svn. Skelta mode seems to refer to apache, however these clients connect \
over ssh.<br> <br>
      --<br>
      Zoran Petkovic<br>
      Industrial Engineering Technology Pty Ltd<br>
      <a href="tel:%2B61%20413%20254%20315" value="+61413254315">+61 413 254 315</a> \
&lt;tel:0413%20254%20315&gt; | <a \
href="mailto:zoran@ietcombustion.com">zoran@ietcombustion.com</a><br> <br>
<br>
      On 14/09/2017, 10:10 PM, &quot;Johan Corveleyn&quot; &lt;<a \
href="mailto:jcorvel@gmail.com">jcorvel@gmail.com</a>&gt; wrote:<br> <br>
            On Wed, Sep 13, 2017 at 7:20 AM, Matt Simmons &lt;<a \
                href="mailto:bandman@gmail.com">bandman@gmail.com</a>&gt; wrote:<br>
            &gt; On Tue, Sep 12, 2017 at 12:14 PM Zoran Petkovic &lt;<a \
href="mailto:zoran@ietcombustion.com">zoran@ietcombustion.com</a>&gt;<br>  &gt; \
wrote:<br>  &gt;&gt;<br>
            &gt;&gt; In the past few days I have been doing extensive testing of \
                Subversion<br>
            &gt;&gt; with different clients, operating systems, client and server \
                versions and<br>
            &gt;&gt; have noticed very strange behaviour with windows clients \
                connecting to Linux<br>
            &gt;&gt; servers, hitting them with excessive CPU usage on the sshd \
process, where<br>  &gt;&gt; the Linux clients do not exhibit this behaviour.<br>
            &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt; A sample test setup is as follows:<br>
            &gt;&gt;<br>
            &gt;&gt; Server Linux Ubuntu 16.04.3 LTS, OpenSSH_7.2p2 \
                Ubuntu-4ubuntu2.2, OpenSSL<br>
            &gt;&gt; 1.0.2g   1 Mar 2016, Subversion version 1.9.3 (and 1.9.7).<br>
            &gt;&gt;<br>
            &gt;&gt; Client TortoiseSVN 1.9.7<br>
            &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt; When checking out large repositories the linux server is hit on \
                the sshd<br>
            &gt;&gt; process, the process running with 100% cpu usage. This in effect \
                slows down<br>
            &gt;&gt; the performance and ultimately the speed at which the checkout \
                runs. Linux<br>
            &gt;&gt; clients connecting to the same server do not cause this load on \
the server.<br>  &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt;<br>
            &gt;&gt; This happens even when compressions is turned off and when \
                encryption<br>
            &gt;&gt; Cyphers are changed, as well as different versions of \
                subversion. The<br>
            &gt;&gt; behaviour is identical. I&#39;m not sure who to address for this \
                issue as this<br>
            &gt;&gt; not only happens with TortoiseSVN but with SlikSVN as well. Any \
direction<br>  &gt;&gt; would be appreciated.<br>
            &gt;<br>
            &gt; Why does iostat show? Could it be that your underlying disk is \
io-saturated<br>  &gt; and your CPU spike is due to iowait?<br>
<br>
            One possible explanation would be that the windows clients perform \
                the<br>
            checkout faster than the linux clients for some reason, thus being<br>
            able to put higher load on the server, saturating its cpu or disk \
I/O.<br> <br>
            Other than that I have no idea.<br>
<br>
            The Linux client you&#39;re testing is also 1.9.7?<br>
<br>
            Perhaps the windows client and the linux client are performing the<br>
            checkout in a different way for some reason (I&#39;m thinking of the<br>
            so-called &quot;skelta mode&quot; vs. &quot;bulk mode&quot;, see<br>
            <a href="http://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default" \
rel="noreferrer" target="_blank">http://subversion.apache.org/<wbr>docs/release-notes/1.8.html#<wbr>serf-skelta-default</a>).<br>
 <br>
            --<br>
            Johan<br>
<br>
<br>
<br>
<br>
</div></blockquote></div><br></div></div>



[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic