[prev in list] [next in list] [prev in thread] [next in thread] 

List:       secure-shell
Subject:    Re: Performance question of file transfer of scp/sftp vs. FTP
From:       Atro Tossavainen <atossava () cc ! helsinki ! fi>
Date:       2002-06-28 10:07:19
[Download RAW message or body]

I wrote earlier:

>> Do remember that compression, whether by gtar z or by ssh -C, uses up
>> significant amounts of CPU and WILL slow down your transfers in most
>> cases involving modern networking.  It may be useful on 10 Mbps and
>> below, but in 100 Mbps and faster networks your machines are most
>> likely working hard enough doing the en/decryption, they don't need
>> an extra compression stage on the CPU slowing things down even more.

Thor Lancelot Simon replied:

> That's a bogus analysis.  Compression reduces the amount of data that
> must be encrypted;

It's an analysis based on observing actual data transfers.  I'll repeat
it for you.

> if the data is compressible, and the amount of CPU time used by the
> compressor is less that that used by encryption, overall throughput
> is increased.

...may be increased, depending on the compression ratio you get, and
also, whether throughput is limited by your network (in which case
you may be able to squeeze in more than the basic rate).

> Test on your loopback interface and see this for yourself -- zlib
> compression level 1 usually speeds up ssh data transfer significantly,
> unless you're moving precompressed data.

(When you're using the loopback interface, you're doing both the
 encryption and decryption (and compression and decompression) on
 the same machine, which (at least in our case) rarely has more
 than one CPU.)

Here's a test. I've got 101,806,080 bytes of linux-2.2.19.tar (source
and object files).  It is not compressed; the level 6 .gz version of
the same file was only 26,339,314 bytes, so the data is compressible.

The machines are running ssh 1.2.27, for whatever it's worth.

time scp file root@localhost:/dev/null

(RSA authentication in use so auth was not interactive and didn't
 increase the time seen by "time")

Scp'ing the file from disk over the loopback interface to /dev/null on
my Celeron 400 MHz, no other CPU-consuming processes running, took
44.4 seconds.  I repeated the test immediately to make sure I was
getting the file from cache instead of disk (since the machine has
plenty of memory), it took 42.9 s.  This was without compression, and
the transfer rate is slightly above 2200 kB/s.

I then repeated the test with compression.  I had set CompressionLevel
to 1 in the /etc/ssh_config file prior to trying it out.  I verified
with scp -v that compression was occurring at the requested level
instead of the default level 6, which was the case.  It took the
machine 1'20" to perform the transfer (>1200 kB/s).

Yes, my desktop machine is an old piece of garbage.  Let's see if we can
get better results with more modern machines.  I then tried the same
operation with an Athlon XP 1800+, running Linux, otherwise unloaded,
plenty of memory.  The two boxes are both on the same 100 Mbps fully
switched Ethernet, where maximum transfer rates obtained on non-
encrypting protocols are near wire speed.

Scp'ing the file to the Athlon from the Celeron took 20.4 s without
compression (>4700 kB/s).  Compressing at level 1, it took 54 s (>1800 kB/s).

Athlon over loopback to /dev/null: 22.2 s, >4500 kB/s.
With compression at level 1: 31.8 s, >3000 kB/s.

Timing via "time", and RSA authentication was used in all cases.

Would you be so kind as to point out exactly where the bogosity lies?

-- 
Atro Tossavainen (Mr.)               / The Institute of Biotechnology at
Systems Analyst, Techno-Amish &     / the University of Helsinki, Finland,
+358-9-19158939  UNIX Dinosaur     / employs me, but my opinions are my own.
< URL : http : / / www . iki . fi / atro . tossavainen / >

File attachments NOT welcome unless agreed to beforehand.
[prev in list] [next in list] [prev in thread] [next in thread]