[Dirvish] CPU bound or disk I/O bound? (rsh: option respected?)
jasonb at edseek.com
Sat Jan 8 23:13:19 PST 2005
On Saturday 08 January 2005 22:40, Keith Lofstrom wrote:
> On Sat, Jan 08, 2005 at 04:17:13PM -0500, Jason Boxman wrote:
> > Does setting 'rsh: rsh' work (or using `ssh` with some additional
> > options) for anyone else in /etc/dirvish/master.conf?
> > (It ought to outright fail unless `rsh` is actually configured and
> > installed.)
> "rsh: rsh" added to /etc/dirvish/master.conf works for me. That
> is, it *tries* to use rsh and fails, because I have rsh service
> turned off. There is a small chance that you set the "rsh:" option
> in both master.conf and in $VAULT/dirvish/default.conf, in which
> case dirvish will use the latter file for the option.
Fortunately, this was an easy one. For non-binary options you can specify
multiple parameters, each on a newline. I assumed such would be the case
with the "rsh: " option. Specifying it in master.conf as a single line works
as expected. In a roundabout way the manual actually mentions this.
Single value options are specified by lines of the form option: value.
Options expecting list must be specified in a multi-line format as shown
here where the lines specifying values are indented by any kind of
whitespace even if only one value is being specified.
> You might try setting "summary: long" in your master.conf file,
> and see what that tells you.
> To speed up transfers when you have mondo network bandwidth, try
> adding "whole-file: 1" to master.conf and see what that does.
> See dirvish.conf(5) for more details.
Yeah, I had thought about that. This most recent run was a whole vault of new
files, so the option would have had no effect.
> You should also try using raw rsync to move some files, and rcp,
> as well as native file transfers between disks. If your target
> is a Linux box, think about experiments with "hdparm" to tune up
> the disks - that can double the speed. If it turns out that disk
> speed is the issue, you may find yourself going to striped RAID.
Yeah, I was a bit light on the details originally. It was really time for
The fileserver is a dual P3 733MHz box on a SuperMicro 370DLE. The array
being backed up is a 4 x 120GB RAID 5 (64K stripe) array on a 3Ware Escalade
7410 controller on a dedicated PCI bus, 64 @ 33MHz. I haven't run bonnie++
or iozone3 on it, but a quick `hdparm -Tt` gives me about a 34MB/s read
speed. The Gigabit card is a rather cheap 32-bit ns83820 based card I plan
on abandoning soon. (I actually emailed you about Dirvish last March and I
think I mentioned I was going to try backups over gigabit and this was one of
the cards I had at the time...) The card's good for at least 20MB/s.
The backup server is a slot 1 P3 600MHz. Its array is on a 3Ware Escalade
7810 with 2 x 300GB in a RAID 0 configuration with a 128K stripe. It has an
old Allied gigabit NIC in the shared 32-bit bus along with the 3Ware. I ran
bonnie++ on it and it reported:
Version1.03------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sarah 2000M:64k 9311 97 84936 81 29314 35 10409 97 60936 31 155.6 6
sarah 2000M:64k 8951 97 84840 81 29563 36 10509 99 62812 25 155.7 7
sarah 2000M:64k 9438 98 84899 81 29923 36 10502 99 62822 24 154.3 6
sarah 2000M:64k 9430 97 85009 81 29618 36 10501 99 62803 25 154.3 7
sarah 2000M:64k 9426 97 84845 81 29728 36 10502 99 62815 24 155.4 6
Both filesystems are XFS.
> All that said, it would be a great addition to the wiki if you
> did some careful experiments and wrote up your results. For those
> of us poking along with 100BT, 11MB/s sounds pretty good.
It seems I did a lot better this time, once I had it running over `rsh`. I
imagine the P3 can handle using `ssh` with arcfour or blowfish without much
change in these numbers.
When I transferred around 20GB of files mostly < 1MB I ended up with the
Total bytes received: 22770511126
sent 2125814 bytes received 22770511126 bytes 8821474.70 bytes/sec
total size is 22761591080 speedup is 1.00
When I copied 200GB of files > 100MB I ended up with just shy of 12MB/s
Am I looking at the latency of the disk heads repositioning themselves for
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff
More information about the Dirvish