[Dirvish] Dirvish speed and security

[Leopold Palomo-Avellaneda]

A Dimecres 28 Maig 2008, Dave Howorth va escriure:
> Leopold Palomo-Avellaneda wrote:
> > A Dimecres 28 Maig 2008, Dave Howorth va escriure:
> >
> > The icy box _only_ offer network connection. So, maybe one option should
> > be use another box (with a Linux) and this external hd connected via nfs.
> > Then I could have a bot with ram and cpu to run rsync. But, by now, I
> > couldn't.
>
> Well, that still leaves the NFS link as a performance killer. If the icy
> box can't be upgraded then the best solution (but maybe not achievable)
> might be to remove the disk(s) from it and connect them to another
> computer.

:-(

ok, I take note.

>
> >> As it is, it looks like most time is spent transferring files, since
> >> your logs say "File list generation time: 882.634 seconds" and "File
> >> list generation time: 927.970 seconds" which is only 15 mins if my
> >> arithmetic is correct. Even the total bytes transferred is only 90 MB or
> >> so. Which points to NFS speed as the main suspect, IMHO.
> >
> > yes, I agree, but let me show you some numbers.
> >
> > I have a directory with dumps of svn. The directory has a size of:
> > ris:/opt/backup# du -sh /srv/seg/repositori/
> > 3,5G    /srv/seg/repositori/
> > ris:/opt/backup# du -s /srv/seg/repositori/
> > 3590972 /srv/seg/repositori/
> >
> > if I do I simple copy of the whole directory to the nfs unit, I got:
> >
> > ris:/opt/backup# time cp -r /srv/seg/repositori /opt/backup/test/
> >
> > real    18m26.502s
> > user    0m0.148s
> > sys     0m6.532s
> >
> > so, it has needed 18m and 26.502 s to copy 3.5G. So, making some
> > division: 3590972/(18*60 + 27) = 3243.8 ~ 3.5 Mb/s
>
> I agree, except that's 3.5 MB/s isn't it, which makes about 28 Mb/s.
> That's believable for a 100 Mbit link, but you said you had a 1 Gbit link?

yes, and I physically have been to the servers room and looked the switch and 
the cables. And I did an error in the units. In the icy box there's a raid1, 
but I don't think that could affect.

> So the first performance problem appears to have nothing to do with
> rsync; it's simply that your network is running slow. Are you absolutely
> sure it is running at 1 Gb and hasn't downspeeded for some reason? I
> think you need to run some network diagnostics.

well I can investigate ...

> (that's assuming you can actually better that rate on local disk
> transfers :)
>
> >> As far as I can see, you haven't posted your NFS export and mount
> >> settings, nor any measurement of actual network transmission rates.
> >
> > the server mounts the nfs unit with:
> > ulises:/mnt/md1/backup  /opt/backup     nfs
> > rsize=32k,wsize=32k,intr,rw,nfsvers=3,retrans=10 0 0
> >
> > and the icy box export the unit:
> >
> > /mnt/md1/backup X.Y.Z.K(rw,no_root_squash,sync)
>
> I think others have suggested using noatime; that will definitely help.
> I believe that using sync will also slow it down but I haven't tried
> that recently.

well is more secure. Proving the noatime at mount, I got:

ris:/opt/backup# time cp -r /srv/seg/repositori /opt/backup/test/

real    18m12.526s
user    0m0.172s
sys     0m6.452s

a 2% more faster ...


How can I test the network with this box? I cannot run the iperf because I 
need to run it in the icy box and it's not possible. Maybe, compiling it 
using crosscomplie, etc ....

Regards,

Leo