[Dirvish] USB drives causing server lockups.(somewhat ontopic)

[Noel Kelly]

Keith's advice notwithstanding, I found a work around for crappy USB 
hardware can be to renice the process and limit the bandwidth:

nice --adjustment=10 rsync -rltDvv --progress --no-whole-file --delete 
--exclude=Temp --bwlimit=1000 /home/shared/Backups/ 
/media/usb-storage-10000E000CC31A5D\:0\:0\:0p1/Backups/

Cheers
Noel

Keith Lofstrom wrote:
> On Wed, Aug 20, 2008 at 11:51:54PM -0500, Richard wrote:
>   
>> ... External USB drives with 2.4 kernel ...
>>     
>
> Known problem, see:   http://wiki.dirvish.org/index.cgi?USB2Drives
>
> This is a problem between some Linux USB2 mass storage drivers and
> some nonstandard implementations of USB2 hard drive controllers. 
> It is not a dirvish or rsync problem - it is repeatable by writing
> large amounts of data to the drive.  It gets better with 2.6, but
> I still see problems.
>
> Suggestion:  consider trying different USB2 hardware, if it is
> cheap; you will need to know the chipset.  Avoid Cypress.  I 
> have some extra USB2 drives that I'm not using anymore.
>
> Better Suggestion: use external SATA instead.  The SATA drivers
> are quite robust in 2.6, and I am told they were adequate in the
> later 2.4 kernels, too.  If you use PATA aka IDE hard drives,
> like me, they can be adapted with SATA to PATA adapters, see: 
>
> http://wiki.dirvish.org/index.cgi?SataDrives
>
> These adapters permit hotplugging on the IDE side, with the
> right hdparm stuff thrown at them.  I have noticed a need for
> some time delay (wait a minute or two) when hotswapping with
> a 2.6.18 kernel;  I'm guessing they were still working out
> bugs with device state signalling in HAL, the hardware
> abstraction layer.  This is not a problem with earlier and
> later 2.6 kernels, and I don't think 2.4.x uses HAL.
> YMMV.
>
> All the adapters with JMICRON converter chips I have tried so
> far have worked, on 4 or 5 different adapter boards.  These
> typically cost $20-$40.  A PCI-slot 4 port SATA controller is
> also around $40.  Since SATA is becoming cheaper than IDE, 
> and is faster than IDE and USB2, this is the best long term
> way to go.
>
> If you learn something new, add it to the wiki.  Good luck!
>
> Keith
>
>