[Dirvish] Baffling rsync/ssh/Cygwin problem

Joseph Lamoree joseph at lamoree.com
Sat Mar 26 21:52:24 PST 2005


Thanks for the thoughtful response, Keith. Your help is much 
appreciated.

On 26 Mar, 2005, at 14:23, Keith Lofstrom wrote:

> Thanks for writing.  I wish I had more to offer, but as you say, this
> is a Windows/Cygwin/Rsync problem.  There are some members of this list
> that will have nothing to do with Windows.  I tend towards that view
> myself, but temper it with sympathy for all the people stuck between
> their wish to run pure Linux and the reality of jobs that involve 
> Windows.

If I had a vote in the matter, there'd be no Windows machines on my 
network. Unfortunately, this is a legacy system that needs to be 
fostered until a Linux solution can replace it.

> Speaking practically, you are probably not going to get an immediate
> solution.  Steve Ramage has had some success, see for example:
> http://www.dirvish.org/pipermail/dirvish/2005-March/000170.html

That was actually the very first hit in my results of searching for 
other poor bastards in my situation. I had not yet tried running the 
rsync daemon on TCP/873 instead of through SSH. [Ed: read further for 
results]

> You will get lots of hits, including some suggestions from jw schultz
> himself.  I would love to see someone take his advice in this posting:
> http://lists.samba.org/archive/rsync/2003-November/007727.html

After reading all the messages in that thread, and others related, I've 
come to the conclusion that rsync doesn't work in the way that I would 
use it for all the other non-Windows machines on the network. I even 
tried having the ssh remote command run in daemon mode, as described in 
the rsync manpage using this style command:

   rsync -av --rsh="ssh -l ssh-user" rsync-user at host::module[/path] 
local-path

When a command like that executes, rsync reports that it will 
effectively be executing the following:

   ssh -l ssh-user host 'rsync --server -daemon .'

And then it'll communicate with the freshly spawned rsync daemon 
through the ssh session. Still, there is something about Cygwin that 
causes the rsync process to get stuck, and not let the whole procedure 
continue (under certain circumstances).


> ... To start with,  you should see if rsync always hangs in the same 
> place, or
> with the same type of data.  Building the smallest possible data
> set that reliably triggers the hang would be invaluable to the folks
> that eventually fix it (and it WILL get fixed with such help).

My experience thus far is that the following things to happen:
  - A rsync job on the "server" that requests to synchronize a small 
amount of data on the "client" (say, one directory with a few files, 
roughly 5 Mb each) works fine

  - A rsync job on the server that requests to synchronize a large 
amount of data on the client (say, several hundred directories in a 
nested hierarchy containing thousands of files) will cause the server 
to create a destination copy of the complete directory structure, then 
copy a single file, and hang forever.

The circumstances of that scenario are that the "server" is a Linux 
machine running rsync over ssh to a Windows 2000 machine running 
Cygwin's sshd to provide a shell from which rsync is started.

If I change the rsync command on the server such that it starts rsync 
in daemon mode on the client, the results are the same, however the 
src/dst are a little different because rsync is reading the 
/etc/rsyncd.conf to load up the modules, and so forth. But basically, 
it's the same.

> Thank you for the quality of your question, though.  You have
> obviously read ESR's guide to asking questions the smart way:
> http://www.catb.org/~esr/faqs/smart-questions.html

I didn't know about that. That's a nice piece of writing.

After doing some experimenting, what does work nicely is to start up 
rsync on the client like so:

   rsync --daemon --no-detach --config="/home/Admin/rsyncd.conf"

In which the configuration file defines a module with the stuff to be 
placed under Dirvish control. My next step will be to implement this as 
a NT service, just like cwRsync does. I would have liked the ssh method 
to work, but since the backup traffic is on a gigabit private backup 
network, it isn't a big security problem. Well, I mean, besides it 
being Windows.

--
Joseph Lamoree



More information about the Dirvish mailing list