[Dirvish] expire and runall config file
keithl at kl-ic.com
Tue Mar 29 08:44:48 PST 2005
On Tue, Mar 29, 2005 at 02:50:25PM +0200, Paul Slootman wrote:
> I'm in the process of setting up a couple of backup systems that will
> eventually backup all 200 systems here. One problem I just ran into is
> that this takes up too much time if the systems are backed up
> sequentially. I'd like dirvish to be able to run X hosts in parallel.
> Of course it would be great if you could say dirvish-runall --parallel=3
> but that would entail a lot of code... A simple solution is to maintain
> a second (and third) master.conf file that eachs lists part of the
> hosts. For dirvish-runall that should work, as you can give a --config
> option to override the default. However, dirvish-expire doesn't accept a
> --config option. That doesn't make much sense to me...
> I've hacked up dirvish-expire to accept --config, tonight I'll know more
I will be very interested in your results. As slow as a single rsync
process might be, I suspect multiple rsync processes will just complete
with each other for resources, if they were using the same RAM and
target disk and network bandwidth. Multiple servers might do better,
but they would still be contending for network bandwidth. You will
probably make some good measurements of where dirvish/rsync is
spending its time, and we will need that profiling data when we start
working on performance optimization.
Version 1.3 scripts will share as much of a common command line and
config file read-in process as possible, so I'm hoping that in the
future, hacking scripts to add features that are already in other
scripts (and --config is a great example) will be a simple matter
of adding another quoted word to an array in the target script.
I would offer more support, but you are a better programmer than I am,
so the best I can offer is to seriously consider incorporating your
discoveries into the next version.
Keith Lofstrom keithl at keithl.com Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs
More information about the Dirvish