[Dirvish] stop dirvish but not loose transfered data

Keith Lofstrom keithl at kl-ic.com
Sat Nov 12 03:05:53 EST 2005


On Fri, Nov 11, 2005 at 11:50:46PM -0200, Joel Franco wrote:

> I've used dirvish a lot time and have a specific need: i wanted to use
> the full bandwith to make the backup at night but, if the data size is
> too big to transfer, i want to kill the process at 7am, at which the
> people start working and need the WAN bandwith to use the web.
> 
> Thinking about it and, supposing that at certain night a have a data
> size that i didn't get to completely transfer, i do not want to loose it
> and start again, but continue at the next night.
...

Probably the easiest way to do this is to partition your data into
smaller vaults and run them sequentially using your own script, not
dirvish-runall.  Let's say you divide things up into 10 smaller
chunks, each taking about an hour on average.  You launch them
in a circular queue, perhaps doing A B C D E F G the first night,
H I J A B C D E the second night, F G H I J A B C D E (all) the
third night because they are light backups, and so forth.  Complete
as many 1 hour segments as you can manage, starting the last one at
530am or so.

You can use the dirvish include and exclude rules to divide up your
data into appropriate-sized groups.

I don't think you want to interrupt the dirvish/rsync process. 
If rsync is so busy that it cannot complete moving a filesystem,
that implies that there is a lot of change to that filesystem.
A 16 hour gap in the middle of the move will make too many files
change or appear or disappear.

This has the additional advantage that you can initialize new
vaults in a reasonable time.  A vault that incrementally requires
60 minutes to move over a slow link may take many hours - perhaps
days - to initialize, a much bigger problem for you.

A perl script, perhaps named "dirvish-rotate", that adds rotation
features to dirvish-runall would be a useful addition to the dirvish
capability.  Alternatively, you can accomplish something similar by
using multiple master.conf files called on alternating days from cron.

I hope that gives you some ideas.  

Keith

-- 
Keith Lofstrom          keithl at keithl.com         Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs


More information about the Dirvish mailing list