[Dirvish] Heads up for DST if you backup to/from FAT/NTFS
em-dirvish-1 at nerim.net
Tue Mar 29 14:04:45 PST 2005
On Monday 28 Mar 2005 19:09, Keith Lofstrom spake thus:
> To clarify, I assume you are talking about backing up the windows
> partition on a dual-boot backup *client* to a dirvish/rsync backup
> server on a different machine.
Err, nope. Same machine.
> This sounds like something that really ought to be fixed at the application
> level rather than the configuration level, perhaps in cygwin+rsync.
OK, maybe I wasn't clear enough. I have a dual-boot machine with 2 hard
drives. The main hard drive has one partition for Windows, and the rest for
Linux. The second HD is where my dirvish backups go. Dirvish only runs when
the machine is booted under Linux - when it does, it backs up part of the
Linux file systems (my home, /etc, etc.) *and* "My Docs" from the Windows
[ So no cygwin. ]
I run the system with the hardware clock set to UTC. Now, if you're a
Unix-only kinda guy, that won't shock you, but Windows expects the hardware
clock to be set to local time. So when I'm booted in Windows, it gives me
UTC - this is only a minor inconvenience, makes life on Linux easier (and
that's my main concern as it is where I spend 99% of my time - there are ways
to handle having a local-time hardware clock in Linux, but I can't be
asked ;-) ...and I'm pretty sure it entails problems - e.g. suppose Linux
gets booted up after switch to DST occurred: how does it guess whether
Windows has already adjusted the clock or not? Maybe there is a solution to
this, I would have to RTFM.).
[ Any O/S stupid enough to want the h/w clock to be set to local time doesn't
really deserve to be called one (I think it was Win95 required you to reboot
the machine when flipping between summer and winter time). ]
I haven't quite figured out yet *exactly* why I've hit a problem in my
configuration. Sure, Linux is making an incorrect assumption about Windows
FAT timestamps being in local time, but from what I can tell, the "incorrect"
adjustments are always "consistent" (i.e. since DST rules are applied
correctly, I always see the same "wrong" time).
Anyway, I know my configuration is probably a little uncommon. However, even
more common setups (running h/w clock on local time, or
client(windows)-server(Un*x) configurations) are going to have problems if
FAT is involved.
> We should be comparing the time data structures on the disk, not the
> interpretation of them by the client or server OS. We should not rely on
> anything that could be affected by the settings of any hardware or software
==> UTC. Which is what rsync and Co are using (see JW's article).
The problem arises when you start "talking" to systems with FAT/NTFS file
- FAT stores the file modification time in local time *and* Windows APIs
adjust for DST based on the *current* date (so the adjustment may be
- NTFS stores UTC timestamps, but conversions to local time again adjust for
DST based on the *current* date... Here, whether you will have problems or
not depends on whether you're using stat() or GetFileTime() if I understand
correctly (http://www.codeproject.com/datetime/dstbugs.asp). Not sure what
rsync does, but I would assume it Does The Right Thing(tm). Not sure what
happens through Windows or Samba shares though (for instance)??
> But it gets worse. If I move a backup client from Oregon to Japan
> (across the date line), I should not get a new copy of the files on
> the dirvish server. Since I can imagine a situation where the reported
> time can change by 23 hours, and I may want to back up every 4 hours,
> the --modify-window strategy will not always work. It is certainly a
> lot better than nothing, though. Thanks for making us aware of both
> the problem and a pretty good solution. Remind me to put something
> in the FAQ about it.
Again, it's only a problem if FAT is involved (and NTFS if UTC isn't being
used to compare file times). O/w you're OK since you're comparing UTC times.
I hope this clarifies a little my reason for bringing the issue up - i.e. it
was not so much my particular configuration as the problem in general that I
thought deserved the heads-up.
JW drew up an excellent summary. If you've got this kind of problem, it
really should be your first stop.
More information about the Dirvish