[Dirvish] Heads up for DST if you backup to/from FAT/NTFS filesystems

Eric Mountain em-dirvish-1 at nerim.net
Tue Mar 29 14:04:45 PST 2005


On Monday 28 Mar 2005 19:09, Keith Lofstrom spake thus:
> To clarify, I assume you are talking about backing up the windows
> partition on a dual-boot backup *client* to a dirvish/rsync backup
> server on a  different machine.  

Err, nope.  Same machine.

> This sounds like something that really ought to be fixed at the application 
> level rather than the configuration level, perhaps in cygwin+rsync.  

OK, maybe I wasn't clear enough.  I have a dual-boot machine with 2 hard 
drives.  The main hard drive has one partition for Windows, and the rest for 
Linux.  The second HD is where my dirvish backups go.  Dirvish only runs when 
the machine is booted under Linux - when it does, it backs up part of the 
Linux file systems (my home, /etc, etc.) *and* "My Docs" from the Windows 
partition.

[ So no cygwin. ]

I run the system with the hardware clock set to UTC.  Now, if you're a 
Unix-only kinda guy, that won't shock you, but Windows expects the hardware 
clock to be set to local time.  So when I'm booted in Windows, it gives me 
UTC - this is only a minor inconvenience, makes life on Linux easier (and 
that's my main concern as it is where I spend 99% of my time - there are ways 
to handle having a local-time hardware clock in Linux, but I can't be 
asked ;-) ...and I'm pretty sure it entails problems - e.g. suppose Linux 
gets booted up after switch to DST occurred: how does it guess whether 
Windows has already adjusted the clock or not?  Maybe there is a solution to 
this, I would have to RTFM.).

[ Any O/S stupid enough to want the h/w clock to be set to local time doesn't 
really deserve to be called one (I think it was Win95 required you to reboot 
the machine when flipping between summer and winter time). ]

I haven't quite figured out yet *exactly* why I've hit a problem in my 
configuration.  Sure, Linux is making an incorrect assumption about Windows 
FAT timestamps being in local time, but from what I can tell, the "incorrect" 
adjustments are always "consistent" (i.e. since DST rules are applied 
correctly, I always see the same "wrong" time).

Anyway, I know my configuration is probably a little uncommon.  However, even 
more common setups (running h/w clock on local time, or 
client(windows)-server(Un*x) configurations) are going to have problems if 
FAT is involved.

> We should be comparing the time data structures on the disk, not the
> interpretation of them by the client or server OS.  We should not rely on
> anything that could be affected by the settings of any hardware or software  
> clock. 

==> UTC.  Which is what rsync and Co are using (see JW's article).

The problem arises when you start "talking" to systems with FAT/NTFS file 
systems:

- FAT stores the file modification time in local time *and* Windows APIs 
adjust for DST based on the *current* date (so the adjustment may be 
incorrect).

- NTFS stores UTC timestamps, but conversions to local time again adjust for 
DST based on the *current* date...  Here, whether you will have problems or 
not depends on whether you're using stat() or GetFileTime() if I understand 
correctly (http://www.codeproject.com/datetime/dstbugs.asp).  Not sure what 
rsync does, but I would assume it Does The Right Thing(tm).  Not sure what 
happens through Windows or Samba shares though (for instance)??

> But it gets worse.  If I move a backup client from Oregon to Japan
> (across the date line), I should not get a new copy of the files on
> the dirvish server.  Since I can imagine a situation where the reported
> time can change by 23 hours, and I may want to back up every 4 hours,
> the --modify-window strategy will not always work.  It is certainly a
> lot better than nothing, though.  Thanks for making us aware of both
> the problem and a pretty good solution.  Remind me to put something
> in the FAQ about it.

Again, it's only a problem if FAT is involved (and NTFS if UTC isn't being 
used to compare file times).  O/w you're OK since you're comparing UTC times.

I hope this clarifies a little my reason for bringing the issue up - i.e. it 
was not so much my particular configuration as the problem in general that I 
thought deserved the heads-up.

JW drew up an excellent summary.  If you've got this kind of problem, it 
really should be your first stop.

Cheers,
Eric
-- 
Eric Mountain


More information about the Dirvish mailing list