[Dirvish] Vault Unsynced

Eric Mountain em-dirvish-1 at nerim.net
Mon Jan 3 14:36:36 PST 2005

On Monday 27 Dec 2004 18:54, Steve Ramage spake thus:
> I've noticed a HUGE jump in disk consumption accross all my backups,
> they seem to be double each of there original size.

As Keith mentionned, it might be good to take a look at the log files, but I 
have a feeling what we're looking for isn't there (at least, not unless you 
have debug/verbose rsync options and are proficient at reading the output - 
which I am not ;-) ).

> I don't know of a way to get du to take two files and see if they are
> symlinked., and then not use those. 

Don't understand that bit.

> Also is there a way to have a program go thru each pair of directories,
> check if the file matches and if it does hardlink the files together, so
> I can repair this?

rsync ;-)  No kidding: just do the same as the backup process does, only:
 src = "bad" backup which didn't create hardlinks as expected
 dest = new backup directory
 --link-dest= "good" backup directory to which you want to hardlink
Untested, but I think it should do the trick.  Check the logs for the rest of 
the relevant rsync options to use based on command that dirvish actually 

Note: I'm assuming you mean you want to fix backups which are good, but for 
which hardlinks didn't occur for some reason.

Note2: This will only work if it is possible to hardlink the files - i.e. all 
the standard rsync requirements for hardlinking are satisfied, i.e. that the 
2 files':
 * timestamp
 * owner uid+gid
 * permissions
 * size
 * others?
*all* match.  And I have a hunch this is where the problem lies: for some 
reason the files are not hardlinkable.

> [02:39:06] root at fermat:/data2/backups/mydocs$du ./ -s --si
> 4.4G    ./
> [02:39:11] root at fermat:/data2/backups/mydocs$./file.sh
> 2.2G    20041026
> 2.2G    20041027
> 2.2G    20041028
> 2.2G    20041029
> 2.2G    20041030
> 2.2G    20041031
> 2.2G    20041101
> 21k     20041102
> 2.2G    20041103
> 21k     20041104
> 2.2G    20041105
> 2.2G    20041106
[... etc ...]

Can't tell much from this really except that it looks like 2 dumps failed.

> Follow Up Question (11-15-04): I found out that the new 'set' starts on
> 20041103, I found this by symlink randomly in another directory and
> checking the filesize of that directory, and found that if I have
> 20041024 and 20041102 the du reports 2 gigs in use, but 20041024 and
> 20041103 reports 4.1 gigs in use. 

Well, 20041102 failed (21k), so it would be a big surprise if 
20041024+20041102 was 4Gb.  Is 20041102 in the dirvish backup history file?  
If so, then it would have been used as the reference for 20041103, so no 
hardlinks would have taken place since it is empty, hence the duplication.

What is the size of 20041024+20041101?

> The question is now that I know where 
> the set is broken, how can I repair it, adn get my disk space back?

See above.  Then rm the "bad" backup once you're sure you've created a good 
copy.  Problem is, you actually have to repeat the procedure for all 
subsequent backups until all are once again based on the same pool of 
hardlinked files.  That is of course assuming files are hardlinkable by rsync 

Anyway, I have looked for a similar problem on my system and did come across 
something: I discovered that one of my vaults (a backup of a CVS repository 
on which *no* changes occur *at*all*) exhibited the exact problem you 
describe.  Turns out, that the owner write permission changes over time in 
the backups.  I have yet to figure out why.  Example:

root at erina:cvs# find . -name SelectYear.jsp\* -exec ls -l {} \;
-r--r--r--  1 500 500 3384 2002-09-22 
18:12 ./200501031850/tree/repository/school/src/java/school/jsp/SelectYear.jsp,v
-rw-r--r--  4 500 500 3384 2002-09-22 
18:12 ./200412302243/tree/repository/school/src/java/school/jsp/SelectYear.jsp,v
-rw-r--r--  4 500 500 3384 2002-09-22 
18:12 ./200412311222/tree/repository/school/src/java/school/jsp/SelectYear.jsp,v
-rw-r--r--  4 500 500 3384 2002-09-22 
18:12 ./200501011647/tree/repository/school/src/java/school/jsp/SelectYear.jsp,v
-rw-r--r--  4 500 500 3384 2002-09-22 
18:12 ./200501020925/tree/repository/school/src/java/school/jsp/SelectYear.jsp,v
root at erina:cvs# ls 
-l /home/cvs/repository/school/src/java/school/jsp/SelectYear.jsp,v
-r--r--r--  1 500 500 3384 2002-09-22 
18:12 /home/cvs/repository/school/src/java/school/jsp/SelectYear.jsp,v

I will try to figure this out, but if anyone has any ideas, they're welcome...

Oh and in case you're interested, I'm attaching a perl script (ahem, not sure 
it deserves being called that) that you can use to get a few stats on a 
directory structure, including the number of files which are hardlinked (i.e. 
inode reference count > 1).  To get a list of files that are not hardlinked, 
look for the line that reads 'print "not hardlinked: $_\n";' and which is 
commented out.

Eric Mountain

More information about the Dirvish mailing list