[Dirvish] Filesystems for big partitions

Patrick Toal ptoal at takeflight.ca
Sat Oct 10 14:34:39 UTC 2015


I have nothing as large as 190TB... That said, here's my experience, to help you decide if you want to look more into it...

I have built a FreeNAS system for my home NAS/backup needs.  It is currently composed of 5 x 3TB disks, running in RAID-Z2. I run my dirvish installation in a FreeBSD Jail.  

I have been running this system for nearly a year, and I have to say that I am very impressed with the FreeBSD / ZFS combination for this application.  It is extremely performant, and rock solid.

Also, an important side-benefit... ZFS offers on the fly compression.  So, if your backups contain a lot files that are compressible, you'll be able to save a lot of disk space, with little performance impact (if you use an appropriate CPU).

If you have a large backup need, you can also create multiple ZFS Pools, and distribute your backup vaults across pools.  Extremely flexible.

http://www.freenas.org/


—
Patrick Toal
ptoal at takeflight.ca







> On Oct 8, 2015, at 3:31 AM, Andreas Nitsche <andi at sixhop.net> wrote:
> 
> Hi Chris,
> 
> maybe you want to think about having more banks with smaller partitions
> in case of a filesystem check or even a resize. I'm still using ext4 for
> my backup filesystem because I didn't knew it better when I setup the
> machine. I resized the partition from 4 to 6 TB for which I first need a
> filesystem check which needs to have 4 GB additional RAM and it took
> about 3 ours to check the partition. After that the resize took again 4
> ours. Maybe that's not needed with XFS or ZFS, but think about it before
> you need to do it is everytime a good idea. :)
> 
> kind regards,
> Andi
> 
> Am 05.10.2015 um 18:22 schrieb Güder, Christian:
>> 
>> 
>> Dear all,
>> 
>> 
>> 
>> We are using Dirvish since a long time to backup numerous Linux
>> applications and NAS servers – and despite development is not that
>> active anymore, it’s still doing a good job.
>> 
>> 
>> 
>> We’re about to expand our backup servers – this time using 6TB disks.
>> This will lead to filesystems of theoretical sizes between ~120 and
>> ~190TB. Right now, we are using XFS as filesystem, but the sheer amount
>> of files is really challenging it. Does anybody have experience with
>> handling /bigger than usual/ filesystems using ext4, BtrFS, GlusterFS or
>> even ZFS (using Linux)? Sharing any advices greatly appreciated.
>> 
>> 
>> 
>> Kind regards,
>> 
>> Chris
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Dirvish mailing list
>> Dirvish at dirvish.org
>> http://www.dirvish.org/mailman/listinfo/dirvish
>> 
> _______________________________________________
> Dirvish mailing list
> Dirvish at dirvish.org
> http://www.dirvish.org/mailman/listinfo/dirvish



More information about the Dirvish mailing list