[linux-elitists] Multi-terrabyte filesystems in Linux?

[Karsten M. Self]

The straightforward approach of creating a multiterrabye (in this case
~16 TB) in Linux, under Debian Sarge, doesn't work.  Kernel is
2.6.22-3-amd64.

  - fdisk maxes at about 355 GB, for some reason.
  - parted creates the partition but ...
  - mkfs.ext2 fails with a "file to large" error (not sure of the exact
    quote).

I'd not ordinarially resort to talking to this crowd but for two
additional facts:

  - The Linux filesystem and large disk howtos don't currently address
    this point.

  - I am the maintainer of a small but highly-Google-ranked Linux
    partitioning FAQ, which also, as it turns out, fails to address this
    situation (I'm currently berating myself for failing to anticipate
    my needs in this departemnt).
    http://linuxmafia.com/~karsten/Linux/FAQs/partition.html

I can, with appropriate information, address the latter issue, at the
least.


WRT the current situation, I strongly suspect sarge is Just Too Old.
We're looking at implementing on lenny (fortunately few if any external
dependencies).  CONFIG_LBD appears to be required.  Neither my Ubuntu
8.10 nor Debian Lenny stock kernels have this option set.


Among the possibly useful docs I am finding:

  - http://www.carltonbale.com/2007/05/how-to-break-the-2tb-2-terabyte-file-system-limit/

    Mentions CONFIG_LBD and suggests GBT partition type is required.


  - http://oss.sgi.com/projects/xfs/

    Quote:
        For Linux 2.4, the maximum accessible file offset is 16TB on 4K
        page size and 64TB on 16K page size. For Linux 2.6, when using
        64 bit addressing in the block devices layer (CONFIG_LBD), file
        size limit increases to 9 million terabytes (or the device
        limits).


  - http://www.linux.com/feature/115767

    Quote:
        The current standard Linux filesystem, Ext3, has a severe
        limitation with its 16 terabyte filesystem size limit. Between
        that and some performance issues, it was decided to to branch
        into Ext4.


  - http://web.archive.org/web/20061205013716rn_1/kerneltrap.org/node/186

    (archived) announcement of Peter Chubb's 2TB filesystem patch, in
    2002.



And the current filesystem and large disk HOWTOs:

    http://tldp.org/HOWTO/Filesystems-HOWTO.html
    http://tldp.org/HOWTO/html_single/Large-Disk-HOWTO/

I'm pinging the author of the LDH separately.


Peace.

-- 
Karsten M. Self <karsten at linuxmafia.com>        http://linuxmafia.com/~karsten
    Ceterum censeo, Caldera delenda est.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: Digital signature
URL: <http://allium.zgp.org/pipermail/linux-elitists/attachments/20090408/3bdfee9f/attachment.pgp>