[linux-elitists] Cluster filesystems

Nathan Poznick kraken@drunkmonkey.org
Fri Jan 2 13:30:54 PST 2004

Thus spake Ben Woodard:
> 3) The lustre developers at CFS want absolutely nothing to do with doing
> support. They want to get paid to code and don't want to do support at
> all. They have been trying to convince Red Hat to productize and support
> CFS and just give them some money to continue developing it.

I'm not disparaging the technical aspects of lustre; it does appear to
be working toward a solid clustered filesystem.  However, the attitude
of CFS leaves a lot to be desired.  I can understand not wanting to do
support, but I think they go too far.  I can't remember how many times
Peter mentioned something about, "but no one is paying us for that"
during the lustre tutorial at SC2003.  The one that really got my
attention was in regards to their website.  (not exact wording:) "People
ask us why we don't update our website, why we have broken code up for
download, and when it comes down to it, the fact is that no one is
paying us to update our website."  When the disdain for support extends
to not putting forth the effort to place non-broken code drops on your
website... I dunno, but that almost seems contemptuous to me.  It's just
my opinion that CFS could put forth a little bit of effort, and make
huge leaps forward in public relations.

> One more thing I can say about lustre. Since this is public money that
> we are spending here. We constantly are reevaluating the various cluster
> file systems and every single time we do this we find that nothing meets
> our needs other than lustre. PVFS is one that we just evaluated and it
> didn't do very well on our tests. I can probably dig up the report.
> However, all of this info is based upon our needs. Your needs might be
> different.

Indeed.  PVFS is not intended for use as a general-purpose clustered
filesystem, and is designed as such.  PVFS1 in particular has an
extremely simple design, with very little attention to fault-tolerance
and other such considerations.  It is however, Really Fast.  On the Jazz
cluster at ANL for instance (where PVFS is developed, in conjunction with
Clemson Univ.), GFS is used for long-term storage.  PVFS1 is used for
scratch space when running jobs -- when the filesystem needs to be very
fast, shared, and a failure simply means that a job must be run again.
PVFS2 makes things a little bit easier to handle in the fault-tolerance
area, but is still designed for speed-not-recovery.

As an aside, have you (or other folks at LLNL) done any evaluation of
Panasas?  It appears to be derived from the same basic principles as
luster (object-based storage in particular), but has taken a different
route.  It's closed-source (they told us they intended to GPL the client
piece), and they sell a complete solution including the hardware - just
plug it into the network and go.  LANL has some Panasas storage in use,
but when I talked to them at SC2003 about it, I got the feeling that
they hadn't really had a chance to really pound on it in production yet.
I'm curious if LLNL has evaluated it, and if so, how you feel it
measures up to the other solutions.

Nathan Poznick <kraken@drunkmonkey.org>

A program is a device used to convert data into error messages.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://allium.zgp.org/pipermail/linux-elitists/attachments/20040102/62ef72a4/attachment.pgp 

More information about the linux-elitists mailing list