[linux-elitists] rsync hacks (fwd from kragen@pobox.com)

Aaron Sherman ajs@ajs.com
Mon Jul 19 13:52:20 PDT 2004

On Mon, 2004-07-19 at 15:09, Andrew Kohlsmith wrote:

> I've only been half-following this thread, but LVM supports filesystem 
> snapshots.  Would this do what you need?  I'm eagerly awaiting PostgreSQL's 
> ability to quiesce its file structure so you can essentially tell it to sync, 
> run the LVM snapshot command, resume normal operation and back up while the 
> DB churns away as normal.

If LVM's snapshot is a true atomic snapshot, then no quiescing of the
database should be required. Every database has to contend with the fact
that the system could crash or a disk could go off line at any time. If
your database became corrupt every time the system crashed, it wouldn't
be very useful.

To that end, databases generally use the same sort of journalling that
filesystems do (a database is really just a high-level filesystem with
relational query semantics, after all).

If you perform a truly atomic snapshot, then your database will have a
consistent journal and, thought the state on disk may have to be
recovered by replaying the journal, that's not a big deal.

And, in fact, here is someone suggesting that exactly that is the case:

        I have a 2 disk stripe LVM on /dev/postgres/pgdata/
        lvcreate -L4000M -s -n pg_backup /dev/postgres/pgdata
        mount /dev/postgres/pg_backup /pg_backup 
        tar cf - /pg_backup | gzip -1 > /squeegit/mb.backup 
        umount /pg_backup;
        lvremove -f /dev/postgres/pg_backup;
        In a nutshell an LVM snapshot is an atomic operation that takes, well, a
        snapshot of hte FS as it was at that instant.  It does not make a 2nd
        copy of the data.   This way you can simply tar up the pgdata directory
        and be happy as the snapshot will not be changing due to db activity.

Things that I would question here: HOW atomic is it (e.g. is there any
negotiation with the scheduler over pending writes, etc) and how much
has it been tested? Would be interesting to find out....

Aaron Sherman <ajs@ajs.com>
Senior Systems Engineer and Perl Toolsmith

More information about the linux-elitists mailing list