[linux-elitists] etcd: A highly-available key value store

Don Marti dmarti at zgp.org
Tue Oct 29 06:20:29 PDT 2013


begin Don Marti quotation of Tue, Aug 13, 2013 at 02:58:02PM -0700:
> begin Greg KH quotation of Tue, Aug 13, 2013 at 10:21:16AM -0700:
> > On Tue, Aug 13, 2013 at 07:30:59AM -0700, Don Marti wrote:
> > > Making the rounds...
> > >   http://coreos.com/docs/etcd/
> > > 
> > >   "A highly-available key value store for shared
> > >   configuration and service discovery."
> > > 
> > > Another useful thing you could do with this is
> > > replicate Git. 
> > 
> > So, I asked Xiang Li, the main developer of etcd, about this yesterday
> > on irc, as I saw you had posted this idea elsewhere.  He said it really
> > wouldn't work, as the raft protocol is only for "small" amounts of data
> > per key (100kb).  This is due to the requirements of syncing the servers
> > together in a specific amount of time (in ms).
> 
> You don't need to maintain the Git objects in
> etcd--just refs.  You can move objects around with a
> DHT or git-send-pack.  The only time you would need to
> test and set a value in Raft is from an update hook.

First whack at an implementation:
  https://github.com/dmarti/piehole

The general idea is that I'll have copies of Git
repositories on multiple VPSs and in-house--push to
one, sync with all the rest.  Putting it up to see
what I've missed -- as far as I can tell, connecting
the dificult Computer Science in etcd to the difficult
Computer Science in git, using a simple hook, gives
you a high-availability versioning system.

-- 
Don Marti                      +1-510-332-1587 (mobile)
http://zgp.org/~dmarti/        Alameda, California, USA
dmarti at zgp.org


More information about the linux-elitists mailing list