[linux-elitists] etcd: A highly-available key value store

Don Marti dmarti at zgp.org
Tue Aug 13 14:58:02 PDT 2013

begin Greg KH quotation of Tue, Aug 13, 2013 at 10:21:16AM -0700:
> On Tue, Aug 13, 2013 at 07:30:59AM -0700, Don Marti wrote:
> > Making the rounds...
> >   http://coreos.com/docs/etcd/
> > 
> >   "A highly-available key value store for shared
> >   configuration and service discovery."
> > 
> > Another useful thing you could do with this is
> > replicate Git.  The way Git is designed makes
> > it easy to replicate (I disclosed this obvious
> > fact just in case USPTO wants to apply its usual
> > low standard of what's obvious and what's not:
> > http://ip.com/IPCOM/000225058 ).  Instead of a complex
> > replication scheme, you only need a highly-available
> > key value store to keep branch references in sync.
> > The testAndSet in etcd looks like what you need.
> So, I asked Xiang Li, the main developer of etcd, about this yesterday
> on irc, as I saw you had posted this idea elsewhere.  He said it really
> wouldn't work, as the raft protocol is only for "small" amounts of data
> per key (100kb).  This is due to the requirements of syncing the servers
> together in a specific amount of time (in ms).

You don't need to maintain the Git objects in
etcd--just refs.  You can move objects around with a
DHT or git-send-pack.  The only time you would need to
test and set a value in Raft is from an update hook.

The update hook would check if the value stored in the
ref being pushed to on that repository is the same as
the value in Raft, and attempt to update the value in
Raft before updating the ref on the repo and accepting
the push.  If this succeeds, then the slower task
of transferring the objects and updating the refs on
the other replicas to match what's in Raft can start.

Every other replicated copy is going to refuse pushes
to that branch until the objects reach it and its
copy of the ref gets updated to match Raft, but Git
users can already deal with it when a push fails.

> > Also, it's in Go.  On that subject, bonus link:
> >   http://blog.lusis.org/blog/2013/08/11/go-for-system-administrators/
> I've played around a bit in Go, and am very impressed.  There are some
> issues with libraries that seem easy for developers to mess up, but as
> long as you don't rely on random github repos for a shipping product
> (i.e. not copying them locally and using specific tags), you should be
> fine.

Tagged releases and locavore build and test servers
-- already making sure to do this.

Don Marti                      +1-510-332-1587 (mobile)
http://zgp.org/~dmarti/        Alameda, California, USA
dmarti at zgp.org

More information about the linux-elitists mailing list