Git and Make: not just for code

Don Marti

pandoc --template templates/default.html \
--to=html5 --smart --standalone --toc 

Other tools

Unoconv

Imagemagick

Ledger

aspell

Working on the command line

Remember command syntax?

Remember sequences of commands?

Tired of rowing through stuff?

Use the same spelling dictionary for both?

Professional Marketing White Paper

Gritty Noir Novel

Basic rules

Makefile has tabs, never spaces

$$ is $

# Comments

\ to break long commands (passed to shell)

Structure of a rule

# quiet comment
target : prerequisite
    # noisy comment
    recipe

This presentation's Makefile

index.html : index.md template.kittenslides
    pandoc --section-divs -t html5 \
    --template template.kittenslides -s -o $@ $< 

clean :
    rm -f index.html

.PHONY : clean

$@
$<

Patterns with %

SOURCES = index.md

PRESENTATIONS = $(SOURCES:%.md=%.html)

all : $(PRESENTATIONS)

%.html : %.md template.kittenslides
    pandoc --section-divs -t html5 \
    --template template.kittenslides -s -o $@ $< 

clean :
    rm -f $(PRESENTATIONS)

.PHONY : clean

Special target

.PHONY

Empty files as targets

deploy :
    # deploy your site here...
    touch deploy

print :
    enscript # obscure enscript arguments here
    touch print

That's great when you're working with a known list of files...

But what about NaNoWriMo?

"shell" and "foreach"

CHAPTERS=$(shell find . -name '??-*' | sort)

index.md : $(CHAPTERS) 
    rm -f $@
    $(foreach c,$(CHAPTERS),cat $(c) >> $@; \
    echo "<hr>" >> $@;)
    echo "The End" >> $@

Fix our aspell problem

# Create the project-specific dictionary
mynovel.pws : 
        echo "personal_ws-1.1 en 0" > $@

# Spell check using the project-specific 
# dictionary
spell : $(CHAPTERS) mynovel.pws
        $(foreach c,$(CHAPTERS), aspell \
        --personal=./mynovel.pws \
        check $(c);)

NaNoWriMo is all about the count

count : $(CHAPTERS)
        wc -w $(CHAPTERS)

"git gets easier once you get the basic idea that branches are homeomorphic endofunctors mapping submanifolds of a Hilbert space." —Isaac Wolkerstorfer

Scratch repositories are your friend

Hooks

pre-commit

update

post-update

Push to deploy or make deploy?

Use make

hooks : .git/hooks/pre-commit

.git/hooks/% : Makefile
        echo "#!/bin/sh" > $@
        echo "make `basename $@`" >> $@
        chmod 755 $@

pre-commit :
        git diff-index --check HEAD

Use make

hooks : .git/hooks/pre-commit

.git/hooks/% : Makefile
        echo "#!/bin/sh" > $@
        echo "make `basename $@`" >> $@
        chmod 755 $@

pre-commit : spell

Check that a ledger balances before committing

pre-commit : company.dat
    ledger -f $< bal > \
    /dev/null

smudge and clean scripts must run as filters: read from stdin, write to stdout

$ head secret-document.docx 
Ideas for a SCaLE talk on Git and Make {.western}
======================================

Two of the most common programming tools are way too useful to be left
to the programmers. We'll cover how to use the Git version control
system and the Make utility to get more accurate and faster results on a
variety of common tasks.

“But what about the people who use MS-Word or Libreoffice? Don't they
want to stick with their tools of choice?” you might ask.

In this case what's actually checked into Git is real .DOCX

git config filter.docx.smudge smudge-docx

git config filter.docx.clean clean-docx

$ cat .gitattributes
*.docx filter=docx

Like all Git configuration this is repo-specific, so you can mangle DOCX in one repository and leave it alone in another one.

Or you could make the Markdown the "real" version and use smudge to put a temporary .DOCX in Dropbox...

makewatch

make $* # Pass all arguments to make
while true; do
    fl=$(prereq_files $*)
    ev=$(inotifywait --quiet --format %e $fl)
    if [ "xOPEN" != "x$ev" ]; then
        sleep 1
        make $*
    fi
done

make_prereqs() {
    # Make "make" figure out what files
    # it's interested in.
    echo "Makefile"
    make -dnr $* | tr ' ' '\n' | \
        grep ".*'.$" | grep -o '\w.*\b'
}

prereq_files() {
    # prerequisites mentioned in a Makefile
    # that are extant files
    for f in `make_prereqs $* | sort -u`; do
        [ -e $f ] && echo -n "$f ";
    done
    echo
}

Replicating Git is remarkably easy*

Really big* Git deployments

etcd

update hook

post-update hook

Update hook: before moving the ref

Atomic test and set in etcd

Post-update hook

Flood the update out to other git repositories

Since etcd is already current, they accept it

Failures

Conflicting pushes: one fails, the other succeeds

Out of date repo: push to it fails, then it tries to pull from other members of repogroup

Git pushes already fail, so users can just deal

Meteor strike 1: you get out with your laptop

Meteor strike 2: laptop and home Git server destroyed right after update but before all objects copied

Summary

Use Makefiles to remember complex commands.

Never believe a Git tutorial.

People are lazy and like to break things.

Smudge and clean can help with format issues.

You don't have to keep typing "make" if you don't want to.

Git is almost meteor-proof because Computer Science.

Further reading

GNU Make manual

Butterick's Practical Typography

Pandoc User's Guide

Raft Consensus Algorithm

piehole: simple highly fault-tolerant Git replication

Creative Commons photos

Kitten: Christian Holmér

Map and compass: Audriusa for Wikimedia Commons

collision d'une comète sur la terre: NASA

Git data illustration from Pro Git (MIT license)

Any questions?

Follow @dmarti on Twitter, or contact via email.