Using Python and Golang in Makefiles

January 2020 · 3 minute read

Makefiles are the main and most common “build system” used by me. A Makefile is the first file I’m creating in every project. With a reasonable default rule (usually “build and run tests”) they make switching between different projects less time-consuming (cause I don’t have to check, which cargo flags I usually use or what name a docker image I want build should have) and, also, the things around the code more uniform (e.g. usually my CI configuration has jobs calling simply make check, make, make test etc.).

Python

Last week, when reading why my Makefiles are wrong I’ve learnt that I should:

Tell Make “this Makefile is written with Bash as the shell” by adding this to the top of the Makefile: SHELL := bash

Which is a thing I’ve spotted many times before during years of using Makefiles, but when I’ve been using Make as “a tool for writing tab-indented named shell snippets that depend on each other” I’ve never thought it’s possible to use a non-shell interpreter for the recipes:

The key message here, of course, is to choose a specific shell. If you’d rather use ZSH, or Python or Node for that matter, set it to that.

What does it mean is that one can both use all the goodies supplied by Makefile (so well-understood target/prerequisites/recipe model and ubiquitousness of make) with one’s favourite language’s syntax and libraries. Wow!

So by using setting SHELL and .ONESHELL one can use Python to generate a target file by downloading (and, possibly, parsing) a website:

# Makefile
SHELL := python
.ONESHELL:

index.html:
	import requests
	with open('$@', 'w') as f:\
		f.write(requests.get('https://hryni.uk').text)

Running make ($@ refers to the target file) will create the index.html file (if it doesn’t exist) with the content of the site under https://hryni.uk.

Go

It seems make just calls passed interpreter with -c flag and the recipe’s source, we can check it in (docs or) a fancy way by running strace -f make (-f makes strace trace child processes):

$ strace -f make
...
[pid 28099] execve("/usr/sbin/python", ["python", "-c", "import requests\nwith open('index"...], 0x55a94d29aca0 /* 42 vars */ <unfinished ...>
...

…so it’s possible to run whatever script one could think of there. So, why not, let’s write Makefile in Go. One way to do so could be to save recipe’s code in a file and then go run this file. Here’s the Makefile:

# Makefile
SHELL := ./rungo.sh
.ONESHELL:

index.html:
	c := make(chan string)
	go func () { c <- "Makefile in go!" }()
	fmt.Println(<-c)

rungo.sh is a script (without most of the things that make your Bash scripts right to make it shorter) that takes source code of the recipe above, wraps it with func main(), saves to a temporary file, then compiles and runs:

#!/bin/bash
# rungo.sh

# $0 == "./rungo.sh"
# $1 == "-c"
# $2 == "c := make(chan string) go func () { c <- "Makefile in go!" }() fmt.Println(<-c)"
SRC="$2"
TMP_FILE="$(mktemp)"

echo -e "package main\nfunc main() { ${SRC} }" > "${TMP_FILE}"
goimports "${TMP_FILE}" > "${TMP_FILE}.go"
go run "${TMP_FILE}.go"

Of course it can be run like a “normal” Makefile:

$ make
Makefile in go!

Ta-da!