Skip to main content

shave: making the autotools output sane

updated: Automake 1.11 has been release with "silent rules" support, a feature that supersedes the hack that shave is. If you can depend on automake 1.11 please consider using its silent rules rather than shave.
updated: add some gtk-doc info
updated: CXX support thanks to Tommi Komulainen

shave


Fed up with endless screens of libtool/automake output? Fed up with having to resort to -Werror to see warnings in your code? Then shave might be for you. shave transforms the messy output of autotools into a pretty Kbuild-like one (Kbuild is the Linux build system). It's composed of a m4 macro and 2 small shell scripts and it's available in a git repository.
git clone git://git.lespiau.name/shave
Hopefully, in a few minutes, you should be able to see your project compile like this:
$ make
Making all in foo
Making all in internal
CC    internal-file0.o
LINK  libinternal.la
CC    lib-file0.o
CC    lib-file1.o
LINK  libfoo.la
Making all in tools
CC    tool0-tool0.o
LINK  tool0
Just like Kbuild, shave supports outputting the underlying commands using:
$ make V=1

Setup


  • Put the two shell scripts shave.in and shave-libtool.in in the directory of your choice (it can be at the root of your autotooled project).
  • add shave and shave-libtool to AC_CONFIG_FILES
  • add shave.m4 either in acinclude.m4 or your macro directory
  • add a call to SHAVE_INIT just before AC_CONFIG_FILES/AC_OUTPUT. SHAVE_INIT takes one argument, the directory where shave and shave-libtool are.

Custom rules


Sometimes you have custom Makefile rules, e.g. to generate a small header, run glib-mkenums or glib-genmarshal. It would be nice to output a pretty 'GEN' line. That's quite easy actually, just add few (portable!) lines at the top of your Makefile.am:
V         = @
Q         = $(V:1=)
QUIET_GEN = $(Q:@=@echo '  GEN   '$@;)
and then it's just a matter of prepending $(QUIET_GEN) to the rule creating the file:
lib-file2.h: Makefile
$(QUIET_GEN)echo "#define FOO_DEFINE 0xbabe" > lib-file2.h

gtk-doc + shave


gtk-doc + shave + libtool 1.x (2.x is fine) is known to have a small issue, a patch is available. Meanwhile I suggest adding a few lines to your autogen.sh script.
sed -e 's#) --mode=compile#) --tag=CC --mode=compile#' gtk-doc.make > gtk-doc.temp \
&& mv gtk-doc.temp gtk-doc.make
sed -e 's#) --mode=link#) --tag=CC --mode=link#' gtk-doc.make > gtk-doc.temp \
&& mv gtk-doc.temp gtk-doc.make

dolt + shave


It's possible to use dolt in conjunction with shave with a surprisingly small patch to dolt.

Real world example: Clutter


$ make
GEN   stamp-clutter-marshal.h
GEN   clutter-marshal.c
GEN   stamp-clutter-enum-types.h
Making all in cogl
Making all in common
CC    cogl-util.o
CC    cogl-bitmap.o
CC    cogl-bitmap-fallback.o
CC    cogl-primitives.o
CC    cogl-bitmap-pixbuf.o
CC    cogl-clip-stack.o
CC    cogl-fixed.o
CC    cogl-color.o
cogl-color.c: In function ‘cogl_set_source_color4ub’:
cogl-color.c:141: warning: implicit declaration of function ‘cogl_set_source_color’
CC    cogl-vertex-buffer.o
CC    cogl-matrix.o
CC    cogl-material.o
LINK  libclutter-cogl-common.la
[...]

Eh! now we can see a warning there!

TODO


This is a first release, shave has not been widely tested aka it may not work for you!
  • test it with a wider range of automake/libtool versions
  • shave won't work without AC_CONFIG_HEADERS due to shell quoting problems
  • see what can be done for make install/dist (they are prettier thanks to make -s, but we probably miss a few actions)
  • there is a '-s' hardcoded in MAKEFLAGS,  I have to find a way to make it more flexible

Comments

  1. [...] to Damien, I finally found a way to shut up libtool and get a clean output for the build of a project: [...]

    ReplyDelete
  2. Shame on you. You are setting people up for hours or days of misery when they cannot debug their automated builds.

    ReplyDelete
  3. Unfortunately 'make clean' is not yet actually clean with this. (Oh irony!)

    ReplyDelete
  4. Hmm the problem is this all doesn't really work on libtool 2.2.

    ReplyDelete
  5. Also, it gets confused by libtool convenience libraries.

    ReplyDelete
  6. And preloaded modules don't work either.

    ReplyDelete
  7. You're doing it wrong, this should be added to automake itself so all projects benefit from it instead of just select ones using shave.

    Please make it dependent on a configuration variable in ~/.automake and default it to off.

    Automatic builds should always show all commands and info.

    BTW, you must have missed this:

    http://kim.tensta.gannert.se/projects/pretty-am/

    Please contact automake upstream about integrating this finally.

    ReplyDelete
  8. I forgott to mention that when warnings/errors occur you *NEED* to output the command too.

    ReplyDelete
  9. Hum, first I just want to say the script is 3 days old. DON'T expect it to work. I know people tend to skip this kind of warnings but I had to try:p Hopefully early adopters will report bugs (or fix them!).

    @Zack, foo: I think you have a point here. You *obviously* need a way to get a verbose build when you get errors and/or when packaging. Now please consider:
    * a workaround could be to export V=1 in your automated build environment; the hardcoded 'make -s' is neither portable nor a good solution, I'm fully aware of that.
    * a distributed package can just disable shave, that's up the to the maintainer,
    * the default behaviour could be set to verbose, and then --enable-shave given to configure would turn shave on.
    This still needs a bit of thinking though.

    @blah: I've tested it with libtool 2.2 and it worked for me (at least at some point), could you point me to the project you are trying to "shave"? (the fix to the unmangling regexp I just pushed could help)

    @foo: I agree with you, the real fix would be to patch automake. However patching automake on your system is just out of the question. You can't require people to patch their own autotools when they want to autoreconf your project. This has to be mainlined. The fact that pretty-am has been around for 2 years and a half tends to prove it's not as easy as it sounds.
    That said the clean/install/... targets have an hardcoded 'echo' and handling those cases requires to patch automake. I might try to revive pretty-am, that's not a promise!

    As a side note, I do know that this problem has been around for years, that people have spend countless hours discussing it and that shave won't solve the problem.

    ReplyDelete
  10. zack's environment and use case is completely flawed (a weird build environment, with weird and proprietary tools).

    the obvious fact that everyone is trying to shut automake and libtool up is that the line noise 9 times out of 10 detracts from the ability to catch an error in the build and in the code. it's as simple as that. solutions to the problem, until the autotools maintainers start integrating stuff like this is using common tools that do not require patching the hell out of automake and/or libtool. once everyone is using the same additions to the build system, with the same ways to restore verbosity level when needed, then the autotools can ship with a centralized solution — possibly using the same syntax.

    that's why shave is a good idea: same syntax, everywhere, to turn on (or off) verbosity, until automake 1.12 is out and everyone has transitioned to it.

    ReplyDelete
  11. I also think shave is a great idea! Thanks for working on this..

    ReplyDelete
  12. Shave looks awesome. That too, gotten without forcing the end user to recompile their Automake. I do have a possilby stupid question (being the autotools n00b I am), how would I apply this to a gnome-autogen-ed project like Anjuta which doesn't have a configure.ac (only a configure.in)?

    ReplyDelete
  13. @Arun Chaganty

    configure.in is just the old name of configure.ac, see this page of the autoconf manual.

    ReplyDelete
  14. Works fine for me out of the box, with libtool 2.2.6.

    I even have a couple of custom build rules and it took me about 5 minutes to shut these bastards up too.

    I think this is brilliant. Thanks.

    ReplyDelete
  15. Automake 1.11 has been release with "silent rules" support, a feature that
    supersedes the hack that shave is. If you can depend on automake 1.11 please
    consider using its silent rules rather than shave.

    ReplyDelete
  16. I used shave for some time now and it worked well. It has a blocking issue with GObject introspection though: my build was failing because trying to execute "'/bin/sh", so I switched to silent-rules.

    Not a big deal... just to let you know it and thank you.

    ReplyDelete

Post a Comment

Popular posts from this blog

Building and using coverage-instrumented programs with Go

tl;dr We can create coverage-instrumented binaries, run them and aggregate the coverage data from running both the program and the unit tests.

In the Go world, unit testing is tightly integrated with the go tool chain. Write some unit tests, run go test and tell anyone that will listen that you really hope to never have to deal with a build system for the rest of your life.

Since Go 1.2 (Dec. 2013), go test has supported test coverage analysis: with the ‑cover option it will tell you how much of the code is being exercised by the unit tests.

So far, so good.

I've been wanting to do something slightly different for some time though. Imagine you have a command line tool. I'd like to be able to run that tool with different options and inputs, check that everything is OK (using something like bats) and gather coverage data from those runs. Even better, wouldn't be neat to merge the coverage from the unit tests with the one from those program runs and have an aggregated view of …

Augmenting mailing-lists with Patchwork - Another try

The mailing-list problem
Many software projects use mailing-lists, which usually means mailman, not only for discussions around that project, but also for code contributions. A lot of open source projects work that way, including the one I interact with the most, the Linux kernel. A contributor sends patches to a mailing list, these days using git send-email, and waits for feedback or for his/her patches to be picked up for inclusion if fortunate enough.

Problem is, mailing-lists are awful for code contribution.

A few of the issues at hand:
Dealing with patches and emails can be daunting for new contributors,There's no feedback that someone will look into the patch at some point,There's no tracking of which patch has been processed (eg. included into the tree). A shocking number of patches are just dropped as a direct consequence,There's no way to add metadata to a submission. For instance, we can't assign a reviewer from a pool of people working on the project. As a re…

Testing for pending migrations in Django

DB migration support has been added in Django 1.7+, superseding South. More specifically, it's possible to automatically generate migrations steps when one or more changes in the application models are detected. Definitely a nice feature!

I've written a small generic unit-test that one should be able to drop into the tests directory of any Django project and that checks there's no pending migrations, ie. if the models are correctly in sync with the migrations declared in the application. Handy to check nobody has forgotten to git add the migration file or that an innocent looking change in models.py doesn't need a migration step generated. Enjoy!

See the code on djangosnippets or as a github gist!