JW-Build

Features

The JW-Build package is basically a bunch of scripts and GNU makefile snippets for organizing software builds and releases.

You can install it to its standard location /opt/jw-build, or put it anywhere you want in your file system, typically next to your own projects in the same directory. To organize their respective builds, you can then include JW-Build's makefile snippets from your own projects' makefiles, like so:

include $(JWBDIR)/make/cpp.mk

where JWBDIR needs to point to JW-Build's installation directory. In this example, the snippet cpp.mk would by default take all C++ files it finds in the directory from where its included, compile them and and add them to a shared library. It would also take all header files and copy them to a central include directory. js.mk would by default minify all JavaSript it finds, java.mk jar up .java files into classes and jar-files, and so on. JW-Build also handles installation and packaging of all of these files, to customizable locations with standardish defaults.

JW-Build is small. It's small enough to be self-documenting. Well, okay, somewhat self-documenting. You have to know GNU Makefile syntax to understand what it does, and dig into its somtimes arcane code, ideally with a working example. You can install it with your distribution's package manager, or you can keep it within your code versioning system, alongside your own code. It's also designed to be the lightest possible touch on any given source code package, in terms of code you need to add to a package you want to build with it, and also in terms of needed prerequisite software packages. This way, it's easily introduced - and it's also easy to get rid of, should you choose to do so at some point in time. You will then have all your settings like file system path definitions and compiler flags in well-defined places already.

JW-Build runs a recursive make, so, with a few exceptions such as submodules, you will need a makefile in every directory with source code. Most, if not all of these makefiles can be three-liners. This buys you convenience, minimal, clean, humanly digestable code, and a structured way to customize builds involving multiple packages - as centrally or as locally as you want. You can override any of JW-Build's default variables - for many packages, for an entire package, or for any subdirectory of a given package, at your option. You can write your own snippets and reuse them in multiple places. You can keep overrides in your versioning system or add them to local.mk-files as needed, which only your machine knows about. Or you can use environment variables, of course. JW-Build avoids makefile code generation as seen with CMake or GNU Autotools. This keeps the code small and readable for easy debugging. Okay, for relatively easy debugging. To achieve this, JW-Build has to detect a couple of things in every directory it enters, but it uses various caching mechanisms to keep builds still reasonably fast.

JW-Build has makefile snippets for building libraries and executables, snippets that output code compiled from C/C++, Python, Java, JavaScript and LaTeX, and it's easily extendible to support any given programming language or task. It's in use at janware for managing sub-builds of Maven, Ant, CMake and others, and for packaging the results. It provides targets to flash binaries onto MCUs, produce Debian, RPM and IPK packages, install them locally or remotely, or feed them into a DevOps pipeline, taking note of released versions within GIT, SVN or CVS. It detects if a package needs a new release because its source code changed. Or because a package it depends on has changed incompatibly. JW-Build has built-in support for collaboration over well-defined sets of remote Git repositories. It supports a simple configuration file per package for specifying package metadata, e.g. its dependencies, license, description, pre- and postinstall scriptlets, and so on. It has a SAT-solver built in, for building multiple packages in the right order, including packages that are not organized with JW-Build, based on that metadata. With the same metadata, it can also automatically generate BitBake recipes and run Yocto-builds incorporating your software. It generates runtime, development and source code package variants. It supports cross compilation with MinGW and the GNU toolchain. It's tested on Debian, Ubuntu, OpenSUSE, Fedora / CentOS / RHEL, and many Unices.

JW-Build is designed to be friendly to both developers and integrators. Developers can cd into any given directory, edit the source code, run make, and expect the resulting binaries to work and be immediately testable, for a workflow that lets you focus on coding in your target language. For integrators on the other hand, a hotfix on a server or an embedded host can be as simple as

TARGET_HOST=myserver.acme.com make pkg-remote-install

And of course, it can build, package and release itself. Without being installed, which is a Good Thing (TM).

Usage

See https://janware.com/wiki/pub/sw/build/ for documentation on how to get and use JW-Build within the janware build tree.

If you want to use it standalone, OTOH, do the following to get a minimal working example:

First, add a make subdirectory to the toplevel directory of your package, containing two files:

1. proj.mk, containing a definition of JWBDIR, pointing to the JW-Build installation directory, e.g. like so:

   JWBDIR ?= $(firstword $(wildcard $(addsuffix /jw-build,$(TOPDIR)/.. /opt)))
   

   TOPDIR points to, you guessed it, the toplevel directory of your package. You will define it yourself in every directory, see below. The right-hand side of the equation is GNU make gibberish for: "Look for a directory named jw-build next to my package, and then below /opt, in that order." Assuming that's how your source code is organized. Note that all directory paths can be relative, which is nice if you want to organize multiple packages in a fixed tree layout, but want them to work wherever you place the tree.

2. project.conf, containing

   [description]
   A frobnicator library
   

Then, add files named Makefile to the directories of your project, containing, e.g., in a C++ directory:

TOPDIR = ../..

include $(TOPDIR)/make/proj.mk
include $(JWBDIR)/make/cpp.mk

Done. Well, in principle. Other notable snippets are topdir.mk for the toplevel directory, dirs.mk for other directories with subdirectories, lib.mk for the directory containing the package's main library, which defaults to $(TOPDIR)/lib, bin.mk for $(TOPDIR)/bin, include.mk for $(TOPDIR)/include, and make.mk for $(TOPDIR)/make. You should add them in the same manner. Once you've added those makefiles, running make will do - something. Try and see what happens. Every snippet supports at least the targets all, install, clean and distclean. The target echo-makefiles shows you all included snippets, cat-makefiles concatenates them. Hitting TAB should show you all targets supported in a particular directory. Good luck!