jw-pkg/README.md

# JW-Build

## Features

The JW-Build package is basically a bunch of scripts and GNU makefile snippets
for organizing software builds and releases.

You can install it to its standard location `/opt/jw-build`, or put it anywhere
you want in your file system, typically next to your own projects in the same
directory. To organize their respective builds, you can then include JW-Build's
makefile snippets from your own projects' makefiles, like so:

    include $(JWBDIR)/make/cpp.mk

where `JWBDIR` needs to point to JW-Build's installation directory. In this
example, the snippet `cpp.mk` would by default take all C++ files it finds in
the directory from where its included, compile them, and and add them to a
central shared library. It would also take all header files and copy them to a
central include directory. `js.mk` would by default minify all JavaSript it
finds, `java.mk` jar up .java files into classes and jar-files, and so on.
JW-Build also handles installation and packaging of all of these files, to
customizable locations with standardish defaults.

JW-Build is small. It's small enough to be self-documenting. Well, okay,
somewhat self-documenting. It better be, given its lack of documentation. You
have to know GNU Makefile syntax to understand what it does, and dig into its
somtimes arcane code, ideally with a working example. You can install it with
your distribution's package manager, or you can keep it within your code
versioning system, alongside your own code. It's also designed to be the
lightest possible touch on any given source code package, in terms of code you
need to add to a package you want to build with it, and also in terms of needed
prerequisite software packages. This way, it's easily introduced - and it's
also easy to get rid of, should you choose to do so at some point in time.  You
will then have all your settings like file system path definitions and compiler
flags in well-defined places already.

JW-Build runs a recursive make, so, with a few exceptions such as submodules,
you will need a makefile in every directory with source code. Most, if not all
of these makefiles can be three-liners. This buys you convenience, minimal,
clean, humanly digestable code, and a structured way to customize builds
involving multiple packages - as centrally or as locally as you want. You can
override any of JW-Build's default variables - for many packages, for an entire
package, or for any subdirectory of a given package, at your option. You can
write your own snippets and reuse them in multiple places. You can keep
overrides in your versioning system or add them where needed in
`local.mk`-files, which only your machine knows about. Or you can use
environment variables, of course. JW-Build intentionally avoids makefile code
generation as seen with CMake or GNU Autotools. This keeps the code small and
readable for easy debugging. Okay, for relatively easy debugging. To achieve
this, JW-Build has to detect a couple of things in every directory it enters,
but it uses various caching mechanisms to keep builds still reasonably fast.

JW-Build has makefile snippets for building libraries and executables, snippets
that output code compiled from C/C++, Python, Java, JavaScript, LaTeX, SWIG and
other input formats, and it's easily extendible to support any given
programming language or task. It's in use at janware for managing sub-builds of
Maven, Ant, CMake and others, and for packaging the results. It provides
targets to flash binaries onto microcontrollers, produce Debian, RPM and IPK
packages, install them locally or remotely, or feed them into DevOps pipelines,
taking note of released versions within GIT, SVN or CVS. It detects whether or
not a package needs a new release because its source code changed. Or because a
package it depends on has changed incompatibly. JW-Build has targets for
collaboration over structured sets of remote Git repositories. It supports a
simple configuration file per package for specifying package metadata, e.g. its
dependencies, license, description, pre- and postinstall scriptlets, and so on.
It has a SAT-solver built in, for building multiple packages in the right
order, based on that metadata. With the same metadata, it can also
automatically generate BitBake recipes and run Yocto-builds incorporating your
software. Packages which are not organized with JW-Build can also be integrated
into the build. It generates runtime, development and source code package
variants. It supports cross compilation with MinGW and the GNU toolchain. It's
tested on Debian, Ubuntu, OpenSUSE, Fedora / CentOS / RHEL, and many Unices.

JW-Build is designed to be friendly to both developers and integrators.
Developers can cd into any given directory, edit the source code, run `make`,
and expect the resulting binaries to work and be immediately testable, for a
workflow that lets you focus on coding in your target language. For integrators
on the other hand, a hotfix on a server or an embedded host can be as simple as

    TARGET_HOST=myserver.acme.com make pkg-remote-install

And of course, it can build, package and release itself. Without being
installed, which is a Good Thing (TM).

## Basic Usage

### For janware builds
See https://janware.com/wiki/pub/sw/build/ for documentation on how to get
and use JW-Build within the janware build infrastructure.

### Standalone
If you want to use it standalone, OTOH, do the following to get a minimal
working example:

First, add a `make` subdirectory to the toplevel directory of your package,
containing two files:

1. `proj.mk`, containing a definition of `JWBDIR`, pointing to the JW-Build
  installation directory, e.g. like so:

        JWBDIR ?= $(firstword $(wildcard $(addsuffix /jw-build,$(TOPDIR)/.. /opt)))

   `TOPDIR` points to, you guessed it, the toplevel directory of your package.
   You will most likely define it yourself in every makefile that uses
   `proj.mk`, see below. The right-hand side of the equation is GNU Make
   gibberish for: *"Look for a directory named `jw-build` next to my package,
   and then below `/opt`, in that order."* Assuming that's where the packages
   are in your file system.

     Things to note:
     - All directory paths can be relative, which is nice if you want to
       organize multiple packages in a fixed tree layout, but want them to work
       wherever you place the tree.
     - It doesn't have to be `$(TOPDIR)/make`. You can choose any other
       subdirectory of your package, but for now let's just assume it's
       `$(TOPDIR)/make`, the default location.

2. `project.conf`, containing

        [description]
        A frobnicator library

Then, add files named `Makefile` to the directories of your project,
containing, e.g., in a C++ directory:

    TOPDIR = ../..

    include $(TOPDIR)/make/proj.mk
    include $(JWBDIR)/make/cpp.mk

Done. Well, in principle. Other notable snippets are `topdir.mk` for the
toplevel directory, `dirs.mk` for other directories with subdirectories,
`lib.mk` for the directory containing the package's main library, which
defaults to `$(TOPDIR)/lib`, `bin.mk` for `$(TOPDIR)/bin`, `include.mk` for
`$(TOPDIR)/include`, and `make.mk` for `$(TOPDIR)/make`. You should add them in
the same manner. Again, these are just default locations. Once you've added
those makefiles, running `make` will do - something. Try and see what happens.
Every snippet supports at least the targets `all`, `install`, `clean` and
`distclean`. The target `echo-makefiles` shows you all included snippets,
`cat-makefiles` concatenates them. Hitting TAB should show you all targets
supported in a particular directory.

Good luck!

## Advanced Usage, Custom Makefiles, Tips and Tricks, Things to Note

- JW-Build will use any `Makefile`, `makefile` or `GNUmakefile` it finds in the
  toplevel directory of your package. It will `cd` into the toplevel directory
  and run `make` during build runs, and subsequently `make install` during
  installation runs. It will also run `make clean` if invoked from outside.
  `make distclean` doesn't need to be supported, but comes in handy, if you
  want your packages to be wiped extra-clean.

  JW-Build will not run `make install` as the `root` user, and will therefore
  expect you to use `$(INSTALL)` instead of `/usr/bin/install`, to which it is
  command line compatible.
  [pseudo](http://git.yoctoproject.org/cgit/cgit.cgi/pseudo) support is
  underway and will allow to use plain `/usr/bin/install`, but is not yet
  implemented.

- If you want GNU Make to do what it's designed for for your custom task, i.e.
  do the right thing based on file modification times, do the following:

        TOPDIR                 = ../..
      
        MY_CREATION_INPUT      = $(wildcard *.tmpl)
        MY_CREATION            = $(patsubst %.tmpl,%,$(MY_CREATION_INPUT))
        MY_INSTALL_DIR         = $(ENV_PREFIX)/var/lib/my-creation
        MY_INSTALLED_CREATION  = $(addprefix $(MY_INSTALL_DIR)/,$(MY_CREATION))
      
        include $(TOPDIR)/make/proj.mk
        include $(JWBDIR)/make/defs.mk
      
        all: $(MY_CREATION)
        install: $(MY_INSTALLED_CREATION)
        clean: clean.my-creation
        distclean: clean
      
        $(MY_CREATION): $(MY_CREATION_INPUT)
                bash ./my-special-stuff-creator.sh $< > $@.tmp
                mv $@.tmp $@
        clean.my-creation:
                rm -f $(MY_CREATION) *.tmp
        $(MY_INSTALL_DIR):
	        $(INSTALL) -o root -g root -m 755 -d $@
        $(MY_INSTALL_DIR)/%: % | $(MY_INSTALL_DIR)
                $(INSTALL) -o root -g root -m 644 $< $@

  Things to note:
    - If the example seems too verbose for you, consider putting the code into
      `$(TOPDIR)/make/my-creation.mk` (for example) and `include` that.
    - `defs.mk` doesn't provide any targets, but only defines central variables.
      It's implicitly included with most snippets, but if you want to define your
      own tasks, you can use it directly.
    - The `MY_XXX` variables have arbitrary names and are not evaluated by
      JW-Build.
    - `$(ENV_PREFIX)` will automatically point to something below your home
      directory if invoked from a package which has `.git`, `.svn` or `CVS` in its
      toplevel directory, and to the proper target root during package creation.
    - Using the intermediate `$@.tmp` is a good idea in case
      `my-special-stuff-creator.sh` can fail.
    - See `info make` for what the `|` pipe - symbol is for.

- If you want to add arbitrary tasks to the build run, consider the following
  makefile:
  ```
  TOPDIR = ../..

  include $(TOPDIR)/make/proj.mk
  include $(JWBDIR)/make/defs.mk

  all: my-task.done
  clean: clean.my-task

  my-task.done:
          cd $(SUBMOD_SRC_DIR) && mvn -DskipTests --settings ../maven-settings.xml
          touch $@
  clean.my-task:
          rm -rf $(addprefix $(SUBMOD_SRC_DIR)/,dependency-reduced-pom.xml externs/target target) $(SOURCE_BASE)
          rm -f *.done
  ```
[logo]: https://janware.com/janware/images/logo-janware/logo-janware-200.png