mirror of
ssh://git.janware.com/srv/git/janware/proj/jw-pkg
synced 2026-01-15 03:53:32 +01:00
225 lines
11 KiB
Markdown
225 lines
11 KiB
Markdown
# JW-Build
|
|
|
|
## Features
|
|
|
|
The JW-Build package is basically a bunch of scripts and GNU makefile snippets
|
|
for organizing software builds and releases.
|
|
|
|
You can install it to its standard location `/opt/jw-build`, or put it anywhere
|
|
you want in your file system, typically next to your own projects in the same
|
|
directory. To organize their respective builds, you can then include JW-Build's
|
|
makefile snippets from your own projects' makefiles, like so:
|
|
|
|
include $(JWBDIR)/make/cpp.mk
|
|
|
|
where `JWBDIR` needs to point to JW-Build's installation directory. In this
|
|
example, the snippet `cpp.mk` would by default take all C++ files it finds in
|
|
the directory from where its included, compile them, and and add them to a
|
|
shared project library. It would also take all header files and copy them to a
|
|
project include directory. `js.mk` would by default minify all JavaSript it
|
|
finds, `java.mk` jar up .java files into classes and jar-files, and so on.
|
|
JW-Build also handles installation and packaging of these files, to
|
|
customizable locations with standardish defaults.
|
|
|
|
JW-Build is small, its tarball is about 200K. It's small enough to be shipped
|
|
with the source code of your project, if you choose to do so. And it's small
|
|
enough to be self-documenting. Well, okay, somewhat self-documenting. You have
|
|
to know GNU Makefile syntax to understand what it does, and dig into its
|
|
somtimes arcane code, ideally with a working example. You can install it with
|
|
your distribution's package manager, or you can keep it within your code
|
|
versioning system, alongside your own code. It's also designed to be the
|
|
lightest possible touch on any given source code package, in terms of code you
|
|
need to add to a given package that should be built with it, and also in terms
|
|
of software packages needed to be installed on your machine. This way, it's
|
|
easily added - and it's also easy to replace, should you choose to do so at
|
|
some point. You will then have all your settings like file system path
|
|
definitions and compiler flags in well-defined places already.
|
|
|
|
JW-Build runs a recursive make, so, with a few exceptions such as submodules,
|
|
you will need a makefile in every directory with source code. Most, if not all
|
|
of these makefiles can be three-liners. This buys you convenience, minimal,
|
|
clean, humanly digestable code, and a structured way to customize builds
|
|
involving multiple packages - as centrally or as locally as you want. You can
|
|
override any of JW-Build's default variables - for many packages, for an entire
|
|
package, or for any subdirectory of a given package, at your option. You can
|
|
write your own snippets and reuse them in multiple places. You can keep
|
|
overrides in your versioning system or add them where needed in
|
|
`local.mk`-files, which only your machine knows about. Or you can use
|
|
environment variables, of course. JW-Build intentionally avoids makefile code
|
|
generation as seen with CMake or GNU Autotools. This keeps the code small and
|
|
readable for easy debugging. Okay, for relatively easy debugging. To achieve
|
|
this, JW-Build has to detect a couple of things in every directory it enters,
|
|
but it uses various caching mechanisms to keep builds still reasonably fast.
|
|
|
|
JW-Build has makefile snippets for building libraries and executables, snippets
|
|
that output code compiled from C/C++, Python, Java, JavaScript, LaTeX, SWIG and
|
|
other input formats, and it's easily extendible to support any given
|
|
programming language or task. It's in use at janware for managing sub-builds of
|
|
Maven, Ant, CMake and others, and for packaging the results. It provides
|
|
targets to flash binaries onto microcontrollers, produce Debian, RPM and IPK
|
|
packages, install them locally or remotely, or feed them into DevOps pipelines,
|
|
taking note of released versions within GIT, SVN or CVS. It detects whether or
|
|
not a package needs a new release because its source code changed. Or because a
|
|
package it depends on has changed incompatibly. JW-Build has targets for
|
|
collaboration over structured sets of remote Git repositories. It supports a
|
|
simple configuration file per package for specifying package metadata, e.g. its
|
|
dependencies, license, description, pre- and postinstall scriptlets, and so on.
|
|
It has a SAT-solver built in, for building multiple packages in the right
|
|
order, based on that metadata. With the same metadata, it can also
|
|
automatically generate BitBake recipes and run Yocto-builds incorporating your
|
|
software. Packages which are not organized with JW-Build can also be integrated
|
|
into the build. It generates runtime, development and source code package
|
|
variants. It supports cross compilation with MinGW and the GNU toolchain. It's
|
|
tested on Debian, Ubuntu, OpenSUSE, Fedora / CentOS / RHEL, and many Unices.
|
|
|
|
JW-Build is designed to be friendly to both developers and integrators.
|
|
Developers can cd into any given directory, edit the source code, run `make`,
|
|
and expect the resulting binaries to work and be immediately testable, for a
|
|
workflow that lets you focus on coding in your target language. For integrators
|
|
on the other hand, a hotfix on a server or an embedded host can be as simple as
|
|
|
|
TARGET_HOST=myserver.acme.com make pkg-remote-install
|
|
|
|
And of course, it can build, package and release itself. Without being
|
|
installed, which is a Good Thing (TM).
|
|
|
|
## Basic Usage
|
|
|
|
### For janware builds
|
|
See https://janware.com/wiki/pub/sw/build/ for documentation on how to get
|
|
and use JW-Build within the janware build infrastructure.
|
|
|
|
### Standalone
|
|
If you want to use it standalone, OTOH, do the following to get a minimal
|
|
working example:
|
|
|
|
First, add a `make` subdirectory to the toplevel directory of your package,
|
|
containing two files:
|
|
|
|
1. `proj.mk`, containing a definition of `JWBDIR`, pointing to the JW-Build
|
|
installation directory, e.g. like so:
|
|
|
|
JWBDIR ?= $(firstword $(wildcard $(addsuffix /jw-build,$(TOPDIR)/.. /opt)))
|
|
|
|
`TOPDIR` points to, you guessed it, the toplevel directory of your package.
|
|
You will most likely define it yourself in every makefile that uses
|
|
`proj.mk`, see below. The right-hand side of the equation is GNU Make
|
|
gibberish for: *"Look for a directory named `jw-build` next to my package,
|
|
and then below `/opt`, in that order."* Assuming that's where the packages
|
|
are in your file system.
|
|
|
|
Things to note:
|
|
- All directory paths can be relative, which is nice if you want to
|
|
organize multiple packages in a fixed tree layout, but want them to work
|
|
wherever you place the tree.
|
|
- It doesn't have to be `$(TOPDIR)/make`. You can choose any other
|
|
subdirectory of your package, but for now let's just assume it's
|
|
`$(TOPDIR)/make`, the default location.
|
|
|
|
2. `project.conf`, containing
|
|
|
|
[description]
|
|
A frobnicator library
|
|
|
|
Then, add files named `Makefile` to the directories of your project,
|
|
containing, e.g., in a C++ directory:
|
|
|
|
TOPDIR = ../..
|
|
|
|
include $(TOPDIR)/make/proj.mk
|
|
include $(JWBDIR)/make/cpp.mk
|
|
|
|
Done. Well, in principle. Other notable snippets are `topdir.mk` for the
|
|
toplevel directory, `dirs.mk` for other directories with subdirectories,
|
|
`lib.mk` for the directory containing the package's main library, which
|
|
defaults to `$(TOPDIR)/lib`, `bin.mk` for `$(TOPDIR)/bin`, `include.mk` for
|
|
`$(TOPDIR)/include`, and `make.mk` for `$(TOPDIR)/make`. You should add them in
|
|
the same manner. Again, these are just default locations. Once you've added
|
|
those makefiles, running `make` will do - something. Try and see what happens.
|
|
Every snippet supports at least the targets `all`, `install`, `clean` and
|
|
`distclean`. The target `echo-makefiles` shows you all included snippets,
|
|
`cat-makefiles` concatenates them. Hitting TAB should show you all targets
|
|
supported in a particular directory.
|
|
|
|
Good luck!
|
|
|
|
## Advanced Usage, Custom Makefiles, Tips and Tricks, Things to Note
|
|
|
|
- JW-Build will use any `Makefile`, `makefile` or `GNUmakefile` it finds in the
|
|
toplevel directory of your package. It will `cd` into the toplevel directory
|
|
and run `make` during build runs, and subsequently `make install` during
|
|
installation runs. It will also run `make clean` if invoked from outside.
|
|
`make distclean` doesn't need to be supported, but comes in handy, if you
|
|
want your packages to be wiped extra-clean.
|
|
|
|
JW-Build will not run `make install` as the `root` user, and will therefore
|
|
expect you to use `$(INSTALL)` instead of `/usr/bin/install`, to which it is
|
|
command line compatible.
|
|
[pseudo](http://git.yoctoproject.org/cgit/cgit.cgi/pseudo) support is
|
|
underway and will allow to use plain `/usr/bin/install`, but is not yet
|
|
implemented.
|
|
|
|
- If you want GNU Make to do what it's designed for for your custom task, i.e.
|
|
do the right thing based on file modification times, do the following, for
|
|
example:
|
|
|
|
TOPDIR = ../..
|
|
|
|
MY_CREATION_INPUT = $(wildcard *.tmpl)
|
|
MY_CREATION = $(patsubst %.tmpl,%,$(MY_CREATION_INPUT))
|
|
MY_INSTALL_DIR = $(ENV_PREFIX)/var/lib/my-creation
|
|
MY_INSTALLED_CREATION = $(addprefix $(MY_INSTALL_DIR)/,$(MY_CREATION))
|
|
|
|
include $(TOPDIR)/make/proj.mk
|
|
include $(JWBDIR)/make/defs.mk
|
|
|
|
all: $(MY_CREATION)
|
|
install: $(MY_INSTALLED_CREATION)
|
|
clean: clean.my-creation
|
|
distclean: clean
|
|
|
|
%: %.tmpl
|
|
bash ./my-special-stuff-creator.sh $< > $@.tmp
|
|
mv $@.tmp $@
|
|
clean.my-creation:
|
|
rm -f $(MY_CREATION) *.tmp
|
|
$(MY_INSTALL_DIR):
|
|
$(INSTALL) -o root -g root -m 755 -d $@
|
|
$(MY_INSTALL_DIR)/%: % | $(MY_INSTALL_DIR)
|
|
$(INSTALL) -o root -g root -m 644 $< $@
|
|
|
|
Things to note:
|
|
- If the example seems too verbose for you, consider putting the code into
|
|
`$(TOPDIR)/make/my-creation.mk` (for example) and `include` that.
|
|
- `defs.mk` doesn't provide any targets, but only defines central variables.
|
|
It's implicitly included with most snippets, but if you want to define your
|
|
own tasks, you can use it directly.
|
|
- The `MY_XXX` variables have arbitrary names and are not evaluated by
|
|
JW-Build.
|
|
- `$(ENV_PREFIX)` will automatically point to something below your home
|
|
directory if invoked from a package which has `.git`, `.svn` or `CVS` in its
|
|
toplevel directory, and to the proper target root during package creation.
|
|
- Using the intermediate `$@.tmp` is a good idea in case
|
|
`my-special-stuff-creator.sh` can fail.
|
|
- See `info make` for how pattern rules work (`%`), what the `|` pipe -
|
|
symbol is for and all that jazz. If you don't know already.
|
|
|
|
- If you want to add arbitrary tasks to the build run, consider the following
|
|
makefile:
|
|
```
|
|
TOPDIR = ../..
|
|
|
|
include $(TOPDIR)/make/proj.mk
|
|
include $(JWBDIR)/make/defs.mk
|
|
|
|
all: my-task.done
|
|
clean: clean.my-task
|
|
|
|
my-task.done:
|
|
cd $(SUBMOD_SRC_DIR) && mvn -DskipTests --settings ../maven-settings.xml
|
|
touch $@
|
|
clean.my-task:
|
|
rm -rf $(addprefix $(SUBMOD_SRC_DIR)/,dependency-reduced-pom.xml externs/target target) $(SOURCE_BASE)
|
|
rm -f *.done
|
|
```
|
|
[logo]: https://janware.com/janware/images/logo-janware/logo-janware-200.png
|