If a project is not initialized via make pkg-init-%, it doesn't contain a VERSION file. When CI tries to build and package such a project, it auto-creates a VERSION file, but a broken one: The revision isn't properly seperated by a dash but by a dot, which makes CI give up while parsing it.
Add a new target git-show-ahead-of-master, which does what git-show-pushable did up to now: List all repos where the currently checked-out branch is ahead of origin/master.
git-show-pushable now shows repos that have their checked-out branch ahead of their respective upstream branches. Which is truer to what the target's name suggests.
Rename command "distro" to "pkg" together with "info", its last remaining subcommand. "distro" is often used in the sense of "Linux distribution", which would be too narrow for the targets jw-pkg could theoretically support.
With the exception of the "info" subcommand, nearly all of distro's subcommands deal with package managing, so push them into their own command category.
pgit.sh get is not invoked with --create-remote-user-repos by default. For the janware Git servers, this makes CI fail over new repos: Repos from the maintainers are pulled, but before pkg-rebuild-reinstall, repos that the devops user has no remote repo for are purged again and are subseqently missing during build.
Passing --create-remote-user-repos to all pgit.sh get invocations should fix the problem, so do that.
CmdCanonicalizeRemotes / canonicalize-remotes and the respective target in topdir.mk remove the /srv/git portion from all remotes' URLs pointing to git.janware.com.
/usr/bin/file <candidate> | grep text is used to detect if a file is a text file or not. Replace that with grep -I., because that adds some files left out by /usr/bin/file, notably systemd service files.
Rename git-show-pushable-master-branches to git-show-pushable, because, the target's recipe doesn't show pushable master branches, but all commits in the working directories current branch not present in origin/master.
Don't use persistent SSH-connections any more for "pkg-" targets,
because, sadly, this hangs after uploading a package.
This commit puts some safeguards against hanging SSH into place, namely setting default SSH timeouts down, SSH keepalive, setting SSH BatchMode to yes.
Use DEP_PROJECTS instead of BUILD_PROJECTS everywhere. DEP_PROJECTS considers more projects, and for all targets it is relevant to, it's desirable to have them operate on the maximum blast radius.
make git-show-pushable-master-branches misses projects which should better be pulled in as dependencies. It searches $(BUILLD_PROJECTS) for repositories with pushable commits, that variable leaves some out, and DEP_PROJECTS has them. It make use of the renovated pkg-requires command.
This move should be extended to other uses of BUILD_PROJECTS as well, after giving it some test runs. And the legacy and redundant command prereq should be removed.
Log to stderr and add some ASCII-art around the output. Also, add a --porcelain option to allow more stable output parsing. Subsequently, use that option in make targets parsing the output, notably make diff and make git-show-xxx.
Move the PY_XXX = true|false variable definitions meant to be preset by including makefiles to the top of py-defs.mk to make the structure of the file clearer.
Add PY_INSTALL_INIT_PY ?= true to py-defs.mk. If set to false, a Python module will not try to attempt installing an existing / generated __init__.py. This is useful when installing into an exiting directory with an existing __init__.py.
jw-pkg supports more than RPM-based package managers, but for historic reasons, lots of its Makefile variables still have "RPM" in their names. This is misleading. Replace "RPM" in variable names by the more generic "PKG" where appropriate.
RPM_REQUIRES_DEVEL is often filled from the current version, which in turn is filled from the version file, so the order of events here is unclear at best.
Add target pkg-release-update-version and make pkg-release-reinstall depend on it to make the order explicit.
Make target pkg-release-reinstall depend on target get-official. It already depends on get-maintainer, but that's not enough in situations where devops built a target on platform A, pushed the new release, then proceeds to build on platform B: It needs to pull its own changes made during release of A.
Use pkg-requires --hide-self to find all prerequisites that should be installed for a test run against packages installed from the repositories, including self-built and self-hosted packages.
In a push to eventually merge the classes, somewhat align the command-line API of CmdRequiredOsPkg to the one of BaseCmdPkgRelations by using dependency flavours as mandatory, first argument.
Move the dependencies listed in BASE_PKGS from projcts-dir.mk and topdir.mk into project.conf.
Due to various hen-and-egg problems on a minimal system, in some situations these packages can't be installed from project.conf. The same is true with BASE_PKGS, however, so remove it, at least that does away with some redundancy.
Add support for --syntax to BaseCmdPkgRelations.pkg_relations(), and default to 'semver', i.e. the current state of affairs. If that's changed to 'debian', relations declared in project.conf as
Don't prefix JW_PKG_PY_PROJECTS with time -p. A timing summary shows up in too many places unexpectedly, e.g. in the context of the target update-text-files. Add back later as more concrete demand comes up.
Make pkg-install-testbuild-deps an alias for pkg-install-release-deps. At this point, they do nearly the same thing, and the distinction between what the implementations should do are blurry at best. This commit removes redundancy but keeps the use cases distinct. Different implementations can be reinstantiated should requirements for different implementations become clearer later on.
Remove the sections pkg.requires.ubuntu|raspbian from project.conf, because their contents is present in pkg.requires.debian and is already evaluated by Ubuntu and Raspbian builds.
Remove dependency inkscape: It's needed by svg-to-pixmap.mk, but not for building jw-pkg. If anything, -devel should depend on it, but that seems a little heavy handed and can be achieved by packages which know that they include svg-to-pixmap.mk.
Add dependency cpio: Needed by pkg.sh to copy a the source file tree. Will be removed again as soon as pkg.sh goes Python.
The code below lib.distro, as left behind by the previous commit, is geared towards being directly used as a command-line API. This commit introduces the abstract base class Distro, a proxy for distribution-specific interactions. The proxy abstracts distro specifics into an API with proper method prototypes, not argparse.Namespace contents, and can thus be more easily driven by arbitrary code.
The Distro class is initialized with a member variable of type ExecContext, another new class introduced by this commit. It is designed to abstract the communication channel to the distribution instance. Currently only one specialization exists, Local, which interacts with the distribution and root file system it is running in, but is planned to be subclassed to support interaction via SSH, serial, chroot, or chains thereof.
PKG_FORMAT is now more straightforward to get with from the revised jw-pkg.py distro info --format '%{cascade}', so do that, and do it in the context where all the other variables are set from the output of that command.
To remove redundancy, get-os.sh needs to be retired in favor of pkg.py distro info. It's needed in platform.mk, but the only definiton of JW_PKG_PY is in projects.mk, so move it, along with the variables essential for the command:
include $(JWBDIR)/make/py-version.mk (defining PYTHON) JW_PKG_PY DEVELOPMENT VERSION_FILE
