The code below lib.distro, as left behind by the previous commit, is
geared towards being directly used as a command-line API. This commit
introduces the abstract base class Distro, a proxy for
distribution-specific interactions. The proxy abstracts distro
specifics into an API with proper method prototypes, not
argparse.Namespace contents, and can thus be more easily driven by
arbitrary code.
The Distro class is initialized with a member variable of type
ExecContext, another new class introduced by this commit. It is
designed to abstract the communication channel to the distribution
instance. Currently only one specialization exists, Local, which
interacts with the distribution and root file system it is running
in, but is planned to be subclassed to support interaction via SSH,
serial, chroot, or chains thereof.
Signed-off-by: Jan Lindemann <jan@janware.com>
HOST_TUPLE is bastardized to contain more info that just the GNU
Triplet. Add HOST_DISTRO_INFO to do the caching job, and leave
HOST_TUPLE alone.
Signed-off-by: Jan Lindemann <jan@janware.com>
PKG_FORMAT is now more straightforward to get with from the revised
jw-pkg.py distro info --format '%{cascade}', so do that, and do it in
the context where all the other variables are set from the output of
that command.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove the now obsolete get-os.sh from jw-pkg. Use "jw-pkg.py distro
info" to get the information it used to provide.
Signed-off-by: Jan Lindemann <jan@janware.com>
To remove redundancy, get-os.sh needs to be retired in favor of
pkg.py distro info. It's needed in platform.mk, but the only
definiton of JW_PKG_PY is in projects.mk, so move it, along with the
variables essential for the command:
include $(JWBDIR)/make/py-version.mk (defining PYTHON)
JW_PKG_PY
DEVELOPMENT
VERSION_FILE
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a VCS-independent rule rule get-%, currently an alias for
git-get-%, with an empty recipe to keep GNU Make satisfied.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't use the master branch as target when getting code from
somebody, merge into the branch currently checked out.
Signed-off-by: Jan Lindemann <jan@janware.com>
Currently git-get-% pulls into the master branch. Change that to pull
into the branch currently checked out in the workspace, because
that's the more likely use case if you want a quick update from
somewhere.
Signed-off-by: Jan Lindemann <jan@janware.com>
"clone" in the Git sense means to copy a remote project over from
scratch. pgit.sh clone has come from that, but has since evolved into
something different, a mixture of clone, pull and fetch, so find a
different name. "get" seems generic enough and doesn't clash with a
Git meaning. Adapt variable names accordingly across the project.
Signed-off-by: Jan Lindemann <jan@janware.com>
To be explicit about what's happening during make fetch, default the
fetch refspec to $(CLONE_FROM_USER):current-branch:current-branch.
Signed-off-by: Jan Lindemann <jan@janware.com>
git-pull-<username> doesn't use pgit.sh if username == login.
pgit.sh should handle that case fine now, so remove the distinction
from topdir.mk and make it in one place, i.e. pgit.sh. This has the
additional advantage that pull as done by pgit.sh conveniently uses
--autostash.
Signed-off-by: Jan Lindemann <jan@janware.com>
Purging the environment of pkg-release-reinstall did a tad too much:
Add SSH_AUTH_SOCK back to allow SSH agent-based authentication to go
through.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change commit message of first commit from "initial checkin" to
"First commit", because that's more to the point.
Signed-off-by: Jan Lindemann <jan@janware.com>
sudo is certainly not needed for the run package (which in itself is
hardly useful at all), so move that dependency into the devel
package. Same for gawk. /opt/jw-pkg/bin/get-os.sh depends on it,
but I don't see where else but in a -devel context that would matter.
And if it breaks something, it is going to be an easy fix without
awk.
Signed-off-by: Jan Lindemann <jan@janware.com>
--source-profile=replace should be passed to all
jw-pkg.py projects build pkg-*install
invocations. Those invocations typically happen in the context of
pkg-%install, so add that target, specializing the pkg-% target.
The problem this solves is that /etc/profile is currently read only
once before bootstrapping all software on a pristine system is
started. This might lead to the situation that package A has
installed environment variable definitions into /etc/profile.d,
package B needs them for building, but never gets to read them.
Signed-off-by: Jan Lindemann <jan@janware.com>
To complement git-pull-maintainer with something more generic, also
suitable for other SCMs, add the target pull-maintainer and make
pkg-release-reinstall depend on it. Currently only visible in the
context of pkg-% targets, scope might be expanded if need be.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use pgit.sh to for the git-pull-% target. This should make
git-pull-maintainer work. To limit the blast radius for now, only use
it if the source user differs from the invoking user.
Signed-off-by: Jan Lindemann <jan@janware.com>
Multiple variables are redundantly defined both for a project and for
the multiple-projects toplevel directory. Add a place to maintain
them centrally, and add PGIT_SH as a first variable.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add an init target. Use it if you want to tell the Makefile: _Just_
initalize the build machinery and nothing else, don't pull and build
everything else you can. Not strictly necessary, most of the time
pulling everything is what's wanted, and that does the init anyway.
Signed-off-by: Jan Lindemann <jan@janware.com>
The target pkg-delete-ours, invoked from the projects directory,
should wipe all packages from the system which have been created and
installed via jw-pkg.
Currently they are selected via url =~ janware. This is a default
string which can be overridden by redefining JANWARE_PACKAGE_FILTER.
This might not be the most generic name, but is kind of consistent
and will be matched once all variables get renamed to a more generic
naming scheme.
This currently does not get all packages: Some are not labeled with
URLs matching "janware", because jw-pkg is only used as a convenient
way to package other people's open source projects.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a package dependency on make for the -devel package. Installing
jw-pkg's Makefiles doesn't make much sense without it.
Signed-off-by: Jan Lindemann <jan@janware.com>
If VERSION_FILE is not found, trying to include makefile snippets
triggers the attempt to regenerate it. This happened for installed
Makefiles of jw-docker-images: It defines TOPDIR to
/opt/jw-docker-images, projects.mk looks there, but the version file
is installed under /usr/share/doc/packages/jw-docker-images/VERSION.
Allow VERSION_FILE to be overridden including code to keep that from
happening.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change package group from System/Libraries to
Development/Tools/Building because that expresses the purpose of the
package better.
Signed-off-by: Jan Lindemann <jan@janware.com>
make complains for plugins that soandso.dll has not been remade. The
problem is that it tries to remake all targets of a multi-target.
Split that up into two rules.
Signed-off-by: Jan Lindemann <jan@janware.com>
.cache-projects.mk is not installed / packaged, which makes builds
against an installed jw-pkg considerably slower. Change that, at the
risk of making the installed jw-pkg-devel less versatile. This commit
installs a cache file cache-projects.mk, renamed from
.cache-projects.mk, because there's no justification for hiding an
installed makefile. At least I can't think of one.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's pkg-manager-refresh already, so by adding pkg-manager-dup the
distribution can be upgraded by distribution agnostic targets only
through the Makefile. This might come in handy for CI, so add it.
Signed-off-by: Jan Lindemann <jan@janware.com>
--quote puts double quotation marks around the listed dependencies,
protecting version requirements (>= 1.0) and parenthesis "perl(GD)"
from the shell.
Signed-off-by: Jan Lindemann <jan@janware.com>
Retire pkg-manager.sh and replace it by the cleaner "jw-pkg.sh
distro" command, essentially providing the same functionality and
nearly the same command-line interface.
Not-so-fun-fact:
jw-pkg > git diff --stat jw-devops/master
...
71 files changed, 732 insertions(+), 340 deletions(-)
400 LOC more. That's what the move from a shell script to the more
maintainable Python versions costs. Still a good idea, and the
enhanced extensibility might pay off in terms of LOC with other shell
scripts in the future.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-projects.py is now a multi-call executable, with "projects" being
just one of its subcommands. Rename it to jw-pkg.py to reflect that.
Signed-off-by: Jan Lindemann <jan@janware.com>
As per info make, it turns out that ifndef SOME_VAR is true for
SOME_VAR defined to an empty value. This is unusable for caching, so
replace it with ifeq ($(origin SOME_VAR),undefined).
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --topdir-format. The option supports several
different values, affecting the console output of App wherever it
knows that the output contains a reference to the projects' toplevel
directory.
- "unaltered" will have it print the toplevel directory in the same
format as passed to the commandline
- "absolute" will try to resolve it to an absolute path before
printing
- make:XXX will return the make-varible $(XXX) instead
To implement this, the proj_dir() member function is turned into the
private member function __proj_dir(), and a new member function
find_dir() is supplied, with two additional parameters:
search_subdirs and search_absdirs, which will try to find an existing
directory relative to the toplevel directory of the given module, or
in the search_absdirs list, respectively.
Command modules in cmds.projects have been updated to use the new
function.
Signed-off-by: Jan Lindemann <jan@janware.com>
Reorganize the Python module structure. Placing the command classes
under jw.cmds.projects instead of jw.build.cmds will allow to add a
nested command structure, with the current commands, being mostly
related to building software, found below a "projects" toplevel
command.
Other conceivable commands could be "package" for packaging, or
"distro" for commands wrapping the distribution's package manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
Define Q ?= @, and replace @<command> in recipes by $(Q)<command>.
Meant to be overridden from the environment for debugging as in
Q= make
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename the variable PROJECTS_PY_EXTRA_ARGS to PROJECTS_PY_EXTRA_OPTS
to be consistent with projects-dir.mk.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove PY_PREREQ_BUILD and PY_PREREQ_BUILD_DIRS from py-defs.mk:
Apparently they're not used anywhere, and are costly in terms of
directory startup time.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit aims at improving speed by using better caching.
- Makefile, cache.mk: Split .cache.mk up
To allow caching of runtime path variables which are
project-specific, split .cache.mk up in .cache-project.mk and
.cache-projects.mk
- ldlibpath.mk: Cache ldlibpath, exepath and pythonpath
Place the output of $(call proj_query ldlibpath), $(call
proj_query, exepath) and $(call proj_query pythonpath) in
JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH
respectively, and cache the variables in make/.project-cache.mk.
- cache.mk: Use = instead of :=
Recursively expanded variables are nearly as fast as := variables
if the assigned value is a fixed string. And sometimes it's not,
rightly so, because variables get assigned below, as with
JW_PKG_XXX for instance.
- cache.mk: Use $(TOPDIR) as variable values
Replace absolute references to project's topdir by $(TOPDIR) with
sed. As soon as the project queries produce absolute paths, they
will be transformed into relative paths which allow the code base
to be moved to a different location and still remain functional
without a rebuild.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove unused code, and code which actually does something:
CACHED_VARS looks as if it's sufficently and more centrally defined
in make.mk, don't override that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add std-targets.mk, meant to be included mostly to make sure all
mandatory targets are there. On clean and distclean, it also removes
stuff that should not be there anymore after clean.
Signed-off-by: Jan Lindemann <jan@janware.com>
