Make App.proj_dir() return an absolute path. This looks like a good idea, because some of the $(call proj_query xxx) paths end up being relative, because they get proj_dir()'s idea of a directory prepended. This prohibits caching them in $(TOPDIR)/make/.cache.mk for make benefit glorious nation of performance.
Derive jw.pkg.App from jw.pkg.lib.App. App.run() dissolves as follows:
- Its sub-command invocation logic is left to the base class - parser.add_arguments() are moved into self._add_arguments() - So is handling of early-parsed arguments - async def _run() is reimplemented to set some member variables
Add App and Cmd as generic base classes for multi-command applications. The code is taken from jw-python: The exising jw.pkg.App is very similar to the more capable jwutils.Cmds class, so, to avoid code duplication, add it here to allow for jwutils.Cmds and jw.pkg.App to derive from it at some point in the future.
Both had to be slightly modified to work within jw-pkg's less equipped context, and will need futher code cleanup.
Types is a container for types, notably classes, which are dynamically loaded from other modules. Which modules are loaded is based on the following criteria passed to its constructor:
- mod_names: A list of modules to load and investigate
- type_name_filter: A regular filter expression or None (default).
If it's None, all types pass this filter.
- type_filter: A list of types the returned types must match.
Defaults to [], in which case all types pass this filter
A dedicated logging module is currently provided by jw-python, but since it's often needed also in jw-pkg, and it's relatively small and maintainable, it seems justified to move it into jw-pkg.
CmdGetval.add_arguments() uses self.app.top_name, which may or may not be initialized at the time this runs. Not using it makes CmdGetval's ctor safe to run in the context of App.__init__().
Remove PY_PREREQ_BUILD and PY_PREREQ_BUILD_DIRS from py-defs.mk: Apparently they're not used anywhere, and are costly in terms of directory startup time.
This commit aims at improving speed by using better caching.
- Makefile, cache.mk: Split .cache.mk up
To allow caching of runtime path variables which are
project-specific, split .cache.mk up in .cache-project.mk and
.cache-projects.mk
- ldlibpath.mk: Cache ldlibpath, exepath and pythonpath
Place the output of $(call proj_query ldlibpath), $(call
proj_query, exepath) and $(call proj_query pythonpath) in
JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH
respectively, and cache the variables in make/.project-cache.mk.
- cache.mk: Use = instead of :=
Recursively expanded variables are nearly as fast as := variables
if the assigned value is a fixed string. And sometimes it's not,
rightly so, because variables get assigned below, as with
JW_PKG_XXX for instance.
- cache.mk: Use $(TOPDIR) as variable values
Replace absolute references to project's topdir by $(TOPDIR) with
sed. As soon as the project queries produce absolute paths, they
will be transformed into relative paths which allow the code base
to be moved to a different location and still remain functional
without a rebuild.
Remove unused code, and code which actually does something: CACHED_VARS looks as if it's sufficently and more centrally defined in make.mk, don't override that.
Add std-targets.mk, meant to be included mostly to make sure all mandatory targets are there. On clean and distclean, it also removes stuff that should not be there anymore after clean.
Change $(check_scm_sync) from "make git pull" to "make git-pull-maintainer", which most notably should delegate devops builds to the maintainer defined in project.conf.
There's little system in the pkg-install-xxx targets, add one more to increase the confusion. It's needed to install all packages needed to do a standalone build against the packages installed into the system via package manager. That said, the respective jw-projects.sh commands need broader refactoring, as well as the pkg-install-xxx target naming.
$(TOPDIR)/scripts contains a symbolic link jw -> ../src/python/jw, to allow jw-python.py access to the in-repo module jw.pkg. Should be fine now even on Windows, OTOH, it's also solvable via __init__.py, so do that.
To support minimal environments, notably minimal Docker containers which don't have /usr/bin/sudo by the time pkg-manager.sh is invoked (possibly to install sudo), support running all commands without invoking sudo first. Of course this only works if invoked as root.
Note that this is still somewhat hacky, command-line parsing needs to be cleaned up.
Make jw-projects.py list-repos support a local directory as base URL of all git repositories, notably used by PROJECTS_DIR_REMOTE_BASE, which can now point to a local directory.
Targets defined by projects-dir.mk are not available before it is included, but make makes up its mind about what targets are available after parsing the included makefiles, so remove that redundancy.
On the other hand, a dependency alone is not enough for make to understand that an included makefile has been remade, it needs a rule, so add a dummy-rule body. In this case only echoing that the include file has been provided.
Kali Linux' default installation doesn't have /usr/bin/time which brings out a but: $(TIME) doesn't expand to nothing but to -p, which fails miserably, of course. Fix that.
PGIT_SH gets added --remote-base, but too late to make it into the non-recursive variable PGIT_SH_CLONE. This leads to --remote-base lacking from the clone invocation, and anonymous Git over HTTP failing because it tries to clone via SSH. Fix that.
make git-show-pushable-master-branches output too litte for two reasons: 1. grep -q returns zero also if no matches are found, and 2. PROJECTS doesn't contain all relevant projects. BUILD_PROJECTS is more meaningful.
