In the os_cascade() function, don't roll own logic, call the richer
jw-projects.py projects os-cascade instead.
Signed-off-by: Jan Lindemann <jan@janware.com>
App.res_cache should be private. It's needed here to cache the
results of a function which is private to CmdBuild anyway, so solve
that with lru_cache, to allow App to do it's private thing with
caching.
Signed-off-by: Jan Lindemann <jan@janware.com>
Replace the jw.pkg.App.debug(), .warn() and .err() methods by the
global log() function. There's no obvious benefit in having App know
what's logged.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --topdir-format. The option supports several
different values, affecting the console output of App wherever it
knows that the output contains a reference to the projects' toplevel
directory.
- "unaltered" will have it print the toplevel directory in the same
format as passed to the commandline
- "absolute" will try to resolve it to an absolute path before
printing
- make:XXX will return the make-varible $(XXX) instead
To implement this, the proj_dir() member function is turned into the
private member function __proj_dir(), and a new member function
find_dir() is supplied, with two additional parameters:
search_subdirs and search_absdirs, which will try to find an existing
directory relative to the toplevel directory of the given module, or
in the search_absdirs list, respectively.
Command modules in cmds.projects have been updated to use the new
function.
Signed-off-by: Jan Lindemann <jan@janware.com>
Reorganize the Python module structure. Placing the command classes
under jw.cmds.projects instead of jw.build.cmds will allow to add a
nested command structure, with the current commands, being mostly
related to building software, found below a "projects" toplevel
command.
Other conceivable commands could be "package" for packaging, or
"distro" for commands wrapping the distribution's package manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make App.proj_dir() return an absolute path. This looks like a good
idea, because some of the $(call proj_query xxx) paths end up being
relative, because they get proj_dir()'s idea of a directory
prepended. This prohibits caching them in $(TOPDIR)/make/.cache.mk
for make benefit glorious nation of performance.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove the --debug option, which is superseded by the semantically
richer --log-level debug in the base class.
Signed-off-by: Jan Lindemann <jan@janware.com>
Derive jw.pkg.App from jw.pkg.lib.App. App.run() dissolves as follows:
- Its sub-command invocation logic is left to the base class
- parser.add_arguments() are moved into self._add_arguments()
- So is handling of early-parsed arguments
- async def _run() is reimplemented to set some member variables
Signed-off-by: Jan Lindemann <jan@janware.com>
Add App and Cmd as generic base classes for multi-command
applications. The code is taken from jw-python: The exising
jw.pkg.App is very similar to the more capable jwutils.Cmds class,
so, to avoid code duplication, add it here to allow for jwutils.Cmds
and jw.pkg.App to derive from it at some point in the future.
Both had to be slightly modified to work within jw-pkg's less
equipped context, and will need futher code cleanup.
Signed-off-by: Jan Lindemann <jan@janware.com>
Types is a container for types, notably classes, which are
dynamically loaded from other modules. Which modules are loaded is
based on the following criteria passed to its constructor:
- mod_names: A list of modules to load and investigate
- type_name_filter: A regular filter expression or None (default).
If it's None, all types pass this filter.
- type_filter: A list of types the returned types must match.
Defaults to [], in which case all types pass this filter
Signed-off-by: Jan Lindemann <jan@janware.com>
A dedicated logging module is currently provided by jw-python, but
since it's often needed also in jw-pkg, and it's relatively small and
maintainable, it seems justified to move it into jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdGetval.add_arguments() uses self.app.top_name, which may or may
not be initialized at the time this runs. Not using it makes
CmdGetval's ctor safe to run in the context of App.__init__().
Signed-off-by: Jan Lindemann <jan@janware.com>
Define Q ?= @, and replace @<command> in recipes by $(Q)<command>.
Meant to be overridden from the environment for debugging as in
Q= make
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename the variable PROJECTS_PY_EXTRA_ARGS to PROJECTS_PY_EXTRA_OPTS
to be consistent with projects-dir.mk.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove PY_PREREQ_BUILD and PY_PREREQ_BUILD_DIRS from py-defs.mk:
Apparently they're not used anywhere, and are costly in terms of
directory startup time.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit aims at improving speed by using better caching.
- Makefile, cache.mk: Split .cache.mk up
To allow caching of runtime path variables which are
project-specific, split .cache.mk up in .cache-project.mk and
.cache-projects.mk
- ldlibpath.mk: Cache ldlibpath, exepath and pythonpath
Place the output of $(call proj_query ldlibpath), $(call
proj_query, exepath) and $(call proj_query pythonpath) in
JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH
respectively, and cache the variables in make/.project-cache.mk.
- cache.mk: Use = instead of :=
Recursively expanded variables are nearly as fast as := variables
if the assigned value is a fixed string. And sometimes it's not,
rightly so, because variables get assigned below, as with
JW_PKG_XXX for instance.
- cache.mk: Use $(TOPDIR) as variable values
Replace absolute references to project's topdir by $(TOPDIR) with
sed. As soon as the project queries produce absolute paths, they
will be transformed into relative paths which allow the code base
to be moved to a different location and still remain functional
without a rebuild.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove unused code, and code which actually does something:
CACHED_VARS looks as if it's sufficently and more centrally defined
in make.mk, don't override that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add std-targets.mk, meant to be included mostly to make sure all
mandatory targets are there. On clean and distclean, it also removes
stuff that should not be there anymore after clean.
Signed-off-by: Jan Lindemann <jan@janware.com>
This reverts commit 98188eab23.
__init__.py alone is not enough to resolve the jw module in all
circumnstances, so go back to the old symbolic links solution.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change $(check_scm_sync) from "make git pull" to "make
git-pull-maintainer", which most notably should delegate devops
builds to the maintainer defined in project.conf.
Signed-off-by: Jan Lindemann <jan@janware.com>
--prefix doesn't denote an "App Path Prefix", "Parent directory of
project source directories" decribes it better.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's little system in the pkg-install-xxx targets, add one more to
increase the confusion. It's needed to install all packages needed to
do a standalone build against the packages installed into the system
via package manager. That said, the respective jw-projects.sh
commands need broader refactoring, as well as the pkg-install-xxx
target naming.
Signed-off-by: Jan Lindemann <jan@janware.com>
$(TOPDIR)/scripts contains a symbolic link jw -> ../src/python/jw, to
allow jw-python.py access to the in-repo module jw.pkg. Should be
fine now even on Windows, OTOH, it's also solvable via __init__.py,
so do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
To support minimal environments, notably minimal Docker containers
which don't have /usr/bin/sudo by the time pkg-manager.sh is invoked
(possibly to install sudo), support running all commands without
invoking sudo first. Of course this only works if invoked as root.
Note that this is still somewhat hacky, command-line parsing needs to
be cleaned up.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make jw-projects.py list-repos support a local directory as base URL
of all git repositories, notably used by PROJECTS_DIR_REMOTE_BASE,
which can now point to a local directory.
Signed-off-by: Jan Lindemann <jan@janware.com>
Targets defined by projects-dir.mk are not available before it is
included, but make makes up its mind about what targets are available
after parsing the included makefiles, so remove that redundancy.
On the other hand, a dependency alone is not enough for make to
understand that an included makefile has been remade, it needs a
rule, so add a dummy-rule body. In this case only echoing that the
include file has been provided.
Signed-off-by: Jan Lindemann <jan@janware.com>
Kali Linux' default installation doesn't have /usr/bin/time which
brings out a but: $(TIME) doesn't expand to nothing but to -p, which
fails miserably, of course. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
PGIT_SH gets added --remote-base, but too late to make it into the
non-recursive variable PGIT_SH_CLONE. This leads to --remote-base
lacking from the clone invocation, and anonymous Git over HTTP
failing because it tries to clone via SSH. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
To avoid network errors while fetching tags, run
git submodule foreach --recursive 'git fetch --tags -f origin
i.e. only fetch tags from origin, which by convention points to
git.janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
make git-show-pushable-master-branches output too litte for two
reasons: 1. grep -q returns zero also if no matches are found, and 2.
PROJECTS doesn't contain all relevant projects. BUILD_PROJECTS is
more meaningful.
Signed-off-by: Jan Lindemann <jan@janware.com>
HTMLOWNER wwwrun is not a good idea with file mode 0644. The web
server process should not be allowed to write its own executable
files.
Signed-off-by: Jan Lindemann <jan@janware.com>
