Add the target sh-syntax-check, which triggers bash syntax linting with shellcheck This commit stops short of making target all depend on it, because the fallout is impressive.
To make EXE_SH useful for automated shell syntax checking, remove all non POSIX shell / bash scripts from that variable, and place the non-shell scripts into EXE_SCRIPTS.
Modifying JW_PKG_XXX_PATH in pre-local.mk is fragile.
Amending PREREQ_RUN in pre-local.mk works, but only with $(JW_PKG_NO_CACHE) == true, or if "undefine JW_PKG_XXX_PATH" is also added in pre-local.mk. Otherwise JW_PKG_XXX_PATH will not be recomputed, because it's already defined from the cache.
Introduce the new variable PREREQ_RUN_ADD to solve that. If it's defined, it automatically invalidates the JW_PKG_XXX_PATH variables and sets them up for recalculation in py-path.mk / ldlibpath.mk.
Reverse inclusion order of .cache-project.mk and cache-projects.mk: Definitions in .cache-project.mk should win over cache-projects.mk, because it's the more specialized include file, and the way the definitions in both files are structured, the later doesn't overwrite the earlier.
The clean-dirs target does not only clean the repos present in PROJECTS, but all repos it finds to be dirty, and clean-all-dirs does the opposite. I suppose that was an oversight, swap their recipes.
Moreover, cleaning all directories goes about its business in an overly complicated and unecessarily time-consuming way, fix that, too.
This commit adds more tweaks to shell command output in order to make it nicer. The biggest patch is in Result.__summarize(), which makes it more versatile, and allows removal of some code in SSHClient.
App sees some independent, minor result format beautification.
This commit makes them use spaces instead, so they can be more easily amended by Makefiles using them. Also define them in a more uniform way, and use the newly introduced PREREQ_RUN variable to fill them, which in turn can also be appended to before that.
Support a --delimiter option to the ldlibpath and exepath commands. Notable use case are the JW_PKG_XXX_PATH variables, which should use spaces instead of colons.
TODO: Merging those two command modules with BaseCmdPkgRelations would have made introducing this redundancy unnecessary, check if that's a possibility.
Some options to the pkg-xxx commands, like flavour, --subsections and --ignore understand a comma as delimiter if multiple option values are specified. The comma character is not very friendly to use in $(call ...) macros, though, so support spaces and pipe characters as well.
At present, the PREREQ-variable is effectively only used to detect if prerequiste packages haven't run "make all" before make is run in a given package. Also, it's only useful in $(TOPDIR). This commit splits the variable up into PREREQ_BUILD and PREREQ_RUN, and makes the variables available in every Makefile of a package by placing them in defs.mk instead of topdir.mk.
This also fixes a problem that PREREQ was cached before being filled, hence empty. Which effectively wasn't much of a problem, because it was basically unused, but still.
make/Makefile is responsible to generate $(TOPDIR)/cache-projects.mk. The variables are taken from .cache-project.mk, with some variables intentionally omitted, but their ifndef / endif blocks remain in place. Not harmful but ugly. Make sed range-delete the left-over blocks entirely.
Aside from PYTHONPATH, ldlibpath.mk runs jw-pkg.py for determining other paths, too, which is often unneeded and can impact performance. Split the PYTHONPATH detection into a dedicated py-path.mk, and include it from ldlibpath.mk, so it can be used instead where needed.
Add support for PY_INIT_SUBMODULES to py-mod.mk. If it is defined in a Makefile including py-mod.mk, the listed submodules will be added to __init__.py and thus included in the list of things that can be imported from a module.
This commit also adds support for --submodules to python-tools.sh for that to happen.
The global --topdir-format make:XXX option to jw-pkg is half-baked at best, and __find_dir() ignores it entirely. Make __find_dir() return some Makefile-syntax-formatted output if the option is present. Not used anywhere, currently, and, hence, badly tested, but still better than the situation before.
Overriding the _run() method entirely in App subclasses is currently only possible if the application supports a subcommand structure. Make it possible to use it as an abstraction for a single-command application.
The .strip property of class Result defaults to True, and its name isn't very clear. Rename it to .strip_output, and default it to False to avoid surprising contents for unsuspecting callers.
The command line jw-pkg.py is run with is logged with level "debug", and reconstructed with ' '.join(sys.argv). Use pretty_cmd() instead, this adds quotes around spaces.
Commands executed by ExecContext and its derived classes don't populate the "cmd" parameter of "Result"'s constructor. Fixing that makes for nicer error messages.
In LoadTypes' constructor, allow the type_filter parameter to be of type Sequence[type[Any]] instead of list[type[T]]. a) Sequence is more generic than list, and b) with T instead of Any, trying to instantiate with an abstract class has mypy complain:
# E: Only concrete class can be given where "type[MyClass]" is
expected [type-abstract]
OpenSUSE leaves installing local packages to the default implementation in lib.Distro._install_local_files(), which passes the package path to the package manager, i.e. zypper in OpenSUSE's case. That has advantages, namely automatic installation of dependencies, but also disadvantages, namely the attempt to install dependencies even if the package manager is disfunctional, possibly because an installed package containing installation sources is broken.
That could lead to a deadlock when trying to install a fixed package. I see two ways out: Support an additional flag to jw-pkg's install command which selects whether or not dependencies shall be resolved along, or just use rpm directly for all local install attempts.
The latter is the less fancy way to handle this, so as a first step make it the default by overriding suse.Distro._install_local_files().
Define default parameter values for Result's constructor, namely None for exit status, stdout and stderr.
Instantiating a Result object without parameters signifies "this object doesn't contain data from a real process's exit event". Up to now, similar meaning has been hand-crafted by ExecContext's run() and friends by using an error exit status (1) to make sure it wasn't mistaken for success. This commit formalizes that into the Result structure itself, but uses None instead for the exit status.
Controlling default values in Result itself also means that the Result class gets better awareness of what it contains, and its log messages and stdin / stdout can be more fitting:
- If a real process failed, make stdout return at least b'' - If a real process succeeded, make stdout return at least b''
Returning something from .stdout on success fixes a real bug: An attempt to access what "rpm -U somepackage.rpm" returns, namely nothing, raises a bogus exception, because stdout is None.
