get_values() splits comma-separated values and strips whitespace but does not filter out empty strings. A value like "a, b, " produces ['a', 'b', ''], with an empty string at the end. This empty string propagates to callers like CmdRequiredOsPkg.py and pollutes output. Add a filter for non-empty stripped values.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
The refactored __get_project_conf() raises FileNotFoundError when project.conf does not exist, whereas the original read_value() returns None. This causes get_value() to crash for projects with missing or incomplete project.conf files.
Catch FileNotFoundError in __get_project_conf() and return None to restore original behavior. Remove redundant @cache from __read_project_conf() since __get_project_conf() already provides caching.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
The __find_circular_deps() and find_circular_deps() methods now return list[str] instead of bool. On a cycle, the path builds up the dependency chain in __find_circular_deps_recursive(), and __find_circular_deps() appends the closing project to complete the cycle. An empty list means no cycle found.
CmdDep prints the cycle as 'a -> b -> c -> a' instead of a generic message.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
The __flip_dep_graph(graph) call sits inside the while loop and performs redundant graph flipping on every iteration. Hoist it outside to compute once and reuse the result.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
Remove App.get_section() which parses raw file sections by scanning for section headers and accumulating lines. This method is no longer needed since ProjectConf now handles all config file parsing.
Introduce ProjectConf module to cleanly parse make/project.conf ini-like configuration files. The new class supports:
- ini-style sections with header comments - Key-value pairs with backslash line continuation - Quoted values preserving spaces and comment delimiters (#) inside - Inline comments outside of quotes - Comma-separated list values with quoted commas - Cached section parsing to avoid re-parsing the same section - .get_section() to return an entire section unparsed
Replace the @functools.lru_cache(maxsize=None) decorator with @functools.cache throughout App. functools.cache is a shorthand for functools.lru_cache(maxsize=None) introduced in Python 3.9 and is more concise and readable with identical behaviour.
Fix shellcheck SC2068 (unquoted array expansions), SC2145 (mixed string/array arguments), SC2328 (redirection in command substitution), SC2173 (untrapable signals), and SC2148 (missing shebang) errors across 14 script files.
Also configure scripts/Makefile with --severity=error so that only errors (not warnings or notes) cause check failures. To be tightened by follow-up commits.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev
Move the unit test hierarchy to below test/unit/python/jw/pkg. Nesting the subdirectories so deeply might be overly careful, but it may as well be not - maybe in the future installable test packages are going to be generated and need simple ways to install without stepping onto each other's toes. If not, it's easier to cut two directory components out than having to reorganize possibly incoherent paths grown over multiple packages.
The Uri class provides URL parsing and manipulation utilities used throughout jw-pkg. Add a unit test covering URL parsing, credential handling, path manipulation, and safe string formatting.
After a pipeline change, CI now runs "make all" in a repo's root, which uncovers two problems:
1. The help integration test only succeeded as long CI didn't run "make all" before "make test". That way, the checked out repository lacked the generated __init__.py files needed for some modular subcommands to be fully loaded, and hence, the test should have failed. The entire machinery only worked because the subcommands in question are not not essential to building jw-pkg itself: "secrets" and "posix". So, this commit adapts the help integration test to the new reality.
2. Regarding python-tools.sh: Commit 55060486 satisfies yapf in some places of the source code, but in others not anylonger. So patch python-tools.sh's newline handling again.
While not thematically similar, both fixes get baked into one commit to satisfy the requirement that every single commit needs to pass "make clean all check test" individually.
Add a test/integration subdirectory as a location for integration tests. The first tests that are added are unproblematic in that they don't need superuser privileges to run:
- help: Recursively check if jw-pkg.py's help messages are unchanged
- packages ls: Check if the bash package exists and contains
Add include file to provide some definitions for a first integration test suite. It provides the shared make variables, most notably TEST_CMD_LINE as handy default for running jw-pkg.py.
Relabel the toplevel command CmdPkg from "pkg" to "packages", because it rolls off the tounge much more nicely. Keep "pkg" as an alias for compatibilty.
Add a property aliases to AbstractCmd in prepeparation for commands to bear multiple names / abbreviations / aliases.
The App.add_cmds_to_parser() function uses parse_known_args() to determine which subcommand was invoked, then conditionally registers nested subcommands. The lookup dictionary (scs) contains only canonical names, not aliases, so add them too, otherwise using the alias instead of the canonical name causes the lookup to fail and nested subcommands to never be registered.
Fix: Register each alias in scs pointing to the same SubCommand object, and deduplicate with id(sc) when iterating in all=True mode to avoid infinite recursion on help output.
Add the target sh-syntax-check, which triggers bash syntax linting with shellcheck This commit stops short of making target all depend on it, because the fallout is impressive.
To make EXE_SH useful for automated shell syntax checking, remove all non POSIX shell / bash scripts from that variable, and place the non-shell scripts into EXE_SCRIPTS.
Modifying JW_PKG_XXX_PATH in pre-local.mk is fragile.
Amending PREREQ_RUN in pre-local.mk works, but only with $(JW_PKG_NO_CACHE) == true, or if "undefine JW_PKG_XXX_PATH" is also added in pre-local.mk. Otherwise JW_PKG_XXX_PATH will not be recomputed, because it's already defined from the cache.
Introduce the new variable PREREQ_RUN_ADD to solve that. If it's defined, it automatically invalidates the JW_PKG_XXX_PATH variables and sets them up for recalculation in py-path.mk / ldlibpath.mk.
Reverse inclusion order of .cache-project.mk and cache-projects.mk: Definitions in .cache-project.mk should win over cache-projects.mk, because it's the more specialized include file, and the way the definitions in both files are structured, the later doesn't overwrite the earlier.
The clean-dirs target does not only clean the repos present in PROJECTS, but all repos it finds to be dirty, and clean-all-dirs does the opposite. I suppose that was an oversight, swap their recipes.
Moreover, cleaning all directories goes about its business in an overly complicated and unecessarily time-consuming way, fix that, too.
This commit adds more tweaks to shell command output in order to make it nicer. The biggest patch is in Result.__summarize(), which makes it more versatile, and allows removal of some code in SSHClient.
App sees some independent, minor result format beautification.
This commit makes them use spaces instead, so they can be more easily amended by Makefiles using them. Also define them in a more uniform way, and use the newly introduced PREREQ_RUN variable to fill them, which in turn can also be appended to before that.
