__read_dep_graph() iterates over the given sections (flavours), but
the loop body does not use the loop variable: it passes the entire
sections list to get_project_refs() on every iteration, so the same
lookup is repeated for each section, and the recursion into the
found dependencies is re-triggered (and skipped) for each of them.
Remove the loop and do the lookup once per project.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__new_with_path() builds the new Uri by deep-copying self and then
replacing __string. A deep copy, however, also carries over any
cached_property values that were already computed on self (e.g. __p,
path, scheme, full), so once __string changes, they stay stale: for a
Uri on which any of those properties had been accessed before,
new_add_path() and new_replace_path() returned objects whose
to_string() showed the new string while path(), hostname(), full() and
friends still described the old one.
Build a fresh instance with object.__new__() and initialize its three
basic attributes instead of copying, so no computed cached state can
be inherited.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M and pi.dev
Signed-off-by: Jan Lindemann <jan@janware.com>
Add $(TOPDIR)/doc with some subdirectories:
- jw-pkg: Using the jw-pkg application, mirroring the subcommand
structure in its subdirectories. At this point only packages,
projects and secrets are added, all empty To be expanded as needed
- python: jw-pkg Python API
Signed-off-by: Jan Lindemann <jan@janware.com>
Add empty target "test" to keep the build from erroring out over make
test in documentation directories.
Signed-off-by: Jan Lindemann <jan@janware.com>
Unprivileged "make install" currently returns an error. That looks bad.
It actually is not a real problem, because unprivileged "make install"
installs files into locations that are good for nothing, in the sense
that they don't constitute a meaningful or usable hierarchy of installed
files. Debugging / inspecting the install machinery itself is a valid
use case, though, and it's easier to explain to the unsuspecting user
that unprivileged installation is something akin to shooting range
training than why "make install" outright errors out, so fix the error
messages.
Signed-off-by: Jan Lindemann <jan@janware.com>
cmd_log_install() defaults $c_logfile to "install.log", so every
invocation without -l appends to an install.log in the current
directory. That's semantical nonsense.
Default $c_logfile to empty instead. The existing empty-$c_logfile
early exit already skips logging, so no -l now means no log, and
callers that want a log pass -l explicitly, as the packaging flow
does through INSTALL_LOG.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M (pi)
Signed-off-by: Jan Lindemann <jan@janware.com>
In a development tree, $(INSTALL) falls back to plain install(1), which
refuses dead symlinks, so unprivileged "make install" fails outright.
Packaging is unaffected, because the spec exports INSTALL_LOG, which
already routes $(INSTALL) through "pkg.sh log-install"
Route the remaining $(INSTALL) defaults through $(LOG_INSTALL_SH) as
well to address the failures: $(SUDO) jw.pkg log-install -p for
DEVELOPMENT=false, plain jw.pkg log-install otherwise. Move -f $(PKG_FORMAT)
out of $(LOG_INSTALL_SH) and into the two logging call sites, so the
no-log paths don't carry a package format they never use.
Signed-off-by: Jan Lindemann <jan@janware.com>
Running "make clean" in some directories can break "make test" in
others. That notably happens by running clean somewhere in or below
$(TOPDIR)/src/python, which removes __init__.py files potentially needed
by unit and integration tests in $(TOPDIR)/test.
This commit makes the failure easily visible by introducing the notion
of "testabiltity" - if the tree is not testable, "make test" logs
exactly that in a clear error message and gives up.
Makefiles which need to check "testability" can use it as a
prerequisite. This commit does that with py-run.mk and jw-py-test.mk.
Testability is asserted by running "make", "make all" or "make test"
from $(TOPDIR). A successful toplevel build coincides with testability,
and leaves a $(TOPDIR)/dirs-all.done behind, which is why that file is
the perfect testability marker.
It is automatically created by a toplevel build and cleared by using the
"invalidate-testability" prerequisite. This commit makes target "clean"
depend on it in py-mod.mk. To be extended to other use cases / makefile
snippets as needed.
Signed-off-by: Jan Lindemann <jan@janware.com>
App.ResultCache is a horrible piece of software, now superseded by
functools.cache, with no measurable performance benefit as of now.
Remove it.
Signed-off-by: Jan Lindemann <jan@janware.com>
The discovery re-parse re-parses the full command line at each level
against the subparsers registered so far. A level's subcommand parsers
are not registered until the re-parse descends into them, so the tokens
after the subcommand name are parsed against the current level's
options. An option meant for a deeper level can then collide with a
same-named option of a shallower level, or fail on a missing value.
Replace the re-parse with a single argv walk. The subcommand names at
each level are known once the commands are materialized, so the invoked
path is found by matching tokens against those names and skipping the
options (and the values they consume) that precede them. Walking the
tokens never parses the tail against a half-built parser, so the
collision cannot occur.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
App() construction builds the entire command tree: every
load_subcommands() call in a command's __init__() constructs its whole
subtree eagerly, so running a single leaf command pays to instantiate
every unrelated command object as well (jw-pkg builds about 50 command
objects for any invocation).
Defer the construction instead. load_subcommands() now records only the
module search path and name filter, and the subcommands are materialized
on first access to the children or child_classes property. The parser
reads children only down the invoked branch on the non-help path, so a
simple run instantiates just that path (jw-pkg builds 5-7 objects),
while the help and completion path expands every node and leaves
rendered help unchanged.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Expose __root_cmd (the command which may or may not be mounted at the
root of the subcommand hierarchy) as _root_cmd() to derived classes.
This supports jw-ev's App base class, which specializes it to get hold
of jw.ev.app.Cmd's specific properties.
Signed-off-by: Jan Lindemann <jan@janware.com>
The application's top-level behavior is defined by overriding
App._add_arguments() and App._run(). The lightweight run-and-options
unit, Cmd, can already be mounted at any node of the command tree, but
the root is reserved for the application itself. An application that
wants to host a plain command at the top level therefore has to
subclass App and carry its full lifecycle implementation.
Add a root parameter to App.__init__(). When it is given a command
class, App instantiates it and uses it as the top level: the command's
options are registered on the top-level parser, it becomes the parent
of the top-level subcommands, and App._run() delegates the run to it.
The command's children are wired as the top-level subcommands, so the
same Cmd can now occupy the root node. When root is not given, the
previous auto-discovery behavior is preserved unchanged.
Keep the top-level subcommand heading as plain "Available subcommands"
whether it is hosted by the application or by a root command, while
nested command levels continue to qualify the heading with the parent
name. Add a unit test that mounts a root command hosting a child and
checks option registration, dispatch, setup and teardown, and
resolution of the application through the parent chain.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Code beautification: add_cmd_to_parser() isn't very telling about its
return type and the fact that it creates an object, hence the name
change. Also, annotate its argument with a private argparse type to
avoid a cast. My concern that argparse will break the private type at
some point in the future is outweighed by the gained clode clarity in
this function.
Signed-off-by: Jan Lindemann <jan@janware.com>
_build_parser() does an early parse_known_args() to configure logging
from the command line, but that call also enforces the top-level
required arguments. --help is registered only after the early parse so
the subcommands are present in the rendered help, so an app with a
required top-level positional, such as a root Cmd that takes a
config-file, exits with "required: config-file" on --help before the
final parse_args() in __run() can handle it.
Skip the early parse when help or shell completion is requested (the
add_all_parsers flag, already set for -h, --help, and argcomplete). The
subcommands and --help are still registered, and the final parse_args()
shows the help without enforcing the required arguments. The log-flag
configuration and the running-command debug line move inside the guard;
neither help nor completion logs, so they do not need them.
Assisted-by: pi <unsloth/Qwen3.8-27B-GGUF:Q4_K_M>
Signed-off-by: Jan Lindemann <jan@janware.com>
LoadTypes() declares mod_names as Iterable[str], so passing a single
module name as a bare string is not caught at call time and the string
is iterated character by character at load time, producing a
ModuleNotFoundError for the first character instead of a clear error.
Normalize a bare string to a one-element list in __init__() and widen
the annotation accordingly, so that both forms work.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't start a fresh project with version 1.0.0-0-dev. By default no
project can honestly claim the level of maturity suggested by 1.0.0-0
with its first commit.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the (untested) module lib.distros.redhat.Distro. It had been
lingering in the source tree, but I've apparently forgotten to add it
to Git. It has never seen any real use because CI still doesn't run
a RedHat distro, so it is to be regarded as a stub. Which is better
than nothing.
Signed-off-by: Jan Lindemann <jan@janware.com>
AsyncRunner is currently unused. This bug was detected and fixed by
AI.
asyncio.Event() raises RuntimeError on Python 3.12+ when created
outside a running event loop. It is created in the sync portion of
loop_in_thread(), before the threaded loop is up.
The fix is to move the Event creation inside the async main()
coroutine and pass it back via a second future. It replaces the
fragile as_completed loop with sequential result() calls, so failures
are immediately visible rather than causing a silent thread hang.
Assisted-by: unsloth/Qwen3.6-27B-MTP-GGUF:Q4_K_M with pi.dev v0.84.1
Signed-off-by: Jan Lindemann <jan@janware.com>
add_subcommands() advertises Cmd and list[Cmd] in its signature, but
a single Cmd instance raises NotImplementedError, and since the list
branch handles every element through the same method, a list of Cmd
instances is broken as well. The only working forms are Types and
lists of Types.
Handle a single Cmd instance by reparenting it to the caller and
appending it to the children, tracking its class like the class-based
path does. Instances whose name is already taken by a child are
rejected, mirroring the duplicate-class handling, because argparse
cannot register two subparsers under the same name.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The os.getenv() calls in __init__() that read the log and backtrace
defaults mostly pass None as the explicit default value, but the one
for the show-backtrace environment variable relies on the implicit
default.
Pass None explicitly there as well, so that all the calls read the
same way.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The shell completion setup in __run() catches every exception and
silently ignores it. If argcomplete is missing or its initialization
fails, the completion is simply not available and there is no trace
of why, even when logging is turned up to debug level.
Log the reason at debug level: one message when the argcomplete
import fails and one with the exception for any other failure.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__run() only accepts a return value from _run() as the process exit
status if it is an int between 0 and 255, and silently drops any other
value. A command that returns, for instance, 300 therefore exits with
status 0, which presents a failure as a success to the caller without
any trace of the mistake.
Log an error when the returned exit status is out of range so that the
programming error is visible, while still exiting with 0 instead of
passing an invalid status to the shell.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The subcommand registration in _build_parser() defines a SubCommand
helper class inside the add_cmds_to_parser() closure, so a fresh class
object is created on every call. It also stores command names and
aliases in a dictionary without checking for duplicates, so a colliding
name or alias is silently overwritten, and it relies on every subparser
level sharing the dest = 'command' attribute to descend one level per
re-parse, an invariant that is not documented anywhere.
Hoist the helper to a module-level _SubCommand NamedTuple, log a
warning when a subcommand name or alias collides with an earlier one at
the same level, and document the dest = 'command' invariant next to the
re-parse.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_add_arguments() adds the global options to self.__parser instead of
to the parser it receives. The two are the same object, because the
only caller passes self.__parser, so the change is not observable.
Use the parser parameter instead, so that the method honors its
argument the way Cmd.add_arguments() does, and so that the global
options can be shared with other parsers, e.g. the subcommand
parsers, without rewriting this method.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_run() receives the parsed arguments as its args parameter, but then
checks the private __args attribute for the func attribute and resolves
the command function through the args property. Both refer to the same
object today, so the mixing is harmless, but it obscures the data flow
and would silently diverge if a caller ever passed a namespace other
than the stored one.
Use the args parameter consistently in _run().
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__init__() builds the parser and the lazy subcommand registration
inside it decides which subcommands to register by re-parsing
sys.argv. run() then parses a different argv, so if the caller passes
an argv that is deeper than the one in sys.argv, the required
subparsers have not been registered and the invocation fails with an
"unrecognized arguments" error. run_sub_commands() passes argv to
run(), so the mismatch is reachable from the public API.
Move the parser construction from __init__() into _build_parser() and
call it from run() when an argv is given, so that registration and
parsing are driven by the same command line. The top-level command
instances are created once in __init__() and reused when the parser
is rebuilt.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
When run() creates an event loop, it installs it with
set_event_loop() but never restores the thread's previous loop, so
after run() returns, the thread is left with the now-closed loop
created by run(). Any code that calls get_event_loop() afterwards
gets a closed loop, and on Python 3.13+ a thread that had no loop at
all starts emitting or raising deprecation errors that run() caused.
Capture the thread's current loop with _get_current_event_loop()
before installing a new one, and restore it in the finally block. If
there was no previous loop, unset the loop with set_event_loop(None)
so that the thread is left without a loop instead of with the closed
one.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
close() closes the application's own event loop, but the AsyncRunner
is only released in the finally block of run(). An application that
creates a runner through call_async() and then calls close(), for
instance through the async context manager, therefore leaks the
runner, and close() does not fulfill its contract of releasing all
resources.
Move the AsyncRunner cleanup from the finally block of run() into
close() and reset the own-loop flag when the loop is closed, so that
close() releases everything and run() only has to call it.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__aenter__() and __aexit__() are empty stubs. Using the application
as an async context manager therefore binds None in the as clause,
and releases nothing on exit: an AsyncRunner created through
call_async() keeps running in its thread, and since that thread is
not a daemon, the process does not exit after the block.
Return self from __aenter__(), and call close() from __aexit__(), so
that the context manager binds the application and releases all
resources on exit, whether the block exits normally or with an
exception.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The --log-level and --log-flags options are added without a type
converter, so argparse never validates their values. _build_parser()
then hands the raw string to set_log_level() and set_log_flags() during
its first parse, and an unparseable value such as "INVALID" crashes
__init__() with a raw ValueError traceback instead of a usage error.
Pass type = parse_log_level() and type = parse_log_flags() when adding
the options, so that argparse reports invalid values with the standard
usage error and exit status 2. argparse only applies the converter to
command-line strings, so the int and LogFlag defaults are unaffected.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
asyncio.get_event_loop() is removed in Python 3.14 when called from
outside an async context. The current code calls it in __init__(),
which crashes on 3.14+.
To fix this, drop the eager loop creation from __init__(). Instead,
lazily create a loop in run() via asyncio.new_event_loop() when no
external loop was provided, and close it in the finally block. This
makes the lifecycle symmetric: run() owns the full create-use-close
cycle and supports re-entrant calls.
Replace __del__() with an explicit close() method, guarding against
double-close via is_closed(). close() always clears __eloop to None
so a closed loop never lingers.
Signed-off-by: Jan Lindemann <jan@janware.com>
The diff-all and diff targets diff all projects in the workspace
without filtering.
Add a target "diff-projects". It sets PGIT_SH_PROJECTS to the list of
projects from build-order, limiting the diff to the dependency
closure around $(PROJECTS).
Signed-off-by: Jan Lindemann <jan@janware.com>
If /usr/bin/isort is found, run it during "make format" to get a
defined way the imports are sorted. tool.isort in pyproject.toml is
updated to match the other fixers.
Commit the fallout of this change. Running the other fixers alone
doesn't change the formatting, so this should be safe.
Signed-off-by: Jan Lindemann <jan@janware.com>
