_read_filtered() logs an error when reading the tar file fails, but
the opening parenthesis of the error detail after the path is never
closed.
Close it.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__init_askpass() generates an askpass script whose echo statement
embeds a newline inside the quoted password: echo -n "PASSWORD
". Bash happily spans the quote over the newline, and the script
therefore prints the password followed by a trailing newline.
run_askpass() returns that output verbatim, so the password passed
on carries a newline and authentication fails.
Write the script so it prints the password and nothing else.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_run_ssh() waits for the remote process's exit status before reading
stdout and stderr. When a command produces more output than the
channel's flow-control window can hold, the server stops sending, the
remote process blocks on its write and never exits, and
recv_exit_status() blocks forever.
Drain stdout and stderr to EOF first - the channels close when the
process exits, so reading them also implies completion - and only
then query the exit status.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_run_ssh() writes cmd_input to the remote stdin channel but never
closes the write side. The channel stays open until the client
process exits, so a remote command that reads stdin (cat, a login
shell, ...) never sees EOF and blocks forever - including for
cmd_input = None, which is supposed to mean non-interactive with
stdin from /dev/null.
Call shutdown_write() on the channel after writing the input, or
immediately when there is none, so the remote command gets EOF.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__client() connects using only the URI's hostname and username. The
URI's port is ignored, so ssh://host:2222/... ends up connecting to
port 22, and a password carried in the URI is never passed to
paramiko, so URI-based password authentication cannot work. The Exec
and AsyncSSH clients both honor the port and the password.
Pass the port and the password to connect(). The port argument is
omitted entirely when the URI carries no port, because
getaddrinfo() would interpret a None port as service port 0; without
the argument, paramiko falls back to its default of 22.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_run() forwards the execution context to run_cmd() when sudo is not
requested, but calls run_sudo(cmd) without the context in the sudo
case. run_sudo() then falls back to a fresh local context, so sudoed
dpkg and dpkg-query commands run on the local machine instead of on
the given context, e.g. Distro._delete() on a remote host.
Pass the context and non-interactive stdin to run_sudo() as well.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
run() remembers the thread's current event loop before creating a new
one only when the app does not own a loop yet. The finally block,
however, always restores the remembered loop or unsets the loop when
there was none. So when a caller passes its own event loop to the App
constructor and has set it as the thread's current loop, run() unsets
the thread's loop on the way out, although it never changed it.
Remember the thread's current event loop unconditionally and restore
it in the finally block; for a caller-provided loop that is a no-op.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_connect_kwargs(hide_secrets = True) is used to log the connection
parameters when a connection fails, without leaking the password. The
filtered dictionary is built before the password is replaced with
'<hidden>', and the replacement is applied to the local kwargs
dictionary afterwards, after the filtered copy has already been made.
The dictionary that ends up in the log therefore still contains the
real password.
Hide the password before building the filtered dictionary.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_put() catches a failure of the remote command sequence and logs
"Failed to get <path> from <root>", a phrase copied over from the
get() path. This path, however, pushes content, so the message
describes the wrong operation.
Log a put-oriented message instead.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_put() writes the content into a temporary file with tee and, when
atomic is set, moves it to the target path with a final mv. The loop
over the command list sets tmp_file to None after each command, on the
assumption stated in the comment that the file has been moved at that
point - which is only true for the last command. When a chown, chmod,
or mv after the tee fails, the finally block finds tmp_file is None
and leaves the temporary file behind.
Reset tmp_file only after all commands have completed, so that the
finally block erases the temporary file whenever a step fails.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
_is_dir() logs a DEBUG message when _stat() is not implemented and it
must guess from the trailing slash whether a path is a directory. The
second part of that message is a plain string rather than an f-string,
so '{path}' is logged verbatim instead of the path.
Make it an f-string so the path is interpolated.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__new_with_path() joins base and path with exactly one '/', assuming a
trailing '/' in base is a path separator. That assumption breaks for
URIs with empty authority, where scheme_plus_authority ends in '://'
(e.g. 'file:///tmp/x'): new_replace_path('/etc/hosts') drops the
leading slash of the path and produces 'file://etc/hosts', which
parses with hostname 'etc' and path '/hosts' instead of path
'/etc/hosts'.
Treat a base ending in '://' separately and keep a leading slash in
the path, adding one if it is missing.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__format_topdir() accepts "absolute", "relative", "unaltered", and
"make:<variable-name>", but the error message it raises for anything
else only lists "unaltered", "absolute", and
"make:<variable-name>", leaving out "relative".
Add "relative" to the list of valid formats in the error message.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__read_dep_graph() iterates over the given sections (flavours), but
the loop body does not use the loop variable: it passes the entire
sections list to get_project_refs() on every iteration, so the same
lookup is repeated for each section, and the recursion into the
found dependencies is re-triggered (and skipped) for each of them.
Remove the loop and do the lookup once per project.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__find_circular_deps_recursive() builds the cycle path by inserting
each visited dependency at the front of the list on the way back up,
and __find_circular_deps() appends the project where the cycle was
detected at the end. The path therefore starts at the first child of
the DFS root instead of at the project that closes the cycle, and the
closing project appears twice: for a dependency cycle between projects
A and B, find_circular_deps() reports "B -> A -> A" instead of
"A -> B -> A".
Keep the DFS stack as an ordered list, and when a dependency is already
on the stack, return the part of the stack from that dependency to the
top, plus the dependency itself, which starts and ends at the same
project. Reverse the result before returning, because an edge
a -> b in the flipped graph means that b depends on a, so the cycle
is reported in the original dependency direction.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__get_project_refs() appends the values of a node to the result buffer
when the scope is Scope.Self. That is wrong: Scope.Self means "no
children", so a node visited with Scope.Self is supposed to contribute
only itself (if add_self is set), never its children. A previous
refactor that turned the scope handling into a match/case block
accidentally changed that, and as a consequence get_project_refs()
with scope=Scope.One now returns the direct dependencies plus their
dependencies. The second-level entries are raw values: they are not
stripped of whitespace, and they are not reduced to module names even
when names_only is True. The spurious self-edges this injects into the
dependency graph built by __read_dep_graph() make find_circular_deps()
report cycles that do not exist in the direct dependency graph.
Restore the original semantics by not recursing into children when the
scope is Scope.Self.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
PY_ALL_PY is derived from PY_SRC_PY, which wildcards *.py in the module
directory, so a checked-in __init__.py enters the install list
unconditionally. PY_INSTALL_INIT_PY only gates the later append of
__init__.py, which covers files generated at build time. Setting it to
false thus had no effect on an existing __init__.py, which was still
installed together with its .pyc.
Filter __init__.py out of PY_ALL_PY. When PY_INSTALL_INIT_PY is true,
the existing append adds it back, so generated and checked-in
__init__.py files are installed as before. When it is false, an
existing __init__.py is now excluded from PY_ALL_PY and therefore also
from PY_INSTALLED_PY and PY_PYC.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__new_with_path() builds the new Uri by deep-copying self and then
replacing __string. A deep copy, however, also carries over any
cached_property values that were already computed on self (e.g. __p,
path, scheme, full), so once __string changes, they stay stale: for a
Uri on which any of those properties had been accessed before,
new_add_path() and new_replace_path() returned objects whose
to_string() showed the new string while path(), hostname(), full() and
friends still described the old one.
Build a fresh instance with object.__new__() and initialize its three
basic attributes instead of copying, so no computed cached state can
be inherited.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M and pi.dev
Signed-off-by: Jan Lindemann <jan@janware.com>
Add $(TOPDIR)/doc with some subdirectories:
- jw-pkg: Using the jw-pkg application, mirroring the subcommand
structure in its subdirectories. At this point only packages,
projects and secrets are added, all empty To be expanded as needed
- python: jw-pkg Python API
Signed-off-by: Jan Lindemann <jan@janware.com>
Add empty target "test" to keep the build from erroring out over make
test in documentation directories.
Signed-off-by: Jan Lindemann <jan@janware.com>
Unprivileged "make install" currently returns an error. That looks bad.
It actually is not a real problem, because unprivileged "make install"
installs files into locations that are good for nothing, in the sense
that they don't constitute a meaningful or usable hierarchy of installed
files. Debugging / inspecting the install machinery itself is a valid
use case, though, and it's easier to explain to the unsuspecting user
that unprivileged installation is something akin to shooting range
training than why "make install" outright errors out, so fix the error
messages.
Signed-off-by: Jan Lindemann <jan@janware.com>
cmd_log_install() defaults $c_logfile to "install.log", so every
invocation without -l appends to an install.log in the current
directory. That's semantical nonsense.
Default $c_logfile to empty instead. The existing empty-$c_logfile
early exit already skips logging, so no -l now means no log, and
callers that want a log pass -l explicitly, as the packaging flow
does through INSTALL_LOG.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M (pi)
Signed-off-by: Jan Lindemann <jan@janware.com>
In a development tree, $(INSTALL) falls back to plain install(1), which
refuses dead symlinks, so unprivileged "make install" fails outright.
Packaging is unaffected, because the spec exports INSTALL_LOG, which
already routes $(INSTALL) through "pkg.sh log-install"
Route the remaining $(INSTALL) defaults through $(LOG_INSTALL_SH) as
well to address the failures: $(SUDO) jw.pkg log-install -p for
DEVELOPMENT=false, plain jw.pkg log-install otherwise. Move -f $(PKG_FORMAT)
out of $(LOG_INSTALL_SH) and into the two logging call sites, so the
no-log paths don't carry a package format they never use.
Signed-off-by: Jan Lindemann <jan@janware.com>
Running "make clean" in some directories can break "make test" in
others. That notably happens by running clean somewhere in or below
$(TOPDIR)/src/python, which removes __init__.py files potentially needed
by unit and integration tests in $(TOPDIR)/test.
This commit makes the failure easily visible by introducing the notion
of "testabiltity" - if the tree is not testable, "make test" logs
exactly that in a clear error message and gives up.
Makefiles which need to check "testability" can use it as a
prerequisite. This commit does that with py-run.mk and jw-py-test.mk.
Testability is asserted by running "make", "make all" or "make test"
from $(TOPDIR). A successful toplevel build coincides with testability,
and leaves a $(TOPDIR)/dirs-all.done behind, which is why that file is
the perfect testability marker.
It is automatically created by a toplevel build and cleared by using the
"invalidate-testability" prerequisite. This commit makes target "clean"
depend on it in py-mod.mk. To be extended to other use cases / makefile
snippets as needed.
Signed-off-by: Jan Lindemann <jan@janware.com>
App.ResultCache is a horrible piece of software, now superseded by
functools.cache, with no measurable performance benefit as of now.
Remove it.
Signed-off-by: Jan Lindemann <jan@janware.com>
The discovery re-parse re-parses the full command line at each level
against the subparsers registered so far. A level's subcommand parsers
are not registered until the re-parse descends into them, so the tokens
after the subcommand name are parsed against the current level's
options. An option meant for a deeper level can then collide with a
same-named option of a shallower level, or fail on a missing value.
Replace the re-parse with a single argv walk. The subcommand names at
each level are known once the commands are materialized, so the invoked
path is found by matching tokens against those names and skipping the
options (and the values they consume) that precede them. Walking the
tokens never parses the tail against a half-built parser, so the
collision cannot occur.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
App() construction builds the entire command tree: every
load_subcommands() call in a command's __init__() constructs its whole
subtree eagerly, so running a single leaf command pays to instantiate
every unrelated command object as well (jw-pkg builds about 50 command
objects for any invocation).
Defer the construction instead. load_subcommands() now records only the
module search path and name filter, and the subcommands are materialized
on first access to the children or child_classes property. The parser
reads children only down the invoked branch on the non-help path, so a
simple run instantiates just that path (jw-pkg builds 5-7 objects),
while the help and completion path expands every node and leaves
rendered help unchanged.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Expose __root_cmd (the command which may or may not be mounted at the
root of the subcommand hierarchy) as _root_cmd() to derived classes.
This supports jw-ev's App base class, which specializes it to get hold
of jw.ev.app.Cmd's specific properties.
Signed-off-by: Jan Lindemann <jan@janware.com>
The application's top-level behavior is defined by overriding
App._add_arguments() and App._run(). The lightweight run-and-options
unit, Cmd, can already be mounted at any node of the command tree, but
the root is reserved for the application itself. An application that
wants to host a plain command at the top level therefore has to
subclass App and carry its full lifecycle implementation.
Add a root parameter to App.__init__(). When it is given a command
class, App instantiates it and uses it as the top level: the command's
options are registered on the top-level parser, it becomes the parent
of the top-level subcommands, and App._run() delegates the run to it.
The command's children are wired as the top-level subcommands, so the
same Cmd can now occupy the root node. When root is not given, the
previous auto-discovery behavior is preserved unchanged.
Keep the top-level subcommand heading as plain "Available subcommands"
whether it is hosted by the application or by a root command, while
nested command levels continue to qualify the heading with the parent
name. Add a unit test that mounts a root command hosting a child and
checks option registration, dispatch, setup and teardown, and
resolution of the application through the parent chain.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Code beautification: add_cmd_to_parser() isn't very telling about its
return type and the fact that it creates an object, hence the name
change. Also, annotate its argument with a private argparse type to
avoid a cast. My concern that argparse will break the private type at
some point in the future is outweighed by the gained clode clarity in
this function.
Signed-off-by: Jan Lindemann <jan@janware.com>
_build_parser() does an early parse_known_args() to configure logging
from the command line, but that call also enforces the top-level
required arguments. --help is registered only after the early parse so
the subcommands are present in the rendered help, so an app with a
required top-level positional, such as a root Cmd that takes a
config-file, exits with "required: config-file" on --help before the
final parse_args() in __run() can handle it.
Skip the early parse when help or shell completion is requested (the
add_all_parsers flag, already set for -h, --help, and argcomplete). The
subcommands and --help are still registered, and the final parse_args()
shows the help without enforcing the required arguments. The log-flag
configuration and the running-command debug line move inside the guard;
neither help nor completion logs, so they do not need them.
Assisted-by: pi <unsloth/Qwen3.8-27B-GGUF:Q4_K_M>
Signed-off-by: Jan Lindemann <jan@janware.com>
LoadTypes() declares mod_names as Iterable[str], so passing a single
module name as a bare string is not caught at call time and the string
is iterated character by character at load time, producing a
ModuleNotFoundError for the first character instead of a clear error.
Normalize a bare string to a one-element list in __init__() and widen
the annotation accordingly, so that both forms work.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't start a fresh project with version 1.0.0-0-dev. By default no
project can honestly claim the level of maturity suggested by 1.0.0-0
with its first commit.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the (untested) module lib.distros.redhat.Distro. It had been
lingering in the source tree, but I've apparently forgotten to add it
to Git. It has never seen any real use because CI still doesn't run
a RedHat distro, so it is to be regarded as a stub. Which is better
than nothing.
Signed-off-by: Jan Lindemann <jan@janware.com>
AsyncRunner is currently unused. This bug was detected and fixed by
AI.
asyncio.Event() raises RuntimeError on Python 3.12+ when created
outside a running event loop. It is created in the sync portion of
loop_in_thread(), before the threaded loop is up.
The fix is to move the Event creation inside the async main()
coroutine and pass it back via a second future. It replaces the
fragile as_completed loop with sequential result() calls, so failures
are immediately visible rather than causing a silent thread hang.
Assisted-by: unsloth/Qwen3.6-27B-MTP-GGUF:Q4_K_M with pi.dev v0.84.1
Signed-off-by: Jan Lindemann <jan@janware.com>
add_subcommands() advertises Cmd and list[Cmd] in its signature, but
a single Cmd instance raises NotImplementedError, and since the list
branch handles every element through the same method, a list of Cmd
instances is broken as well. The only working forms are Types and
lists of Types.
Handle a single Cmd instance by reparenting it to the caller and
appending it to the children, tracking its class like the class-based
path does. Instances whose name is already taken by a child are
rejected, mirroring the duplicate-class handling, because argparse
cannot register two subparsers under the same name.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
