_run() receives the parsed arguments as its args parameter, but then
checks the private __args attribute for the func attribute and resolves
the command function through the args property. Both refer to the same
object today, so the mixing is harmless, but it obscures the data flow
and would silently diverge if a caller ever passed a namespace other
than the stored one.
Use the args parameter consistently in _run().
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__init__() builds the parser and the lazy subcommand registration
inside it decides which subcommands to register by re-parsing
sys.argv. run() then parses a different argv, so if the caller passes
an argv that is deeper than the one in sys.argv, the required
subparsers have not been registered and the invocation fails with an
"unrecognized arguments" error. run_sub_commands() passes argv to
run(), so the mismatch is reachable from the public API.
Move the parser construction from __init__() into _build_parser() and
call it from run() when an argv is given, so that registration and
parsing are driven by the same command line. The top-level command
instances are created once in __init__() and reused when the parser
is rebuilt.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
When run() creates an event loop, it installs it with
set_event_loop() but never restores the thread's previous loop, so
after run() returns, the thread is left with the now-closed loop
created by run(). Any code that calls get_event_loop() afterwards
gets a closed loop, and on Python 3.13+ a thread that had no loop at
all starts emitting or raising deprecation errors that run() caused.
Capture the thread's current loop with _get_current_event_loop()
before installing a new one, and restore it in the finally block. If
there was no previous loop, unset the loop with set_event_loop(None)
so that the thread is left without a loop instead of with the closed
one.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
close() closes the application's own event loop, but the AsyncRunner
is only released in the finally block of run(). An application that
creates a runner through call_async() and then calls close(), for
instance through the async context manager, therefore leaks the
runner, and close() does not fulfill its contract of releasing all
resources.
Move the AsyncRunner cleanup from the finally block of run() into
close() and reset the own-loop flag when the loop is closed, so that
close() releases everything and run() only has to call it.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__aenter__() and __aexit__() are empty stubs. Using the application
as an async context manager therefore binds None in the as clause,
and releases nothing on exit: an AsyncRunner created through
call_async() keeps running in its thread, and since that thread is
not a daemon, the process does not exit after the block.
Return self from __aenter__(), and call close() from __aexit__(), so
that the context manager binds the application and releases all
resources on exit, whether the block exits normally or with an
exception.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The --log-level and --log-flags options are added without a type
converter, so argparse never validates their values. _build_parser()
then hands the raw string to set_log_level() and set_log_flags() during
its first parse, and an unparseable value such as "INVALID" crashes
__init__() with a raw ValueError traceback instead of a usage error.
Pass type = parse_log_level() and type = parse_log_flags() when adding
the options, so that argparse reports invalid values with the standard
usage error and exit status 2. argparse only applies the converter to
command-line strings, so the int and LogFlag defaults are unaffected.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
asyncio.get_event_loop() is removed in Python 3.14 when called from
outside an async context. The current code calls it in __init__(),
which crashes on 3.14+.
To fix this, drop the eager loop creation from __init__(). Instead,
lazily create a loop in run() via asyncio.new_event_loop() when no
external loop was provided, and close it in the finally block. This
makes the lifecycle symmetric: run() owns the full create-use-close
cycle and supports re-entrant calls.
Replace __del__() with an explicit close() method, guarding against
double-close via is_closed(). close() always clears __eloop to None
so a closed loop never lingers.
Signed-off-by: Jan Lindemann <jan@janware.com>
The diff-all and diff targets diff all projects in the workspace
without filtering.
Add a target "diff-projects". It sets PGIT_SH_PROJECTS to the list of
projects from build-order, limiting the diff to the dependency
closure around $(PROJECTS).
Signed-off-by: Jan Lindemann <jan@janware.com>
If /usr/bin/isort is found, run it during "make format" to get a
defined way the imports are sorted. tool.isort in pyproject.toml is
updated to match the other fixers.
Commit the fallout of this change. Running the other fixers alone
doesn't change the formatting, so this should be safe.
Signed-off-by: Jan Lindemann <jan@janware.com>
The __init__() method reads default log configuration directly
from environment variables, making it impossible for subclasses to
change defaults.
Extract the defaults into _default_log_flags(), _default_log_level(),
_default_log_file(), and _default_show_backtrace() methods, then call
them from __init__(). Subclasses can now override these methods to
customize defaults without needing to override the entire __init__()
method.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add _default_*_env() helper methods that return the environment variable
names, and use them in __init__() instead of hard-coded strings. This
allows subclasses to override the names.
Signed-off-by: Jan Lindemann <jan@janware.com>
Replace the set of flag strings with a Flag enum for better type
safety and bitwise operations. Add parse_flags() to convert
comma-separated strings into a LogFlag value. Update set_flags() and
set_log_flags() to accept str | LogFlag | None.
set_flags() and set_log_flags() don't return str anylonger which is a
breaking change, but shouldn't be a problem because it's not used
anywhere.
Signed-off-by: Jan Lindemann <jan@janware.com>
The argument string builder in log() previously iterated with
enumerate and checked per-iteration whether to prepend a space.
Replace the loop with ' '.join(str(a) for a in args), which is a
single C-level operation.
Move the leading-space logic after the only_printable block so the
regex transformations see the raw joined content.
Signed-off-by: Jan Lindemann <jan@janware.com>
When prefix flags (position, prio, date) are combined with message
arguments, log() would double-space the output (e.g. '<N> Created')
or omit the separator entirely (e.g. '<N>Created'). The root cause
was that log() prepended a space to every argument, regardless of
whether the prefix already ended with one.
Fix: only prepend a space before the first argument when the prefix
doesn't already end with one. This produces exactly one separator
between prefix and content regardless of which flags are active.
Also fix log_m(): skip empty strings (the sentinel '') so it doesn't
contribute a space. And fix the early return that dropped messages
when no prefix flags were set (previous commit).
Signed-off-by: Jan Lindemann <jan@janware.com>
When position, prio, and date are all absent from the log flags, the
msg variable is empty. The early return 'if not len(msg): return'
would then skip printing the actual message in margs.
Fix by checking both msg and margs before returning. This ensures
messages are still printed even when no prefix flags are set.
Signed-off-by: Jan Lindemann <jan@janware.com>
Fix fallout created by enabling the "strict" option:
lib/App.py:196: error: Class cannot subclass
"BaseCompleter" (has type "Any") [misc]
Signed-off-by: Jan Lindemann <jan@janware.com>
Add type annotations and casts to functions that were returning Any
where a specific type was declared, satisfying the new warn_return_any
mypy rule.
Fixes:
- log.py: get_caller_pos return type via cast
- AsyncRunner.py: cast T for fut.result()
- util.py: cast for getattr result, str() for args.username
- FileContext.py: verbose_default bool annotation
- SSHClient.py: cast SSHClient for dynamic import
- lib/App.py: cast ArgumentParser, add return types to inner funcs
- pm/rpm.py, dpkg.py: cast Iterable[Package]
- App.py: cast for self.args.func(), add return types to inner funcs
- BaseCmdPkgRelations.py: cast str for args.delimiter
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Add missing type annotations to functions that are called from typed
contexts, satisfying the new disallow_untyped_calls mypy rule.
Fixes:
- Local.py: __log() with typed parameters
- lib/App.py: _add_arguments(), add_cmd_to_parser(), add_cmds_to_parser()
- pm/rpm.py, dpkg.py: meta_map() return type
- Exec.py: __init_askpass() return type
- App.py: strip_module_from_spec(), __get_project_refs_cached(),
ResultCache.__init__ and run(), _add_arguments()
- Added Collection type for truthy-iterable compliance
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Add 'case _: pass' to match statements that are intentionally
non-exhaustive, satisfying the new exhaustive-match mypy rule.
Also replaced 'case '_':' (a string literal) with 'case _: pass' in
pkg_relations.py since it was an unreachable case (syntax is a
VersionSyntax enum, not a str).
Added 'case VersionSyntax.names_only:' to the match in pkg_relations.py
to handle the missing enum value.
Files modified:
- util.py: Two match statements for askpass env vars
- Distro.py: Three match statements for backend/os detection
- pkg_relations.py: Match on VersionSyntax enum
- CmdListRepos.py: Match on URL scheme
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Change parameter types from Iterable[str] to Collection[str] wherever
the parameter is tested for emptiness (if not names). This satisfies
the new truthy-iterable mypy rule, since bare Iterable values are
always truthy even when empty.
Affected files:
- Distro.py: install, delete, select, _select, _select_by_name
- rpm.py: query_packages
- suse/Distro.py: _select_by_name
- Cmd.py (secrets): _match_files, _list_template_files, etc.
- DistroContext.py: list_template_files, list_secret_paths, etc.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Some '# type: ignore' comments are needed because they complain about missing
but optional third-party packages: argcomplete, paramiko, asyncssh. The next
commit will enable warn_unused_ignores, and since nor mypy nor pyright have a
way of knowing that this is a tolerable lack of packages, this commit teaches
them in advance.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add explicit type arguments to all generic type annotations that were
previously bare, satisfying the new disallow_any_generics mypy rule.
Fixes:
- Distro.py: Iterable[str] for expand_macros fmt parameter
- Cmd.py: Types[Any] for add_subcommands cmds parameter
- AsyncSSH.py: dict[str, Any] for _connect_kwargs return type
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The comparison ret != 0 is comparing a Result object against the
integer 0, which is always True since Result has no __eq__ defined.
This commit fixes the bug by comparing ret.status instead, which is
the actual exit code we care about. This satisfies the new
strict_equality mypy rule which enables the comparison-overlap check.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The cast to 'T' in run_curl_into was redundant: mypy already narrows
the return value of json.loads() to type T after the isinstance(ret,
expected_type) check (where expected_type: type[T]).
Remove the unnecessary cast and clean up the unused 'cast' import.
This satisfies the new warn_redundant_casts mypy rule.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The _select() method in Distro.py has an assert on a PackageFilter
parameter that doesn't define __bool__ or __len__. Since the type
signature already guarantees it is not None, the assertion is
redundant and is now removed.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The log_m() function has margs conditionally assigned inside an if
block but used unconditionally afterwards. Initialize margs outside
the if to guarantee it is always defined.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Annotate the one existing bare "# type: ignore" in AsyncRunner.py
with arg-type and var-annotated codes to prepare for
mypy.ignore-without-code.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
This commit adds @override decorators to approximately 300 methods
across 76 files that inherit from base classes such as AbstractCmd,
FileContext, ExecContext, Distro, SSHClient, and others.
The decorator ensures the type checker can verify that overridden
methods have compatible signatures and prevents accidental shadowing
of inherited methods without intent.
Files modified include command classes, library modules, distro
implementations, and SSH client implementations.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v
Signed-off-by: Jan Lindemann <jan@janware.com>
A couple of "# export" markers after function prototypes have been
pushed along with the closing parenthesis onto the wrong line by the
code formatter, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Tolera$te missing paramiko imports. jw-pkg is designed to
work with what it finds and use plain /usr/bin/ssh if need
be.
Signed-off-by: Jan Lindemann <jan@janware.com>
An Uri instance constructed from "~/some/path" returns
"file://~/some/path" as the .full property, i.e. with the "~" in the
authority part. That's not accurate and doesn't make the intended
sense in an URL context, see RFC 8089:
Common UNIX shells such as the Bourne-Again SHell (bash) and Z
SHell (zsh) provide a function known as "tilde expansion"
[Bash-Tilde] or "filename expansion" [Zsh-Tilde], where a path
that begins with a tilde character "~" can be expanded out to a
special directory name. No such facility exists using the file
URI scheme; a tilde in a file URI is always just a tilde.
The "fix" introduced by this commit makes .full return the path
without file:// prepended. That's conservative. It could also chose
to expand the tilde, which is arguably cleaner. To be introduced by a
later change after this commit has seen more coverage.
Signed-off-by: Jan Lindemann <jan@janware.com>
In non-interactive mode, "zypper dup" runs with the options
--force-resolution and --auto-agree-with-licenses duplicated, fix
that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Two error messages in Distro.__init__() contain the typo 'instaniate'
instead of 'instantiate'. Fix the spelling.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The scheme property returns 'file://' for local paths instead of
'file', causing scheme_plus_authority to produce 'file://://' with a
triple slash.
Also fix the test to expect 'file' instead of 'file://'.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The CopyContext._run() calls await self._run(), which would create
infinite recursion raising RecursionError at runtime. Make it a pure
virtual.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Introduce ProjectConf module to cleanly parse make/project.conf ini-like
configuration files. The new class supports:
- ini-style sections with header comments
- Key-value pairs with backslash line continuation
- Quoted values preserving spaces and comment delimiters (#) inside
- Inline comments outside of quotes
- Comma-separated list values with quoted commas
- Cached section parsing to avoid re-parsing the same section
- .get_section() to return an entire section unparsed
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a property aliases to AbstractCmd in prepeparation for commands
to bear multiple names / abbreviations / aliases.
The App.add_cmds_to_parser() function uses parse_known_args() to
determine which subcommand was invoked, then conditionally registers
nested subcommands. The lookup dictionary (scs) contains only
canonical names, not aliases, so add them too, otherwise using the
alias instead of the canonical name causes the lookup to fail and
nested subcommands to never be registered.
Fix: Register each alias in scs pointing to the same SubCommand
object, and deduplicate with id(sc) when iterating in all=True mode
to avoid infinite recursion on help output.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit adds more tweaks to shell command output in order to make
it nicer. The biggest patch is in Result.__summarize(), which makes
it more versatile, and allows removal of some code in SSHClient.
App sees some independent, minor result format beautification.
Signed-off-by: Jan Lindemann <jan@janware.com>
Overriding the _run() method entirely in App subclasses is currently
only possible if the application supports a subcommand structure.
Make it possible to use it as an abstraction for a single-command
application.
Signed-off-by: Jan Lindemann <jan@janware.com>
The .strip property of class Result defaults to True, and its name
isn't very clear. Rename it to .strip_output, and default it to False
to avoid surprising contents for unsuspecting callers.
Signed-off-by: Jan Lindemann <jan@janware.com>
The command line jw-pkg.py is run with is logged with level "debug",
and reconstructed with ' '.join(sys.argv). Use pretty_cmd() instead,
this adds quotes around spaces.
Signed-off-by: Jan Lindemann <jan@janware.com>
Commands executed by ExecContext and its derived classes don't
populate the "cmd" parameter of "Result"'s constructor. Fixing that
makes for nicer error messages.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the (mostly) call-compatible log implementation from jw-python.
This buys us much nicer logs with color and source-code positon
annotations.
Signed-off-by: Jan Lindemann <jan@janware.com>
In LoadTypes' constructor, allow the type_filter parameter to be of
type Sequence[type[Any]] instead of list[type[T]]. a) Sequence is
more generic than list, and b) with T instead of Any, trying to
instantiate with an abstract class has mypy complain:
# E: Only concrete class can be given where "type[MyClass]" is
expected [type-abstract]
- type_filter: list[type[T]] = [],
+ type_filter: Sequence[type[Any]] | None = None,
Not that this makes mypy complain that it needs an annotation at the
places where LoadTypes is used.
Signed-off-by: Jan Lindemann <jan@janware.com>
OpenSUSE leaves installing local packages to the default
implementation in lib.Distro._install_local_files(), which passes the
package path to the package manager, i.e. zypper in OpenSUSE's case.
That has advantages, namely automatic installation of dependencies,
but also disadvantages, namely the attempt to install dependencies
even if the package manager is disfunctional, possibly because an
installed package containing installation sources is broken.
That could lead to a deadlock when trying to install a fixed package.
I see two ways out: Support an additional flag to jw-pkg's install
command which selects whether or not dependencies shall be resolved
along, or just use rpm directly for all local install attempts.
The latter is the less fancy way to handle this, so as a first step
make it the default by overriding suse.Distro._install_local_files().
Signed-off-by: Jan Lindemann <jan@janware.com>
Define default parameter values for Result's constructor, namely None
for exit status, stdout and stderr.
Instantiating a Result object without parameters signifies "this
object doesn't contain data from a real process's exit event". Up to
now, similar meaning has been hand-crafted by ExecContext's run() and
friends by using an error exit status (1) to make sure it wasn't
mistaken for success. This commit formalizes that into the Result
structure itself, but uses None instead for the exit status.
Controlling default values in Result itself also means that the
Result class gets better awareness of what it contains, and its log
messages and stdin / stdout can be more fitting:
- If a real process failed, make stdout return at least b''
- If a real process succeeded, make stdout return at least b''
Returning something from .stdout on success fixes a real bug: An
attempt to access what "rpm -U somepackage.rpm" returns, namely
nothing, raises a bogus exception, because stdout is None.
Signed-off-by: Jan Lindemann <jan@janware.com>
