Fix fallout created by enabling the "strict" option:
lib/App.py:196: error: Class cannot subclass
"BaseCompleter" (has type "Any") [misc]
Signed-off-by: Jan Lindemann <jan@janware.com>
Add type annotations and casts to functions that were returning Any
where a specific type was declared, satisfying the new warn_return_any
mypy rule.
Fixes:
- log.py: get_caller_pos return type via cast
- AsyncRunner.py: cast T for fut.result()
- util.py: cast for getattr result, str() for args.username
- FileContext.py: verbose_default bool annotation
- SSHClient.py: cast SSHClient for dynamic import
- lib/App.py: cast ArgumentParser, add return types to inner funcs
- pm/rpm.py, dpkg.py: cast Iterable[Package]
- App.py: cast for self.args.func(), add return types to inner funcs
- BaseCmdPkgRelations.py: cast str for args.delimiter
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Add missing type annotations to functions that are called from typed
contexts, satisfying the new disallow_untyped_calls mypy rule.
Fixes:
- Local.py: __log() with typed parameters
- lib/App.py: _add_arguments(), add_cmd_to_parser(), add_cmds_to_parser()
- pm/rpm.py, dpkg.py: meta_map() return type
- Exec.py: __init_askpass() return type
- App.py: strip_module_from_spec(), __get_project_refs_cached(),
ResultCache.__init__ and run(), _add_arguments()
- Added Collection type for truthy-iterable compliance
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Add 'case _: pass' to match statements that are intentionally
non-exhaustive, satisfying the new exhaustive-match mypy rule.
Also replaced 'case '_':' (a string literal) with 'case _: pass' in
pkg_relations.py since it was an unreachable case (syntax is a
VersionSyntax enum, not a str).
Added 'case VersionSyntax.names_only:' to the match in pkg_relations.py
to handle the missing enum value.
Files modified:
- util.py: Two match statements for askpass env vars
- Distro.py: Three match statements for backend/os detection
- pkg_relations.py: Match on VersionSyntax enum
- CmdListRepos.py: Match on URL scheme
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Change parameter types from Iterable[str] to Collection[str] wherever
the parameter is tested for emptiness (if not names). This satisfies
the new truthy-iterable mypy rule, since bare Iterable values are
always truthy even when empty.
Affected files:
- Distro.py: install, delete, select, _select, _select_by_name
- rpm.py: query_packages
- suse/Distro.py: _select_by_name
- Cmd.py (secrets): _match_files, _list_template_files, etc.
- DistroContext.py: list_template_files, list_secret_paths, etc.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Some '# type: ignore' comments are needed because they complain about missing
but optional third-party packages: argcomplete, paramiko, asyncssh. The next
commit will enable warn_unused_ignores, and since nor mypy nor pyright have a
way of knowing that this is a tolerable lack of packages, this commit teaches
them in advance.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add explicit type arguments to all generic type annotations that were
previously bare, satisfying the new disallow_any_generics mypy rule.
Fixes:
- Distro.py: Iterable[str] for expand_macros fmt parameter
- Cmd.py: Types[Any] for add_subcommands cmds parameter
- AsyncSSH.py: dict[str, Any] for _connect_kwargs return type
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The comparison ret != 0 is comparing a Result object against the
integer 0, which is always True since Result has no __eq__ defined.
This commit fixes the bug by comparing ret.status instead, which is
the actual exit code we care about. This satisfies the new
strict_equality mypy rule which enables the comparison-overlap check.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The cast to 'T' in run_curl_into was redundant: mypy already narrows
the return value of json.loads() to type T after the isinstance(ret,
expected_type) check (where expected_type: type[T]).
Remove the unnecessary cast and clean up the unused 'cast' import.
This satisfies the new warn_redundant_casts mypy rule.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The _select() method in Distro.py has an assert on a PackageFilter
parameter that doesn't define __bool__ or __len__. Since the type
signature already guarantees it is not None, the assertion is
redundant and is now removed.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The log_m() function has margs conditionally assigned inside an if
block but used unconditionally afterwards. Initialize margs outside
the if to guarantee it is always defined.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Annotate the one existing bare "# type: ignore" in AsyncRunner.py
with arg-type and var-annotated codes to prepare for
mypy.ignore-without-code.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL and pi.dev 0.81.1
This commit adds @override decorators to approximately 300 methods
across 76 files that inherit from base classes such as AbstractCmd,
FileContext, ExecContext, Distro, SSHClient, and others.
The decorator ensures the type checker can verify that overridden
methods have compatible signatures and prevents accidental shadowing
of inherited methods without intent.
Files modified include command classes, library modules, distro
implementations, and SSH client implementations.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v
Signed-off-by: Jan Lindemann <jan@janware.com>
A couple of "# export" markers after function prototypes have been
pushed along with the closing parenthesis onto the wrong line by the
code formatter, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Tolera$te missing paramiko imports. jw-pkg is designed to
work with what it finds and use plain /usr/bin/ssh if need
be.
Signed-off-by: Jan Lindemann <jan@janware.com>
An Uri instance constructed from "~/some/path" returns
"file://~/some/path" as the .full property, i.e. with the "~" in the
authority part. That's not accurate and doesn't make the intended
sense in an URL context, see RFC 8089:
Common UNIX shells such as the Bourne-Again SHell (bash) and Z
SHell (zsh) provide a function known as "tilde expansion"
[Bash-Tilde] or "filename expansion" [Zsh-Tilde], where a path
that begins with a tilde character "~" can be expanded out to a
special directory name. No such facility exists using the file
URI scheme; a tilde in a file URI is always just a tilde.
The "fix" introduced by this commit makes .full return the path
without file:// prepended. That's conservative. It could also chose
to expand the tilde, which is arguably cleaner. To be introduced by a
later change after this commit has seen more coverage.
Signed-off-by: Jan Lindemann <jan@janware.com>
In non-interactive mode, "zypper dup" runs with the options
--force-resolution and --auto-agree-with-licenses duplicated, fix
that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Two error messages in Distro.__init__() contain the typo 'instaniate'
instead of 'instantiate'. Fix the spelling.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The scheme property returns 'file://' for local paths instead of
'file', causing scheme_plus_authority to produce 'file://://' with a
triple slash.
Also fix the test to expect 'file' instead of 'file://'.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
The CopyContext._run() calls await self._run(), which would create
infinite recursion raising RecursionError at runtime. Make it a pure
virtual.
Assisted-by: unsloth/Qwen3.6-35B-A3B-GGUF:IQ4_NL with pi.dev v0.80.2
Signed-off-by: Jan Lindemann <jan@janware.com>
Introduce ProjectConf module to cleanly parse make/project.conf ini-like
configuration files. The new class supports:
- ini-style sections with header comments
- Key-value pairs with backslash line continuation
- Quoted values preserving spaces and comment delimiters (#) inside
- Inline comments outside of quotes
- Comma-separated list values with quoted commas
- Cached section parsing to avoid re-parsing the same section
- .get_section() to return an entire section unparsed
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a property aliases to AbstractCmd in prepeparation for commands
to bear multiple names / abbreviations / aliases.
The App.add_cmds_to_parser() function uses parse_known_args() to
determine which subcommand was invoked, then conditionally registers
nested subcommands. The lookup dictionary (scs) contains only
canonical names, not aliases, so add them too, otherwise using the
alias instead of the canonical name causes the lookup to fail and
nested subcommands to never be registered.
Fix: Register each alias in scs pointing to the same SubCommand
object, and deduplicate with id(sc) when iterating in all=True mode
to avoid infinite recursion on help output.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit adds more tweaks to shell command output in order to make
it nicer. The biggest patch is in Result.__summarize(), which makes
it more versatile, and allows removal of some code in SSHClient.
App sees some independent, minor result format beautification.
Signed-off-by: Jan Lindemann <jan@janware.com>
Overriding the _run() method entirely in App subclasses is currently
only possible if the application supports a subcommand structure.
Make it possible to use it as an abstraction for a single-command
application.
Signed-off-by: Jan Lindemann <jan@janware.com>
The .strip property of class Result defaults to True, and its name
isn't very clear. Rename it to .strip_output, and default it to False
to avoid surprising contents for unsuspecting callers.
Signed-off-by: Jan Lindemann <jan@janware.com>
The command line jw-pkg.py is run with is logged with level "debug",
and reconstructed with ' '.join(sys.argv). Use pretty_cmd() instead,
this adds quotes around spaces.
Signed-off-by: Jan Lindemann <jan@janware.com>
Commands executed by ExecContext and its derived classes don't
populate the "cmd" parameter of "Result"'s constructor. Fixing that
makes for nicer error messages.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the (mostly) call-compatible log implementation from jw-python.
This buys us much nicer logs with color and source-code positon
annotations.
Signed-off-by: Jan Lindemann <jan@janware.com>
In LoadTypes' constructor, allow the type_filter parameter to be of
type Sequence[type[Any]] instead of list[type[T]]. a) Sequence is
more generic than list, and b) with T instead of Any, trying to
instantiate with an abstract class has mypy complain:
# E: Only concrete class can be given where "type[MyClass]" is
expected [type-abstract]
- type_filter: list[type[T]] = [],
+ type_filter: Sequence[type[Any]] | None = None,
Not that this makes mypy complain that it needs an annotation at the
places where LoadTypes is used.
Signed-off-by: Jan Lindemann <jan@janware.com>
OpenSUSE leaves installing local packages to the default
implementation in lib.Distro._install_local_files(), which passes the
package path to the package manager, i.e. zypper in OpenSUSE's case.
That has advantages, namely automatic installation of dependencies,
but also disadvantages, namely the attempt to install dependencies
even if the package manager is disfunctional, possibly because an
installed package containing installation sources is broken.
That could lead to a deadlock when trying to install a fixed package.
I see two ways out: Support an additional flag to jw-pkg's install
command which selects whether or not dependencies shall be resolved
along, or just use rpm directly for all local install attempts.
The latter is the less fancy way to handle this, so as a first step
make it the default by overriding suse.Distro._install_local_files().
Signed-off-by: Jan Lindemann <jan@janware.com>
Define default parameter values for Result's constructor, namely None
for exit status, stdout and stderr.
Instantiating a Result object without parameters signifies "this
object doesn't contain data from a real process's exit event". Up to
now, similar meaning has been hand-crafted by ExecContext's run() and
friends by using an error exit status (1) to make sure it wasn't
mistaken for success. This commit formalizes that into the Result
structure itself, but uses None instead for the exit status.
Controlling default values in Result itself also means that the
Result class gets better awareness of what it contains, and its log
messages and stdin / stdout can be more fitting:
- If a real process failed, make stdout return at least b''
- If a real process succeeded, make stdout return at least b''
Returning something from .stdout on success fixes a real bug: An
attempt to access what "rpm -U somepackage.rpm" returns, namely
nothing, raises a bogus exception, because stdout is None.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make the log delimiter look more consistent: Whether a CallContext
was constructed with a title parameter or without, prefix its
.log_delimiter property with a "----".
Signed-off-by: Jan Lindemann <jan@janware.com>
Ignore newline at the end of Result.stdout_str if only one line of
output is wanted from an executed shell command. The output of both
uname and mktemp are used wrongly in that regard.
Signed-off-by: Jan Lindemann <jan@janware.com>
- Remove package_name and package_path from the prototype of
detect_modules(). They can and should be deduced from
namespace['__name__'] and namespace['__path__'], respectively.
- Make prefix default to None, which signifies "Don't filter by
prefix".
- Add an optional extend_namespace parameter, which will make the
function append the module's __name__ to its __path__. This
defaults to True, thereby adding a side effect to the function.
Which is always wanted in the case for all callers of this
function.
Signed-off-by: Jan Lindemann <jan@janware.com>
.lib.Result has grown enourmously in size and merits its own module.
For now, reexport it from .lib.base to not break all code containing
"from jw.lib.base import Result"
Signed-off-by: Jan Lindemann <jan@janware.com>
Accept if AsyncSSH is missing. The package would be nice to have,
i.e. a good candidate for a "recommends" section, but until there's
support for that, better be able to do without and fall back to
command-line ssh.
Signed-off-by: Jan Lindemann <jan@janware.com>
Accept if argcomplete is missing. The package would be nice to have,
i.e. a good candidate for a "recommends" section, but until there's
support for that, better be able to do without.
Signed-off-by: Jan Lindemann <jan@janware.com>
Fix another regression of commit 6db73873e7:
lib.ExecContext.CallContext.__exit__() returns True, which swallows
all exceptions thrown in the context of _run() and _sudo(). Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Updates running in non-interactive mode are passed
--force-resolution
--auto-agree-with-licenses
to get more snakes out of the way, as recently during CI:
# make pkg-install-testbuild-deps
/usr/bin/which: no xdg-open in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
/usr/bin/python3.13 ./scripts/jw-pkg.py -p ./.. -t .
--topdir-format absolute --interactive=false pkg install "make"
"time" "xdg-utils" "coreutils" "cpio" "git-core" "bash" "python3"
"sudo" "gawk" "pkg-config" "python3-isort" "python3-yapf"
"python3-ruff" "python3-pyright" "rpmbuild" "python3-base"
,---- file://local: Running /usr/bin/zypper --non-interactive --gpg-auto-import-keys --no-gpg-checks install make time xdg-utils coreutils cpio git-core bash python3 sudo gawk pkg-config python3-isort python3-yapf python3-ruff python3-pyright rpmbuild python3-base - >
| Loading repository data...
| Reading installed packages...
| 'sudo' is already installed.
| No update candidate for 'sudo-1.9.17p2-2.2.x86_64'. The highest available version is already installed.
| 'bash' is already installed.
| No update candidate for 'bash-5.3.9-6.4.x86_64'. The highest available version is already installed.
| 'python3' not found in package names. Trying capabilities.
| 'python313' providing 'python3' is already installed.
| 'coreutils' is already installed.
| No update candidate for 'coreutils-9.11-3.1.x86_64'. The highest available version is already installed.
| 'pkg-config' not found in package names. Trying capabilities.
| 'make' is already installed.
| No update candidate for 'make-4.4.1-3.5.x86_64'. The highest available version is already installed.
| 'python3-base' not found in package names. Trying capabilities.
| 'python313-base' providing 'python3-base' is already installed.
| 'rpmbuild' not found in package names. Trying capabilities.
| 'git-core' is already installed.
| No update candidate for 'git-core-2.54.0-2.1.x86_64'. The highest available version is already installed.
| 'python3-pyright' not found in package names. Trying capabilities.
| 'python3-ruff' not found in package names. Trying capabilities.
| 'python3-isort' not found in package names. Trying capabilities.
| 'python3-yapf' not found in package names. Trying capabilities.
| Resolving package dependencies...
|
| Problem: 1: the installed busybox-gawk-1.37.0-41.4.noarch conflicts with 'gawk' provided by the to be installed gawk-5.4.0-1.1.x86_64
| Solution 1: Following actions will be done:
| do not install gawk-5.4.0-1.1.x86_64
| do not ask to install a solvable providing rpmbuild
| Solution 2: deinstallation of busybox-gawk-1.37.0-41.4.noarch
|
| Choose from above solutions by number or cancel [1/2/c/d/?] (c): c
`---- file://local: Running /usr/bin/zypper --non-interactive --gpg-auto-import-keys --no-gpg-checks install make time xdg-utils coreutils cpio git-core bash python3 sudo gawk pkg-config python3-isort python3-yapf python3-ruff python3-pyright rpmbuild python3-base - <
Signed-off-by: Jan Lindemann <jan@janware.com>
Fix a regregression breaking run_curl() / run_curl_into(), introduced
by commit 6db73873e7. A missing indentation raises a non-existing
Error after successful JSON parsing, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
