Cosmetics: The "packages" parameter to some DistroContext's methods has a confusing name in same contexts, notably when mixed with a list of Package instances, so rename it to "pkg_names".
Add an optional "filter: PackageFilter|None" parameter to .select(), and if it's not None, call a new version of ._select() with it.
._select() is not abstract anylonger. Its default implementation filters the results of ._select_by_name(), can be reimplemented by deriving classes for better performance, but doesn't have to.
Rename ._select() to _select_by_name() in Distro and its subclasses. Don't rename .select() itself, because it's going to be a broader interface supporting more select criteria than just package names.
Add global --pkg-filter argument, defaulting to JW_DEFAULT_PKG_FILTER. If it's specified, instantiate a PackageFilterString from it, and initialize App's Distro instance with it.
Add a default_pkg_filter parameter to Distro's constructor defaulting to None, and expose it via the .default_pkg_filter property. As of this commit, no code in jw-pkg does anything meaningful with it.
Commit a19679fec reverted the first attempt to make AsyncSSH reuse one connection during an instance lifetime. That failed because a lot of distribution-specific properties were filled in a new event loop thread started by AsyncRunner, and AsyncSSH didn't like that.
The last commit provided the needed properties as members of the Distro class. This commit is the second part of the solution: Keep one connection around as a class member and reuse it on every _run() invocation.
Commit a19679fec reverted the first attempt to make AsyncSSH reuse one connection during an instance lifetime. That failed because a lot of distribution-specific properties were filled in a new event loop thread started by AsyncRunner, and AsyncSSH didn't like that.
This commit is the first part of the solution: Move those properties from the App class to the Distro class, and load the Distro class in an async loader. As soon as it's instantiated, it can provide all its properties without cluttering the code with async keywords.
The name of the env parameter to ExecContext.run() and .sudo() is not descriptive enough for which environment is supposed to be modified and how, so rename and split it up as follows:
- .run(): env -> mod_env
- .sudo(): env -> mod_env_sudo and mod_env_cmd
The parameters have the following meaning:
- "mod_env*" means that the environment is modified, not replaced
- "mod_env" and "mod_env_cmd" modify the environment "cmd" runs in
- "mod_env_sudo" modifies the environment sudo runs in
Fix the fallout of the API change all over jw-pkg.
The "secrets" class of commands currently only works on the host it's invoked on. Use the current FileContext to allow using the existing commands on a target host.
All methods are async and call their protected counterpart, which is designed to be overridden. If possible, default implementations do something meaningful, if not, they just raise plain NotImplementedError.
Add the basic type StatResult. It is something akin to os.stat_result, but with user and group string members instead of st_uid and st_gid. The latter can't be expected to be stable across remote contexts.
Don't pass mode as a string to put(). Given the multitunde of possible string representations for numbers, some understood by int(string, 0) and some not, there's too much room for passing strings which are unparseable, or worse, prone to be parsed wrongly.
However, pass mode down to _put() as a string for convenience, because that's what most _put() implementations will need to use. If they don't, converting to int is easy from the one defined string format.
Add the parameter "atomic" to put() / _put(). If instructs the implementation to take extra precautions to make sure the operation either succeeds or fails entirely, i.e. doesn't leave a broken target file behind.
.put() has some commands to _run(), and it uses its own CallContext for them. Since that pattern only replicates what run() does anyway, we could just as well use run() itself with less code, so do that.
cmd_input is passed as None to _run(), which is legal, but then used in a call to cmd_run(), which is a public API and, hence, illegal. InputMode.NonInteractive should be used instead, do that.
run_cmd() is a thin layer over the public ExecContext API, which falls back to using a Local instance if not other ExecContext is specified explicitly. Both the default Local context as the subsequent call to run() should have the same idea about interactivity, so allowing to specify it in two parameters ("interactive" and "cmd_input") is a bad idea. Remove "interactive".
Allow to configure via the environment which class ssh_client() picks. Can currently be exec, asyncssh, paramiko or a comma-separated search list. The list will be tried through until a class is found that can be instantiated.
Allow to configure logging of LoadTypes' decisions whether or not a class is elegible for loading. Currently, the respective log level is "off", allow to set it via JW_LOG_LEVEL_LOAD_TYPES in the environment.
App.distro_info() accepts and returns str instances, interpret anything passed as fmt parameter which is not a str as iterable, and return lists of expanded strings in that case.
Reusing AsyncSSH's connection is fine and fast, but only if it's not combined with the AsyncRunner. See commit 67e51cf0 why it was introduced in the first place, along with a reasoning why it may be a bad idea. Looks like we're now reaping what we sowed.
The current plan to get this to fly is to sprinkle async / await all over the code paths to App.os_release(). That is a lot of churn, so postpone and revert for now to keep CI working.
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 463, in _run_ssh
return await self._run_on_conn(
^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 403, in _run_on_conn
proc = await conn.create_process(
^^^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4492, in create_process
chan, process = await self.create_session(
^^^^^^^^^^^^^^^^^^^^^^^^^^
SSHClientProcess, *args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4385, in create_session
session = await chan.create(session_factory, command, subsystem,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
bool(self._agent_forward_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 1149, in create
packet = await self._open(b'session')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 717, in _open
return await self._open_waiter
^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Task <Task pending name='Task-1' coro=<App.__run() running at ~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/App.py:137> cb=[_run_until_complete_cb() at /usr/lib64/python3.13/asyncio/base_events.py:181]> got Future <Future pending> attached to a different loop
