Add the parameter "atomic" to put() / _put(). If instructs the implementation to take extra precautions to make sure the operation either succeeds or fails entirely, i.e. doesn't leave a broken target file behind.
.put() has some commands to _run(), and it uses its own CallContext for them. Since that pattern only replicates what run() does anyway, we could just as well use run() itself with less code, so do that.
cmd_input is passed as None to _run(), which is legal, but then used in a call to cmd_run(), which is a public API and, hence, illegal. InputMode.NonInteractive should be used instead, do that.
run_cmd() is a thin layer over the public ExecContext API, which falls back to using a Local instance if not other ExecContext is specified explicitly. Both the default Local context as the subsequent call to run() should have the same idea about interactivity, so allowing to specify it in two parameters ("interactive" and "cmd_input") is a bad idea. Remove "interactive".
Allow to configure via the environment which class ssh_client() picks. Can currently be exec, asyncssh, paramiko or a comma-separated search list. The list will be tried through until a class is found that can be instantiated.
Allow to configure logging of LoadTypes' decisions whether or not a class is elegible for loading. Currently, the respective log level is "off", allow to set it via JW_LOG_LEVEL_LOAD_TYPES in the environment.
App.distro_info() accepts and returns str instances, interpret anything passed as fmt parameter which is not a str as iterable, and return lists of expanded strings in that case.
Reusing AsyncSSH's connection is fine and fast, but only if it's not combined with the AsyncRunner. See commit 67e51cf0 why it was introduced in the first place, along with a reasoning why it may be a bad idea. Looks like we're now reaping what we sowed.
The current plan to get this to fly is to sprinkle async / await all over the code paths to App.os_release(). That is a lot of churn, so postpone and revert for now to keep CI working.
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 463, in _run_ssh
return await self._run_on_conn(
^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 403, in _run_on_conn
proc = await conn.create_process(
^^^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4492, in create_process
chan, process = await self.create_session(
^^^^^^^^^^^^^^^^^^^^^^^^^^
SSHClientProcess, *args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4385, in create_session
session = await chan.create(session_factory, command, subsystem,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
bool(self._agent_forward_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 1149, in create
packet = await self._open(b'session')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 717, in _open
return await self._open_waiter
^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Task <Task pending name='Task-1' coro=<App.__run() running at ~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/App.py:137> cb=[_run_until_complete_cb() at /usr/lib64/python3.13/asyncio/base_events.py:181]> got Future <Future pending> attached to a different loop
As of now, install() passes the "names" parameter on to _install(), which is expected to pass the list of package names on to the package manager for having it handle package download and installation. This commit makes it easier for Distro instances to support directly installing packages via an URL instead by providing a few callback methods to be overridden, in case the package manager doesn't handle package URLs the same way as package names.
Add class Curl as the first pure FileTransfer class without _run() / _sudo(). It doesn't use any PycURL / libcurl-like binding, but runs the curl binary in a subprocess. That looks the most portable still.
ExecContext has get() / _get() and put() / _put(), which make a fine API for a file transfer class. A class supporting file transfer should not, however, be forced to implement _run() and _sudo(), so place this functionality in a new class FileTransfer, and derive ExecContext from it.
For now, move the definiions of Result, Input and InputMode from ExecContext into lib.base. Having to import them from the ExecContect module is too heavy-handed for those simple types.
Add CmdCopy, designed to copy data from the filesystem to another location in the filesystem. Not necessarily local file systems, the URLs can be all URLs supported by ExecContext.run().
Add a new group of commands - "posix". The current command categories are not a good fit for that: "projects" is for CI, "distro" is distribution-specific for CD, and secrets is for handling secrets specifically. Introduce the more general command group "posix", a class of commands not POSIX compliant in the exposed API, but primarily using POSIX utilities as workhorse.
Add wrapper methods get() and put(), plus their wrapped methods _get() and _put(). The wrapped methods have default implementations, using POSIX utilities on the target machine over _run().
ExecContext.create() relies on properly formed URLs with a schema for deciding which backend gets created. Create a Local instance if an URL doesn't have schema.
The Input instance passed as cmd_input to ExecContext.run() and .sudo() currently may be of type str. Allow to pass bytes, too.
At the same time, disallow None to be passed as cmd_input. Force the caller to be more explicit how it wants input to be handled, notably with respect to interactivity.
Along the way fix a bug: Content in cmd_input should result in CallContext.interactive == False but doesn't. Fix that.
To have a pattern in lib.ExecContext and avoid future churn: If a
public wrapper calls a protected method, define the protected
method above the respective wrapper.
- sudo(): Make cmd_input default equal to run(): InputMode.OptInteractive
- CallContext: Expose parameters throw, wd, cmd as properties for
later use
