Apply some style changes:
- Replace double by single quotes for consistency
- Add spaces around equal signs in long parameter lists
Signed-off-by: Jan Lindemann <jan@janware.com>
The name of the env parameter to ExecContext.run() and .sudo() is not
descriptive enough for which environment is supposed to be modified
and how, so rename and split it up as follows:
- .run(): env -> mod_env
- .sudo(): env -> mod_env_sudo and mod_env_cmd
The parameters have the following meaning:
- "mod_env*" means that the environment is modified, not replaced
- "mod_env" and "mod_env_cmd" modify the environment "cmd" runs in
- "mod_env_sudo" modifies the environment sudo runs in
Fix the fallout of the API change all over jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the property .username, backed by the protected _username()
callback. It should return the user run()'s cmd parameter is executed
as.
Signed-off-by: Jan Lindemann <jan@janware.com>
The "secrets" class of commands currently only works on the host it's
invoked on. Use the current FileContext to allow using the existing
commands on a target host.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the following methods, meant to do the obvious:
unlink(self, path: str) -> None
erase(self, path: str) -> None
rename(self, src: str, dst: str) -> None
mktemp(self, tmpl: str, directory: bool=False) -> None
chown(self, path: str, owner: str|None=None, group: str|None=None) -> None
chmod(self, path: str, mode: int) -> None
stat(self, path: str, follow_symlinks: bool=True) -> StatResult
file_exists(self, path: str) -> bool
is_dir(self, path: str) -> bool
All methods are async and call their protected counterpart, which is
designed to be overridden. If possible, default implementations do
something meaningful, if not, they just raise plain
NotImplementedError.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the basic type StatResult. It is something akin to
os.stat_result, but with user and group string members instead of
st_uid and st_gid. The latter can't be expected to be stable across
remote contexts.
Signed-off-by: Jan Lindemann <jan@janware.com>
Pass LC_ALL="C" to _run() by default. This is necessary to be able to
parse error messages and raise FileNotFound if need be.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't pass mode as a string to put(). Given the multitunde of
possible string representations for numbers, some understood by
int(string, 0) and some not, there's too much room for passing
strings which are unparseable, or worse, prone to be parsed wrongly.
However, pass mode down to _put() as a string for convenience,
because that's what most _put() implementations will need to use. If
they don't, converting to int is easy from the one defined string
format.
Signed-off-by: Jan Lindemann <jan@janware.com>
Swap the positions of the "path" and "content" parameters of put().
Path comes always first, in every path related function I know.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the parameter "atomic" to put() / _put(). If instructs the
implementation to take extra precautions to make sure the operation
either succeeds or fails entirely, i.e. doesn't leave a broken target
file behind.
Signed-off-by: Jan Lindemann <jan@janware.com>
.put() has some commands to _run(), and it uses its own CallContext
for them. Since that pattern only replicates what run() does anyway,
we could just as well use run() itself with less code, so do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Prepend the class name to .log_name. Not sure if that makes logs more
legible, but we'll try it out for a while.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename class FileTransfer to FileContext because that's the better
name. It's the base class of ExecContext and also a context.
Signed-off-by: Jan Lindemann <jan@janware.com>
An env argument environment passed to Local._run() entirely replaces
the environment. Make it modify the enviroment instead.
Signed-off-by: Jan Lindemann <jan@janware.com>
AsyncSSH's implementation already supports modifying the execution
environment via env, so declare it to the base class with Caps.Env.
Signed-off-by: Jan Lindemann <jan@janware.com>
Exec's _run_ssh() ignores its "interactive" parameter and uses the
instances' default instead, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
cmd_input is passed as None to _run(), which is legal, but then used
in a call to cmd_run(), which is a public API and, hence, illegal.
InputMode.NonInteractive should be used instead, do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_cmd() is a thin layer over the public ExecContext API, which
falls back to using a Local instance if not other ExecContext is
specified explicitly. Both the default Local context as the
subsequent call to run() should have the same idea about
interactivity, so allowing to specify it in two parameters
("interactive" and "cmd_input") is a bad idea. Remove "interactive".
Signed-off-by: Jan Lindemann <jan@janware.com>
Allow to configure via the environment which class ssh_client()
picks. Can currently be exec, asyncssh, paramiko or a comma-separated
search list. The list will be tried through until a class is found
that can be instantiated.
Signed-off-by: Jan Lindemann <jan@janware.com>
Allow to configure logging of LoadTypes' decisions whether or not a
class is elegible for loading. Currently, the respective log level is
"off", allow to set it via JW_LOG_LEVEL_LOAD_TYPES in the
environment.
Signed-off-by: Jan Lindemann <jan@janware.com>
cmd_input is passed as None to _run(), which is legal, but then used
in a call to cmd_run(), which is a public API and, hence, illegal.
Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
App.distro_info() accepts and returns str instances, interpret
anything passed as fmt parameter which is not a str as iterable, and
return lists of expanded strings in that case.
Signed-off-by: Jan Lindemann <jan@janware.com>
This reverts commit 04fef1e67a.
Reusing AsyncSSH's connection is fine and fast, but only if it's not
combined with the AsyncRunner. See commit 67e51cf0 why it was
introduced in the first place, along with a reasoning why it may be a
bad idea. Looks like we're now reaping what we sowed.
The current plan to get this to fly is to sprinkle async / await all
over the code paths to App.os_release(). That is a lot of churn, so
postpone and revert for now to keep CI working.
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 463, in _run_ssh
return await self._run_on_conn(
^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/ec/ssh/AsyncSSH.py", line 403, in _run_on_conn
proc = await conn.create_process(
^^^^^^^^^^^^^^^^^^^^^^^^^^
...<7 lines>...
)
^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4492, in create_process
chan, process = await self.create_session(
^^^^^^^^^^^^^^^^^^^^^^^^^^
SSHClientProcess, *args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/connection.py", line 4385, in create_session
session = await chan.create(session_factory, command, subsystem,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
bool(self._agent_forward_path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 1149, in create
packet = await self._open(b'session')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/asyncssh/channel.py", line 717, in _open
return await self._open_waiter
^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Task <Task pending name='Task-1' coro=<App.__run() running at ~/local/src/jw.dev/proj/jw-pkg/scripts/jw/pkg/lib/App.py:137> cb=[_run_until_complete_cb() at /usr/lib64/python3.13/asyncio/base_events.py:181]> got Future <Future pending> attached to a different loop
Signed-off-by: Jan Lindemann <jan@janware.com>
Expand platform macros (as in %{codename}) in the src and dst command
arguments. Expansion can be turned off by -F --fixed-strings.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a method distro_info() to App, essentially providing CmdInfo's
macro-expansion of platform information to other interested code.
Signed-off-by: Jan Lindemann <jan@janware.com>
As of now, install() passes the "names" parameter on to _install(),
which is expected to pass the list of package names on to the package
manager for having it handle package download and installation. This
commit makes it easier for Distro instances to support directly
installing packages via an URL instead by providing a few callback
methods to be overridden, in case the package manager doesn't handle
package URLs the same way as package names.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add class Curl as the first pure FileTransfer class without _run()
/ _sudo(). It doesn't use any PycURL / libcurl-like binding, but runs
the curl binary in a subprocess. That looks the most portable still.
Signed-off-by: Jan Lindemann <jan@janware.com>
ExecContext has get() / _get() and put() / _put(), which make a fine
API for a file transfer class. A class supporting file transfer
should not, however, be forced to implement _run() and _sudo(), so
place this functionality in a new class FileTransfer, and derive
ExecContext from it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add lib.base to provide basic definitions.
For now, move the definiions of Result, Input and InputMode from
ExecContext into lib.base. Having to import them from the ExecContect
module is too heavy-handed for those simple types.
Signed-off-by: Jan Lindemann <jan@janware.com>
With CmdCopy as test case and ExecContext.close() in place, we can
actually implement connection reuse, so do it for AsyncSSH.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add CmdCopy, designed to copy data from the filesystem to another
location in the filesystem. Not necessarily local file systems, the
URLs can be all URLs supported by ExecContext.run().
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a new group of commands - "posix". The current command categories
are not a good fit for that: "projects" is for CI, "distro" is
distribution-specific for CD, and secrets is for handling secrets
specifically. Introduce the more general command group "posix", a
class of commands not POSIX compliant in the exposed API, but
primarily using POSIX utilities as workhorse.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add copy(src_uri, dst_uri), instatiating two ExecContext instances,
and doing the obvious with them - copying from src_uri to dst_uri.
Signed-off-by: Jan Lindemann <jan@janware.com>
The property App.__os_release uses _run(['cat', '/etc/os-release']),
use ExecContext.get() instead as the default way to fetch content.
Signed-off-by: Jan Lindemann <jan@janware.com>
App currently has no hook to close async resources. Call it as
context manager, so that __aexit__() gets invoked if
run_sub_commands() exits.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add ExecContext.close() as a hook to clean up async resources living
longer than an ExecContext method call.
Also, implement __aenter__() and __aexit__(), to allow using
ExecContext as context manager. close() is invoked it goes out of
scope.
Signed-off-by: Jan Lindemann <jan@janware.com>
