Since commit 02697af5, ExecContext.run() returns bytes for stdout and stderr and fixes that in calling code. The thing it did not fix was the code calling run_cmd(), which also made return bytes. This commit catches up on that.
Distro's sudo() and run() wrappers are not flagged async. It still works, because throughout jw-pkg all callers expect a coroutine return value, but flagging them as async makes the return value obvious.
Move the code of SSHClientInternal and SSCClientCmd into lib.ec.ssh, as "Paramiko" and "Exec", respectively. This makes the class layout a little more modular, and along the way fixes a bug where SSHClientInternal could be instantiated but was unusable (if the Paramiko is not installed).
ExecContext's .sudo() omits many of run()'s parameters, and this commit adds them. To avoid redundancy around repeating and massaging the long parameter list of both functions and their return values, it also adds some deeper changes:
- Make run(), _run(), sudo() and _sudo() always return instances of
Result. Before it was allowed to return a triplet of stdout,
stderr, and exit status.
- Have ExecContext stay out of the business of decoding the result
entirely. Result provides a convenience method .decode()
operating on stdout and stderr and leaves the decision to the
caller.
This entails miniscule adaptations in calling code, namely in
App.os_release, util.get_profile_env() and CmdListRepos._run().
- Wrap the _run() and _sudo() callbacks in a context manager object
of type CallContext to avoid code duplication.
- Consistently name the first argument to run(), _run(), sudo() and
_sudo() "cmd", not "args". The latter suggests that the caller is
omitting the executable, which is not the case.
Don't open and parse /etc/os-release with Python built-in functions. Spawn "cat /etc/os-release" as a subprocess and capture the output for parsing instead. The obvious advantage is that this also works with a remote shell.
Use the AsyncRunner class introduced in the previous commit to add a call_async() method, allowing to run async functions from sync functions by spawning an extra event loop.
Add class AsyncRunner. This is a wrapper around the ceremony needed to spawn an extra event loop in a synchronous function which wants to call an async function.
Guido van Rossum considers it bad design that such a function exists in the first place. While that may be true in the long run also for jw-pkg, at this point I'm unwilling to flag every lazyly initialized App property as async. It's not clear, yet, which will be async and which not, and I dread the churn. So I will accept this as a minimally invasive helper for now. When the API has stabilized, a design without it may be better.
Remove the key_filename parameter from the call to Paramiko's connect(). It's user-dependent, and the current DevOps implementation relies on having a SSH_AUTH_SOCK in the environment, anyway.
Take a positional uri argument to the constructor of ExecContext, forcing SSHClient to follow suit. The latter was instantiated with a hostname as only argument up to now, which still works as a special case of an uri.
Align the prototype of SSHClient.run_cmd() to ExecContext.run(). This is a push towards making the SSHClient code an ExceContext, too. Some arguments still log a warning or outright raise NotImplementedError.
CmdInfo._expand_macros() raises a custom exception during exception handling. Replace that by logging some details and raising the original exception to keep the stack trace readable.
Add the name of the operating system ID as taken from /etc/os-release to $INSTALL_LOG. This should allow for easier post-mortem debugging if multiple builds used the same file in /tmp and possibly also prevent conflicts.
Move the PY_XXX = true|false variable definitions meant to be preset by including makefiles to the top of py-defs.mk to make the structure of the file clearer.
Add PY_INSTALL_INIT_PY ?= true to py-defs.mk. If set to false, a Python module will not try to attempt installing an existing / generated __init__.py. This is useful when installing into an exiting directory with an existing __init__.py.
jw-pkg supports more than RPM-based package managers, but for historic reasons, lots of its Makefile variables still have "RPM" in their names. This is misleading. Replace "RPM" in variable names by the more generic "PKG" where appropriate.
RPM_REQUIRES_DEVEL is often filled from the current version, which in turn is filled from the version file, so the order of events here is unclear at best.
Add target pkg-release-update-version and make pkg-release-reinstall depend on it to make the order explicit.
