Use the AsyncRunner class introduced in the previous commit to add a call_async() method, allowing to run async functions from sync functions by spawning an extra event loop.
Add class AsyncRunner. This is a wrapper around the ceremony needed to spawn an extra event loop in a synchronous function which wants to call an async function.
Guido van Rossum considers it bad design that such a function exists in the first place. While that may be true in the long run also for jw-pkg, at this point I'm unwilling to flag every lazyly initialized App property as async. It's not clear, yet, which will be async and which not, and I dread the churn. So I will accept this as a minimally invasive helper for now. When the API has stabilized, a design without it may be better.
Remove the key_filename parameter from the call to Paramiko's connect(). It's user-dependent, and the current DevOps implementation relies on having a SSH_AUTH_SOCK in the environment, anyway.
Take a positional uri argument to the constructor of ExecContext, forcing SSHClient to follow suit. The latter was instantiated with a hostname as only argument up to now, which still works as a special case of an uri.
Align the prototype of SSHClient.run_cmd() to ExecContext.run(). This is a push towards making the SSHClient code an ExceContext, too. Some arguments still log a warning or outright raise NotImplementedError.
In commands taking lists of packages, namely install, delete and pkg_files, don't bother asking the backend. Uniformly log a warning and return successfully.
Add the --verbose global option, which is made available as the App.verbose property.
Some functions still take a verbose parameter, but the type of these parameters is converted from bool to bool|None. The idea is that, if they are None, their verbosity falls back to the global default.
Allow to specify the ExecContext in a call to run_cmd(). This effectively makes run_cmd() an thin wrapper around ExecContext.run(), which is what's going to be used in the future. The wrapper is for backwards-compatibility.
The code below lib.distro, as left behind by the previous commit, is geared towards being directly used as a command-line API. This commit introduces the abstract base class Distro, a proxy for distribution-specific interactions. The proxy abstracts distro specifics into an API with proper method prototypes, not argparse.Namespace contents, and can thus be more easily driven by arbitrary code.
The Distro class is initialized with a member variable of type ExecContext, another new class introduced by this commit. It is designed to abstract the communication channel to the distribution instance. Currently only one specialization exists, Local, which interacts with the distribution and root file system it is running in, but is planned to be subclassed to support interaction via SSH, serial, chroot, or chains thereof.
Functions abstracting the distribution are not only needed in the context of the distro subcommand, but also by other code, so make the bulk of the code abstracting the distribution available in some place more universally useful than below cmds.distro.
This commit leaves the source files mostly unchanged. They are only patched to fix import paths, so that functionality is preserved. Refactoring the code from command-line API to library API will be done by the next commit.
jw-pkg distro dup got hung in a chroot environment. strace shows that write(2) into a pipe is the hanging syscall, with the write buffer hinting at zypper dup output.
I strongly suspect that run_cmd() tries to write stdout into the pipe which read_stream() fails to empty. So, make read_stream() more resilient by using read(4096) instead of readline(), which I suspect to be prone to hang on overlong lines.
run_cmd() with cmd_input == mode:interactive and verbose == true logs output too often. First, __log() is called, then pty.spawn() writes everything it reads from the PTY master to the terminal.
The fix it to not call __log() from _read() for the PTY reader.
Most run_xxx() return stdout and stderr. There's no way, really, for the caller to get hold of the exit code of the spawned executable. It can pass throw=true, catch, and assume a non-zero exit status. But that's not semantically clean, since the spawned function can well be a test function which is expected to return a non-zero status code, and the caller might be interested in what code that was, exactly.
The clearest way to solve this is to return the exit code as well. This commit does that.
This is a code maintenance commit: some run_xxx() helper functions take a string, some a list, and some just digest all arguments and pass them on as a list to exec() to be executed. That's highly inconsistent. This commit changes that to list-only.
Except for the run_cmd() method of SSHClient, which is still run as a shell method, because, erm, it's a shell. Might be changed in the future for consistency reasons.
jw.pkg.lib.Cmd._run() is abstract, but it's nice to give it a default implementation which calls self.parent._run() in case parent is also a command class. That allows for some default processing in _run() for each node up the parent chain.
The children / derived classes just need to make sure all classes in the hierarchy do:
async def _run(self, args):
return await super()._run(args)
... add main command logic here ..
Add a function get_profile_env(), a function returning environment variables from /etc/profile. Pass add=True to add its contents to the existing environment dictionary, overwriting old entries, or pass False to get the pristine content.
Add a parameter "output_encoding" to run_cmd(). The parameter allows the caller to specify if the output encoding should be detected as is by passing None (the default), if the output should be returned as undecoded bytes by passing the special string "bytes", or if the output should be treated as the encoding with the specified name and decoded to strings.
Don't log an Exception as {e} but as str(e) producing nicer output. Or as repr(e) if a backtrace is requested, because to people who can read backtraces, type info might be of interest. Also, remove pointless time stamps, those belong into the logging framework.
Move the body of BackendCmd.sudo() into a function. The rationale behind that is that its functionality is independent of the calling object for the most part, so having it in a function instead of a method is the more modular pattern.
Define run(), which calls _run() in the abstract base class Cmd, not in lib.Cmd. Otherwise lib.Cmd is not abstract, which will predictably confuse including code outside of jw-pkg.
During __init__(), commands have no idea of their parent. This is not a problem as of now, but is easy to fix, and it's architecturally desirable to be prepared just in case, so add the parent argument to the ctor before more commands are added.
