By default, argcomplete uses argcomplete.FilesCompleter as default
for every argument. This mixes accessible files into the list of
possible completions. For most of jw-pkg's commands, that's unwanted,
so turn it off by defining a NoopCompleter class which does nothing,
and by set every arguments's default completer to a NoopCompleter
instance. If desired, completing files can be restored for an
argument by
parser.add_argument("some-arg").completer = FilesCompleter()
Signed-off-by: Jan Lindemann <jan@janware.com>
Every module derived from lib.Cmd implements its own
parser.add_argument() logic. As a consequence, all Cmd derived
modules need to be loaded to have the full argument tree available.
This is avoided during normal program startup because it takes some
time. It's not necessary during normal program execution, nor for
showing help messages. It is, however, needed for argcomplete to do
its thing, so fully parse the command line if the program runs in
argcomplete mode, as determined by checking if _ARGCOMPLETE is
present in the environment.
Signed-off-by: Jan Lindemann <jan@janware.com>
lib.ExecContext.log_delim() logs a header not designed for enclosing
command output, and, hence, no footer should be output. This commit
suppresses it.
Signed-off-by: Jan Lindemann <jan@janware.com>
print() should be used to output information requested by a certain
command, but not for logging the process to achieve it. log() should
be used for the latter. The current code has the distinction not down
clearly, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_sudo() is a thin wrapper around ExecContext.sudo(), so don't try
to make sense more arguments than necessary.
Signed-off-by: Jan Lindemann <jan@janware.com>
Request a remote PTY from AsyncSSH, and wire the local terminal's
stdin up with it if interactive == True. This gives a real
interactive session if local stdin belongs to a terminal. Also,
thanks to AsyncSSH understanding that, forward terminal size changes
to the remote end.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a SSHClient implementation using AsyncSSH. This is the first and
currently only class derived from SSHClient which implements
SSHClient.Cap.LogOutput, designed to consume and log command output
as it streams in. It felt like the lower hanging fruit not to do that
with Paramiko: Paramiko doesn't provide a native async API, so it
would need to spawn additional worker threads. I think.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add an optional caps ("capabilities") argument to the constructor of
SSHClient. It is meant to be used by derived classes in order to
declare that they don't want the base class to handle a default
behaviour for a certain capability, but that they want to implement
it themselves instead.
Also, give the _run_ssh() callbacks the necessary info as parameters,
so that the derived classes have the means to do so.
Signed-off-by: Jan Lindemann <jan@janware.com>
lib.pm.query_packages() uses a TTY for doing its thing and outputs
half-digested stuff to the terminal, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit introduces two new types, Input and InputMode. They
replace the more error-prone special strings cmd_input could be used
with. InputMode is an Enum, and Input can be either IntputMode, a
string or None.
Signed-off-by: Jan Lindemann <jan@janware.com>
Whether or not the CallContext.interactive property should be True or
False, and hence, a call should be processed interactively, depends
on multiple factors, constituting matrix of options with multiple
preferences.
--interactive is the application default and can be true, false,
or auto
- A call can be explicitly invoked as interactive, non-interactive
or auto via the cmd_input parameter to ExecContext.run()
This commit adds more "mode:" options to make the latter more
explicit. It takes preference over the global --interactive
parameter: Global --interactive is only given a chance to decide if
cmd_input is None (default) or mode:opt-interactive.
This commit also fixes a bug: --interactive is ignored because the
interactive argument passed to ExecContext's constructor is ignored
later on in calls to the wrapped _run() and _sudo() methods.
Signed-off-by: Jan Lindemann <jan@janware.com>
mod_env can be None. Make it an empty dict in that case to take a
little burden off the implementations in the derived classes.
Signed-off-by: Jan Lindemann <jan@janware.com>
Since commit 02697af5, ExecContext.run() returns bytes for stdout and
stderr and fixes that in calling code. The thing it did not fix was
the code calling run_cmd(), which also made return bytes. This commit
catches up on that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Distro's sudo() and run() wrappers are not flagged async. It still
works, because throughout jw-pkg all callers expect a coroutine
return value, but flagging them as async makes the return value
obvious.
Signed-off-by: Jan Lindemann <jan@janware.com>
Move the code of SSHClientInternal and SSCClientCmd into lib.ec.ssh,
as "Paramiko" and "Exec", respectively. This makes the class layout
a little more modular, and along the way fixes a bug where
SSHClientInternal could be instantiated but was unusable (if the
Paramiko is not installed).
Signed-off-by: Jan Lindemann <jan@janware.com>
ExecContext's .sudo() omits many of run()'s parameters, and this
commit adds them. To avoid redundancy around repeating and massaging
the long parameter list of both functions and their return values, it
also adds some deeper changes:
- Make run(), _run(), sudo() and _sudo() always return instances of
Result. Before it was allowed to return a triplet of stdout,
stderr, and exit status.
- Have ExecContext stay out of the business of decoding the result
entirely. Result provides a convenience method .decode()
operating on stdout and stderr and leaves the decision to the
caller.
This entails miniscule adaptations in calling code, namely in
App.os_release, util.get_profile_env() and CmdListRepos._run().
- Wrap the _run() and _sudo() callbacks in a context manager object
of type CallContext to avoid code duplication.
- Consistently name the first argument to run(), _run(), sudo() and
_sudo() "cmd", not "args". The latter suggests that the caller is
omitting the executable, which is not the case.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the AsyncRunner class introduced in the previous commit to add a
call_async() method, allowing to run async functions from sync
functions by spawning an extra event loop.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add class AsyncRunner. This is a wrapper around the ceremony needed
to spawn an extra event loop in a synchronous function which wants to
call an async function.
Guido van Rossum considers it bad design that such a function exists
in the first place. While that may be true in the long run also for
jw-pkg, at this point I'm unwilling to flag every lazyly initialized
App property as async. It's not clear, yet, which will be async and
which not, and I dread the churn. So I will accept this as a
minimally invasive helper for now. When the API has stabilized, a
design without it may be better.
Signed-off-by: Jan Lindemann <jan@janware.com>
Expose App's __eloop member containing the application's main event
loop to allow outside async event loop trickery.
Signed-off-by: Jan Lindemann <jan@janware.com>
Take implementation burden from the derived classes _run() callback
by moving the respective code into the run() wrapper methods of the
base class.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove the key_filename parameter from the call to Paramiko's
connect(). It's user-dependent, and the current DevOps implementation
relies on having a SSH_AUTH_SOCK in the environment, anyway.
Signed-off-by: janware DevOps <devops@janware.com>
Add a wrapper around urlparse() and Paramiko's connect() function, in
order to log some more info in case an exception is thrown.
Signed-off-by: Jan Lindemann <jan@janware.com>
Take a positional uri argument to the constructor of ExecContext,
forcing SSHClient to follow suit. The latter was instantiated with a
hostname as only argument up to now, which still works as a special
case of an uri.
Signed-off-by: Jan Lindemann <jan@janware.com>
Code beautification chore: Move the public methods of SSHClients to
the bottom of the class to be consistent with other classes.
Signed-off-by: Jan Lindemann <jan@janware.com>
Align the prototype of SSHClient.run_cmd() to ExecContext.run(). This
is a push towards making the SSHClient code an ExceContext, too. Some
arguments still log a warning or outright raise NotImplementedError.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make SSHClient accept a list of strings for the cmd argument to align
with the other run_cmd() functions in jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make all backend package manager prototypes have the same arguments:
yum(self, args: list[str], verbose: bool=True, sudo: bool=True)
This also implies having them behave equally verbose, unless
otherwise specified by the caller. This changes the default for
Debian.
Signed-off-by: Jan Lindemann <jan@janware.com>
In commands taking lists of packages, namely install, delete and
pkg_files, don't bother asking the backend. Uniformly log a warning
and return successfully.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't mention "error" in log message for exit codes > 0 from spawned
processes, because sometimes they don't mean an error.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the --verbose global option, which is made available as the
App.verbose property.
Some functions still take a verbose parameter, but the type of these
parameters is converted from bool to bool|None. The idea is that, if
they are None, their verbosity falls back to the global default.
Signed-off-by: Jan Lindemann <jan@janware.com>
Functions in lib.pm (i.e. run_dpkg(), run_rpm() and friends) also get
an ExecContext-type parameter. Use them in lib/distros/*/Distro.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove defaults from protected callback function parameters. They
have to be decided by the base class's public API.
Signed-off-by: Jan Lindemann <jan@janware.com>
Allow to specify the ExecContext in a call to run_cmd(). This
effectively makes run_cmd() an thin wrapper around ExecContext.run(),
which is what's going to be used in the future. The wrapper is for
backwards-compatibility.
Signed-off-by: Jan Lindemann <jan@janware.com>
The code below lib.distro, as left behind by the previous commit, is
geared towards being directly used as a command-line API. This commit
introduces the abstract base class Distro, a proxy for
distribution-specific interactions. The proxy abstracts distro
specifics into an API with proper method prototypes, not
argparse.Namespace contents, and can thus be more easily driven by
arbitrary code.
The Distro class is initialized with a member variable of type
ExecContext, another new class introduced by this commit. It is
designed to abstract the communication channel to the distribution
instance. Currently only one specialization exists, Local, which
interacts with the distribution and root file system it is running
in, but is planned to be subclassed to support interaction via SSH,
serial, chroot, or chains thereof.
Signed-off-by: Jan Lindemann <jan@janware.com>
Functions abstracting the distribution are not only needed in the
context of the distro subcommand, but also by other code, so make the
bulk of the code abstracting the distribution available in some place
more universally useful than below cmds.distro.
This commit leaves the source files mostly unchanged. They are only
patched to fix import paths, so that functionality is preserved.
Refactoring the code from command-line API to library API will be
done by the next commit.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg distro dup got hung in a chroot environment. strace shows that
write(2) into a pipe is the hanging syscall, with the write buffer
hinting at zypper dup output.
I strongly suspect that run_cmd() tries to write stdout into the pipe
which read_stream() fails to empty. So, make read_stream() more
resilient by using read(4096) instead of readline(), which I suspect
to be prone to hang on overlong lines.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_cmd() with cmd_input == mode:interactive and verbose == true
logs output too often. First, __log() is called, then pty.spawn()
writes everything it reads from the PTY master to the terminal.
The fix it to not call __log() from _read() for the PTY reader.
Signed-off-by: Jan Lindemann <jan@janware.com>
