Don't pass mode as a string to put(). Given the multitunde of
possible string representations for numbers, some understood by
int(string, 0) and some not, there's too much room for passing
strings which are unparseable, or worse, prone to be parsed wrongly.
However, pass mode down to _put() as a string for convenience,
because that's what most _put() implementations will need to use. If
they don't, converting to int is easy from the one defined string
format.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_cmd() is a thin layer over the public ExecContext API, which
falls back to using a Local instance if not other ExecContext is
specified explicitly. Both the default Local context as the
subsequent call to run() should have the same idea about
interactivity, so allowing to specify it in two parameters
("interactive" and "cmd_input") is a bad idea. Remove "interactive".
Signed-off-by: Jan Lindemann <jan@janware.com>
Add lib.base to provide basic definitions.
For now, move the definiions of Result, Input and InputMode from
ExecContext into lib.base. Having to import them from the ExecContect
module is too heavy-handed for those simple types.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add copy(src_uri, dst_uri), instatiating two ExecContext instances,
and doing the obvious with them - copying from src_uri to dst_uri.
Signed-off-by: Jan Lindemann <jan@janware.com>
The Input instance passed as cmd_input to ExecContext.run() and
.sudo() currently may be of type str. Allow to pass bytes, too.
At the same time, disallow None to be passed as cmd_input. Force the
caller to be more explicit how it wants input to be handled, notably
with respect to interactivity.
Along the way fix a bug: Content in cmd_input should result in
CallContext.interactive == False but doesn't. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_curl() has no clear API of whether or not the return values should
be decoded. It has parse_json, which should imply decoding, but there's
no way to specify that explicitly. Moreover, when it tries to decode, it
decodes on the coroutine returned from run_cmd(), not the awaited
coroutine return value.
Add a decode parameter, defaulting to False, change the parse_json
parameter's default from True to False, and fix the run_cmd() return
value evaluation.
Signed-off-by: Jan Lindemann <jan@janware.com>
print() should be used to output information requested by a certain
command, but not for logging the process to achieve it. log() should
be used for the latter. The current code has the distinction not down
clearly, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_sudo() is a thin wrapper around ExecContext.sudo(), so don't try
to make sense more arguments than necessary.
Signed-off-by: Jan Lindemann <jan@janware.com>
Since commit 02697af5, ExecContext.run() returns bytes for stdout and
stderr and fixes that in calling code. The thing it did not fix was
the code calling run_cmd(), which also made return bytes. This commit
catches up on that.
Signed-off-by: Jan Lindemann <jan@janware.com>
ExecContext's .sudo() omits many of run()'s parameters, and this
commit adds them. To avoid redundancy around repeating and massaging
the long parameter list of both functions and their return values, it
also adds some deeper changes:
- Make run(), _run(), sudo() and _sudo() always return instances of
Result. Before it was allowed to return a triplet of stdout,
stderr, and exit status.
- Have ExecContext stay out of the business of decoding the result
entirely. Result provides a convenience method .decode()
operating on stdout and stderr and leaves the decision to the
caller.
This entails miniscule adaptations in calling code, namely in
App.os_release, util.get_profile_env() and CmdListRepos._run().
- Wrap the _run() and _sudo() callbacks in a context manager object
of type CallContext to avoid code duplication.
- Consistently name the first argument to run(), _run(), sudo() and
_sudo() "cmd", not "args". The latter suggests that the caller is
omitting the executable, which is not the case.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the --verbose global option, which is made available as the
App.verbose property.
Some functions still take a verbose parameter, but the type of these
parameters is converted from bool to bool|None. The idea is that, if
they are None, their verbosity falls back to the global default.
Signed-off-by: Jan Lindemann <jan@janware.com>
Allow to specify the ExecContext in a call to run_cmd(). This
effectively makes run_cmd() an thin wrapper around ExecContext.run(),
which is what's going to be used in the future. The wrapper is for
backwards-compatibility.
Signed-off-by: Jan Lindemann <jan@janware.com>
The code below lib.distro, as left behind by the previous commit, is
geared towards being directly used as a command-line API. This commit
introduces the abstract base class Distro, a proxy for
distribution-specific interactions. The proxy abstracts distro
specifics into an API with proper method prototypes, not
argparse.Namespace contents, and can thus be more easily driven by
arbitrary code.
The Distro class is initialized with a member variable of type
ExecContext, another new class introduced by this commit. It is
designed to abstract the communication channel to the distribution
instance. Currently only one specialization exists, Local, which
interacts with the distribution and root file system it is running
in, but is planned to be subclassed to support interaction via SSH,
serial, chroot, or chains thereof.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg distro dup got hung in a chroot environment. strace shows that
write(2) into a pipe is the hanging syscall, with the write buffer
hinting at zypper dup output.
I strongly suspect that run_cmd() tries to write stdout into the pipe
which read_stream() fails to empty. So, make read_stream() more
resilient by using read(4096) instead of readline(), which I suspect
to be prone to hang on overlong lines.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_cmd() with cmd_input == mode:interactive and verbose == true
logs output too often. First, __log() is called, then pty.spawn()
writes everything it reads from the PTY master to the terminal.
The fix it to not call __log() from _read() for the PTY reader.
Signed-off-by: Jan Lindemann <jan@janware.com>
Most run_xxx() return stdout and stderr. There's no way, really, for
the caller to get hold of the exit code of the spawned executable. It
can pass throw=true, catch, and assume a non-zero exit status. But
that's not semantically clean, since the spawned function can well be
a test function which is expected to return a non-zero status code,
and the caller might be interested in what code that was, exactly.
The clearest way to solve this is to return the exit code as well.
This commit does that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make some incomprensible parser error messages if run_curl() returns
nothing slightly less incomprehensible.
Signed-off-by: Jan Lindemann <jan@janware.com>
This is a code maintenance commit: some run_xxx() helper functions
take a string, some a list, and some just digest all arguments and
pass them on as a list to exec() to be executed. That's highly
inconsistent. This commit changes that to list-only.
Except for the run_cmd() method of SSHClient, which is still run as a
shell method, because, erm, it's a shell. Might be changed in the
future for consistency reasons.
Signed-off-by: Jan Lindemann <jan@janware.com>
Replace the boolean parameter "add" with the richer "keep":
- False -> Don't keep anything
- True -> Keep what's in the current environment
- List of strings -> Keep those variables
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a function get_profile_env(), a function returning environment
variables from /etc/profile. Pass add=True to add its contents to the
existing environment dictionary, overwriting old entries, or pass
False to get the pristine content.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a parameter "output_encoding" to run_cmd(). The parameter allows
the caller to specify if the output encoding should be detected as is
by passing None (the default), if the output should be returned as
undecoded bytes by passing the special string "bytes", or if the
output should be treated as the encoding with the specified name and
decoded to strings.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't log an Exception as {e} but as str(e) producing nicer output.
Or as repr(e) if a backtrace is requested, because to people who can
read backtraces, type info might be of interest. Also, remove
pointless time stamps, those belong into the logging framework.
Signed-off-by: Jan Lindemann <jan@janware.com>
Move the body of BackendCmd.sudo() into a function. The rationale
behind that is that its functionality is independent of the calling
object for the most part, so having it in a function instead of a
method is the more modular pattern.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_cmd() is synchronous. Now that all commands are asynchronous, we
can await it, so rewrite it to be asynchronous, too.
Other changes:
- Make it return stderr as well in case its needed
- Drop into a pseuto-tty if
- cmd_input == "mode:interactive" or
- cmd_input == "mode:auto" and stdin is a TTY
- Add argument env, defaulting to None. If it's a dict, it will be
the environment the command is run in
This entails making all functions using run_cmd() async, too,
including run_curl(), get_username() and get_password().
Signed-off-by: Jan Lindemann <jan@janware.com>
