Add ExecContext.close() as a hook to clean up async resources living
longer than an ExecContext method call.
Also, implement __aenter__() and __aexit__(), to allow using
ExecContext as context manager. close() is invoked it goes out of
scope.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add wrapper methods get() and put(), plus their wrapped methods
_get() and _put(). The wrapped methods have default implementations,
using POSIX utilities on the target machine over _run().
Signed-off-by: Jan Lindemann <jan@janware.com>
ExecContext.create() relies on properly formed URLs with a schema for
deciding which backend gets created. Create a Local instance if an
URL doesn't have schema.
Signed-off-by: Jan Lindemann <jan@janware.com>
The Input instance passed as cmd_input to ExecContext.run() and
.sudo() currently may be of type str. Allow to pass bytes, too.
At the same time, disallow None to be passed as cmd_input. Force the
caller to be more explicit how it wants input to be handled, notably
with respect to interactivity.
Along the way fix a bug: Content in cmd_input should result in
CallContext.interactive == False but doesn't. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
- Move _sudo() above sudo()
To have a pattern in lib.ExecContext and avoid future churn: If a
public wrapper calls a protected method, define the protected
method above the respective wrapper.
- sudo(): Make cmd_input default equal to run(): InputMode.OptInteractive
- CallContext: Expose parameters throw, wd, cmd as properties for
later use
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdCanonicalizeRemotes / canonicalize-remotes and the respective
target in topdir.mk remove the /srv/git portion from all remotes'
URLs pointing to git.janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
Without --backtrace, the outmost try-catch block logs exceptions
plainly as their text. If it catches a key error, the exception text
only consists of the key itself, which can be easily mistaken for a
normal program output, so prefix it with a "Failed:".
Signed-off-by: Jan Lindemann <jan@janware.com>
To be able to use secret handling code from other modules, move the
bulk of it from the "secrets"-command centric implementation in
cmds.secrets.Cmd into a new module cmds.secrets.lib.util.
Signed-off-by: Jan Lindemann <jan@janware.com>
_run_ssh() of ssh.Exec doesn't pass throw=False to run_cmd(), which
makes it throw exceptions, and effectively strips the caller of any
chance to get hold of stdout and stderr. Pass throw=False and let
run() decide according the the caller-provided throw parameter
whether or not a problem should propagate up as exception or return
value.
Signed-off-by: Jan Lindemann <jan@janware.com>
ssh_client() tries a predefined order of client class implementations
until it finds a workable candidate. For testing all, it's desirable
to be able to target the exact class. Add a "type" parameter to
achieve that.
I'm aware that type is also a function. But the semantics look so
compelling to me that I'm using the variable name anyway.
Signed-off-by: Jan Lindemann <jan@janware.com>
Naively join()ing a command list to be executed remotely via SSH also
quotes shell operators which doesn't work, of course. Work around
that. The workaround will not always work but covers lots of cases.
Signed-off-by: Jan Lindemann <jan@janware.com>
Instantiating a SSHClient-derived class with an invalid or missing
uri parameter is accepted and fails later down the road. Raise an
Exception early on to make the error log more comprehensible.
Signed-off-by: Jan Lindemann <jan@janware.com>
The SSHClient classes Paramiko and Exec are exported via # export.
This is a bad idea, because if Paramiko is not installed, none of the
other's can be instantiated either: On the attempt to load them,
__init__.py is loaded first and fails. SSHClient.ssh_client() knows
what to do, no need to auto-import them into the lib.ec.ssh module.
Signed-off-by: Jan Lindemann <jan@janware.com>
Installation type SCRIPT has beed disabled long ago because of its
overlap with EXE. Remove the dead code around it.
Signed-off-by: Jan Lindemann <jan@janware.com>
/usr/bin/file <candidate> | grep text is used to detect if a file is
a text file or not. Replace that with grep -I., because that adds
some files left out by /usr/bin/file, notably systemd service files.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg is copied into $(TOPDIR)/bin during build, that's wrong.
Write a rule precisely targeted at installing /usr/bin/jw-pkg, and
cut all the scripts.mk machinery.
Also, make jw-pkg a relative link to avoid the respective RPM
warning.
Signed-off-by: Jan Lindemann <jan@janware.com>
Call run_curl() with parse_json=True to make that explicit, and be a
little more verbose about the outcome.
Signed-off-by: Jan Lindemann <jan@janware.com>
run_curl() has no clear API of whether or not the return values should
be decoded. It has parse_json, which should imply decoding, but there's
no way to specify that explicitly. Moreover, when it tries to decode, it
decodes on the coroutine returned from run_cmd(), not the awaited
coroutine return value.
Add a decode parameter, defaulting to False, change the parse_json
parameter's default from True to False, and fix the run_cmd() return
value evaluation.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the global --verbose and --interactive command-line options as
defaults for constructing a SSHClient instance for use with listing
repos over SSH.
Signed-off-by: Jan Lindemann <jan@janware.com>
PROJECTS_DIR_REMOTE_BASE and JANWARE_USER is determined by jw-pkg.py
get-auth-info. Interactivity hurts here, though, turn it off.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support --hide-jw-pkg. This is a step towards replacing
required-os-pkg, which leaves out packages from pkg.requires.jw
sections.
Signed-off-by: Jan Lindemann <jan@janware.com>
If a package P is added, only those of its dependendencies are added
along which are in the same os-cascade section as P. That's wrong,
fix it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make variable names a little more readable and searchable within the
long pkg_relations_list() method by making their names longer and
truer to what they actually mean.
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdPrereq was mostly redundant to PkgRequired all along. CmdPrereq
has grown more versatile, and CmdPrereq is not used throughout jw-pkg
anylonger by now. Remove it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Enclose sections / keys taken from project.conf in [square.brackets],
hinting at what they are supposed to mean.
Signed-off-by: Jan Lindemann <jan@janware.com>
log_start_stop() is responsible for logging markers at the beginning
and end of a decorated log. They should not be applied if pgit.sh is
run with --porcelain. In fact, they are, and vice versa. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Set -o pipefail at the start of the script. This makes pgit.sh commit
work. Before it didn't, because run_git() doesn't return a proper
return value when it's used in a pipe with a cosmetic sed afterwards.
Signed-off-by: Jan Lindemann <jan@janware.com>
