Add a sub-module for code that's too specific to jw.pkg.cmds.projects
to go into jw.pkg.lib but too generic to go into a command module.
Long-term, it might be a good idea to create a place for code which
jw-pkg doesn't exclusively use for its own purposes. jw.lib, for
example. Then, liberated from the burden to be generally useful also
externally, jw.pkg.lib might be a better fit for code currently in
jw.pkg.cmds.xxx.lib, and a more natural place usable across
subcommands.
Signed-off-by: Jan Lindemann <jan@janware.com>
Lots of sub- and sub-subcommands are derived from the base class of
the invoking command, notably below cmds.projects. That provides some
properties shared across the ancestor hierarchy of a command, but is
semantically unsound. Introduce jw.pkg.BaseCmd class as a place to
provide basic helpers shared across all commands used in a
jw.pkg.App's context. Also add cmds.projects.Cmd to be used by other
commands in cmds.projects in later commits.
Signed-off-by: Jan Lindemann <jan@janware.com>
Not all __init__.py modules are generated by python-tools.sh, some
are needed early to make jw-pkg useful without generation, notably in
jw.pkg.cmds.
Add detect_modules() to unify that detection, and place it into a
minimal module lib.init to not increase startup time cost.
Signed-off-by: Jan Lindemann <jan@janware.com>
janware upstream development server moved from git.janware.com to
devgit.janware.com. This commit follows the move with pretty much a
simple
s/git.janware.com/devgit.janware.com/
over jw-pkg. It found 14 matches, that's pretty bad.
FIXME: Reduce the redundancy, or, better, replace the
devgit.janware.com goodies by a more generally useful concept
altogether.
Signed-off-by: Jan Lindemann <jan@janware.com>
Exceptions raised by the build command are handled and changed,
messing up the stack trace. Re-raise the original exception from the
exception handler to fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Enclose ExecContext._run() in an open() / close() - pair. This is
convenient for the caller in that it doesn't need to take care of
opening and closing for one call only, and inconvenient in that it
forces the caller to conciously add an open() / close() - pair around
multiple run() calls where it wants the context to stay open in
between. Or use the ExecContext as a context manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
There are a couple of assert statements in the codebase which can
make jw-pkg fail without any detail whatsoever if --backtrace is not
specified, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Commit a19679fec reverted the first attempt to make AsyncSSH reuse
one connection during an instance lifetime. That failed because a lot
of distribution-specific properties were filled in a new event loop
thread started by AsyncRunner, and AsyncSSH didn't like that.
This commit is the first part of the solution: Move those properties
from the App class to the Distro class, and load the Distro class
in an async loader. As soon as it's instantiated, it can provide all
its properties without cluttering the code with async keywords.
Signed-off-by: Jan Lindemann <jan@janware.com>
The name of the env parameter to ExecContext.run() and .sudo() is not
descriptive enough for which environment is supposed to be modified
and how, so rename and split it up as follows:
- .run(): env -> mod_env
- .sudo(): env -> mod_env_sudo and mod_env_cmd
The parameters have the following meaning:
- "mod_env*" means that the environment is modified, not replaced
- "mod_env" and "mod_env_cmd" modify the environment "cmd" runs in
- "mod_env_sudo" modifies the environment sudo runs in
Fix the fallout of the API change all over jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add lib.base to provide basic definitions.
For now, move the definiions of Result, Input and InputMode from
ExecContext into lib.base. Having to import them from the ExecContect
module is too heavy-handed for those simple types.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add ExecContext.close() as a hook to clean up async resources living
longer than an ExecContext method call.
Also, implement __aenter__() and __aexit__(), to allow using
ExecContext as context manager. close() is invoked it goes out of
scope.
Signed-off-by: Jan Lindemann <jan@janware.com>
The Input instance passed as cmd_input to ExecContext.run() and
.sudo() currently may be of type str. Allow to pass bytes, too.
At the same time, disallow None to be passed as cmd_input. Force the
caller to be more explicit how it wants input to be handled, notably
with respect to interactivity.
Along the way fix a bug: Content in cmd_input should result in
CallContext.interactive == False but doesn't. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdCanonicalizeRemotes / canonicalize-remotes and the respective
target in topdir.mk remove the /srv/git portion from all remotes'
URLs pointing to git.janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
Call run_curl() with parse_json=True to make that explicit, and be a
little more verbose about the outcome.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the global --verbose and --interactive command-line options as
defaults for constructing a SSHClient instance for use with listing
repos over SSH.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support --hide-jw-pkg. This is a step towards replacing
required-os-pkg, which leaves out packages from pkg.requires.jw
sections.
Signed-off-by: Jan Lindemann <jan@janware.com>
If a package P is added, only those of its dependendencies are added
along which are in the same os-cascade section as P. That's wrong,
fix it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make variable names a little more readable and searchable within the
long pkg_relations_list() method by making their names longer and
truer to what they actually mean.
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdPrereq was mostly redundant to PkgRequired all along. CmdPrereq
has grown more versatile, and CmdPrereq is not used throughout jw-pkg
anylonger by now. Remove it.
Signed-off-by: Jan Lindemann <jan@janware.com>
print() should be used to output information requested by a certain
command, but not for logging the process to achieve it. log() should
be used for the latter. The current code has the distinction not down
clearly, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
print() should be used to output information requested by a certain
command, but not for logging the process to achieve it. log() should
be used for the latter. The current code has the distinction not down
clearly, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Since commit 02697af5, ExecContext.run() returns bytes for stdout and
stderr and fixes that in calling code. The thing it did not fix was
the code calling run_cmd(), which also made return bytes. This commit
catches up on that.
Signed-off-by: Jan Lindemann <jan@janware.com>
ExecContext's .sudo() omits many of run()'s parameters, and this
commit adds them. To avoid redundancy around repeating and massaging
the long parameter list of both functions and their return values, it
also adds some deeper changes:
- Make run(), _run(), sudo() and _sudo() always return instances of
Result. Before it was allowed to return a triplet of stdout,
stderr, and exit status.
- Have ExecContext stay out of the business of decoding the result
entirely. Result provides a convenience method .decode()
operating on stdout and stderr and leaves the decision to the
caller.
This entails miniscule adaptations in calling code, namely in
App.os_release, util.get_profile_env() and CmdListRepos._run().
- Wrap the _run() and _sudo() callbacks in a context manager object
of type CallContext to avoid code duplication.
- Consistently name the first argument to run(), _run(), sudo() and
_sudo() "cmd", not "args". The latter suggests that the caller is
omitting the executable, which is not the case.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use SSHClient as an ExecContext, i.e. use the .run() method instead
of .run_cmd(). Also, let SSHClient decide which implementation to
use.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make SSHClient accept a list of strings for the cmd argument to align
with the other run_cmd() functions in jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
pkg_relations_list() has an intricate case distinction around
expand_semver_revision_range, clean that up.
Signed-off-by: Jan Lindemann <jan@janware.com>
In a push to eventually merge the classes, somewhat align the
command-line API of CmdRequiredOsPkg to the one of
BaseCmdPkgRelations by using dependency flavours as mandatory, first
argument.
Signed-off-by: Jan Lindemann <jan@janware.com>
To support the pkg-install-testbuild-deps target, a selector is
needed listing all prerequisites to be installed except the project
under test. --hide-self should be useful for that.
Signed-off-by: Jan Lindemann <jan@janware.com>
In a push to eventually merge class CmdRequiredOsPkg into this class,
add the --skip-excluded option required by it.
Signed-off-by: Jan Lindemann <jan@janware.com>
In a push to eventually merge class CmdRequiredOsPkg into this class,
add the --quote option required by it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a function pkg_relations_list(), doing pretty much the same as
pkg_relations(), but taking individual arguments instead of an
argparse.Namespace args argument, in order to provide the
functionality to derived classes.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --syntax to BaseCmdPkgRelations.pkg_relations(), and
default to 'semver', i.e. the current state of affairs. If that's
changed to 'debian', relations declared in project.conf as
pkg.requires.os.devel = jw-pkg-devel > 1.2.3
will be output as
jw-pkg-devel >> 1.2.3
which is what Debian expects.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --expand-semver-revision-range to
cmds.projects.BaseCmdPkgRelations.pkg_relations(). The options turns
a dependency in projects.conf
pkg.requires.os.devel = jw-pkg-devel = VERSION
into
jw-pkg-devel >= 1.2.3, jw-pkg-devel < 1.2.4
Note that this will break as soon as a real range is specified in
projects.conf. To be fixed later, this commit is at least less
breakage than before.
Signed-off-by: Jan Lindemann <jan@janware.com>
The output of
jw.pkg.py projects required-os-pkg --flavours release
should include all packages required by flavour devel, because during
the release process, -devel and -run packages are both installed, and
installing the -devel package is only possible if its dependencies
are installed.
Signed-off-by: Jan Lindemann <jan@janware.com>
Most run_xxx() return stdout and stderr. There's no way, really, for
the caller to get hold of the exit code of the spawned executable. It
can pass throw=true, catch, and assume a non-zero exit status. But
that's not semantically clean, since the spawned function can well be
a test function which is expected to return a non-zero status code,
and the caller might be interested in what code that was, exactly.
The clearest way to solve this is to return the exit code as well.
This commit does that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support --remote-owner-base in the command CmdGetAuthInfo. Make it
return what currently --remote-base returned: A URL including the
user / organization specific prefix of the Git remote's URL that
jw-pkg was pulled from.
https://my-company.com/src/theowner/jw-pkg.git
would have a --remote-owner-base of
https://my-company.com/src/theowner
Also, change what --remote-base returns to
https://my-company.com/src
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename the --from-user option to --from-owner. Forgejo supports users
and organizations under the more general term "owner", so that's the
better term.
Signed-off-by: Jan Lindemann <jan@janware.com>
list-repos tries /users/ and /orgs/ to find a working repo URL in a
Forgejo instance, logs a failure and doesn't log anything if it finds
one that works. In the context, that can be mildly confusing,
beautify the output somewhat.
Signed-off-by: Jan Lindemann <jan@janware.com>
base-url is not used as a prefix in its entirety, but massaged in a
janware-specific way. Still is, but at least this commit is a step
towards being more generic.
Signed-off-by: Jan Lindemann <jan@janware.com>
This is a code maintenance commit: some run_xxx() helper functions
take a string, some a list, and some just digest all arguments and
pass them on as a list to exec() to be executed. That's highly
inconsistent. This commit changes that to list-only.
Except for the run_cmd() method of SSHClient, which is still run as a
shell method, because, erm, it's a shell. Might be changed in the
future for consistency reasons.
Signed-off-by: Jan Lindemann <jan@janware.com>
Replace --source-profile by --env-reinit and --env-keep to allow more
fine-grained control over environment manipulation.
Signed-off-by: Jan Lindemann <jan@janware.com>
