BaseCmdPkgRelations contains pkg_relations(), a function doing
package graph analysis code. The function needs to be made available
to code outside BaseCmdPkgRelations, so move it to
cmds.projects.lib.pkg_relations.
The commit also applies style fixes to both BaseCmdPkgRelations and
pkg_relations which anticipate broader style changes to jw-pkg in
general.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use tmpl_render() to render the file template. Better for centralized
template definition.
The commit also applies style fixes to CmdCreatePkgConfig which
anticipate broader style changes to jw-pkg in general.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add tmpl_render(), a function to provide a primitive template
renderer. It takes a dictionary for values to replace variables
shaped {some-variable} in templates found by their name. For now, the
templates are defined in the templates module instead of being read
from a template directory. The values may be lists, in which case
they are rendered with a delimiter, defaulting to ",".
Using an existing template engine like jinja2 is tempting but would
introduce additional dependencies jw-pkg is trying hard to avoid.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a sub-module for code that's too specific to jw.pkg.cmds.projects
to go into jw.pkg.lib but too generic to go into a command module.
Long-term, it might be a good idea to create a place for code which
jw-pkg doesn't exclusively use for its own purposes. jw.lib, for
example. Then, liberated from the burden to be generally useful also
externally, jw.pkg.lib might be a better fit for code currently in
jw.pkg.cmds.xxx.lib, and a more natural place usable across
subcommands.
Signed-off-by: Jan Lindemann <jan@janware.com>
Lots of sub- and sub-subcommands are derived from the base class of
the invoking command, notably below cmds.projects. That provides some
properties shared across the ancestor hierarchy of a command, but is
semantically unsound. Introduce jw.pkg.BaseCmd class as a place to
provide basic helpers shared across all commands used in a
jw.pkg.App's context. Also add cmds.projects.Cmd to be used by other
commands in cmds.projects in later commits.
Signed-off-by: Jan Lindemann <jan@janware.com>
Not all __init__.py modules are generated by python-tools.sh, some
are needed early to make jw-pkg useful without generation, notably in
jw.pkg.cmds.
Add detect_modules() to unify that detection, and place it into a
minimal module lib.init to not increase startup time cost.
Signed-off-by: Jan Lindemann <jan@janware.com>
janware upstream development server moved from git.janware.com to
devgit.janware.com. This commit follows the move with pretty much a
simple
s/git.janware.com/devgit.janware.com/
over jw-pkg. It found 14 matches, that's pretty bad.
FIXME: Reduce the redundancy, or, better, replace the
devgit.janware.com goodies by a more generally useful concept
altogether.
Signed-off-by: Jan Lindemann <jan@janware.com>
Argument ordering in the call to distro.select() is mixed up, the
filter is mistaken for names, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Exceptions raised by the build command are handled and changed,
messing up the stack trace. Re-raise the original exception from the
exception handler to fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add Cmd.print_help(). By default, it prints a help message. If passed
an integer exit_status, it also calls sys.exit(exit_status).
Signed-off-by: Jan Lindemann <jan@janware.com>
Push cmds.Cmd._add_subcommands() as lib.Cmd.load_subcommands() one
step up to the top of the type hierarchy ladder.
By default, it does the same thing, i.e. load subcommands matching
frobnicate.Cmd* if called on class CmdFrobnicate.
This commit also replaces invocations of Cmd._add_subcommands() by
invocations of this new method.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make "secrets install" digest archives with a more elaborate layout.
It selects secrets from archives containing:
default/path/to/secret.jw-secret
com/janware/grautvornix/path/to/secret.jw-secret
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a module cmds.secrets.lib.tar. Secrets handling demands treating
tar archive members more individually, jw.pkg.lib.TarIo is not a good
fit for that, so try with a different module. To be merged
eventually.
Signed-off-by: Jan Lindemann <jan@janware.com>
Enclose ExecContext._run() in an open() / close() - pair. This is
convenient for the caller in that it doesn't need to take care of
opening and closing for one call only, and inconvenient in that it
forces the caller to conciously add an open() / close() - pair around
multiple run() calls where it wants the context to stay open in
between. Or use the ExecContext as a context manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
There are a couple of assert statements in the codebase which can
make jw-pkg fail without any detail whatsoever if --backtrace is not
specified, fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add DistroContext.install(). It takes a tar file containing secrets,
decrypts it, and installs all secrets needed on target and present in
that file. For every file that should be extracted, it logs if it
acutally did something or didn't.
It also features an only_missing argument, which is just a stub for
"allow to define somy extraction policy with respect to replacing all
/ some / not replace / whatever. Not thought through.
Signed-off-by: Jan Lindemann <jan@janware.com>
Allow to pass ready-made FileContext objects to CopyContext's
destructor so it doesn't need to instantiate them itself.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename command "distro" to "pkg" together with "info", its last
remaining subcommand. "distro" is often used in the sense of "Linux
distribution", which would be too narrow for the targets jw-pkg could
theoretically support.
Signed-off-by: Jan Lindemann <jan@janware.com>
With the exception of the "info" subcommand, nearly all of distro's
subcommands deal with package managing, so push them into their own
command category.
Signed-off-by: Jan Lindemann <jan@janware.com>
Cosmetics: The "packages" parameter to some DistroContext's methods
has a confusing name in same contexts, notably when mixed with a list
of Package instances, so rename it to "pkg_names".
Signed-off-by: Jan Lindemann <jan@janware.com>
Commit a19679fec reverted the first attempt to make AsyncSSH reuse
one connection during an instance lifetime. That failed because a lot
of distribution-specific properties were filled in a new event loop
thread started by AsyncRunner, and AsyncSSH didn't like that.
This commit is the first part of the solution: Move those properties
from the App class to the Distro class, and load the Distro class
in an async loader. As soon as it's instantiated, it can provide all
its properties without cluttering the code with async keywords.
Signed-off-by: Jan Lindemann <jan@janware.com>
The name of the env parameter to ExecContext.run() and .sudo() is not
descriptive enough for which environment is supposed to be modified
and how, so rename and split it up as follows:
- .run(): env -> mod_env
- .sudo(): env -> mod_env_sudo and mod_env_cmd
The parameters have the following meaning:
- "mod_env*" means that the environment is modified, not replaced
- "mod_env" and "mod_env_cmd" modify the environment "cmd" runs in
- "mod_env_sudo" modifies the environment sudo runs in
Fix the fallout of the API change all over jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
The "secrets" class of commands currently only works on the host it's
invoked on. Use the current FileContext to allow using the existing
commands on a target host.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't pass mode as a string to put(). Given the multitunde of
possible string representations for numbers, some understood by
int(string, 0) and some not, there's too much room for passing
strings which are unparseable, or worse, prone to be parsed wrongly.
However, pass mode down to _put() as a string for convenience,
because that's what most _put() implementations will need to use. If
they don't, converting to int is easy from the one defined string
format.
Signed-off-by: Jan Lindemann <jan@janware.com>
Expand platform macros (as in %{codename}) in the src and dst command
arguments. Expansion can be turned off by -F --fixed-strings.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a method distro_info() to App, essentially providing CmdInfo's
macro-expansion of platform information to other interested code.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add lib.base to provide basic definitions.
For now, move the definiions of Result, Input and InputMode from
ExecContext into lib.base. Having to import them from the ExecContect
module is too heavy-handed for those simple types.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add CmdCopy, designed to copy data from the filesystem to another
location in the filesystem. Not necessarily local file systems, the
URLs can be all URLs supported by ExecContext.run().
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a new group of commands - "posix". The current command categories
are not a good fit for that: "projects" is for CI, "distro" is
distribution-specific for CD, and secrets is for handling secrets
specifically. Introduce the more general command group "posix", a
class of commands not POSIX compliant in the exposed API, but
primarily using POSIX utilities as workhorse.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add ExecContext.close() as a hook to clean up async resources living
longer than an ExecContext method call.
Also, implement __aenter__() and __aexit__(), to allow using
ExecContext as context manager. close() is invoked it goes out of
scope.
Signed-off-by: Jan Lindemann <jan@janware.com>
The Input instance passed as cmd_input to ExecContext.run() and
.sudo() currently may be of type str. Allow to pass bytes, too.
At the same time, disallow None to be passed as cmd_input. Force the
caller to be more explicit how it wants input to be handled, notably
with respect to interactivity.
Along the way fix a bug: Content in cmd_input should result in
CallContext.interactive == False but doesn't. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdCanonicalizeRemotes / canonicalize-remotes and the respective
target in topdir.mk remove the /srv/git portion from all remotes'
URLs pointing to git.janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
To be able to use secret handling code from other modules, move the
bulk of it from the "secrets"-command centric implementation in
cmds.secrets.Cmd into a new module cmds.secrets.lib.util.
Signed-off-by: Jan Lindemann <jan@janware.com>
Call run_curl() with parse_json=True to make that explicit, and be a
little more verbose about the outcome.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use the global --verbose and --interactive command-line options as
defaults for constructing a SSHClient instance for use with listing
repos over SSH.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support --hide-jw-pkg. This is a step towards replacing
required-os-pkg, which leaves out packages from pkg.requires.jw
sections.
Signed-off-by: Jan Lindemann <jan@janware.com>
If a package P is added, only those of its dependendencies are added
along which are in the same os-cascade section as P. That's wrong,
fix it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make variable names a little more readable and searchable within the
long pkg_relations_list() method by making their names longer and
truer to what they actually mean.
Signed-off-by: Jan Lindemann <jan@janware.com>
