FileContext's _open() and _close() are called everytime their wrapper
is called, which tasks the caller with keeping track of whether they
were already called or not. Be a little easier on the caller, keep
track in an open count, and call _open() only once for multiple calls
to open(), and close() likewise. The caller still needs to make sure
the number of open() and close() calls matches.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename command "distro" to "pkg" together with "info", its last
remaining subcommand. "distro" is often used in the sense of "Linux
distribution", which would be too narrow for the targets jw-pkg could
theoretically support.
Signed-off-by: Jan Lindemann <jan@janware.com>
With the exception of the "info" subcommand, nearly all of distro's
subcommands deal with package managing, so push them into their own
command category.
Signed-off-by: Jan Lindemann <jan@janware.com>
--uri is unnecessarily generic in that it could mean the URI of
anything. --target makes it clearer that operations are to be
exectuted on that target.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add keyword-argument description to Cmd.__init__(), and default it to
help. Also, add a property .description returning it, and add it to
add_parser() so that it shows up in the usage message.
Signed-off-by: Jan Lindemann <jan@janware.com>
pgit.sh get is not invoked with --create-remote-user-repos by
default. For the janware Git servers, this makes CI fail over new
repos: Repos from the maintainers are pulled, but before
pkg-rebuild-reinstall, repos that the devops user has no remote repo
for are purged again and are subseqently missing during build.
Passing --create-remote-user-repos to all pgit.sh get invocations
should fix the problem, so do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a CopyContext class. At this point it mostly acts as a context
manager for two FileContext instances, and copying data is the
canonical case to use it, hence the name.
Signed-off-by: Jan Lindemann <jan@janware.com>
- Add optional in_pipe and out_pipe parameters to __init__()
- Add a add_proc_filter() method
Add possibilites to attach input / output pipes to a FileContext
instance. Data will be passed through the input pipe between ._get()
and .get(), and through the output pipe between .put() and _put().
Signed-off-by: Jan Lindemann <jan@janware.com>
Add an async open() method which should allow to do what __init__()
couldn't, because it's not async, and to match the already existing
.close(). It's called by __aenter__() __aexit__() if the FileContext
is instantiated as context manager, or at will when the user finds it
a good idea.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add .mkdir() to the API which should do the expected, and implement
it in ExecContext and Local specializations.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a bool parameter "chroot" to __init__(). If passed, all
path-centric operations act as if .root was prepended to it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a .root property, returning the path part of the instantiating
URI. Nothing meaningful beyond returning it is done with it so far.
Signed-off-by: Jan Lindemann <jan@janware.com>
Each wrapper method throws a NotImplementedError by default. Make
that error a little more descriptive in case it really gets thrown.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add .schema_from_uri(), .split_uri(), .id() to define some
standardish way to dissect an URI the same way FileContext does.
Signed-off-by: Jan Lindemann <jan@janware.com>
Code beautification: __init__() doesn't use the arguments it grabs by
name from its parameter list, use *args and **kwargs instead.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a ProcFilter implementation for decrypting GPG blobs. It needs an
ExecContext passed to __init__() in order to run /usr/bin/gpg.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the ProcFilter module, containing the following classes:
- ProcFilter:
Abstract base class defining a run(data: bytes) -> Result method,
allowing to do arbitrary data manipulation in subclasses.
- ProcFilterIdenty
A ProcFilter specialization which passes through the input
unchanged.
- ProcPipeline
A container for multiple chained ProcFilter classes
Signed-off-by: Jan Lindemann <jan@janware.com>
Cosmetics: The "packages" parameter to some DistroContext's methods
has a confusing name in same contexts, notably when mixed with a list
of Package instances, so rename it to "pkg_names".
Signed-off-by: Jan Lindemann <jan@janware.com>
Add an optional "filter: PackageFilter|None" parameter to .select(),
and if it's not None, call a new version of ._select() with it.
._select() is not abstract anylonger. Its default implementation
filters the results of ._select_by_name(), can be reimplemented by
deriving classes for better performance, but doesn't have to.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename ._select() to _select_by_name() in Distro and its subclasses.
Don't rename .select() itself, because it's going to be a broader
interface supporting more select criteria than just package names.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add global --pkg-filter argument, defaulting to
JW_DEFAULT_PKG_FILTER. If it's specified, instantiate a
PackageFilterString from it, and initialize App's Distro instance
with it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a default_pkg_filter parameter to Distro's constructor defaulting
to None, and expose it via the .default_pkg_filter property. As of
this commit, no code in jw-pkg does anything meaningful with it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a package filter abstraction designed to replace the package
filter string tossed around various functions througout jw-pkg.
Signed-off-by: Jan Lindemann <jan@janware.com>
- Remove _run_on_conn() because it doesn't add any value
- Add verbose try-except block around connect()
- Add try-except block around failing close
- Prefix private member variables with "__"
Signed-off-by: Jan Lindemann <jan@janware.com>
Commit a19679fec reverted the first attempt to make AsyncSSH reuse
one connection during an instance lifetime. That failed because a lot
of distribution-specific properties were filled in a new event loop
thread started by AsyncRunner, and AsyncSSH didn't like that.
The last commit provided the needed properties as members of the
Distro class. This commit is the second part of the solution: Keep
one connection around as a class member and reuse it on every _run()
invocation.
Signed-off-by: Jan Lindemann <jan@janware.com>
Commit a19679fec reverted the first attempt to make AsyncSSH reuse
one connection during an instance lifetime. That failed because a lot
of distribution-specific properties were filled in a new event loop
thread started by AsyncRunner, and AsyncSSH didn't like that.
This commit is the first part of the solution: Move those properties
from the App class to the Distro class, and load the Distro class
in an async loader. As soon as it's instantiated, it can provide all
its properties without cluttering the code with async keywords.
Signed-off-by: Jan Lindemann <jan@janware.com>
