Commit graph

420 commits

Author SHA1 Message Date
f6f6e56943
make / scripts: git.janware.com -> devgit.janware.com

janware upstream development server moved from git.janware.com to devgit.janware.com. This commit follows the move with pretty much a simple

s/git.janware.com/devgit.janware.com/

over jw-pkg. It found 14 matches, that's pretty bad.

FIXME: Reduce the redundancy, or, better, replace the devgit.janware.com goodies by a more generally useful concept altogether.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-12 15:15:29 +02:00
237096cac9
lib.Uri: Add new convenience properties

Add a couple of properties and methods that come in handy when manipulating URIs:

.path .safe_full_with_username

do the obvious, and these return new Uri objects with modified paths:

.new_add_path() .new_replace_path()

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-06 15:26:14 +02:00
c9b48880a9
cmds.pkg.CmdSelect: Fix argument order in select()

Argument ordering in the call to distro.select() is mixed up, the filter is mistaken for names, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-02 07:19:41 +02:00
fc3ac69bd4
cmds.projects.CmdBuild: Don't change handled Exception

Exceptions raised by the build command are handled and changed, messing up the stack trace. Re-raise the original exception from the exception handler to fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-02 06:59:24 +02:00
30abb227c7
lib.Cmd.print_help(): Add method

Add Cmd.print_help(). By default, it prints a help message. If passed an integer exit_status, it also calls sys.exit(exit_status).

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-01 15:57:16 +02:00
7e2099877c
lib.Cmd.load_subcommands(): Add method

Push cmds.Cmd._add_subcommands() as lib.Cmd.load_subcommands() one step up to the top of the type hierarchy ladder.

By default, it does the same thing, i.e. load subcommands matching frobnicate.Cmd* if called on class CmdFrobnicate.

This commit also replaces invocations of Cmd._add_subcommands() by invocations of this new method.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-05-01 15:56:08 +02:00
7dfe733865
cmds.secrets.lib.DistroContext.install(): Full archive

Make "secrets install" digest archives with a more elaborate layout. It selects secrets from archives containing:

default/path/to/secret.jw-secret com/janware/grautvornix/path/to/secret.jw-secret

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:30 +02:00
530efa1427
cmds.secrets.lib.tar: Add module

Add a module cmds.secrets.lib.tar. Secrets handling demands treating tar archive members more individually, jw.pkg.lib.TarIo is not a good fit for that, so try with a different module. To be merged eventually.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:30 +02:00
5bb7fe96db
lib.util.get(): Add function

Add get(), which does pretty much what FileContext.get() does, but with auto-instantiating a FileContext instance. Input processing filters can be passed, too, all *args and **kwargs are passed unchanged to the FileContext's constructor.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
fe1eb0014a
lib.ProcFilter: Add run()

Add a run() function. It constructs a pipeline from various possible types, and runs its data parameter through it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
1c11503dfb
lib.Distro._install(): Implement remote installation

lib.Distro._install()'s default implementation allows to install packages specified as direct links, but only to the local machine. Implement installation to arbitrary hosts specified with --target.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
60de1b0ca0
lib.Distro.pkg_ext: Fix call to os_cascade

The Distro.pkg_ext property calls .os_cascade, which is a property, with parenthesis. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
5837d10a1c
lib.util.copy(): Tolerate list[str] and FileContext args

Modify copy():

- Allow the source argument src_uri to be a list of URIs, in which case copy should return a list of paths instead of one path
- Change the dst_uri parameter to dst, signalling that it now also tolerates a FileContext
- Use the CopyContext class to manage the source and target FileContext instances
Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
d9746cd20b
lib + cmds.projects: Use lib.Uri

Remove the feeble attempts at unifying URI handling, and use class Uri from lib.Uri instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
f0eeb14a97
lib.base.Uri: Add module

Add a URI abstraction module. The class is designed to replace less unified attempts at URI parsing throughout the jw-pkg code base.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
6cfb86d2a7
lib.FileContext.is_dir(): Add follow_symlinks

Make FileContext.is_dir() usable: - Add follow_symlinks parameter meant to do the obvious - Fix missing await

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
b5762116a1
lib.FileContext.get(): Remove bogus owner, group, mode

owner, group or mode don't belong in .get()'s signature, remove them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:29 +02:00
6ced64202e
lib.ExecContext._stat(): Don't decode stderr twice

The output of /usr/bin/stat is decoded once the Result object is decoded as a whole, and then again individually, which fails of course. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-28 13:00:28 +02:00
f23c0560e5
cmds.secrets.lib.util.compile_template_file(): Fix

Fix broken call to FilesContext(ctx).compile_template_file() after trying it out the first time.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 13:51:12 +02:00
37fcf56ad2
lib.FileContext._mktemp(): Fix missing self before _chroot()

_mktemp() call _chroot() instead of self._chroot(), fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 13:50:42 +02:00
66b968c51c
TarIo.TarIoTarFile._extract(): Fix typo

TarIoTarFile is currently unused but should still be correct, so fix a typo in _extract().

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 08:47:45 +02:00
069899e48e
lib: Remove leftover breakpoints

Remove two breakpoints not belonging in the code of TarIo and Types.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 08:47:21 +02:00
a9475de48e
lib.ExecContext: Add open() + close() around _run()

Enclose ExecContext._run() in an open() / close() - pair. This is convenient for the caller in that it doesn't need to take care of opening and closing for one call only, and inconvenient in that it forces the caller to conciously add an open() / close() - pair around multiple run() calls where it wants the context to stay open in between. Or use the ExecContext as a context manager.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 08:45:29 +02:00
3ac3aff997
lib: Fix silent assertitons

There are a couple of assert statements in the codebase which can make jw-pkg fail without any detail whatsoever if --backtrace is not specified, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-25 08:44:49 +02:00
120a6e4d34
cmds.secrets.CmdInstall: Add command

Support

jw-pkg secrets install mypasswords.tar.gpg [package-1 [package-2 ...]]

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
b88a8ee299
cmds.secrets.lib.DistroContext.install(): Add method

Add DistroContext.install(). It takes a tar file containing secrets, decrypts it, and installs all secrets needed on target and present in that file. For every file that should be extracted, it logs if it acutally did something or didn't.

It also features an only_missing argument, which is just a stub for "allow to define somy extraction policy with respect to replacing all / some / not replace / whatever. Not thought through.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
db22f11b32
cmds.posix.tar.CmdExtract: Log number of extracted files

Add a little log line at execution end, saying how many files were actually extracted.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
7309ec687e
lib.TarIo.extract(): Return list of extracted files

Make TarIo.extract return the list of files that were actually extracted.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
f2ca32a343
CopyContext: Replace src_uri, dst_uri by src, dst

Allow to pass ready-made FileContext objects to CopyContext's destructor so it doesn't need to instantiate them itself.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
772512aee0
FileContext.open() / close(): Only call wrapped once

FileContext's _open() and _close() are called everytime their wrapper is called, which tasks the caller with keeping track of whether they were already called or not. Be a little easier on the caller, keep track in an open count, and call _open() only once for multiple calls to open(), and close() likewise. The caller still needs to make sure the number of open() and close() calls matches.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
514d66dac1
cmds.distro.CmdInfo: Rename to cmds.platform.CmdInfo

Rename command "distro" to "pkg" together with "info", its last remaining subcommand. "distro" is often used in the sense of "Linux distribution", which would be too narrow for the targets jw-pkg could theoretically support.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
5830984bfc
cmds.distro.CmdXxx: Rename to cmds.pkg.CmdXxx

With the exception of the "info" subcommand, nearly all of distro's subcommands deal with package managing, so push them into their own command category.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
c19111e5b6
App: Rename global option --uri -> --target

--uri is unnecessarily generic in that it could mean the URI of anything. --target makes it clearer that operations are to be exectuted on that target.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:55 +02:00
563ec7463a
cmds.secrets.Cmd: Fix type hints

cmds.secrets.Cmd has a CmdDistro-typed parent, should be CmdSecrets, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:54 +02:00
cc6febcda3
lib.Cmd: Add description property

Add keyword-argument description to Cmd.__init__(), and default it to help. Also, add a property .description returning it, and add it to add_parser() so that it shows up in the usage message.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-24 16:53:54 +02:00
7357092576
cmds.posix.Tar: Add command

Add a "jw-pkg posix tar" command. Currently the only thing that can be done with it is extracting.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 20:18:42 +02:00
a5e7647026
lib.TarIo: Add module

Add a class providing a method to read tar files from a FileContext, and extract them to another.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 20:18:42 +02:00
d82bc20663
lib.CopyContext: Add module

Add a CopyContext class. At this point it mostly acts as a context manager for two FileContext instances, and copying data is the canonical case to use it, hence the name.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 20:18:42 +02:00
4c81647bbe
lib.FileContext: Support attaching data filters

- Add optional in_pipe and out_pipe parameters to __init__()

- Add a add_proc_filter() method

Add possibilites to attach input / output pipes to a FileContext instance. Data will be passed through the input pipe between ._get() and .get(), and through the output pipe between .put() and _put().

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 20:18:40 +02:00
58f7997bc6
lib.FileContext.open(): Add method

Add an async open() method which should allow to do what __init__() couldn't, because it's not async, and to match the already existing .close(). It's called by __aenter__() __aexit__() if the FileContext is instantiated as context manager, or at will when the user finds it a good idea.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:32:34 +02:00
432830a5c5
lib.FileContext.mkdir(): Add method

Add .mkdir() to the API which should do the expected, and implement it in ExecContext and Local specializations.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:30:22 +02:00
7547d13a6d
lib.FileContext: Support chroot

Add a bool parameter "chroot" to __init__(). If passed, all path-centric operations act as if .root was prepended to it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:30:08 +02:00
b52264dfad
lib.FileContext.root: Add property

Add a .root property, returning the path part of the instantiating URI. Nothing meaningful beyond returning it is done with it so far.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:29:20 +02:00
6a3cf7a283
lib.FileContext: Beautify NotImplementedErrors

Each wrapper method throws a NotImplementedError by default. Make that error a little more descriptive in case it really gets thrown.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:28:54 +02:00
3360ec86cb
lib.FileContext: Add URI manipulation methods

Add .schema_from_uri(), .split_uri(), .id() to define some standardish way to dissect an URI the same way FileContext does.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:27:38 +02:00
efe047fa50
lib.ExecContext: Strip unnecessary args from __init__()

Code beautification: __init__() doesn't use the arguments it grabs by name from its parameter list, use *args and **kwargs instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:26:42 +02:00
d70454b32a
lib.ProcFilterGpg: Add module

Add a ProcFilter implementation for decrypting GPG blobs. It needs an ExecContext passed to __init__() in order to run /usr/bin/gpg.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:25:57 +02:00
411bfd41ca
lib.ProcFilter: Add module

Add the ProcFilter module, containing the following classes:

- ProcFilter:

Abstract base class defining a run(data: bytes) -> Result method, allowing to do arbitrary data manipulation in subclasses.

- ProcFilterIdenty

A ProcFilter specialization which passes through the input unchanged.

- ProcPipeline

A container for multiple chained ProcFilter classes
Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-23 15:23:27 +02:00
32c3477102
cmds.secrets.lib.DistroContext.list_template_files(): List more

Default to Distro's idea of relevant packages if no packages are specified on the command line.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-21 21:52:10 +02:00
655b17bfec
cmds.secrets.lib.DistroContext: packages -> pkg_names

Cosmetics: The "packages" parameter to some DistroContext's methods has a confusing name in same contexts, notably when mixed with a list of Package instances, so rename it to "pkg_names".

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-21 21:52:10 +02:00