The commit message of 29abf14a6 clearly states
As of this commit, this is the case by default, i.e. PY_UPDATE_INIT_PY is
set to false in py-ns-dir.mk, maintaining the current behaviour.
The commit's diff then proceeds to set PY_UPDATE_INIT_PY to true. Which is
a clear-cut case of my brain doing something different from what my fingers
do.
Correcting the commit. Some downstream projects have already consumed this
blunder. Not quite sure where that leaves us, but we need to start
somewhere with fixing, so I decide to make the code match the documented
behaviour, and wait for the fallout to fix.
Signed-off-by: Jan Lindemann <jan@janware.com>
The framework instantiates command classes in two places: the top-level
commands in App and the lazily materialized subcommands in
Cmd.add_subcommands(). Both pass the parent parameter positionally.
Change that to passing parent by keyword at both call sites. Every command
class names its first __init__() parameter parent, so the keyword form
binds the same slot for all of them regardless of parameter order, and it
allows a command class to declare parent as keyword-only.
Signed-off-by: Jan Lindemann <jan@janware.com>
The top-level all, test, check, check-pre, and check-post targets invoke
the build tool without --dep-flavours, so it falls back to the auto default
and only resolves dependencies of the flavour "build". The build-order-%
target, by contrast, already passes the run,build,test,release set via a
single BUILD_ORDER_DEP_FLAVOURS variable. Planning and building therefore
rely on different dependency flavours, and none of the targets can be tuned
individually.
Introduce one flavour variable per target group, all of them defaulting to
the run,build,test,release set:
- BUILD_DEP_FLAVOURS for all
- BUILD_DEP_FLAVOURS_LINT for check, check-pre, and check-post
- BUILD_DEP_FLAVOURS_TEST for test
- BUILD_DEP_FLAVOURS_ORDER, renamed from BUILD_ORDER_DEP_FLAVOURS, used by
build-order-%
Split the combined all test check check-pre check-post rule into three
rules so that each group passes its own --dep-flavours to the build tool.
all, test, and the check targets now also resolve run, test, and release
dependencies, as build-order already does, and each target's flavour set
can be overridden independently.
Note that they all default the whole shebang: run,build,test,release. And
this opens up the door to dependency cycles that don't exist with the
subset of projects currently used. It's still better to ask for all and
then cut down as needed instead of starting small.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
The spec generator only supports Requires, Conflicts and Provides from
the [pkg.requires.*], [pkg.conflicts.*] and [pkg.provides.*] sections of
make/project.conf. A package that works without but is better with another
package, like jw-amavis-run without jw-clamav-run, must hard-require it, so
the dependency is pulled in on every install and cannot be removed without
dragging the depending package along.
This commit adds Recommends support to the spec generation chain:
- CmdPkgRecommends: new pkg-recommends command that reads the
[pkg.recommends.<flavour>] sections, same base as pkg-requires
- pkg-dist.mk: compute PKG_RECOMMENDS_RUN and pass it to pkg.sh via a new
-E option
- pkg.sh: accept -E and forward it to the mkspec wrapper
- mkspec-wrapper.sh: accept -E and export RECOMMENDS_RUN
- create-mkspec.sh: emit a Recommends: line when RECOMMENDS_<subpkg> is set
- create-mkdebian.sh: emit a Recommends: field in the debian control file
Projects without a [pkg.recommends.*] section generate unchanged specs:
every new line is guarded by a non-empty test. zypper follows Recommends by
default, so existing OpenSUSE installations are unaffected; a recommended
package can be removed without dragging the depending package along.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
A workspace project directory that has a Makefile but no package proofs -
no project.conf, no VERSION - is buildable with a plain make, yet the build
fails on it: calculate_order() walks the requested modules, and the walk in
App.__get_project_refs() raises an unmet dependency, because the
installation state of such a project is unprovable. The walk has to stay
strict: its proofs are package proofs, and loosening App.is_installed()
would weaken the unmet dependency detection for every get_project_refs()
caller.
This commit therefore keeps the tolerance in the build: before calculating
the order, run() partitions the requested modules. A module that is not
installed in any flavour but resolves to a workspace directory with a
Makefile is appended to the build order as a dependency-less leaf and built
with a plain make; modules without any directory at all still raise the
unmet dependency error. A jw dependency declared in a project.conf on such
a project still raises, which is correct: it is a declared dependency on a
package that does not exist.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The "Remove everything non-essential for make clean all" commit (bc883deed)
dropped disabled.mk, judged non-essential from the jw-pkg tree's own
perspective. Legacy projects still include it, though: their Makefiles use
it to replace a directory build with no-op targets that report the target
as disabled. Without the file, make fails in such projects in any workspace
that mounts a current jw-pkg.
Re-add it verbatim. The post-purge re-adds dummy.mk and dummy-topdir.mk
cover the current-style stubs; disabled.mk covers the legacy includes.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a run-success.mk which provides a $(RUN_SUCCESS_CREATE_MARKER)
variable, and include it from run.mk and py-run.mk. If it's not overridden,
it logs the current commit to a marker file if the run succeeds. The marker
file is ignored from Git and cleaned in the context of the distclean
target.
The machinery can be customized by variables:
RUN_SUCCESS_MARKER is the name of the marker file
RUN_SUCCESS_RECORD_CMD is the generator for its content
For example, putting this into local.mk or jw-pkg/make/local.mk will record
the hashes of every repository in the entire workspace:
define RUN_SUCCESS_RECORD_CMD
make git-log-1
make git-log-tree-1
endef
Signed-off-by: Jan Lindemann <jan@janware.com>
The git targets of the workspace Makefile cover push, diff, and status, but
not history: inspecting commits means running git log by hand in a project
directory, and no target shows the history of all projects of a tree at
once.
Add git-log targets at both levels; the target suffix is passed to git log
as-is, so make git-log-5 asks for the last five commits.
- git-log-% in projects-dir.mk runs pgit.sh log over every project of the
tree, one section per project
- git-log-% in rules.mk runs git log -$* in the current project directory
- git-log-tree-% in rules.mk runs git-log-$* in $(PROJECTS_DIR), giving the
whole-tree view from inside a project; it prints a notice if PROJECTS_DIR
is missing
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
The source hash is an md5sum over all versioned files in the working tree.
md5sum follows symlinks, so a project with links to files outside the tree
may produce different hashes depending on files outside the repo.
Compute the hash over the VCS file ids instead. Add a -s option to scm.sh
ls-files that lists each versioned file together with its content id. The
listing is sorted by path rather than by id, so an entry keeps its position
when its content changes.
For Git repos, git ls-tree -r HEAD reports the blob of a symlink's target
string and the pinned commit of a submodule, so the resulting hash depends
only on the committed tree, never on the working tree or on the machine.
calculate_hash() and the HASH variable in make/pkg-dist.mk feed the id
listing, minus the release metadata files, directly to md5sum.
calculate_hash() fails loudly when the pipeline fails, with its callers
refusing to proceed on an empty result.
The scheme change invalidates the stored HASH values, so every project
gets one more release the first time it runs against the new code, and the
hashes are stable again afterwards.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
In CVS, the -f option keeps directory entries out of scm.sh ls-file's
listing; in Git, it's a silent no-op. Notably symlinks are not excluded
despite -f.
Implement -f for git with a mode filter on git ls-files -s output: only
100644 and 100755 entries are listed, so the option means the same thing in
both code paths.
For Git, this actually means a behaviour change which needs to be fixed:
The dist archive targets tar-files and cpio-files in make/list-files.mk and
the scm_files() helper in scripts/pkg.sh then may no longer pass it to get
unchanged behaviour.
The targets list-files and list-text-files keep it, so the text-files cache
now matches its documented regular-files-only membership.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Use Version.next() in __version_boundaries, which steps the last existing
part, so '= 1.0' spans '>= 1.0, < 1.1'.
Tests written by AI.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Range expansion needs a bound that steps the last existing part of a
version, add that.
next() increments the last part, whatever it is: next of '1' is '2', of
'1.0' is '1.1', of '1.2.3' is '1.2.4', of '1.2.3-45' is '1.2.3-46'.
Tests written by AI.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
A multi-boundary constraint is rendered as 'foo >= 1.2.3-45 foo < 1.2.4',
which is invalid: the RPM spec template writes the Requires: line from it
verbatim, and RPM entries are comma-separated, so the second clause is
swallowed into one bad entry. Debian's format_depends() normalizes runs of
whitespace to commas, so it tolerates the space join, but the comma is the
only delimiter that is correct for both.
Join the boundary clauses with ', ' instead of a space.
Also:
- Drop the stray trailing comma in the version_boundaries() call.
- Fix the untemplated docstring, which describes the opposite of the code:
untemplated = True (the default) resolves the macros, untemplated = False
keeps them as written.
Tests written by AI.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Full version dependency specs ("= 1.2.3-4" or "= VERSION-REVISION") don't
need expansion, they pin the wanted version with an = fine exactly.
__version_boundary has that the wrong way around, fix that.
Also fix the is_full and version_boundaries() docstrings, which still
describe the old behavior.
Tests written by AI.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
__resolved_id() returns the full version if only VERSION was specified, fix
that.
Also: raise Version.Error instead of a bare Exception.
Tests written by AI.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
Running make format with the new isort setting rewrites the two wrapped
import blocks:
- The argparse import is split one-per-line, since the logical line exceeds
88 columns
- A trailing comma is added to both imports, which makes yapf keep the
split layout instead of re-joining it
The result is a fixed point that both isort and yapf accept, so make format
and make check agree.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
The isort configuration leaves include_trailing_comma at its default, false,
so isort strips the trailing commas from wrapped imports. However, yapf's
split_arguments_when_comma_terminated setting depends on those commas:
without them, yapf re-joins the one-per-line import layout that isort just
produced, and the two formatters disagree on files such as
src/python/jw/pkg/lib/App.py.
Set include_trailing_comma so that isort keeps trailing commas and yapf
honors them, making both tools converge on the same layout.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
conf/templates/gitignore contains duplicates (local.mk) and typos
(-test-out.txt). Fix them and run make gitignore again.
Signed-off-by: Jan Lindemann <jan@janware.com>
TarIo._match() and _filter_tar_file() are pure logic without test coverage.
Add unit tests for the exact path matching, the unfiltered round trip,
filtered extraction with a matched list, and a filter without hits.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
PackageFilterString is pure regex logic without test coverage.
Add unit tests for the url=~ filter, packages without a url, whitespace
around the operator, and the rejection of unsupported filter definitions.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
ProcPipeline and the run() helper are pure logic without test coverage.
Add unit tests for the identity filter, chained filter execution order, the
append() layouts, and the chain handling of run().
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
parse_spec_str(), parse_specs_str(), order_tags(), and __repr__ are pure
text processing without test coverage, shared by the dpkg and rpm package
manager backends.
Add unit tests for valid and invalid spec strings, multi-line input with
and without a trailing newline, tag ordering with default values, and the
repr layout.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
__init_askpass() embeds the password verbatim in a generated bash script,
with a newline included inside the double quotes of the echo -n. The
askpass program therefore always appends a newline to the password, and
any password containing quotes or shell metacharacters either breaks the
script or injects commands into it.
Embed the password as base64 and decode it with printf piped into base64
-d, so the script is safe for any password and prints the password exactly,
byte for byte.
Make __del__() idempotent: it deletes the environment variables and the
script file, so a second call, e.g. an explicit one followed by garbage
collection, raises KeyError and FileNotFoundError.
Add a unit test that executes the generated script and compares its output
with the password byte for byte.
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
expand_macros() looks up each %{name} via getattr with '-' turned into '_',
but macro_names advertises two names that don't resolve: 'os-release' has
no corresponding attribute, and 'os-cascade' resolves to the list property
os_cascade, which str.replace() rejects. Both macros are listed in the
platform info help output, yet always fail when used in a format string.
The space-joined cascade string is the cascade property, and 'cascade' is
the macro name the make recipes and scripts already use, together with the
--format default in cmds/platform/CmdInfo.py.
Before the App.distro_* properties moved to Distro, the advertised macro
names were derived from the distro_* attribute names, so the advertised
macro was %{cascade}; the hand-written macro_names replaced it with the
list attribute's name.
- Add the missing os_release attribute as an alias of os_release_str
- Advertise the 'cascade' macro in macro_names instead of 'os-cascade',
leaving os_cascade as the purely programmatic list
- Update the platform info help expected output accordingly
- Add unit tests for the os-release field parsers, the derived properties,
and the expansion of every advertised macro
Signed-off-by: Jan Lindemann <jan@janware.com>
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
JWB_SCRIPT_DIR bypasses $(JWBDIR) for its scripts candidate, re-assembling
./$(JWBDIR_NAME)/scripts from the bare name. That works, but is redundant
resolution machinery which should be removed per se.
Resolve the scripts directory from $(JWBDIR) instead. Aside from
simplifying, that fixes the resolution when projects-dir.mk is included
from Makefiles other than projects-dir-miminimal.mk, and complements work
to commit f3f0ef10ca which also aims at allowing to run make from a
bind-mounted dependency closure of repositories.
Signed-off-by: Jan Lindemann <jan@janware.com>
Update .gitignore by running "make gitignore. Add a couple of jw-pkg-
specific build-artifacts to GITIGNORE_PATTERNS_END before:
.cache-projects.mk
make/ssh-wrapper.sh
Also cancel the automatic inclusion of file paths containing jw-pkg with
an empty GITIGNORE_PATTERNS_TOP.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg-create-project.sh contains its own .gitignore template, which is
redundant to the new gitignore mechanism. Replace it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a target gitignore to topdir.mk, which blasts jw-pkg's default idea of
how .gitignore should look like into $(TOPDIR)/.gitignore. It's not a
prerequisite of any other target, because it's not meant to run
automatically. The idea is to keep some central maintenance of what should
be in there from jw-pkg's perspective, make it convenvient to update, but
leave the final decision to the commiter of any project using the
machinery.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add variables to shorten typical invocations of "jw-pkg project update
--format tmpl", and use them in py-topdir.mk
JW_PKG_TMPL_SEARCH_PATH
JW_PKG_TMPL_UPDATE
Signed-off-by: Jan Lindemann <jan@janware.com>
create-pkg-config's output is not covered by any test, even though the
template and the Requires cleanup produce exact text that pkg-config
consumes, so regressions in the generated file went unnoticed for a long
time.
Add an integration test that runs create-pkg-config with a full option set,
with space- and comma-mixed Requires input, and with a
Requires.private-only input, and compares the output against a reference
file.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
__cleanup_requires() replaces every run of whitespace with ", " before
re-pairing the version constraints, so input that is already
comma-separated, e.g. "jw-core >= 1.0, jw-base", comes out with a double
comma, "jw-core >= 1.0,, jw-base". And the Requires line is appended
without a trailing newline, so a following Requires.private line runs
straight into it.
Split the input on commas and whitespace, treating the version constraint
operators as delimiters that are re-paired with the preceding name and the
following version, so that comma- and space-separated input alike comes out
as a clean ", "-joined list. Add the missing newline after the Requires
line.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>
CmdCreateFile renders a file from project metadata, yet nothing runs it end
to end. The "tmpl" format substitutes --field values into template markers,
and the "pyright" format computes extra paths from the jw run dependencies
of a module. Both formats are untested, and so is the handling of missing
templates, unknown --field keys, and malformed command-line arguments.
Add an integration test under test/integration/jw-pkg/projects that drives
the command through the real CLI and diffs the rendered output against
a reference. It covers single- and multi-value field substitution, the
default and custom field separators including the %n newline escape, value
quoting, and --field-keys filling in keys the caller omitted. The pyright
case uses a relative topdir so the computed project paths stay relocatable.
The error cases check that the command rejects a missing template name, an
unknown template file, a --field key outside --field-keys, an unsupported
pyright separator, and invalid --format and --field arguments with the
expected message and exit status.
Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.85.1
Signed-off-by: Jan Lindemann <jan@janware.com>
