After release, pkg.sh pushes the changes to VERSION and HASH
upstream. Failures are masked, though, propagate them.
Unclear what motivated masking the error. Tracking that down with git
blame leads to build-package.sh, which was inherited from ytools,
where the change was introduced 2014 with the trust-inspiring
comment:
attempted fix for error during commit of version files in
build-package.sh
Signed-off-by: Jan Lindemann <jan@janware.com>
The generated code doesn't pass "make check": It would like to see a
newline after the import statement. Add that.
Signed-off-by: Jan Lindemann <jan@janware.com>
The previous commits have put rules for linting and formatting via
ruff, yapf, mypy and pyright into place. They are checked with the
make check target, and this commit adds the fixes for the target to
succeed.
It does some refactoring where type checking dug up dirty bits, and
also adds lots of churn in the Python code. To a good deal, that's
owed to mere formatting changes. It would have been better to
seperate those from syntax and refactoring fixes into multiple
commits, so that the interesting changes don't drown in the
formatting nose. However, that would have been a lot of additional
work only to be thrown away by later commits, hence this commit has a
big diff in one piece. The size of the diff is regrettable but
hopefully a one-off: What it buys is automatic format checking for CI
and predictble formats for smaller diffs in the future.
Rules that "make check" enforces are, in the following order
- Syntax checkers:
- ruff check .
- mypy .
- pyright
- Format check:
- yapf --diff --recursive .
The refactoring includes:
- Turn the Result class into a more elaborate object, capable of
doing more heavy lifting around stderr and stdout decoding,
summarizing outcome, and matching error strings.
Aside from fixing broken type checks, this also removes lots of
boilerplate calling code which is currently used for handling
possible call outcome scenarios. Trying to access an inexistent,
decoded string should raise a meaningful exception by itself now,
which removes lots of code with case distinctions.
- Fix Cmd type hierarchy:
- Add the AbstractCmd class above Cmd. This is necessary because
the checker rightfully complains it can't instantiate a Cmd
instance where constructor arguments were needed. They never
were, but the type used at the instantiating code's location in
jw.pkg.App so claims.
- Lots of sub- and sub-subcommands are derived from the base
class of the invoking command. That provides some properties
shared across the ancestor hierarchy of a command, but is
semantically unsound. Fix that by introducing jw.pkg.BaseCmd
class as a place to provide basic helpers shared across all
commands used in a jw.pkg.App's context, and derive all command
classes from that afresh. The parent command is still reachable
via a common parent property.
Formatting changes are conforming to PEP-8, mostly, with minor
tweaks. All in all they include the following changes.
- Remove # -*- coding: utf-8 -*-
The line was needed by Python 2 which is not supported anylonger.
For Python 3, the default encoding is UTF-8, anyway.
- Allow to run "make py-format" without having it produce any
changes. It's basically "yapf --in-place --recursive ." with some
code style settings, see conf/topdir/pyproject.toml. The settings
may be debatable. I've had custom tweaks in place on that target,
too, but then again, IDEs would have more hassle to integrate
that.
- Introduce a 88 character line length limit
- One import per line, reshuffle them semantically, see
[tool.isort] in pyproject.toml.
- Hide imports needed for type-checking only behind
if TYPE_CHECKING
- Spaces around assignments accounts for much churn. Having having
no spaces in inline parameter list assignments and default
parameter values would arguably be more compact where it's
useful. On the other hand, I have not found a code formatter
which allows spaces around assignments in parameter lists broken
into one per line and that's often better than a wall of text.
- Add two spaces before # export, as this seems to be mandated by
PEP-8
- Use single quotes by default
Signed-off-by: Jan Lindemann <jan@janware.com>
py-topdir.mk: Use ruff and yapf
- Use pyright for the target py-check-syntax
- Generate a $(TOPDIR)/pyrightconfig.json for that
- Add pyrightconfig-base.json because it's used by
pyrightconfig.json
- Add python3-pyright to pkg.requires.release, anticipating the use
of the py-syntax-check target by CI
Signed-off-by: Jan Lindemann <jan@janware.com>
- Use ruff and yapf for the targets py-check-syntax, py-format and
py-check-format.
- Add a pyproject.toml for those. It also includes configuration
for isort, albeit not being directly used in the linter targets.
- Make .gitignore igore that in newly created projects.
- Add ruff, yapf and isort to pkg.requires.release, anticipating
their use by CI.
Signed-off-by: Jan Lindemann <jan@janware.com>
The __init__.py files as gnerated by python-tools.sh contain multiple
issues, fix them:
- Make the machinery fail if the same type name is imported from
different modules
- Support relative imports from .Module import Module instead of
having to use the entire module path as import source
- Import types explicitly re-exported with "as":
from .Module import Module as Module
Otherwise ruff will regard the type as "imported but not used"
- Add "# ruff: noqa: E501" near the top. The import lines can get
long and are beyond manual control (except for renaming the
modules themselves, that is). This can cause ruff to fail, so get
it to accept long lines in __init__.py. The style violation
doesn't make much of a difference in generated code, anyway,
because nobody reads that. Plus what's happening in the code
isn't rocket science, so good style wouldn't help much with
understanding, either.
This promptly digs up two symbol name conflicts lib.pm.dpkg and
lib.pm.rpm. Fix them along with this commit to keep it from breaking the build.
Signed-off-by: Jan Lindemann <jan@janware.com>
Currently, completing a release works with a plain git push. It may
push to several repos, depending on how the client repo's origin's
pushurl is configured. Those repos may have different user names, and
if the ssh wrapper added -l via JW_PKG_SSH_EXTRA_OPTS, the push would
fail. Hence, disable JW_PKG_SSH_EXTRA_OPTS for that case.
Signed-off-by: Jan Lindemann <jan@janware.com>
janware upstream development server moved from git.janware.com to
devgit.janware.com. This commit follows the move with pretty much a
simple
s/git.janware.com/devgit.janware.com/
over jw-pkg. It found 14 matches, that's pretty bad.
FIXME: Reduce the redundancy, or, better, replace the
devgit.janware.com goodies by a more generally useful concept
altogether.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename command "distro" to "pkg" together with "info", its last
remaining subcommand. "distro" is often used in the sense of "Linux
distribution", which would be too narrow for the targets jw-pkg could
theoretically support.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg is copied into $(TOPDIR)/bin during build, that's wrong.
Write a rule precisely targeted at installing /usr/bin/jw-pkg, and
cut all the scripts.mk machinery.
Also, make jw-pkg a relative link to avoid the respective RPM
warning.
Signed-off-by: Jan Lindemann <jan@janware.com>
log_start_stop() is responsible for logging markers at the beginning
and end of a decorated log. They should not be applied if pgit.sh is
run with --porcelain. In fact, they are, and vice versa. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Set -o pipefail at the start of the script. This makes pgit.sh commit
work. Before it didn't, because run_git() doesn't return a proper
return value when it's used in a pipe with a cosmetic sed afterwards.
Signed-off-by: Jan Lindemann <jan@janware.com>
Log to stderr and add some ASCII-art around the output. Also, add a
--porcelain option to allow more stable output parsing. Subsequently,
use that option in make targets parsing the output, notably make diff
and make git-show-xxx.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the name of the operating system ID as taken from /etc/os-release
to $INSTALL_LOG. This should allow for easier post-mortem debugging
if multiple builds used the same file in /tmp and possibly also
prevent conflicts.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg supports more than RPM-based package managers, but for
historic reasons, lots of its Makefile variables still have "RPM" in
their names. This is misleading. Replace "RPM" in variable names by
the more generic "PKG" where appropriate.
Signed-off-by: Jan Lindemann <jan@janware.com>
Maintainer scripts often mess with systemd services via systemctl. In
Docker containers, chroot environments or other environments not
governed by Systemd, systemctl will not exist or complain. This is a
frequent use case, worthy of providing a wrapper to catch and ignore
these cases conveniently.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's no "Homepage" meta tag in the .deb files created by jw-pkg,
add one.
Also, generate an e-mail address <global.jw-maintainer>@janware.com
to go into the Maintainer field. Not ideal, but a low-hanging fruit.
Signed-off-by: Jan Lindemann <jan@janware.com>
pgit.sh logs "Running $0 $@ GIT_SSH=" which is not the exact command
line. Fix that, and prefix log messages with "pgit.sh".
Signed-off-by: Jan Lindemann <jan@janware.com>
A compiled release package is currently uploaded to a fixed
directory, determine it dynamically for every distro seperately.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove the now obsolete get-os.sh from jw-pkg. Use "jw-pkg.py distro
info" to get the information it used to provide.
Signed-off-by: Jan Lindemann <jan@janware.com>
pkg.run is not evaluated on Debian, fix that. For now it's hacked
into pkg.sh, which is bound to be replaced by Python. The limited
hassle is still worth the detour.
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename the --from-user option to --from-owner. Forgejo supports users
and organizations under the more general term "owner", so that's the
better term.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a command "exec", which takes its arguments and runs it as a
shell command within all projects in $PGIT_SH_PROJECTS.
Signed-off-by: Jan Lindemann <jan@janware.com>
PGIT_SH_PROJECTS currently only sets the projects to operate on for
the get command. Extend this to all commands. And replace the
environment variable by a command-line option, likely after this
script has been ported to Python.
Signed-off-by: Jan Lindemann <jan@janware.com>
"clone" in the Git sense means to copy a remote project over from
scratch. pgit.sh clone has come from that, but has since evolved into
something different, a mixture of clone, pull and fetch, so find a
different name. "get" seems generic enough and doesn't clash with a
Git meaning. Adapt variable names accordingly across the project.
Signed-off-by: Jan Lindemann <jan@janware.com>
Continue to name variables in pgit.sh somewhat more consistently,
notably turn somevar into some_var. Plus some additional cleanup.
Still not a beauty.
Signed-off-by: Jan Lindemann <jan@janware.com>
The fat marker with counter (# ==== [1/1] Fetching ...) is
distracting if shown for only one project, so only show a regular
marker without counter.
Signed-off-by: Jan Lindemann <jan@janware.com>
If "current-branch" is specified within --refspec, either as from-ref
or as to-ref, expand that to the branch the working directory has
currently checked out.
Signed-off-by: Jan Lindemann <jan@janware.com>
The cases from-user == login and from-user != login have different
code paths and can be streamlined somewhat, so do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
dc945537 (pkg.sh log-install: Log %attr(0777, ...) for links) fixed
packaging symlink for Debian-based distros, but produces a warning on
RPM based distros: Links may not have explicit attributes, so go back
to not specifying them at all for RPM.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit allows pgit.sh to target not only multiple projects below
a projects-directory, but also one single project. If invoked from
the toplevel directory of a project, it uses that as the only project
it should deal with. This is meant to facilitate running the same VCS
abstraction logic for one project as for many projects. The project
or projects to deal with should probably be specified on the command
line, but changing the auto-detection mechanism buys us what we want
for now with low hassle.
Signed-off-by: Jan Lindemann <jan@janware.com>
Some variable names are too short for global scope ($p, $pdir,
$pdirs), among others. For those mentioned: Make them longer and more
descriptive.
Also add a variable project_name, which denotes what a project is in
a remote repository, and which is currently but not necessarily
always the same as the project directory.
Signed-off-by: Jan Lindemann <jan@janware.com>
Among other atrocities, the Debian path filter contains some horrible
redundancies in an sed regular expression. Be a little less redundant
about it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Not logging any attribute for links, as it's now, breaks Debian's
parser. So, log %attr(0777, $owner, $mode). This fixes the parser on
the Debian side and hopefully leaves the RPM side intact.
Signed-off-by: Jan Lindemann <jan@janware.com>
