Commit graph

707 commits

Author SHA1 Message Date
31537a0bd6 build.cmds.CmdCreatePkgConfig: Add module
Implement the functionality of create-pkg-config.sh in a Python
module CmdCreatePkgConfig.py. This allows to remove
create-pkg-config.sh and jw-build-functions.sh.

Note that the translation was done pretty literally to play it safe.
More code can and should be removed by taking advantage of the fact
that jw-projects.py knows more about the project than the shell
scripts.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-16 17:31:16 +01:00
fbc9e3a001 make, scripts: Rename projects.py -> jw-projects.py
projects.py is too unspecific, rename it to jw-projects.py.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 12:06:38 +01:00
aa3e2efd84 projects.py: Move bulk of code into module
Move nearly all of projects.py into src/python/jw/build/App.py. The
plan is to split the script into parts and import them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 11:08:52 +01:00
7a315442d4 create-mkdebian.sh: Re-add create-mkdebian.sh
pkg.sh's operating system abstraction doesn't make much sense with
the debian compatibilty script removed, so add it back.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-16 16:31:52 +01:00
3842e98e11 pkg-manager.sh: Re-add pkg-manager.sh
pkg-manager.sh is needed by rollout cronjob, so re-add it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 11:04:13 +01:00
d7d4ffa640 py-run.mk, python-tools.sh: Re-add files necessary for jw-client-devops-web
jw-client-devops-web needs py-run.mk and python-tools.sh, add them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 17:15:24 +01:00
bf16c6714b make, scripts: Re-add files necessary for building ytools
Re-add everthing needed for building and packaging ytools. This is a
big commit, 2002 lines of code. It mostly consists of C/C++ machinery,
plus some documentation-related stuff.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:59:49 +01:00
5b758023ac pgit.sh: Re-add scripts/pgit.sh
pgit.sh is not indispensable for building a projects directory, but
it is necessary for some convenience targets in projects-dir.mk.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:38:54 +01:00
82eb80979d bin, make, scripts: Re-add files necessary for packaging jw-build
Re-add all files necessary to package jw-build itself, i.e.
sucessfully run make pkg-rebuild-reinstall. This adds 1892 lines of
code.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 11:22:50 +01:00
bc883deed4 Everywhere: Remove everything non-essential for "make clean all"
This commit removes everything not strictly necessary for running
"make clean all" inside jw-build.

packaging jw-devtest. This cuts the repo down from 24077 to 4725
lines of code.

The idea is to

1) Further remove bloat from the remaining bits

2) Re-add what's necessary to build and package other essential repos.

   The decision should be based on whether or not jw-build can also be
   useful in a non-janware context.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:02:56 +01:00
c36c83c164 scripts/Makefile: Make jannet jcs jw-pkg optional
Make a few scripts optional which don't belong into a minimal
jw-build, increasing compatibility with a branch which removes them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 13:46:40 +01:00
069aa5bc1b qemu-boot.sh: Boot from LAN by default
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-13 15:26:08 +01:00
f82988dcc0 projects.py: Fix exception in error message (missing -devel package)
Missing -devel packages during build often lead to this:

	WARNING:  No project directory for module "dw-pi-playlist: No project
	path found for module "dw-pi-playlist"

but actually, the warning isn't correctly logged. Instead, an exception is
thrown. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-10-19 10:30:47 +00:00
eb3949d085 jw-pkg: Support -c option (template file attribute definition pattern)
jw-pkg determines the file attributes of template compilation output via a line
matching the '^#\s*conf' regular expression. Allow that regex to be configured.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-21 11:43:46 +00:00
9b546592e1 jw-pkg: Honour #conf: line at top of template
Up to now, jw-pkg only evaluates config lines at the top of secret-tables. Look
for config patterns at the top of templates, too.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-13 10:46:53 +00:00
a0842c6297 qemu-boot.sh: Replace deprecated brctl by iproute2 (ip)
/sbin/brctl is not part of openSUSE anymore, use ip link commands instead.

Also, remove the vlan option, now unsupported by qemu.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-16 13:23:21 +00:00
8e7ceb6444 jw-pkg: Support command list-changed-files
list-changed-files does the obvious: List all files locally diverging from its
packaged version.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-05 13:44:59 +00:00
8b8f1cc306 projects.py: Allow projects with non-existing project directory
proj_dir() raises an exception for projects which don't have a dedicated
project directory, even though for some projects this is legal. php-cli, for
instance, only installs stuff below /srv/www/proj/php-cli.

This commit makes projects.py tolerate that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-03-30 12:24:10 +00:00
144cb341b7 create-mkspec.sh: Remove -n $NAME from %prep
%prep is passed -n <name> option. This is considered an error by rpm
Release 4.20.1-1.1. It is superfluous anyway, because the name is communicated to rpm
via %setup -n <name> later in the %prep section. Remove the -n option.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-03-26 16:58:07 +00:00
856dae7efa collect-installed-pkg-changes.sh: Ddefault packages to jw-pkg list
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-02-19 16:49:36 +00:00
fd6607de15 projects.py: Os-cascade: Fix regex
Signed-off-by: Jan Lindemann <jan@janware.com>
2024-12-03 12:26:35 +00:00
531e0eab19 scm.sh: Ls-files: Add --recurse-submodules for git
Make scm.sh ls-files list all files belonging to submodules, too. The use case
is to have list-text-files return them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-07-16 10:57:52 +00:00
531798ed0b jw-pkg: Beautify logging
Remove superfluous comma from "saving backup".

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-07-16 10:57:51 +00:00
8702a63323 jw-pkg: Fix ignored target file metadata
During an invokation owner, group and mode definitions of one target file could
leak into another.

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-07-05 11:20:55 +00:00
b87099bd3a pkg-manager.sh: Remove eval from shell commands
Installing packages breaks with eval, because it removes quotes,
notably around perl(XYZ) capabilities. Works just as well without
eval, remove it

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-06-30 16:55:44 +00:00
c2ad910e91 jw-pkg compile-templates: Beautify logging
Don't renew files with nothing to change, and be verbose about it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-06-28 05:40:12 +00:00
5e66868557 jw-pkg: Add command get-value-from-tables
Signed-off-by: Jan Lindemann <jan@janware.com>
2024-06-25 11:19:23 +00:00
6109703ccb projects.py: Warn about modules which were not found
Signed-off-by: Jan Lindemann <jan@janware.com>
2024-06-04 08:56:21 +00:00
94e1c29ede projects.py proj-dir: Fall back on /opt
Search in /opt, if a project is not found in projects-dir

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-05-25 15:38:50 +00:00
d63b40082e cvs-admin.sh: Add support for $HOME/cvs-list.txt
If $HOME/cvs-list.txt exists, it is used as whitelist for the
list-projects command

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-04-30 09:50:54 +00:00
b77d92794e python-tools.sh: Fix __path__ type check in __init__.py
Don't check __path__ as Interable[str] anylonger, this somehow broke (Don't
remember how). Use type: ignore instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-04-30 09:00:45 +00:00
3f5a3d004a projects.py: Add command modules
projects.py modules shows either all modules, or modules which have one of the
key-value pairs requested for filtering via the -F option.

Signed-off-by: Jan Lindemann <jan@janware.com>
2024-04-30 08:00:39 +00:00
a4c3ca6f2b projects.py: Add some comments to dependency ordering code
Signed-off-by: Jan Lindemann <jan@janware.com>
2023-12-18 13:00:45 +00:00
ce7984e2e8 jw-build-create-project.sh: Add *tmp* to .gitignore by default
Signed-off-by: Jan Lindemann <jan@janware.com>
2023-12-18 13:00:44 +00:00
62abfc4c35 jw-pckg: Fix duplicates in compile-templates again
Signed-off-by: Jan Lindemann <jan@janware.com>
2023-03-21 16:26:30 +00:00
32450c47bf jw-pckg: Fix duplicates in compile-templates
Files were listed twice in compile-templates, because apparently, during
postinstall, rpm -ql returns the list of a packages files twice.

Signed-off-by: Jan Lindemann <jan@janware.com>
2023-03-19 14:28:10 +00:00
b633e9637c pkg.sh: Filter out system installation directories below /usr
System installation directories should not belong to jw-build-generated
packages, so exclude them via regex.

Signed-off-by: Jan Lindemann <jan@janware.com>
2021-09-26 19:14:58 +00:00
e3489d6222 purge-stale-projects.sh: Ignore untracked files
Untracked files flag a git repository as unfit for purging, don't do that. We
don't care for untracked files.

Signed-off-by: Jan Lindemann <jan@janware.com>
2021-01-30 13:05:58 +00:00
b3c093fd94 pkg.sh: Don't exclude .in files from source package
Building with .in files should work just fine, so don't exclude them anylonger.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-12-01 16:21:22 +00:00
c37f49a170 project.conf + pkg-manager.sh: Add support for raspbian
Signed-off-by: Jan Lindemann <jan@janware.com>
2020-06-16 17:05:03 +00:00
5e9168480a project-dirs.mk / pgit.sh: Add support for offlining projects
This adds support for the variable OFFLINE_PROJECTS in projects-dir.mk and the
PGIT_IGNORE environment variable. Both go hand in hand and do what their names
insinuate. OFFLINE_PROJECTS is initialized from EXCLUDE_FROM_BUILD, which in
turn is initialized from exclude.txt and friends.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-05-10 08:07:22 +00:00
a404be15b9 make, scripts: Add support for static Python typechecking
This commit adds support for static typechecking with mypy.

Notable additions:

  - A new target "check" which does the type checking

  - Py-mods.mk, meant to be included from a directory containing python modules
    in subdirectories, but not being a python module itself. It makes the all
    target depend on check only if PY_RUN_CHECK_AFTER_BUILD is defined and
    true.  That's because pypy is under heavy development, and the Ubuntu 18.04
    version is too old to work for lots of the code.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-10 12:13:12 +00:00
089aebeaec scm.sh clean: Make git clean ignore .gitignore
scm.sh clean doesn't remove files in .gitignore, which is kind of beside the
point. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-10 12:03:43 +00:00
5f3c4a6320 jw-build-create-project.sh: Add __init__.py and __pychache__ to .gitignore
Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-06 07:03:32 +00:00
5f9258bdf8 jw-build-create-project.sh: Add *.swp to .gitignore
Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-05 12:54:49 +00:00
7fb117a4c4 python-tools.sh: Make __init__py recognize multiple package modules
import my.mod.thing, my.mod.thang wouldn't work, if the modules where not in
the same directory hierarchy, e.g. in $(HOME)/blah/my/mod/thing.py and
$(HOME)/blub/my/mod/thang.py, not even with PYTHONPATH=$(HOME)/blah:$(HOME)/blub.

This commit fixes that.

See https://stackoverflow.com/questions/1675734

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-05 12:51:11 +00:00
1239fb9f34 projects.py: Support line continuation in project.conf
This commit makes lines with backslashes at the end behave as
if the next line was appended

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-03-19 17:36:58 +00:00
e71661425c projects.py: Fix integer comparison
Python 3.8 complains about integers compared with 'is not'
instead of '!=', fix that

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-03-19 17:34:35 +00:00
17244e9a3f pgit.sh: Add support for target refspec to PGIT_CLONE_FROM_USER
PGIT_CLONE_CLONE_FROM_USER now supports optional suffixes of the form
[:src-ref[:dst-ref]]. If present, src-ref specifies the remote branch pgit.sh
tries to fetch, and dst-ref the local branch it tries to fetch into. src-ref
defaults to "master" (as it was before) and dst-ref to nothing, i.e. no
head-update of any local branch.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-02-24 14:35:26 +00:00
3a0d85ff89 git-srv-admin.sh: Fix group ownership
Fix group ownership of newly created git directories.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-01-21 15:35:06 +00:00