Commit graph

744 commits

Author SHA1 Message Date
4431993665 pgit.sh pull: Rebase target branch before merge

Before merging the remote branch, do a rebase. This may fail and prompt conflict resolution, but that seems the canonical outcome for the common use case "interactive make git pull-xxx" with master out-of-sync.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-31 11:13:41 +01:00
b8e8ecf2f1 pkg-manager.sh: Replace by jw-pkg.py distro

Retire pkg-manager.sh and replace it by the cleaner "jw-pkg.sh distro" command, essentially providing the same functionality and nearly the same command-line interface.

Not-so-fun-fact:

jw-pkg > git diff --stat jw-devops/master ... 71 files changed, 732 insertions(+), 340 deletions(-)

400 LOC more. That's what the move from a shell script to the more maintainable Python versions costs. Still a good idea, and the enhanced extensibility might pay off in terms of LOC with other shell scripts in the future.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-28 18:11:09 +01:00
132dce8b3f jw-projects.py: Rename it to jw-pkg.py

jw-projects.py is now a multi-call executable, with "projects" being just one of its subcommands. Rename it to jw-pkg.py to reflect that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-28 15:24:16 +01:00
dd5adbfcfc create-mkspec.sh: Use jw-projects.py os-cascade

In the os_cascade() function, don't roll own logic, call the richer jw-projects.py projects os-cascade instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-26 17:58:23 +01:00
0b83c863a2 jw.build.cmds: Move build.cmds -> cmds.projects

Reorganize the Python module structure. Placing the command classes under jw.cmds.projects instead of jw.build.cmds will allow to add a nested command structure, with the current commands, being mostly related to building software, found below a "projects" toplevel command.

Other conceivable commands could be "package" for packaging, or "distro" for commands wrapping the distribution's package manager.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-26 17:58:23 +01:00
1183aed3f5 Revert "scripts: Add __init__.py"

This reverts commit 98188eab23.

__init__.py alone is not enough to resolve the jw module in all circumnstances, so go back to the old symbolic links solution.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-26 17:17:23 +01:00
98188eab23 scripts: Add __init__.py

$(TOPDIR)/scripts contains a symbolic link jw -> ../src/python/jw, to allow jw-python.py access to the in-repo module jw.pkg. Should be fine now even on Windows, OTOH, it's also solvable via __init__.py, so do that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-20 14:56:08 +01:00
d678bab05a pkg-manager.sh: Support running without sudo

To support minimal environments, notably minimal Docker containers which don't have /usr/bin/sudo by the time pkg-manager.sh is invoked (possibly to install sudo), support running all commands without invoking sudo first. Of course this only works if invoked as root.

Note that this is still somewhat hacky, command-line parsing needs to be cleaned up.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-15 15:05:23 +00:00
57cbb42fcd pkg-manager.sh: Add command dup

Add command dup (dist-upgrade), currently only supported on OpenSUSE.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-15 15:05:23 +00:00
93052882de pgit.sh clone: Fetch submodule tags only from origin

To avoid network errors while fetching tags, run

git submodule foreach --recursive 'git fetch --tags -f origin

i.e. only fetch tags from origin, which by convention points to git.janware.com.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-06 19:11:01 +01:00
3af712705d pgit.sh: Run git fetch --tags on submodules

On pull / clone operations, run

git submodule foreach --recursive 'git fetch --tags'

Notably the Bootstrap package needs the tags to check out different Bootstrap versions.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-12-23 14:06:41 +00:00
e42e3b783b pkg.sh build: Support -a / --include-vcs-files

pkg.sh by default doesn't pack up version control metadata. Passing -a or --include-vcs-files includes them in the source packages.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-12-01 11:31:54 +01:00
e55793f61a scm.sh ls-files: Support -a (all files)

scm.sh ls-files by defaults does not list the VSC metadata files. Passing -a includes them in the output.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-12-01 11:29:53 +01:00
016c1f4220 pgit.sh: Don't fetch with target refspec

"git fetch $remote $fromref:$toref" fails if the $fromref is behind $toref.

Unrolling the syntax into "git fetch" followed by "git merge --ff-only $remote/$fromref $toref" is accepted, though, and saves some otherwise necessary case distinction code around it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-29 15:35:39 +01:00
c3c6cdc446 purge-stale-projects.sh: Support --vcs

Support option --vcs. CVS is retired, but worked well as a test case for mixing multiple version-control systems in one tree.

purge-stale-projects.sh is still pretty ugly and will have to go, but its API might still serve as a working template.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-29 13:28:51 +01:00
9217d38964 Everywhere: Rename package "jw-build" to "jw-pkg"

jw-build doesn't stop at building software, packaging it afterwards is also a core feature, so this commit gives the package a better name.

The commit replaces strings s/jw-build/jw-pkg/ in text files and file names. Fallout to the functionality is fixed, variable names are left as they are, though. To be adjusted by later commits.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-28 13:35:56 +01:00
e31e50036d pgit.sh: push doesn't understand --login

--login is not understood by pgit.sh push. Solve that by allowing all commands a --login option. This addresses our use case, but isn't ideal of course. Will be finally fixed by moving pgit.sh's functionality into Python code.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-26 20:55:35 +01:00
0d95ccc5a0 pgit.sh: clone --login configures broken remote

pgit.sh clone --login <username> fails to insert said username into a remotes url while adding it: Of ssh://<username>@git.janware.com/srv/git, only ssh://@git.janware.com/srv/git makes it into the config. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-26 20:00:51 +01:00
f9f0041790 pgit.sh: Support --login

In the move away from environment variables, replace JANWARE_USER support in pgit.sh by the --login option.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-24 10:35:07 +01:00
d047ad650c pgit.sh: Support --refspec

In the attempt to move away from communicating options via environment variables from one part of jw-build software to another, replace PGIT_CLONE_FROM_USER with the clearer --refspec option. Which is also more versatile.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-24 10:04:47 +01:00
9283479489 make, scripts: Don't unconditionally clone with proactiveAuth=basic

Don't unconditionally add proactiveAuth=basic to Git's config during clone, but only if cloning happens after authentication.

This saves unauthenticated users funny password prompts. On the other hand, this makes a server setting persistent which could be changed on the server.

URL =~ /api/ (or so) => 401, followed by Basic Auth URL !=~ /api/ (or so) => Redirect or free access, depending on resource

Currently all resources, including API, are accessible by either basic auth or a Cookie, but basic auth needs to be present in the first request, which throws off some clients (notably Git without proactiveAuth=basic).

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-20 17:32:04 +01:00
c37f2aaca3 pgit.sh: Deprecate CentOS 7

CentOS 7's Git doesn\'t understand -C, consider that obsolete and don't cd into a directory any longer.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-20 16:15:26 +01:00
e46ab382e9 pgit.sh: Set http.proactiveAuth=basic

For git clone, use and persist the http.proactiveAuth=basic config option, because it's needed for janware.com servers.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-20 15:07:17 +01:00
b7833c928a pgit.sh fix: --create-remote-user-repos parsed incorrectly

--create-remote-user-repos is not recognized because of is not recognized because of a typo, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-18 21:12:54 +01:00
dc5515df09 pgit.sh clone: Use jw-projects.py instead of git-srv-admin.sh

Replace git-srv-admin.sh list-personal-projects by the more universal "jw-projects.py list-repos" for enumerating repo names. This is a step towards supporting Git servers other than janware.com.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-18 15:40:27 +01:00
1b21e33ded pgit.sh: Support --remote-base

pgit.sh has ssh://$login@git.janware.com/srv/git/$fromuser/proj/$reponame hard-coded as the remote Git URLs of every cloned project.

This commit adds support for the global option --remote-base. Passing it changes the URL to <remote-base>/$fromuser/$reponame..

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-18 15:25:46 +01:00
3ed25d5654 scripts: Add symbolic link jw -> ../src/python/jw

Adding a symbolic link to src/python/jw allows jw-projects.py to run without pointing PYTHONPATH to that module.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-18 17:52:20 +01:00
aed14fd480 pgit.sh clone: Support --create-remote-user-repos

Add support for --create-user-repos to pgit.sh. It controls whether or not personal remote repositories on janware.com are created when cloning from another user.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-18 15:37:03 +01:00
e6757727fa purge-stale-projects.sh: Fix git-srv-admin.sh relies on PATH

git-srv-admin.sh is not necessarily in PATH, notably because jw-build doesn't add itself to it anylonger, so invoke it with itsits full path.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 13:15:02 +01:00
4074486735 purge-stale-projects.sh: Re-add purge-stale-projects.sh, needed for purging

Without purge-stale-projects.sh, projects not longer in the upstream directory don't get purged, so add it back.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 13:10:54 +01:00
0c9fd98009 projects-dir.mk, scripts/Makefile, jw-pkg: Move jw-pkg from jw-build to jw-base

jw-pkg is related to, but strictly speaking not indispensible for building and packaging software. So, in the attempt have a minimal jw-build, move jw-pkg to jw-base, and fix all packages that use it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 13:05:43 +01:00
3f8c1ad573 scm.sh: Be more explicit about where ls-files fails

scm.sh sometimes complains about "no VCS here" without detailing what "here" actually means. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 12:17:04 +01:00
b82d4249f9 pgit.sh: Remove CVS support

Don't identify projects directory by a the presence of a correct CVS subdirectory any longer.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 11:30:59 +01:00
4839f60cb8 jw-build-create-project.sh: Re-add jw-build-create-project.sh for toplevel git-init-%

Re-add jw-build-create-project.sh, because it's needed for the toplevel Makefile's git-init-% target.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 10:37:56 +01:00
e589caf73f Re-add git-srv-admin.sh: Needed for enumerating repos via SSH

Since currently remote SSH git repos are identified via git-srv-admin.sh, we still need it to run make over a bare toplevel Makefile.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 08:52:12 +01:00
342030ee32 upload.sh: Re-add upload.sh, needed for release pipeline

upload.sh is needed for uploading a built package to the repo server, so re-add it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-17 02:19:08 +01:00
0f3c45c1d3 Re-add jw-pkg: Use by many postinstall scripts

There's a certain logic that jw-pkg is part of jw-build, because it comes in handy to compile config file templates in a post-install stage. On the other hand, jw-base looks like a better place for that.

Re-adding for now, to not break too many packages.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-16 20:19:32 +01:00
31537a0bd6 build.cmds.CmdCreatePkgConfig: Add module

Implement the functionality of create-pkg-config.sh in a Python module CmdCreatePkgConfig.py. This allows to remove create-pkg-config.sh and jw-build-functions.sh.

Note that the translation was done pretty literally to play it safe. More code can and should be removed by taking advantage of the fact that jw-projects.py knows more about the project than the shell scripts.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-16 17:31:16 +01:00
fbc9e3a001 make, scripts: Rename projects.py -> jw-projects.py

projects.py is too unspecific, rename it to jw-projects.py.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 12:06:38 +01:00
aa3e2efd84 projects.py: Move bulk of code into module

Move nearly all of projects.py into src/python/jw/build/App.py. The plan is to split the script into parts and import them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 11:08:52 +01:00
7a315442d4 create-mkdebian.sh: Re-add create-mkdebian.sh

pkg.sh's operating system abstraction doesn't make much sense with the debian compatibilty script removed, so add it back.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-16 16:31:52 +01:00
3842e98e11 pkg-manager.sh: Re-add pkg-manager.sh

pkg-manager.sh is needed by rollout cronjob, so re-add it.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-15 11:04:13 +01:00
d7d4ffa640 py-run.mk, python-tools.sh: Re-add files necessary for jw-client-devops-web

jw-client-devops-web needs py-run.mk and python-tools.sh, add them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 17:15:24 +01:00
bf16c6714b make, scripts: Re-add files necessary for building ytools

Re-add everthing needed for building and packaging ytools. This is a big commit, 2002 lines of code. It mostly consists of C/C++ machinery, plus some documentation-related stuff.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:59:49 +01:00
5b758023ac pgit.sh: Re-add scripts/pgit.sh

pgit.sh is not indispensable for building a projects directory, but it is necessary for some convenience targets in projects-dir.mk.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:38:54 +01:00
82eb80979d bin, make, scripts: Re-add files necessary for packaging jw-build

Re-add all files necessary to package jw-build itself, i.e. sucessfully run make pkg-rebuild-reinstall. This adds 1892 lines of code.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 11:22:50 +01:00
bc883deed4 Everywhere: Remove everything non-essential for "make clean all"

This commit removes everything not strictly necessary for running "make clean all" inside jw-build.

packaging jw-devtest. This cuts the repo down from 24077 to 4725 lines of code.

The idea is to

1) Further remove bloat from the remaining bits

2) Re-add what's necessary to build and package other essential repos.

The decision should be based on whether or not jw-build can also be useful in a non-janware context.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 15:02:56 +01:00
c36c83c164 scripts/Makefile: Make jannet jcs jw-pkg optional

Make a few scripts optional which don't belong into a minimal jw-build, increasing compatibility with a branch which removes them.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-14 13:46:40 +01:00
069aa5bc1b qemu-boot.sh: Boot from LAN by default
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-13 15:26:08 +01:00
f82988dcc0 projects.py: Fix exception in error message (missing -devel package)

Missing -devel packages during build often lead to this:

WARNING: No project directory for module "dw-pi-playlist: No project path found for module "dw-pi-playlist"

but actually, the warning isn't correctly logged. Instead, an exception is thrown. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-10-19 10:30:47 +00:00