dc945537 (pkg.sh log-install: Log %attr(0777, ...) for links) fixed
packaging symlink for Debian-based distros, but produces a warning on
RPM based distros: Links may not have explicit attributes, so go back
to not specifying them at all for RPM.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit allows pgit.sh to target not only multiple projects below
a projects-directory, but also one single project. If invoked from
the toplevel directory of a project, it uses that as the only project
it should deal with. This is meant to facilitate running the same VCS
abstraction logic for one project as for many projects. The project
or projects to deal with should probably be specified on the command
line, but changing the auto-detection mechanism buys us what we want
for now with low hassle.
Signed-off-by: Jan Lindemann <jan@janware.com>
Some variable names are too short for global scope ($p, $pdir,
$pdirs), among others. For those mentioned: Make them longer and more
descriptive.
Also add a variable project_name, which denotes what a project is in
a remote repository, and which is currently but not necessarily
always the same as the project directory.
Signed-off-by: Jan Lindemann <jan@janware.com>
Among other atrocities, the Debian path filter contains some horrible
redundancies in an sed regular expression. Be a little less redundant
about it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Not logging any attribute for links, as it's now, breaks Debian's
parser. So, log %attr(0777, $owner, $mode). This fixes the parser on
the Debian side and hopefully leaves the RPM side intact.
Signed-off-by: Jan Lindemann <jan@janware.com>
Calling make git-pull-xxx from a projects directory stops iterating
projects if one has a dirty workspace. Calling --autostash fixes
that.
With this in place, a failed rebase leaves the local changes behind
stashed. So, after manually fixing the rebase, the stash needs to be
manually reapplied. The commands that led up to the failure are
logged right before, so I have hope that this is learnable, and not
too much of a footgun.
Signed-off-by: Jan Lindemann <jan@janware.com>
Before merging the remote branch, do a rebase. This may fail and
prompt conflict resolution, but that seems the canonical outcome for
the common use case "interactive make git pull-xxx" with master
out-of-sync.
Signed-off-by: Jan Lindemann <jan@janware.com>
Retire pkg-manager.sh and replace it by the cleaner "jw-pkg.sh
distro" command, essentially providing the same functionality and
nearly the same command-line interface.
Not-so-fun-fact:
jw-pkg > git diff --stat jw-devops/master
...
71 files changed, 732 insertions(+), 340 deletions(-)
400 LOC more. That's what the move from a shell script to the more
maintainable Python versions costs. Still a good idea, and the
enhanced extensibility might pay off in terms of LOC with other shell
scripts in the future.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-projects.py is now a multi-call executable, with "projects" being
just one of its subcommands. Rename it to jw-pkg.py to reflect that.
Signed-off-by: Jan Lindemann <jan@janware.com>
In the os_cascade() function, don't roll own logic, call the richer
jw-projects.py projects os-cascade instead.
Signed-off-by: Jan Lindemann <jan@janware.com>
Reorganize the Python module structure. Placing the command classes
under jw.cmds.projects instead of jw.build.cmds will allow to add a
nested command structure, with the current commands, being mostly
related to building software, found below a "projects" toplevel
command.
Other conceivable commands could be "package" for packaging, or
"distro" for commands wrapping the distribution's package manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
This reverts commit 98188eab23.
__init__.py alone is not enough to resolve the jw module in all
circumnstances, so go back to the old symbolic links solution.
Signed-off-by: Jan Lindemann <jan@janware.com>
$(TOPDIR)/scripts contains a symbolic link jw -> ../src/python/jw, to
allow jw-python.py access to the in-repo module jw.pkg. Should be
fine now even on Windows, OTOH, it's also solvable via __init__.py,
so do that.
Signed-off-by: Jan Lindemann <jan@janware.com>
To support minimal environments, notably minimal Docker containers
which don't have /usr/bin/sudo by the time pkg-manager.sh is invoked
(possibly to install sudo), support running all commands without
invoking sudo first. Of course this only works if invoked as root.
Note that this is still somewhat hacky, command-line parsing needs to
be cleaned up.
Signed-off-by: Jan Lindemann <jan@janware.com>
To avoid network errors while fetching tags, run
git submodule foreach --recursive 'git fetch --tags -f origin
i.e. only fetch tags from origin, which by convention points to
git.janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
On pull / clone operations, run
git submodule foreach --recursive 'git fetch --tags'
Notably the Bootstrap package needs the tags to check out different
Bootstrap versions.
Signed-off-by: Jan Lindemann <jan@janware.com>
pkg.sh by default doesn't pack up version control metadata. Passing
-a or --include-vcs-files includes them in the source packages.
Signed-off-by: Jan Lindemann <jan@janware.com>
scm.sh ls-files by defaults does not list the VSC metadata files.
Passing -a includes them in the output.
Signed-off-by: Jan Lindemann <jan@janware.com>
"git fetch $remote $fromref:$toref" fails if the $fromref is behind
$toref.
Unrolling the syntax into "git fetch" followed by
"git merge --ff-only $remote/$fromref $toref" is accepted, though, and saves
some otherwise necessary case distinction code around it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support option --vcs. CVS is retired, but worked well as a test case
for mixing multiple version-control systems in one tree.
purge-stale-projects.sh is still pretty ugly and will have to go, but
its API might still serve as a working template.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-build doesn't stop at building software, packaging it afterwards
is also a core feature, so this commit gives the package a better
name.
The commit replaces strings s/jw-build/jw-pkg/ in text files and file
names. Fallout to the functionality is fixed, variable names are left
as they are, though. To be adjusted by later commits.
Signed-off-by: Jan Lindemann <jan@janware.com>
--login is not understood by pgit.sh push. Solve that by allowing all
commands a --login option. This addresses our use case, but isn't
ideal of course. Will be finally fixed by moving pgit.sh's
functionality into Python code.
Signed-off-by: Jan Lindemann <jan@janware.com>
pgit.sh clone --login <username> fails to insert said username into a
remotes url while adding it: Of ssh://<username>@git.janware.com/srv/git,
only ssh://@git.janware.com/srv/git makes it into the config. Fix
that.
Signed-off-by: Jan Lindemann <jan@janware.com>
In the move away from environment variables, replace JANWARE_USER
support in pgit.sh by the --login option.
Signed-off-by: Jan Lindemann <jan@janware.com>
In the attempt to move away from communicating options via
environment variables from one part of jw-build software to another,
replace PGIT_CLONE_FROM_USER with the clearer --refspec option. Which
is also more versatile.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't unconditionally add proactiveAuth=basic to Git's config during
clone, but only if cloning happens after authentication.
This saves unauthenticated users funny password prompts. On the other
hand, this makes a server setting persistent which could be changed
on the server.
URL =~ /api/ (or so) => 401, followed by Basic Auth
URL !=~ /api/ (or so) => Redirect or free access, depending on resource
Currently all resources, including API, are accessible by either
basic auth or a Cookie, but basic auth needs to be present in the
first request, which throws off some clients (notably Git without
proactiveAuth=basic).
Signed-off-by: Jan Lindemann <jan@janware.com>
For git clone, use and persist the http.proactiveAuth=basic config
option, because it's needed for janware.com servers.
Signed-off-by: Jan Lindemann <jan@janware.com>
Replace git-srv-admin.sh list-personal-projects by the more universal
"jw-projects.py list-repos" for enumerating repo names. This is a
step towards supporting Git servers other than janware.com.
Signed-off-by: Jan Lindemann <jan@janware.com>
pgit.sh has ssh://$login@git.janware.com/srv/git/$fromuser/proj/$reponame
hard-coded as the remote Git URLs of every cloned project.
This commit adds support for the global option --remote-base. Passing
it changes the URL to <remote-base>/$fromuser/$reponame..
Signed-off-by: Jan Lindemann <jan@janware.com>
Adding a symbolic link to src/python/jw allows jw-projects.py to run
without pointing PYTHONPATH to that module.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --create-user-repos to pgit.sh. It controls whether
or not personal remote repositories on janware.com are created when
cloning from another user.
Signed-off-by: Jan Lindemann <jan@janware.com>
git-srv-admin.sh is not necessarily in PATH, notably because
jw-build doesn't add itself to it anylonger, so invoke it with itsits
full path.
Signed-off-by: Jan Lindemann <jan@janware.com>
Without purge-stale-projects.sh, projects not longer in the upstream
directory don't get purged, so add it back.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg is related to, but strictly speaking not indispensible for building and
packaging software. So, in the attempt have a minimal jw-build, move jw-pkg to
jw-base, and fix all packages that use it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Since currently remote SSH git repos are identified via
git-srv-admin.sh, we still need it to run make over a bare toplevel
Makefile.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's a certain logic that jw-pkg is part of jw-build, because it
comes in handy to compile config file templates in a post-install
stage. On the other hand, jw-base looks like a better place for that.
Re-adding for now, to not break too many packages.
Signed-off-by: Jan Lindemann <jan@janware.com>
Implement the functionality of create-pkg-config.sh in a Python
module CmdCreatePkgConfig.py. This allows to remove
create-pkg-config.sh and jw-build-functions.sh.
Note that the translation was done pretty literally to play it safe.
More code can and should be removed by taking advantage of the fact
that jw-projects.py knows more about the project than the shell
scripts.
Signed-off-by: Jan Lindemann <jan@janware.com>
Move nearly all of projects.py into src/python/jw/build/App.py. The
plan is to split the script into parts and import them.
Signed-off-by: Jan Lindemann <jan@janware.com>
pkg.sh's operating system abstraction doesn't make much sense with
the debian compatibilty script removed, so add it back.
Signed-off-by: Jan Lindemann <jan@janware.com>
