Remove the now obsolete get-os.sh from jw-pkg. Use "jw-pkg.py distro
info" to get the information it used to provide.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a VCS-independent rule rule get-%, currently an alias for
git-get-%, with an empty recipe to keep GNU Make satisfied.
Signed-off-by: Jan Lindemann <jan@janware.com>
Don't use the master branch as target when getting code from
somebody, merge into the branch currently checked out.
Signed-off-by: Jan Lindemann <jan@janware.com>
"clone" in the Git sense means to copy a remote project over from
scratch. pgit.sh clone has come from that, but has since evolved into
something different, a mixture of clone, pull and fetch, so find a
different name. "get" seems generic enough and doesn't clash with a
Git meaning. Adapt variable names accordingly across the project.
Signed-off-by: Jan Lindemann <jan@janware.com>
To be explicit about what's happening during make fetch, default the
fetch refspec to $(CLONE_FROM_USER):current-branch:current-branch.
Signed-off-by: Jan Lindemann <jan@janware.com>
Purging the environment of pkg-release-reinstall did a tad too much:
Add SSH_AUTH_SOCK back to allow SSH agent-based authentication to go
through.
Signed-off-by: Jan Lindemann <jan@janware.com>
--source-profile=replace should be passed to all
jw-pkg.py projects build pkg-*install
invocations. Those invocations typically happen in the context of
pkg-%install, so add that target, specializing the pkg-% target.
The problem this solves is that /etc/profile is currently read only
once before bootstrapping all software on a pristine system is
started. This might lead to the situation that package A has
installed environment variable definitions into /etc/profile.d,
package B needs them for building, but never gets to read them.
Signed-off-by: Jan Lindemann <jan@janware.com>
Multiple variables are redundantly defined both for a project and for
the multiple-projects toplevel directory. Add a place to maintain
them centrally, and add PGIT_SH as a first variable.
Signed-off-by: Jan Lindemann <jan@janware.com>
The target pkg-delete-ours, invoked from the projects directory,
should wipe all packages from the system which have been created and
installed via jw-pkg.
Currently they are selected via url =~ janware. This is a default
string which can be overridden by redefining JANWARE_PACKAGE_FILTER.
This might not be the most generic name, but is kind of consistent
and will be matched once all variables get renamed to a more generic
naming scheme.
This currently does not get all packages: Some are not labeled with
URLs matching "janware", because jw-pkg is only used as a convenient
way to package other people's open source projects.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's pkg-manager-refresh already, so by adding pkg-manager-dup the
distribution can be upgraded by distribution agnostic targets only
through the Makefile. This might come in handy for CI, so add it.
Signed-off-by: Jan Lindemann <jan@janware.com>
--quote puts double quotation marks around the listed dependencies,
protecting version requirements (>= 1.0) and parenthesis "perl(GD)"
from the shell.
Signed-off-by: Jan Lindemann <jan@janware.com>
Retire pkg-manager.sh and replace it by the cleaner "jw-pkg.sh
distro" command, essentially providing the same functionality and
nearly the same command-line interface.
Not-so-fun-fact:
jw-pkg > git diff --stat jw-devops/master
...
71 files changed, 732 insertions(+), 340 deletions(-)
400 LOC more. That's what the move from a shell script to the more
maintainable Python versions costs. Still a good idea, and the
enhanced extensibility might pay off in terms of LOC with other shell
scripts in the future.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-projects.py is now a multi-call executable, with "projects" being
just one of its subcommands. Rename it to jw-pkg.py to reflect that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Reorganize the Python module structure. Placing the command classes
under jw.cmds.projects instead of jw.build.cmds will allow to add a
nested command structure, with the current commands, being mostly
related to building software, found below a "projects" toplevel
command.
Other conceivable commands could be "package" for packaging, or
"distro" for commands wrapping the distribution's package manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
Define Q ?= @, and replace @<command> in recipes by $(Q)<command>.
Meant to be overridden from the environment for debugging as in
Q= make
Signed-off-by: Jan Lindemann <jan@janware.com>
Kali Linux' default installation doesn't have /usr/bin/time which
brings out a but: $(TIME) doesn't expand to nothing but to -p, which
fails miserably, of course. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
PGIT_SH gets added --remote-base, but too late to make it into the
non-recursive variable PGIT_SH_CLONE. This leads to --remote-base
lacking from the clone invocation, and anonymous Git over HTTP
failing because it tries to clone via SSH. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
make git-show-pushable-master-branches output too litte for two
reasons: 1. grep -q returns zero also if no matches are found, and 2.
PROJECTS doesn't contain all relevant projects. BUILD_PROJECTS is
more meaningful.
Signed-off-by: Jan Lindemann <jan@janware.com>
--create-remote-user-repos had been disabled in
4053451bfd on the grounds that it's
hard to test and possibly superflous. It actually is not superfluous,
as devops builds show, and that's a valid test-case, so re-enable it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Support option --vcs. CVS is retired, but worked well as a test case
for mixing multiple version-control systems in one tree.
purge-stale-projects.sh is still pretty ugly and will have to go, but
its API might still serve as a working template.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-build doesn't stop at building software, packaging it afterwards
is also a core feature, so this commit gives the package a better
name.
The commit replaces strings s/jw-build/jw-pkg/ in text files and file
names. Fallout to the functionality is fixed, variable names are left
as they are, though. To be adjusted by later commits.
Signed-off-by: Jan Lindemann <jan@janware.com>
In the move away from environment variables, replace JANWARE_USER
support in pgit.sh by the --login option.
Signed-off-by: Jan Lindemann <jan@janware.com>
In the attempt to move away from communicating options via
environment variables from one part of jw-build software to another,
replace PGIT_CLONE_FROM_USER with the clearer --refspec option. Which
is also more versatile.
Signed-off-by: Jan Lindemann <jan@janware.com>
Creating user repositories is hard to test. Not sure if the concept
will survive the current workflow changes, disable it for now.
Signed-off-by: Jan Lindemann <jan@janware.com>
JW_BUILD_SSH_EXTRA_OPTS contains a -l $(JANWARE_USER), which make ssh
break if $(JANWARE_USER) is empty. Fix that case.
Signed-off-by: Jan Lindemann <jan@janware.com>
The GIT_ASKPASS environment variable can point to a relative path,
and since pgit.sh currently does a chdir, it needs to be made an
absolute path.
Signed-off-by: Jan Lindemann <jan@janware.com>
Currently, the primary discriminating criterion on how to handle a
set of remote repositories is whether or not JANWARE_USER is defined.
The canonical way to do that is PROJECTS_DIR_REMOTE_BASE, though, so
go from that definition.
Signed-off-by: Jan Lindemann <jan@janware.com>
Group variables related with pgit.sh (PGIT_SH and PGIT_SH_CLONE)
closer together. Define CLONE_FROM_USER early on.
Signed-off-by: Jan Lindemann <jan@janware.com>
The usage comments heading projects-dir-minimal.mk and
projects-dir.mk state that for cloning all repositories, JANWARE_USER
needs to be defined. That restriction is now gone, so reflect that in
the comment.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add the variable PROJECTS_DIR_REMOTE_BASE, defaulting to
ssh://git.janware.com/srv/git if REMOTE_USER is defined, and to
https://janware.com/code in case it isn't.
Signed-off-by: Jan Lindemann <jan@janware.com>
Do not set JANWARE_USER to $(id -un) in case it's undefined. Instead,
rely on it being set explicitly in the environment if so desired.
Signed-off-by: Jan Lindemann <jan@janware.com>
Pass --create-remote-user-repos to pgit.sh clone in case JANWARE_USER
is defined, restoring the original behaviour.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-pkg is related to, but strictly speaking not indispensible for building and
packaging software. So, in the attempt have a minimal jw-build, move jw-pkg to
jw-base, and fix all packages that use it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make list-files target ignore errors. This catches errors from
git ls-files | xargs realpath if git ls-files lists dead symbolic
links, as in my-project/blah/dev/fd -> ./proc/self/fd.
Implemented by passing -q (quiet) to realpath, not sure what else
this suppresses.
Signed-off-by: Jan Lindemann <jan@janware.com>
Move nearly all of projects.py into src/python/jw/build/App.py. The
plan is to split the script into parts and import them.
Signed-off-by: Jan Lindemann <jan@janware.com>
Re-add everything necessary for recursively building all repos in
a directory, e.g. as a build controlled by janware.com/Makefile or
any other installation.
This adds 489 lines of code which can (and should) be massively
reduced, notably removing code supporting CVS.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit removes everything not strictly necessary for running
"make clean all" inside jw-build.
packaging jw-devtest. This cuts the repo down from 24077 to 4725
lines of code.
The idea is to
1) Further remove bloat from the remaining bits
2) Re-add what's necessary to build and package other essential repos.
The decision should be based on whether or not jw-build can also be
useful in a non-janware context.
Signed-off-by: Jan Lindemann <jan@janware.com>
The projects toplevel Makefile is not maintained in CVS anylonger but
in Git, so don't update it from CVS any longer.
Signed-off-by: Jan Lindemann <jan@janware.com>
