git-pull-<username> doesn't use pgit.sh if username == login.
pgit.sh should handle that case fine now, so remove the distinction
from topdir.mk and make it in one place, i.e. pgit.sh. This has the
additional advantage that pull as done by pgit.sh conveniently uses
--autostash.
Signed-off-by: Jan Lindemann <jan@janware.com>
Purging the environment of pkg-release-reinstall did a tad too much:
Add SSH_AUTH_SOCK back to allow SSH agent-based authentication to go
through.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change commit message of first commit from "initial checkin" to
"First commit", because that's more to the point.
Signed-off-by: Jan Lindemann <jan@janware.com>
sudo is certainly not needed for the run package (which in itself is
hardly useful at all), so move that dependency into the devel
package. Same for gawk. /opt/jw-pkg/bin/get-os.sh depends on it,
but I don't see where else but in a -devel context that would matter.
And if it breaks something, it is going to be an easy fix without
awk.
Signed-off-by: Jan Lindemann <jan@janware.com>
--source-profile=replace should be passed to all
jw-pkg.py projects build pkg-*install
invocations. Those invocations typically happen in the context of
pkg-%install, so add that target, specializing the pkg-% target.
The problem this solves is that /etc/profile is currently read only
once before bootstrapping all software on a pristine system is
started. This might lead to the situation that package A has
installed environment variable definitions into /etc/profile.d,
package B needs them for building, but never gets to read them.
Signed-off-by: Jan Lindemann <jan@janware.com>
To complement git-pull-maintainer with something more generic, also
suitable for other SCMs, add the target pull-maintainer and make
pkg-release-reinstall depend on it. Currently only visible in the
context of pkg-% targets, scope might be expanded if need be.
Signed-off-by: Jan Lindemann <jan@janware.com>
Use pgit.sh to for the git-pull-% target. This should make
git-pull-maintainer work. To limit the blast radius for now, only use
it if the source user differs from the invoking user.
Signed-off-by: Jan Lindemann <jan@janware.com>
Multiple variables are redundantly defined both for a project and for
the multiple-projects toplevel directory. Add a place to maintain
them centrally, and add PGIT_SH as a first variable.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add an init target. Use it if you want to tell the Makefile: _Just_
initalize the build machinery and nothing else, don't pull and build
everything else you can. Not strictly necessary, most of the time
pulling everything is what's wanted, and that does the init anyway.
Signed-off-by: Jan Lindemann <jan@janware.com>
The target pkg-delete-ours, invoked from the projects directory,
should wipe all packages from the system which have been created and
installed via jw-pkg.
Currently they are selected via url =~ janware. This is a default
string which can be overridden by redefining JANWARE_PACKAGE_FILTER.
This might not be the most generic name, but is kind of consistent
and will be matched once all variables get renamed to a more generic
naming scheme.
This currently does not get all packages: Some are not labeled with
URLs matching "janware", because jw-pkg is only used as a convenient
way to package other people's open source projects.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add a package dependency on make for the -devel package. Installing
jw-pkg's Makefiles doesn't make much sense without it.
Signed-off-by: Jan Lindemann <jan@janware.com>
If VERSION_FILE is not found, trying to include makefile snippets
triggers the attempt to regenerate it. This happened for installed
Makefiles of jw-docker-images: It defines TOPDIR to
/opt/jw-docker-images, projects.mk looks there, but the version file
is installed under /usr/share/doc/packages/jw-docker-images/VERSION.
Allow VERSION_FILE to be overridden including code to keep that from
happening.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change package group from System/Libraries to
Development/Tools/Building because that expresses the purpose of the
package better.
Signed-off-by: Jan Lindemann <jan@janware.com>
make complains for plugins that soandso.dll has not been remade. The
problem is that it tries to remake all targets of a multi-target.
Split that up into two rules.
Signed-off-by: Jan Lindemann <jan@janware.com>
.cache-projects.mk is not installed / packaged, which makes builds
against an installed jw-pkg considerably slower. Change that, at the
risk of making the installed jw-pkg-devel less versatile. This commit
installs a cache file cache-projects.mk, renamed from
.cache-projects.mk, because there's no justification for hiding an
installed makefile. At least I can't think of one.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's pkg-manager-refresh already, so by adding pkg-manager-dup the
distribution can be upgraded by distribution agnostic targets only
through the Makefile. This might come in handy for CI, so add it.
Signed-off-by: Jan Lindemann <jan@janware.com>
--quote puts double quotation marks around the listed dependencies,
protecting version requirements (>= 1.0) and parenthesis "perl(GD)"
from the shell.
Signed-off-by: Jan Lindemann <jan@janware.com>
Retire pkg-manager.sh and replace it by the cleaner "jw-pkg.sh
distro" command, essentially providing the same functionality and
nearly the same command-line interface.
Not-so-fun-fact:
jw-pkg > git diff --stat jw-devops/master
...
71 files changed, 732 insertions(+), 340 deletions(-)
400 LOC more. That's what the move from a shell script to the more
maintainable Python versions costs. Still a good idea, and the
enhanced extensibility might pay off in terms of LOC with other shell
scripts in the future.
Signed-off-by: Jan Lindemann <jan@janware.com>
jw-projects.py is now a multi-call executable, with "projects" being
just one of its subcommands. Rename it to jw-pkg.py to reflect that.
Signed-off-by: Jan Lindemann <jan@janware.com>
As per info make, it turns out that ifndef SOME_VAR is true for
SOME_VAR defined to an empty value. This is unusable for caching, so
replace it with ifeq ($(origin SOME_VAR),undefined).
Signed-off-by: Jan Lindemann <jan@janware.com>
Add support for --topdir-format. The option supports several
different values, affecting the console output of App wherever it
knows that the output contains a reference to the projects' toplevel
directory.
- "unaltered" will have it print the toplevel directory in the same
format as passed to the commandline
- "absolute" will try to resolve it to an absolute path before
printing
- make:XXX will return the make-varible $(XXX) instead
To implement this, the proj_dir() member function is turned into the
private member function __proj_dir(), and a new member function
find_dir() is supplied, with two additional parameters:
search_subdirs and search_absdirs, which will try to find an existing
directory relative to the toplevel directory of the given module, or
in the search_absdirs list, respectively.
Command modules in cmds.projects have been updated to use the new
function.
Signed-off-by: Jan Lindemann <jan@janware.com>
Reorganize the Python module structure. Placing the command classes
under jw.cmds.projects instead of jw.build.cmds will allow to add a
nested command structure, with the current commands, being mostly
related to building software, found below a "projects" toplevel
command.
Other conceivable commands could be "package" for packaging, or
"distro" for commands wrapping the distribution's package manager.
Signed-off-by: Jan Lindemann <jan@janware.com>
Define Q ?= @, and replace @<command> in recipes by $(Q)<command>.
Meant to be overridden from the environment for debugging as in
Q= make
Signed-off-by: Jan Lindemann <jan@janware.com>
Rename the variable PROJECTS_PY_EXTRA_ARGS to PROJECTS_PY_EXTRA_OPTS
to be consistent with projects-dir.mk.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove PY_PREREQ_BUILD and PY_PREREQ_BUILD_DIRS from py-defs.mk:
Apparently they're not used anywhere, and are costly in terms of
directory startup time.
Signed-off-by: Jan Lindemann <jan@janware.com>
This commit aims at improving speed by using better caching.
- Makefile, cache.mk: Split .cache.mk up
To allow caching of runtime path variables which are
project-specific, split .cache.mk up in .cache-project.mk and
.cache-projects.mk
- ldlibpath.mk: Cache ldlibpath, exepath and pythonpath
Place the output of $(call proj_query ldlibpath), $(call
proj_query, exepath) and $(call proj_query pythonpath) in
JW_PKG_LD_LIBRARY_PATH, JW_PKG_EXE_PATH, and JW_PKG_PYTHON_PATH
respectively, and cache the variables in make/.project-cache.mk.
- cache.mk: Use = instead of :=
Recursively expanded variables are nearly as fast as := variables
if the assigned value is a fixed string. And sometimes it's not,
rightly so, because variables get assigned below, as with
JW_PKG_XXX for instance.
- cache.mk: Use $(TOPDIR) as variable values
Replace absolute references to project's topdir by $(TOPDIR) with
sed. As soon as the project queries produce absolute paths, they
will be transformed into relative paths which allow the code base
to be moved to a different location and still remain functional
without a rebuild.
Signed-off-by: Jan Lindemann <jan@janware.com>
Remove unused code, and code which actually does something:
CACHED_VARS looks as if it's sufficently and more centrally defined
in make.mk, don't override that.
Signed-off-by: Jan Lindemann <jan@janware.com>
Add std-targets.mk, meant to be included mostly to make sure all
mandatory targets are there. On clean and distclean, it also removes
stuff that should not be there anymore after clean.
Signed-off-by: Jan Lindemann <jan@janware.com>
Change $(check_scm_sync) from "make git pull" to "make
git-pull-maintainer", which most notably should delegate devops
builds to the maintainer defined in project.conf.
Signed-off-by: Jan Lindemann <jan@janware.com>
There's little system in the pkg-install-xxx targets, add one more to
increase the confusion. It's needed to install all packages needed to
do a standalone build against the packages installed into the system
via package manager. That said, the respective jw-projects.sh
commands need broader refactoring, as well as the pkg-install-xxx
target naming.
Signed-off-by: Jan Lindemann <jan@janware.com>
Make jw-projects.py list-repos support a local directory as base URL
of all git repositories, notably used by PROJECTS_DIR_REMOTE_BASE,
which can now point to a local directory.
Signed-off-by: Jan Lindemann <jan@janware.com>
Targets defined by projects-dir.mk are not available before it is
included, but make makes up its mind about what targets are available
after parsing the included makefiles, so remove that redundancy.
On the other hand, a dependency alone is not enough for make to
understand that an included makefile has been remade, it needs a
rule, so add a dummy-rule body. In this case only echoing that the
include file has been provided.
Signed-off-by: Jan Lindemann <jan@janware.com>
Kali Linux' default installation doesn't have /usr/bin/time which
brings out a but: $(TIME) doesn't expand to nothing but to -p, which
fails miserably, of course. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
PGIT_SH gets added --remote-base, but too late to make it into the
non-recursive variable PGIT_SH_CLONE. This leads to --remote-base
lacking from the clone invocation, and anonymous Git over HTTP
failing because it tries to clone via SSH. Fix that.
Signed-off-by: Jan Lindemann <jan@janware.com>
make git-show-pushable-master-branches output too litte for two
reasons: 1. grep -q returns zero also if no matches are found, and 2.
PROJECTS doesn't contain all relevant projects. BUILD_PROJECTS is
more meaningful.
Signed-off-by: Jan Lindemann <jan@janware.com>
HTMLOWNER wwwrun is not a good idea with file mode 0644. The web
server process should not be allowed to write its own executable
files.
Signed-off-by: Jan Lindemann <jan@janware.com>
For a project to supply templates, it needs to advertise their
location. For this, the tmpl_dir make variable is added to
projects.mk. If other-project wants to get hold of some-project's
templates, it can do, e.g.:
TEMPLATES = $(wilcard $(call tmpl_dir,some-project)/*.tmpl)
To achieve this, support for the tmpls-dir command is added to
jw-projects.py.
Signed-off-by: Jan Lindemann <jan@janware.com>
Templates (i.e. text files ending as .tmpl) are not part of jw-pkg
anylonger, but controlling the way they are installed is beneficial
to other packages, so add tmpl.mk back.
That said, the variable names will need some tweaking to avoid
collisions. Postponed.
Signed-off-by: Jan Lindemann <jan@janware.com>
To avoid name collisions, rename svg.mk to the more specialized
svg-to-pixmap.mk, because that's what it does. To the same end, rename $(SVG)
to $(PIXMAP_TO_SVG_SRC_SVG).
Signed-off-by: Jan Lindemann <jan@janware.com>
--create-remote-user-repos had been disabled in
4053451bfd on the grounds that it's
hard to test and possibly superflous. It actually is not superfluous,
as devops builds show, and that's a valid test-case, so re-enable it.
Signed-off-by: Jan Lindemann <jan@janware.com>
Running "make install" from an arbitrary source directory currently
by default either installs to a user-accessible ENV_PREFIX, or, if
DEVELOPMENT is set to false, tries to install into the system's root
filesystem, but fails over permission errors. This was by design: To
now, I considered trying the latter ill-conceived, because installing
without package manager control bears the risk of leaving unversioned
files in the system.
Actually, thinking again, during development this looks like a valid
use case: Having run pkg-rebuild-reinstall before, installing from a
source directory will leave a trace in the package manager's hash
check output, will be handled during the next clean install, and
might be a useful shortcut for trying things in the root file system.
So make this possible by:
$ DEVELOPMENT=false make install
Signed-off-by: Jan Lindemann <jan@janware.com>
