make: Improve make variable caching machinery #50

Merged
Jan Lindemann merged 4 commits from jan/feature/20260722-make-improve-make-variable-caching-machinery into master 2026-07-22 14:53:17 +02:00 AGit

This PR improves the generation of makefile snippets caching variables:

  • Most importantly, it makes generation of these files the first thing that happens in a recursive build. Up to now, it happens only after the build entered the make subdirectory.
  • It also names the cache more consistently: .cache-project.mk (project scope) and .cache-projects.mk (caching across all managed projects)

platform.mk: Fix make -C $(TOPDIR)/make clean all

If run from $(TOPDIR), "make clean all" runs fine, because it recurses twice into $(TOPDIR)/make, once for every target. If invoked directly from $(TOPDIR)/make, it can break in two different ways:

If the cache files don't exist, "make clean all" in $(TOPDIR)/make tries to create them too early as implicit make target. This leaves variables empty which should have a value.

If the cache files do exist, "make clean all" in $(TOPDIR)/make includes them, cleans them, and re-creates them from the same variables just read from cache. Undesirable for cache purging.

py-topdir.mk: PY_CHECK_ROOTS: Don't += assign

py-topdir.mk has this:

ifndef PY_CHECK_ROOTS
  PY_CHECK_ROOTS += ...
endif

That is too involved: Either PY_CHECK_ROOTS is defined, then nothing is appended, or it's undefined, then a simple "=" would be just fine. Use that instead.

cache.mk: Include from topdir.mk, not make.mk

During a topdir "make all", caching variables is currently not the first thing that happens. Instead, variables are cached as soon as a project recurses into the make subdirectory. That was necessary, because some makefiles were regenerated in the make subdirectory by autoconf, potentially contributing variables that needed to be cached.

As of now, autoconf is long gone and this is no longer true. And for some variables, the two step process becomes involved, notably for PYTHONPATH, which coding agents would like to look at from the topdir very early on.

This commit moves the .project-cache.mk creation to topdir.mk, and .projects-cache.mk creation to jw-pkg/Makefile to address that.

py-xxx.mk: Clean more python tool caches

.mypy_cache, .ruff_cache and .pytest_cache are not consistently cleaned in all subdirectories. Fix that.

This PR improves the generation of makefile snippets caching variables: - Most importantly, it makes generation of these files the first thing that happens in a recursive build. Up to now, it happens only after the build entered the make subdirectory. - It also names the cache more consistently: `.cache-project.mk` (project scope) and `.cache-projects.mk` (caching across all managed projects) #### platform.mk: Fix make -C $(TOPDIR)/make clean all If run from $(TOPDIR), "make clean all" runs fine, because it recurses twice into $(TOPDIR)/make, once for every target. If invoked directly from $(TOPDIR)/make, it can break in two different ways: If the cache files don't exist, "make clean all" in $(TOPDIR)/make tries to create them too early as implicit make target. This leaves variables empty which should have a value. If the cache files do exist, "make clean all" in $(TOPDIR)/make includes them, cleans them, and re-creates them from the same variables just read from cache. Undesirable for cache purging. #### py-topdir.mk: PY_CHECK_ROOTS: Don't += assign py-topdir.mk has this: ``` ifndef PY_CHECK_ROOTS PY_CHECK_ROOTS += ... endif ``` That is too involved: Either PY_CHECK_ROOTS is defined, then nothing is appended, or it's undefined, then a simple "=" would be just fine. Use that instead. #### cache.mk: Include from topdir.mk, not make.mk During a topdir "make all", caching variables is currently not the first thing that happens. Instead, variables are cached as soon as a project recurses into the make subdirectory. That was necessary, because some makefiles were regenerated in the make subdirectory by autoconf, potentially contributing variables that needed to be cached. As of now, autoconf is long gone and this is no longer true. And for some variables, the two step process becomes involved, notably for PYTHONPATH, which coding agents would like to look at from the topdir very early on. This commit moves the .project-cache.mk creation to topdir.mk, and .projects-cache.mk creation to jw-pkg/Makefile to address that. #### py-xxx.mk: Clean more python tool caches .mypy_cache, .ruff_cache and .pytest_cache are not consistently cleaned in all subdirectories. Fix that.
If run from $(TOPDIR), "make clean all" runs fine, because it
recurses twice into $(TOPDIR)/make, once for every target. If invoked
directly from $(TOPDIR)/make, it can break in two different ways:

If the cache files don't exist, "make clean all" in $(TOPDIR)/make
tries to create them too early as implicit make target. This leaves
variables empty which should have a value.

If the cache files do exist, "make clean all" in $(TOPDIR)/make
includes them, cleans them, and re-creates them from the same
variables just read from cache. Undesirable for cache purging.

Signed-off-by: Jan Lindemann <jan@janware.com>
py-topdir.mk has this:

  ifndef PY_CHECK_ROOTS
    PY_CHECK_ROOTS += ...
  endif

That is too involved: Either PY_CHECK_ROOTS is defined, then nothing
is appended, or it's undefined, then a simple "=" would be just fine.
Use that instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
Jan Lindemann scheduled this pull request to auto merge when all checks succeed 2026-07-22 12:34:07 +02:00
Jan Lindemann force-pushed jan/feature/20260722-make-improve-make-variable-caching-machinery from 7bea55e926
Some checks failed
CI / Packaging - Kali Linux (pull_request) Failing after 3m45s
CI / Packaging - OpenSUSE Tumbleweed (pull_request) Failing after 3m36s
CI / Packaging test (pull_request) Failing after 0s
to 36af4a590f
All checks were successful
CI / Packaging - Kali Linux (pull_request) Successful in 4m6s
CI / Packaging - OpenSUSE Tumbleweed (pull_request) Successful in 4m6s
CI / Packaging test (pull_request) Successful in 0s
CI / Packaging - Kali Linux (push) Successful in 4m2s
CI / Packaging - OpenSUSE Tumbleweed (push) Successful in 3m54s
CI / Packaging test (push) Successful in 0s
2026-07-22 14:44:58 +02:00
Compare
Sign in to join this conversation.
No reviewers
No labels
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
janware/jw-pkg!50
No description provided.