lib.App / lib.Cmd: Build command tree lazily

Jan Lindemann commented

2026-08-18 01:06:04 +02:00

Owner

This PR's purpose is performance optimization. jw-pkg parses the command line twice, and then loads the entire command tree, including subtrees that are unneeded for an invocation. Both can be done more efficiently.

lib.Cmd: Build the command tree lazily

App() construction builds the entire command tree: every load_subcommands() call in a command's init() constructs its whole subtree eagerly, so running a single leaf command pays to instantiate every unrelated command object as well (jw-pkg builds about 50 command objects for any invocation).

Defer the construction instead. load_subcommands() now records only the module search path and name filter, and the subcommands are materialized on first access to the children or child_classes property. The parser reads children only down the invoked branch on the non-help path, so a simple run instantiates just that path (jw-pkg builds 5-7 objects), while the help and completion path expands every node and leaves rendered help unchanged.

lib.App: Find invoked path by walking argv

The discovery re-parse re-parses the full command line at each level against the subparsers registered so far. A level's subcommand parsers are not registered until the re-parse descends into them, so the tokens after the subcommand name are parsed against the current level's options. An option meant for a deeper level can then collide with a same-named option of a shallower level, or fail on a missing value.

Replace the re-parse with a single argv walk. The subcommand names at each level are known once the commands are materialized, so the invoked path is found by matching tokens against those names and skipping the options (and the values they consume) that precede them. Walking the tokens never parses the tail against a half-built parser, so the collision cannot occur.

This PR's purpose is performance optimization. jw-pkg parses the command line twice, and then loads the entire command tree, including subtrees that are unneeded for an invocation. Both can be done more efficiently. #### lib.Cmd: Build the command tree lazily App() construction builds the entire command tree: every load_subcommands() call in a command's __init__() constructs its whole subtree eagerly, so running a single leaf command pays to instantiate every unrelated command object as well (jw-pkg builds about 50 command objects for any invocation). Defer the construction instead. load_subcommands() now records only the module search path and name filter, and the subcommands are materialized on first access to the children or child_classes property. The parser reads children only down the invoked branch on the non-help path, so a simple run instantiates just that path (jw-pkg builds 5-7 objects), while the help and completion path expands every node and leaves rendered help unchanged. #### lib.App: Find invoked path by walking argv The discovery re-parse re-parses the full command line at each level against the subparsers registered so far. A level's subcommand parsers are not registered until the re-parse descends into them, so the tokens after the subcommand name are parsed against the current level's options. An option meant for a deeper level can then collide with a same-named option of a shallower level, or fail on a missing value. Replace the re-parse with a single argv walk. The subcommand names at each level are known once the commands are materialized, so the invoked path is found by matching tokens against those names and skipping the options (and the values they consume) that precede them. Walking the tokens never parses the tail against a half-built parser, so the collision cannot occur.

Jan Lindemann added 2 commits

2026-08-18 01:06:04 +02:00

lib.Cmd: Build the command tree lazily 23622fedc6

App() construction builds the entire command tree: every
load_subcommands() call in a command's __init__() constructs its whole
subtree eagerly, so running a single leaf command pays to instantiate
every unrelated command object as well (jw-pkg builds about 50 command
objects for any invocation).

Defer the construction instead. load_subcommands() now records only the
module search path and name filter, and the subcommands are materialized
on first access to the children or child_classes property. The parser
reads children only down the invoked branch on the non-help path, so a
simple run instantiates just that path (jw-pkg builds 5-7 objects),
while the help and completion path expands every node and leaves
rendered help unchanged.

Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>

lib.App: Find invoked path by walking argv

CI / Packaging - Kali Linux (pull_request) Successful in 4m9s

Details

CI / Packaging - OpenSUSE Tumbleweed (pull_request) Successful in 4m23s

Details

CI / Packaging test (pull_request) Successful in 0s

Details

105c171908

The discovery re-parse re-parses the full command line at each level
against the subparsers registered so far. A level's subcommand parsers
are not registered until the re-parse descends into them, so the tokens
after the subcommand name are parsed against the current level's
options. An option meant for a deeper level can then collide with a
same-named option of a shallower level, or fail on a missing value.

Replace the re-parse with a single argv walk. The subcommand names at
each level are known once the commands are materialized, so the invoked
path is found by matching tokens against those names and skipping the
options (and the values they consume) that precede them. Walking the
tokens never parses the tail against a half-built parser, so the
collision cannot occur.

Assisted-by: unsloth/Qwen3.8-27B-GGUF:Q4_K_M with pi.dev v0.84.2
Signed-off-by: Jan Lindemann <jan@janware.com>

Jan Lindemann scheduled this pull request to auto merge when all checks succeed

2026-08-18 01:06:05 +02:00

Jan Lindemann merged commit e3cff8104f into master

2026-08-18 01:14:40 +02:00

Rows
Columns

lib.App / lib.Cmd: Build command tree lazily #68

lib.Cmd: Build the command tree lazily

lib.App: Find invoked path by walking argv