Commit graph

192 commits

Author SHA1 Message Date
30c81af1b0 stree.serdes: Only ignore ENOENT for -include

-include ignores too many failures. It should only ignore ENOENT, all others should throw an exception regardless.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-03 18:03:15 +02:00
1347a34481 stree.serdes: Support glob patterns for includes

Allow -include /some/dir/app.d/*.conf style includes.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-03 18:02:10 +02:00
bdb7127d27 ldap.Connection.walk(): Support decode & unroll

The entry instance passed to the walk callback contains raw results of LDAP search operations, i.e. all attribute values are lists, and all attribute values are bytes. Add the boolean parameters decode and unroll to walk() as a convenience method to get decoded values. They default to False, representing current behaviour.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-03 18:01:02 +02:00
988d420e44 ldap.Connection: Support uri config option

Currently the configuration passed to the Connection constructor needs to contain an ldap_uri entry. Add "uri" as alias, because ldap_uri for the LDAP config in many contexts represents a tautology and is left out.

Signed-off-by: Jan Lindemann <jan@janware.com>
2026-04-03 18:00:24 +02:00
9b5cd1b857 Cmd / Cmds: Update legacy type annotation
Signed-off-by: Jan Lindemann <jan@janware.com>
2026-01-20 18:56:15 +01:00
d8d514a4d1 Fix errors reported by mypy
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-02 09:42:21 +01:00
e071df39ab log._clean_str_regex: Fix missing r-qualifier

Fix missing raw string qualifier missing from regex with backslashes.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-11-02 09:41:14 +01:00
9dd957c9d3 jwutils.Options: Increase option parsing error verbosity
Signed-off-by: Jan Lindemann <jan@janware.com>
2024-01-18 12:22:08 +01:00
82c0e6fe2e jwutils.Options: Add support for duplicate keys
Signed-off-by: Jan Lindemann <jan@janware.com>
2023-01-17 13:33:44 +01:00
12bf59b3cd os.misc: Add update_symlink()

update_symlink(target, link_name) does the equivalent of ln -sf, i.e. force replacement of link_name with a new sysmlink, if it existed before the call.

Signed-off-by: Jan Lindemann <jan@janware.com>
2023-01-17 11:40:58 +01:00
a4fb791649 Cmd / Cmds: Add config file support

An application based on Cmds will now try to read StringTree formatted config data from a file $HOME/.<name>rc, and make the data available via the newly added conf_value() method.

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-12-08 16:44:49 +01:00
b7de950750 Add log.add_log_file()

log.add_log_file(path) allows to pass a file path log messages will be written into. Special characters (e.g. console color codes) will be removed from the messages.

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-12-08 16:41:27 +01:00
8642b73ba1 misc: Add multi_regex_edit()
Signed-off-by: Jan Lindemann <jan@janware.com>
2022-12-04 12:34:52 +01:00
1ffdb8728a Add class Options

The Options constructor takes an options string, parses it and makes the options available via __getitem__().

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-12-01 14:29:01 +01:00
a52557570b misc.py: from . import log

Do "from . import log" instead of "from jwutils import log". It works and looks cleaner than a module referencing itself. Not entirely sure if that's the way to go, I must admit.

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-12-02 12:14:00 +01:00
56357ae0a3 Add asyncio.ShellCmd

Add class ShellCmd, an easy API to run shell commands as async subprocesses, capture their exit status, and have them log to slog().

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-11-03 12:25:45 +01:00
f8a31e01f8 Remove asyncio/__init__.py

__init__.py is an automatically generated file, so don't include it in git.

Signed-off-by: Jan Lindemann <jan@janware.com>
2022-11-03 12:39:16 +01:00
bfd0544ff8 Fix errors reported by mypy
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-10-13 12:45:51 +02:00
d7db0cac47 ldap.Auth: Fix commit 72d56d9

Commit 72d56d9 (Move type argument into config) swapped the order of arguments in Auth.load(), but the ctor of ldap.Auth didn't follow the change, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-18 23:11:22 +02:00
72d56d9b63 auth.Auth.load(): Move type argument into config

Auth.load() takes a "tp" argument, specifying the Auth module name to load. In case of jw-devops, this is part of the configuration, which is also passed as an argument. Make taking 'type' from the config the default. Note that this changes the prototype.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-12 08:25:16 +02:00
c941ef307c jwutils.ldap: Introduce 'password' alias for 'bind_pw'

jw-client-devops configures its databases with a set of config keys which are identical - except for the 'bind_pw' key, which is called 'password'. Allow that, too.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-11 22:33:47 +02:00
bc9fa57a2b ldap.Auth: Fix half-baked use of jwutils.ldap module

ldap.Auth lost a lot of code when jwutils.ldap was introduced, and rightfully so, because jwutils.ldap contains most of it. OTOH, it was used wrongly, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-09-11 21:35:01 +02:00
1e43cdc715 jwutils.ldap: Add module
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-26 13:51:42 +02:00
e4cee86482 Config.branch(): Add throw argument

In case a branch doesn't exist, branch() throws an error. Allow to return None instead.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-27 18:58:28 +02:00
89578b788f graph.yed.MapAttr2Shape: Add Class

MapAttr2Shape is a class which allows processing GraphML files, such that nodes get added shape attributes understood by the yEd editor, based on the values of other node attributes. This allows to programmatically visualize node attributes.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-27 18:12:05 +02:00
b2b726b632 Bunch: Add class

jwutils.Bunch is a simple wrapper around a dictionary, which allows member-type access to to dictionary data.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-27 14:11:42 +02:00
ceea557a61 algo.ShuntingYard: Increase debug logging

Make ShuntingYard log more. Might be overkill in which case the change should be reversed.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-11 10:46:19 +02:00
1e16c0ffb1 Cmds: Add argcomplete support

argcomplete takes the arguments added to argparse, and builds bash completion with it. Add it to all Cmds based executables.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-11 10:43:47 +02:00
428692ea3a Streamline Python file headers somewhat
- Add coding statement - Import all modules in one line where possible - Order: __future__, typing, plain imports, from imports, janware modules
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-10 05:14:06 +02:00
862a5dbc95 ArgsContainer.specified_args: Add property

ArgsContainer.specified_args is a list of arguments that were explicitly specified, i.e. not defaults, in the order they were specified in.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-06 07:12:52 +02:00
d35a5588cd Cmds.__run(): Don't exit(0)

WSGI doesn't like sys.exit() being called, so avoid it. Two cases need to be taken into consideration:

1. No exception thrown by self.args.func()

The variable exit_status is zero, we can check that and _not_ call sys.exit()

2. Exception thrown by self.args.func()

In that case, the exception should be raised, to be caught by WSGI / ASGI whatever to do what they want with it. The code calling sys.exit() is never reached. Hence, we need to add --backtrace to the invocation options.
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-05 12:27:11 +02:00
1a267dd2e5 cast.from_env(): Add function

cast.from_env() takes an environment variable name instead of the string as the first argument and does the obvious thing. It also takes a default and saves the conversion in case the environment variable doesn't exist.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-05 10:54:30 +02:00
8b6504f01b fixup! algo.ShuntingYard: Use slog() for debugging
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-04 10:22:14 +02:00
49e9b5ab5d algo.ShuntingYard.tokenize(): Allow mixed quotes

"A string which contains 'single quotes'" or 'a string containing "double quotes"' is not tokenized as quoted. Fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-04 08:59:58 +02:00
27e2a49d1e algo.ShuntingYard: Use slog() for debugging
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-07-04 08:58:42 +02:00
271ea32dcd algo.ShuntingYard.tokenize(): Support double quotes

tokenize() only sorts single quoted strings into QUOTED, fix that.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-27 09:46:12 +02:00
29f57ec187 algo.ShuntingYard: Add operator()
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-09 09:16:48 +02:00
2efe2f97fa Config.__getitem__(): Raise KeyError if need be
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-06 13:39:24 +02:00
8a316ead21 auth: Add LDAP support
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-05 20:48:14 +02:00
0a1a6e5e17 log.get_caller_pos(): Fix None module

<W> jw-client-devops-web.py [ 29] Exiting jw-client-devops-web.py Exception ignored in: <function App.__del__ at 0x7f3ddeb8f560> Traceback (most recent call last): File "/home/jan/local/src/jw.dev/proj/jw-client-devops-web/src/python/jw/devops/client/web/cmds/App.py", line 43, in __del__ File "/home/jan/local/src/jw.dev/proj/jw-python/tools/python/jwutils/log.py", line 167, in slog File "/home/jan/local/src/jw.dev/proj/jw-python/tools/python/jwutils/log.py", line 126, in get_caller_pos AttributeError: 'NoneType' object has no attribute '__name__'

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-05 22:57:55 +02:00
958c077da3 Cmds: Support --write-profile <filename>

Add an option which makes the program write profiling statistics in pstats format to a specified file.

Visualize e.g. with:

gprof2dot -f pstats <filename> -o <dotfile> dot -Tpng | display <dotfile>

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-05 16:58:26 +02:00
d2ec56336e ArgsContainer: Add add_argument() function

add_argument() at the face of it looks much like ArgumentParser.add_argument(), but is a function, not a class method. It takes the ArgsContainer|ArgParser instance as first argument, then decides what type it is, and proceeds to use this knowledge to decide whether or not the argument to be added already has a definition.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-04 04:55:32 +02:00
6e70529f7a log.(add|rm)_capture_stream(): Add functions

Add functions log.(add|rm)_capture_stream(). The functions are meant to capture everything passed to syslog into the stream objects installed by it. Not sure about future semantics changes: add_capture_stream() takes a currently unused flags argument, and it suppresses everything else logged until the stream is removed again.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-03 15:25:44 +02:00
748b850532 log.set_level(): Return current flags

Make log.set_level() return the flags that were set before setting the flags passed as argument.

Support None as flags argument, in which case it doesn't change anything and only returns the currently set flags.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-03 15:17:08 +02:00
cf122e44be log.slog(): Support log flag "module"

If module is present in the log flags string, the module is prepended to each log message. The length of the module prefix can be set via the new set_module_name_length() function.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-06-01 15:14:48 +02:00
5ee88c1a81 cmd.Cmds: Support --backtrace

Wrap all execution in try-block, don't raise in except block unless --backtrace is passed to the command line.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-05-30 15:53:43 +02:00
b464f20e6c stree.StringTree.find(): Add method

StringTree.find(key, val) (and Config.find(), for that matter) returns a list of paths with sections containing children matching key / val pairs. One of them can be None, which acts as a wildcard.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-05-29 12:05:10 +02:00
a9fa274bb1 stree.StringTree: Add property path
Signed-off-by: Jan Lindemann <jan@janware.com>
2025-05-29 12:04:52 +02:00
9c13381e7a cast.cast_str_to_timedelta(): Add conversion method

Make cast.from_str() accept time strings.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-05-28 11:12:49 +02:00
296dde387c cast.from_str(): Add as alias to cast_str()

Using cast.from_str('blah') seems more logical than cast.cast_str(), so add that alias and see how it fares.

Signed-off-by: Jan Lindemann <jan@janware.com>
2025-05-28 11:11:11 +02:00