Commit graph

167 commits

Author SHA1 Message Date
c5b964a5bb jwutils.Cmd: Add method add_subcommands()
Add method jwutils.Cmd.add_subcommands(). cmd.add_subcommands(C) will
parse and invoke C as a subcommand of cmd, if C is of type jwutils.cmd.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 16:32:39 +02:00
ad67dd3832 jwutils.Cmds: Beautify log invocations
Add from jwutils import log to avoid unnecessary jwutils.log module
path.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 13:43:08 +02:00
e48fa1b00a Cmds.py: Add early evaluation of --log-xxx
jwutils.log.set_flags() and .set_level() is not set until after all
commands are loaded. This can make debugging complicated. OTOH,
command-line parsing is also not done until all commands are loaded,
because they define the command line options. This commit adds some
poor man's command line parsing for fetching --log-level and
--log-flags only, and applying them before the commands are all
loaded.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:39:01 +02:00
dc532704d0 StringTree: Add support for adding StringTree children
Add support for adding children of type StringTree. If a StringTree
argument is passed to .add(), this should do the right thing. It
makes use of the newly added .__add_children() method and involves
quite a bit of hairy case distinctions.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:35:26 +02:00
dead9d9561 StringTree: add child_list()
Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:31:25 +02:00
160667b6a9 StringTree: Re-implement __str__()
Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:29:14 +02:00
020bb970bd stree/serdes.py: Add support for kwarg "root_content"
parse() and read() up to present automatically set the root content
of a StringTree object to "root". This can now be optionally defined
by the "root_content" keyword argument.

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:14:33 +02:00
79ebb798c7 slog_m(): Support kwarg "caller"
Treat slog_m() to a caller keyword argument, up to now only supported
by slog().

Signed-off-by: Jan Lindemann <jan@janware.com>
2020-04-04 11:12:11 +02:00
08f9aae61a log.get_caller_pos(): Implement frames > 1
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-30 17:46:20 +01:00
26d322023a log.py: Improve console color implementation
o Remove unnecessary function prio_colors()
o Don't output color codes on non-TTY output

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-23 18:01:53 +01:00
2171e1e16e log.py: Add support for flag "date"
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-21 19:08:51 +00:00
a85f0929c6 Consistently replace def run() by async def run()
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-20 08:41:58 +01:00
d3faa8ee85 log.py: Add utilities, better list-support in slog_m()
- Add functions: append_to_prefix(),
  remove_from_prefix() set_filename_length()
- Make slog_m() log one list item per line
- Add console_color_chars(prio)

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-30 15:46:26 +01:00
c96ffe52c0 Add Process and Signals support
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-29 11:25:03 +01:00
6123a68195 Cmds.py: Run all commands in asyncio event loop
This commit makee Cmds run all sub-commands in an asyncio event loop.
The event loop is currently passed to the commands, which seems
unnecessary and looks like it's using a feature which is bound to be
deprecated in the future.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 13:12:18 +01:00
036a978130 log.py: Add slog_m() (multiline logging)
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 12:50:17 +01:00
1cdb38b767 log.py: Make some variables private
Add leading underscore to variables that should not be visible
outside of log.py. Also do some indentation beautification.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 10:59:20 +01:00
ddfb500fa2 Add misc.object_builtin_name()
object_builtin_name() is meant to derive an object name from its
class name. Doesn't work well, but adding the code nonetheless for
improving it later.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-24 18:45:05 +02:00
1f122fae82 Rename the MODDIR makefile variable to JWBDIR
Rename the omnipresent MODDIR variable to JWBDIR, since that's more to the
point.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-29 23:32:20 +02:00
4471ddfd0f Add forgotten jwutils/CppState.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-03 10:00:04 +02:00
faa85f7c69 Flip some more switches from pathon 2 to 3 (untested!)
This commit flips some more switches from Python 2 to Python 3 in makefiles and
Python code. Build runs through, but it's still likely to break things.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-02 14:28:47 +02:00
1c0c5ab884 Attempt to support parsing full lines into StringTree
This commit tries to add support for the

  [section]
  first line
  second line

syntax, i.e. non-assignments, without equal sign. Half-baked, but
still better than before. Also support some more Python 3, i.e.
dict.items() instead of iteritems()

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-05-19 13:15:27 +00:00
6dd594d47b Fix multiple Python 3 compatibility issues
Changes in Python 3 that made the code choke:

  o basestring is merged into str
  o print() needs parentesis
  o Class inheritance syntax changed
  o Abstract baseclass (ABCMeta) syntax changed
  o map.iteritems() is replaced by map.items()
  o Inconsistent use of tabs and spaces are no longer tolerated

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-03-10 16:38:59 +01:00
fd125f07fe Add class Cmds
run_sub_commands() was the only way to access subcommands up to now,
Cmds.run() adds another interface, an object as a place to add
customizations affecting all commands, e.g. global command line
options.

Signed-off-by: Jan Lindemann <jan@janware.com>
2018-12-11 14:10:01 +01:00
2b23026bfa Add HDRDIR_SCOPE_SUFFIX to ytools include files
Change ytools header location from #include <header.h> to
#include <ytools/header.h>.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-12-18 19:32:49 +01:00
838ac06ebb Fix: StringTree misparsed key = "val # blah"
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-26 21:25:01 +01:00
ef79027b06 misc.py: Make process cleanup more robust
Eliminate a far chance of an exception thrown during process cleanup.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:52:22 +01:00
3d93d534db StringTree: Respect quoted quotes
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:34:26 +01:00
4a200c8851 misc.py: Add atomic_store()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:33:18 +01:00
5a87040e10 stree/serdes.py: Allow quoted = in rhs of assignment
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-12 16:08:34 +01:00
c2c409ed4b Move grammar-related stuff into package jw-grammar
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-12 16:08:26 +01:00
930efae39f grammar.py: Add grammar_fix_list_recursion()
This replaces the mechanism to always construct lists with a leading
empty rule. This leads to many RR-conflicts, so this empty rule is omitted
and later added conditionally by grammar_fix_list_recursion(). Sadly,
the condition is bollocks, since it's always true.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-06 12:01:06 +01:00
a2e5f9e1c9 grammar.py: Fix duplicate and missing rules
- unroll_lists() sometimes introduced identical lists into a rule set
- unroll_options() sometimes chopped off a production's last rule

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 18:30:57 +01:00
7007c894b8 StringTree fix: Comments were not ignored in quotes
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:35:41 +01:00
07a328d9a8 grammar.py: Reimplement configured types
Add function grammar_add_configured_types() and keep configured types as
symbols as part of the grammar like all the others

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:34:00 +01:00
c7ecfbff3a grammar.py: Add support for --start-symbols
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-04 11:05:39 +01:00
3cba245b77 grammar.py and friends: Improve line numbers and headers
Line numbers of parsed input are now automatically maintained. More
needed data structures are now automatically created in the API header
file.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-03 12:54:19 +01:00
a3a8313ce8 grammar.py and friends: Make list parsing run through
First time parsing doesn't error out with a syntax error. No usable AST
is produced, strings are not returned from lexer, and AST lists aren't
lists, really.

TEXT:="Hello world!"; had to be excluded from the example, because I
don't get how this could be parsed with the given syntax. There's a
special sequence "all visible characters", but any lexer regex I could
think of will also match the types defining "alphabetic character" and
return the respective tokens (e.g. T_A) or vice-versa, depending on the
order in the lexer input file. I suppose, the only sensible way to
handle this, is to define "all visible characters" by defining the
tokens for the missing characters, and then use them along T_A ... T_Z
or their derived types.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 13:47:19 +01:00
6ca2eeef61 StringTree parser: Add support for comments
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 11:47:04 +01:00
1a7a34f73c grammar.py and friends: Implement config file support
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 08:54:39 +01:00
214c222002 Add StringTree and friends
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-01 13:26:10 +01:00
16ce7abd93 grammar.py and friends: Make test/grammar compile and start
Doesn't successfully parse grammartest.code, yet, it errors out with a
syntax error on whitespace. But at least it compiles and starts.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 13:05:22 +01:00
8c5934725c grammar.py: Add support for --irrelevant-symbols
Arguments to --irrelevant-symbols are not meant to be represented in the
AST resulting from parsing.

Also, add pad() to misc.py.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 10:00:25 +01:00
6297f10f55 grammar.py et al: Centralize more code
More code is removed from the special parser directories and centralized
into grammar.py, Cmd.py, and generate-flex-bison.mk.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 18:25:55 +01:00
fee94deb48 grammar.py: Don't create productions for special sequences
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 14:18:52 +01:00
4fee6ac06e grammar.py: Add support for --replace-whitespace
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:56:52 +01:00
cbdd343f85 grammar.py: Add grammar_unroll_alternatives()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:13:14 +01:00
432d78cdc5 grammar.py: Add grammar_parse_ebnf_tokens()
Add grammar_parse_ebnf_tokens(), to be used by external grammar parsers
and grammar_parse_ebnf()

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-28 19:16:00 +02:00
860f7d8cab Implement caller keyword argument to slog()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-27 18:58:53 +02:00
40e6add5ad Continue implementation of grammar.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 13:45:44 +02:00