Commit graph

57 commits

Author SHA1 Message Date
2171e1e16e log.py: Add support for flag "date"
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-21 19:08:51 +00:00
a85f0929c6 Consistently replace def run() by async def run()
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-12-20 08:41:58 +01:00
d3faa8ee85 log.py: Add utilities, better list-support in slog_m()
- Add functions: append_to_prefix(),
  remove_from_prefix() set_filename_length()
- Make slog_m() log one list item per line
- Add console_color_chars(prio)

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-30 15:46:26 +01:00
c96ffe52c0 Add Process and Signals support
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-29 11:25:03 +01:00
6123a68195 Cmds.py: Run all commands in asyncio event loop
This commit makee Cmds run all sub-commands in an asyncio event loop.
The event loop is currently passed to the commands, which seems
unnecessary and looks like it's using a feature which is bound to be
deprecated in the future.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 13:12:18 +01:00
036a978130 log.py: Add slog_m() (multiline logging)
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 12:50:17 +01:00
1cdb38b767 log.py: Make some variables private
Add leading underscore to variables that should not be visible
outside of log.py. Also do some indentation beautification.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-28 10:59:20 +01:00
ddfb500fa2 Add misc.object_builtin_name()
object_builtin_name() is meant to derive an object name from its
class name. Doesn't work well, but adding the code nonetheless for
improving it later.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-10-24 18:45:05 +02:00
1f122fae82 Rename the MODDIR makefile variable to JWBDIR
Rename the omnipresent MODDIR variable to JWBDIR, since that's more to the
point.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-29 23:32:20 +02:00
4471ddfd0f Add forgotten jwutils/CppState.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-03 10:00:04 +02:00
faa85f7c69 Flip some more switches from pathon 2 to 3 (untested!)
This commit flips some more switches from Python 2 to Python 3 in makefiles and
Python code. Build runs through, but it's still likely to break things.

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-06-02 14:28:47 +02:00
1c0c5ab884 Attempt to support parsing full lines into StringTree
This commit tries to add support for the

  [section]
  first line
  second line

syntax, i.e. non-assignments, without equal sign. Half-baked, but
still better than before. Also support some more Python 3, i.e.
dict.items() instead of iteritems()

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-05-19 13:15:27 +00:00
6dd594d47b Fix multiple Python 3 compatibility issues
Changes in Python 3 that made the code choke:

  o basestring is merged into str
  o print() needs parentesis
  o Class inheritance syntax changed
  o Abstract baseclass (ABCMeta) syntax changed
  o map.iteritems() is replaced by map.items()
  o Inconsistent use of tabs and spaces are no longer tolerated

Signed-off-by: Jan Lindemann <jan@janware.com>
2019-03-10 16:38:59 +01:00
fd125f07fe Add class Cmds
run_sub_commands() was the only way to access subcommands up to now,
Cmds.run() adds another interface, an object as a place to add
customizations affecting all commands, e.g. global command line
options.

Signed-off-by: Jan Lindemann <jan@janware.com>
2018-12-11 14:10:01 +01:00
2b23026bfa Add HDRDIR_SCOPE_SUFFIX to ytools include files
Change ytools header location from #include <header.h> to
#include <ytools/header.h>.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-12-18 19:32:49 +01:00
838ac06ebb Fix: StringTree misparsed key = "val # blah"
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-26 21:25:01 +01:00
ef79027b06 misc.py: Make process cleanup more robust
Eliminate a far chance of an exception thrown during process cleanup.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:52:22 +01:00
3d93d534db StringTree: Respect quoted quotes
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:34:26 +01:00
4a200c8851 misc.py: Add atomic_store()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-22 09:33:18 +01:00
5a87040e10 stree/serdes.py: Allow quoted = in rhs of assignment
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-12 16:08:34 +01:00
c2c409ed4b Move grammar-related stuff into package jw-grammar
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-12 16:08:26 +01:00
930efae39f grammar.py: Add grammar_fix_list_recursion()
This replaces the mechanism to always construct lists with a leading
empty rule. This leads to many RR-conflicts, so this empty rule is omitted
and later added conditionally by grammar_fix_list_recursion(). Sadly,
the condition is bollocks, since it's always true.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-06 12:01:06 +01:00
a2e5f9e1c9 grammar.py: Fix duplicate and missing rules
- unroll_lists() sometimes introduced identical lists into a rule set
- unroll_options() sometimes chopped off a production's last rule

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 18:30:57 +01:00
7007c894b8 StringTree fix: Comments were not ignored in quotes
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:35:41 +01:00
07a328d9a8 grammar.py: Reimplement configured types
Add function grammar_add_configured_types() and keep configured types as
symbols as part of the grammar like all the others

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:34:00 +01:00
c7ecfbff3a grammar.py: Add support for --start-symbols
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-04 11:05:39 +01:00
3cba245b77 grammar.py and friends: Improve line numbers and headers
Line numbers of parsed input are now automatically maintained. More
needed data structures are now automatically created in the API header
file.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-03 12:54:19 +01:00
a3a8313ce8 grammar.py and friends: Make list parsing run through
First time parsing doesn't error out with a syntax error. No usable AST
is produced, strings are not returned from lexer, and AST lists aren't
lists, really.

TEXT:="Hello world!"; had to be excluded from the example, because I
don't get how this could be parsed with the given syntax. There's a
special sequence "all visible characters", but any lexer regex I could
think of will also match the types defining "alphabetic character" and
return the respective tokens (e.g. T_A) or vice-versa, depending on the
order in the lexer input file. I suppose, the only sensible way to
handle this, is to define "all visible characters" by defining the
tokens for the missing characters, and then use them along T_A ... T_Z
or their derived types.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 13:47:19 +01:00
6ca2eeef61 StringTree parser: Add support for comments
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 11:47:04 +01:00
1a7a34f73c grammar.py and friends: Implement config file support
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 08:54:39 +01:00
214c222002 Add StringTree and friends
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-01 13:26:10 +01:00
16ce7abd93 grammar.py and friends: Make test/grammar compile and start
Doesn't successfully parse grammartest.code, yet, it errors out with a
syntax error on whitespace. But at least it compiles and starts.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 13:05:22 +01:00
8c5934725c grammar.py: Add support for --irrelevant-symbols
Arguments to --irrelevant-symbols are not meant to be represented in the
AST resulting from parsing.

Also, add pad() to misc.py.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 10:00:25 +01:00
6297f10f55 grammar.py et al: Centralize more code
More code is removed from the special parser directories and centralized
into grammar.py, Cmd.py, and generate-flex-bison.mk.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 18:25:55 +01:00
fee94deb48 grammar.py: Don't create productions for special sequences
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 14:18:52 +01:00
4fee6ac06e grammar.py: Add support for --replace-whitespace
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:56:52 +01:00
cbdd343f85 grammar.py: Add grammar_unroll_alternatives()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:13:14 +01:00
432d78cdc5 grammar.py: Add grammar_parse_ebnf_tokens()
Add grammar_parse_ebnf_tokens(), to be used by external grammar parsers
and grammar_parse_ebnf()

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-28 19:16:00 +02:00
860f7d8cab Implement caller keyword argument to slog()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-27 18:58:53 +02:00
40e6add5ad Continue implementation of grammar.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 13:45:44 +02:00
5b76358238 Add grammar.py for generating compiler-compilers
grammar.py is meant as a compiler compiler compiler, taking in a grammar
structure and returning flex and bison files. It can also parse EBNF.
Currently there are still remnants of VHDL-specific stuff in the code,
because parsing VHDL was what it was initially about.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 12:41:51 +02:00
4e3a6630da log.py: Add set_flags()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 10:06:29 +02:00
51bbed559d Cmd.py: Fix log inclusion
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-16 12:23:24 +02:00
d15242600a log.py: Fix non-working slog()
No idea how this ever worked. Still not sure about the best way.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-16 12:22:49 +02:00
22cabb4eac Add Cmd.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-14 17:25:24 +02:00
0aecc8d076 Make jwutils.log module somewhat usable
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-08-07 18:05:53 +02:00
421ff284cb ShuntingYard.py: Improve tokenize() quote handling
Quoted arguments were not grouped into one token, firstly, because the
parser didn't pay any attention to them, and secondly, because the data
structure carrying the postfixed string converted it back to being a
string, not a list, in column 2 of the result table.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-08-03 10:00:29 +02:00
36aac73b5d Add Object.py and log.py
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-30 18:51:19 +02:00
751afbe93d ShuntingYard: Decrease logging
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 17:53:55 +02:00
da6c32255e Make jwutils.algo.Operator a class
This was a namedtuple before, but I couldn't figure out how to ex-/import it
over module boundaries, so I've made it a class.

Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 16:51:48 +02:00