7007c894b8
StringTree fix: Comments were not ignored in quotes
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:35:41 +01:00
07a328d9a8
grammar.py: Reimplement configured types
...
Add function grammar_add_configured_types() and keep configured types as symbols as part of the grammar like all the others
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-05 16:34:00 +01:00
c7ecfbff3a
grammar.py: Add support for --start-symbols
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-04 11:05:39 +01:00
3cba245b77
grammar.py and friends: Improve line numbers and headers
...
Line numbers of parsed input are now automatically maintained. More needed data structures are now automatically created in the API header file.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-03 12:54:19 +01:00
a3a8313ce8
grammar.py and friends: Make list parsing run through
...
First time parsing doesn't error out with a syntax error. No usable AST is produced, strings are not returned from lexer, and AST lists aren't lists, really.
TEXT:="Hello world!"; had to be excluded from the example, because I don't get how this could be parsed with the given syntax. There's a special sequence "all visible characters", but any lexer regex I could think of will also match the types defining "alphabetic character" and return the respective tokens (e.g. T_A) or vice-versa, depending on the order in the lexer input file. I suppose, the only sensible way to handle this, is to define "all visible characters" by defining the tokens for the missing characters, and then use them along T_A ... T_Z or their derived types.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 13:47:19 +01:00
6ca2eeef61
StringTree parser: Add support for comments
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 11:47:04 +01:00
1a7a34f73c
grammar.py and friends: Implement config file support
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-02 08:54:39 +01:00
214c222002
Add StringTree and friends
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-11-01 13:26:10 +01:00
16ce7abd93
grammar.py and friends: Make test/grammar compile and start
...
Doesn't successfully parse grammartest.code, yet, it errors out with a syntax error on whitespace. But at least it compiles and starts.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 13:05:22 +01:00
8c5934725c
grammar.py: Add support for --irrelevant-symbols
...
Arguments to --irrelevant-symbols are not meant to be represented in the AST resulting from parsing.
Also, add pad() to misc.py.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-30 10:00:25 +01:00
6297f10f55
grammar.py et al: Centralize more code
...
More code is removed from the special parser directories and centralized into grammar.py, Cmd.py, and generate-flex-bison.mk.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 18:25:55 +01:00
fee94deb48
grammar.py: Don't create productions for special sequences
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 14:18:52 +01:00
4fee6ac06e
grammar.py: Add support for --replace-whitespace
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:56:52 +01:00
cbdd343f85
grammar.py: Add grammar_unroll_alternatives()
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-29 13:13:14 +01:00
432d78cdc5
grammar.py: Add grammar_parse_ebnf_tokens()
...
Add grammar_parse_ebnf_tokens(), to be used by external grammar parsers and grammar_parse_ebnf()
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-28 19:16:00 +02:00
860f7d8cab
Implement caller keyword argument to slog()
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-27 18:58:53 +02:00
40e6add5ad
Continue implementation of grammar.py
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 13:45:44 +02:00
5b76358238
Add grammar.py for generating compiler-compilers
...
grammar.py is meant as a compiler compiler compiler, taking in a grammar structure and returning flex and bison files. It can also parse EBNF. Currently there are still remnants of VHDL-specific stuff in the code, because parsing VHDL was what it was initially about.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 12:41:51 +02:00
4e3a6630da
log.py: Add set_flags()
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-25 10:06:29 +02:00
51bbed559d
Cmd.py: Fix log inclusion
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-16 12:23:24 +02:00
d15242600a
log.py: Fix non-working slog()
...
No idea how this ever worked. Still not sure about the best way.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-16 12:22:49 +02:00
22cabb4eac
Add Cmd.py
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-10-14 17:25:24 +02:00
0aecc8d076
Make jwutils.log module somewhat usable
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-08-07 18:05:53 +02:00
421ff284cb
ShuntingYard.py: Improve tokenize() quote handling
...
Quoted arguments were not grouped into one token, firstly, because the parser didn't pay any attention to them, and secondly, because the data structure carrying the postfixed string converted it back to being a string, not a list, in column 2 of the result table.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-08-03 10:00:29 +02:00
36aac73b5d
Add Object.py and log.py
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-30 18:51:19 +02:00
751afbe93d
ShuntingYard: Decrease logging
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 17:53:55 +02:00
da6c32255e
Make jwutils.algo.Operator a class
...
This was a namedtuple before, but I couldn't figure out how to ex-/import it over module boundaries, so I've made it a class.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 16:51:48 +02:00
1259b16837
Re-implement ShuntingYard.infix_to_postfix()
...
The previous implementation had no concept of associativity.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 15:18:17 +02:00
81ca793a6e
Add algo.ShuntingYard
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-27 09:14:45 +02:00
6f1fcdc693
Rename MuteStdIO to RedirectStdIO
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-26 11:40:35 +02:00
3451ea0d83
MuteStdIO: Partly support ctor arguments
...
ctor now understands stdio='off' or ='something' and mutes stdout in the first case.
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-26 11:38:38 +02:00
f6e5d25df7
Add misc.py
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-25 17:06:03 +02:00
4c076691a6
Rename module jwutil to jwutils
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-24 13:00:37 +02:00
5755d14d84
initial checkin
...
Signed-off-by: Jan Lindemann <jan@janware.com>
2017-07-24 12:19:34 +02:00
