jw-python/test/grammar/generate.conf

[symbols]

  [white_space[
    type = token
    lex_extra_action = "if memchr(yytext, '\n', yyleng) context->line++;"
    regex = "[ \n\t\r]+" 
  ]

  [all_characters[
    type = non-terminal
    regex = "[[:print:]]"
    #lex_as = yytext[0]
  ]

  [test[
    type = token
    dings = bums
    regex = "bumsdings"
  ]
grammar.py and friends: Implement config file support Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 08:54:39 +01:00			`[symbols]`
grammar.py and friends: Make list parsing run through First time parsing doesn't error out with a syntax error. No usable AST is produced, strings are not returned from lexer, and AST lists aren't lists, really. TEXT:="Hello world!"; had to be excluded from the example, because I don't get how this could be parsed with the given syntax. There's a special sequence "all visible characters", but any lexer regex I could think of will also match the types defining "alphabetic character" and return the respective tokens (e.g. T_A) or vice-versa, depending on the order in the lexer input file. I suppose, the only sensible way to handle this, is to define "all visible characters" by defining the tokens for the missing characters, and then use them along T_A ... T_Z or their derived types. Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 13:47:19 +01:00
grammar.py and friends: Implement config file support Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 08:54:39 +01:00			`[white_space[`
grammar.py and friends: Make list parsing run through First time parsing doesn't error out with a syntax error. No usable AST is produced, strings are not returned from lexer, and AST lists aren't lists, really. TEXT:="Hello world!"; had to be excluded from the example, because I don't get how this could be parsed with the given syntax. There's a special sequence "all visible characters", but any lexer regex I could think of will also match the types defining "alphabetic character" and return the respective tokens (e.g. T_A) or vice-versa, depending on the order in the lexer input file. I suppose, the only sensible way to handle this, is to define "all visible characters" by defining the tokens for the missing characters, and then use them along T_A ... T_Z or their derived types. Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 13:47:19 +01:00			`type = token`
grammar.py and friends: Improve line numbers and headers Line numbers of parsed input are now automatically maintained. More needed data structures are now automatically created in the API header file. Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-03 12:54:19 +01:00			`lex_extra_action = "if memchr(yytext, '\n', yyleng) context->line++;"`
grammar.py and friends: Implement config file support Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 08:54:39 +01:00			`regex = "[ \n\t\r]+"`
			`]`
grammar.py and friends: Make list parsing run through First time parsing doesn't error out with a syntax error. No usable AST is produced, strings are not returned from lexer, and AST lists aren't lists, really. TEXT:="Hello world!"; had to be excluded from the example, because I don't get how this could be parsed with the given syntax. There's a special sequence "all visible characters", but any lexer regex I could think of will also match the types defining "alphabetic character" and return the respective tokens (e.g. T_A) or vice-versa, depending on the order in the lexer input file. I suppose, the only sensible way to handle this, is to define "all visible characters" by defining the tokens for the missing characters, and then use them along T_A ... T_Z or their derived types. Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 13:47:19 +01:00
			`[all_characters[`
			`type = non-terminal`
			`regex = "[[:print:]]"`
			`#lex_as = yytext[0]`
			`]`

grammar.py and friends: Implement config file support Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 08:54:39 +01:00			`[test[`
grammar.py and friends: Make list parsing run through First time parsing doesn't error out with a syntax error. No usable AST is produced, strings are not returned from lexer, and AST lists aren't lists, really. TEXT:="Hello world!"; had to be excluded from the example, because I don't get how this could be parsed with the given syntax. There's a special sequence "all visible characters", but any lexer regex I could think of will also match the types defining "alphabetic character" and return the respective tokens (e.g. T_A) or vice-versa, depending on the order in the lexer input file. I suppose, the only sensible way to handle this, is to define "all visible characters" by defining the tokens for the missing characters, and then use them along T_A ... T_Z or their derived types. Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 13:47:19 +01:00			`type = token`
grammar.py and friends: Implement config file support Signed-off-by: Jan Lindemann <jan@janware.com> 2017-11-02 08:54:39 +01:00			`dings = bums`
			`regex = "bumsdings"`
			`]`