Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

some updates for Python 3.13.1 parser #4375

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

some updates for Python 3.13.1 parser #4375

wants to merge 2 commits into from

Conversation

RobEin
Copy link
Contributor

@RobEin RobEin commented Jan 7, 2025

  • parser grammar update for Python 3.13.1
  • added ENCODING token
  • complete rewrite of fstring tokenizer in lexer grammar and PythonLexerBase class
    • now correctly tokenizes the followings in fstring:
      • escape sequences
      • walrus operator
      • dictionary comprehension
      • set comprehension
  • soft keywords changes:
    • no embedded code (semantic predicates) in parser grammar for soft keywords
    • no need for PythonParserBase class
    • no need for transformGrammar.py
    • BREAKING CHANGES:
      • dedicated tokens for soft keywords instead of NAME token:
        • NAME_OR_TYPE
        • NAME_OR_MATCH
        • NAME_OR_CASE
        • NAME_OR_WILDCARD

mlorek and others added 2 commits January 7, 2025 18:31
@mlorek @RobEin
exclude clause (antlr#4374)
48ba78a 
Signed-off-by: Robert Einhorn <[email protected]>
@RobEin
some updates
eba4650 
- parser grammar update for Python 3.13.1
- added ENCODING token
- complete rewrite of fstring tokenizer in lexer grammar and PythonLexerBase class
  - now correctly tokenizes the followings in fstring:
    - escape sequences
    - walrus operator
    - dictionary comprehension
    - set comprehension
- soft keywords changes:
  - no embedded code (semantic predicates) in parser grammar for soft keywords
  - no need for PythonParserBase class
  - no need for transformGrammar.py
  - BREAKING CHANGES:
    - dedicated tokens for soft keywords instead of NAME token:
      - NAME_OR_TYPE
      - NAME_OR_MATCH
      - NAME_OR_CASE
      - NAME_OR_WILDCARD

Signed-off-by: Robert Einhorn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@RobEin @mlorek