Skip to content

Commit

Permalink
Fixed:
Browse files Browse the repository at this point in the history
- Fixed leading zero issue in decimal number for lex token interpreter, e.g. +012345 is now interpreted correctly.
 - Added support for UTF-8, UTF-16, UTF-32, ISO encoding for A2L input files.
- Fixed handling of empty blocks inside IF_DATA, e.g. "/begin TEST_X /end TEST_X" (allowing and saving to AST now).

Refactor:
- Moved a2lparser/a2lparser.py to a2lparser/main.py
- Moved a2lparser/a2l/parser.py to a2lparser/a2lparser.py
- Moved a2lparser/a2l/parsing_exception.py to a2lparser/a2lparser_exception.py
- Moved Loguru format initialization to A2LParser constructor.
- Changed PyPI and build workflow names
  • Loading branch information
mrom1 committed Jul 17, 2024
1 parent 92b8246 commit 855884b
Show file tree
Hide file tree
Showing 24 changed files with 651 additions and 367 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: pypi
name: build

# Controls when the action will run.
on:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI
name: pypi publish

on: push

Expand Down
78 changes: 44 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,39 @@
# Python A2L Parser

![main_workflow](https://github.com/mrom1/a2lparser/actions/workflows/main.yml/badge.svg)
![build_workflow](https://github.com/mrom1/a2lparser/actions/workflows/build.yml/badge.svg)
![flake8_workflow](https://github.com/mrom1/a2lparser/actions/workflows/flake8.yml/badge.svg)
![Main Workflow](https://github.com/mrom1/a2lparser/actions/workflows/main.yml/badge.svg)
![PyPI Workflow](https://github.com/mrom1/a2lparser/actions/workflows/publish-to-pypi.yml/badge.svg)
![Build Workflow](https://github.com/mrom1/a2lparser/actions/workflows/build.yml/badge.svg)
![Flake8 Workflow](https://github.com/mrom1/a2lparser/actions/workflows/flake8.yml/badge.svg)
[![codecov](https://codecov.io/gh/mrom1/a2lparser/branch/main/graph/badge.svg?token=CZ74J83NO2)](https://codecov.io/gh/mrom1/a2lparser)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)

This A2L Parser, implemented in Python using [PLY](https://ply.readthedocs.io/en/latest/index.html), serves the purpose of reading A2L files according to the [ASAM MCD-2 MC](https://www.asam.net/standards/detail/mcd-2-mc/) Data Model for ECU Measurement and Calibration Standard. All resources utilized in developing this project are derived from publicly available information, such as the [ASAM Wiki](https://www.asam.net/standards/detail/mcd-2-mc/wiki/).
## Overview

This Python module enables the parsing of A2L files into an Abstract Syntax Tree with dictionary access in Python. Moreover, it provides functionality to convert the parsed A2L file into simpler formats, currently supporting XML, JSON, or YAML conversions. It's important to note that this project focuses solely on parsing the A2L grammar and does not provide mapping capabilities. It has been entirely rewritten from the original codebase and now fully supports ASAM MCD-2 MC Version 1.7.1.
The Python A2L Parser is a tool designed for reading A2L files compliant with the [ASAM MCD-2 MC](https://www.asam.net/standards/detail/mcd-2-mc/) Data Model for ECU Measurement and Calibration. This parser, implemented in Python using [PLY](https://ply.readthedocs.io/en/latest/index.html), constructs an Abstract Syntax Tree (AST) from A2L files, allowing for structured data access and utility functions like searching.

Released under the GPL license with no warranty, it is recommended primarily for educational purposes. For professional solutions, consider exploring specialized companies in this domain, such as the [MATLAB Vehicle Network Toolbox](https://www.mathworks.com/help/vnt/index.html) or the [Vector ASAP2 Toolset](https://www.vector.com/int/en/products/products-a-z/software/asap2-tool-set/).
This project supports ASAM MCD-2 MC Version 1.7.1 and focuses on parsing A2L grammar, not providing mapping capabilities. The module also includes functionality for converting parsed A2L files into simpler formats like XML, JSON, and YAML.

## Installation

```console
pip install a2lparser
```
You can use this repository to interpret A2L files, build upon this functionality, or for educational purposes.

## Usage from CLI
**Note:** This project is released under the GPL license with no warranty and is recommended for educational purposes. For professional solutions, consider exploring specialized tools such as the [MATLAB Vehicle Network Toolbox](https://www.mathworks.com/help/vnt/index.html) or the [Vector ASAP2 Toolset](https://www.vector.com/int/en/products/products-a-z/software/asap2-tool-set/).

```console
a2lparser --help
usage: a2lparser [-h] [-x] [-j] [-y] [--no-prompt] [--no-optimize] [--no-validation] [--output-dir [OUTPUT_DIR]]
[--gen-ast [GEN_AST]] [--version]
[filename]
## Installation

positional arguments:
filename A2L file(s) to parse
To install the A2L Parser, run:

options:
-h, --help show this help message and exit
-x, --xml Converts an A2L file to a XML output file
-j, --json Converts an A2L file to a JSON output file
-y, --yaml Converts an A2L file to a YAML output file
--no-prompt Disables CLI prompt after parsing
--no-validation Disables possible A2L validation warnings
--output-dir [OUTPUT_DIR]
Output directory for converted files
--gen-ast [GEN_AST] Generates python file containing AST node classes
--version show program's version number and exit
```
```console
pip install -i https://test.pypi.org/simple/ a2lparser --extra-index-url https://pypi.org/simple/
```

## Usage as Module

```python
from a2lparser.a2l.parser import Parser
from a2lparser.a2lparser import A2LParser
from a2lparser.a2lparser_exception import A2LParserException

try:
# Create Parser and parse files
ast = Parser().parse_files(files="./data/test.a2l")
ast = A2LParser(quiet=True).parse_file(files="./data/test.a2l")

# Dictionary access on abstract syntax tree
module = ast["test.a2l"]["PROJECT"]["MODULE"]
Expand All @@ -58,6 +42,32 @@ try:
measurements = ast.find_sections("MEASUREMENT")
print(measurements)

except Exception as ex:
except A2LParserException as ex:
print(ex)
```

## Usage from CLI

```console
a2lparser --help
usage: a2lparser [-h] [-x] [-j] [-y] [--output-dir [PATH]] [--prompt] [--quiet] [--no-optimize] [--no-validation]
[--gen-ast [CONFIG]] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--version]
[file]

positional arguments:
file A2L files to parse

options:
-h, --help show this help message and exit
-x, --xml Converts an A2L file to a XML output file
-j, --json Converts an A2L file to a JSON output file
-y, --yaml Converts an A2L file to a YAML output file
--output-dir [PATH] Output directory for converted files
--prompt Enables CLI prompt after parsing
--quiet Disables console output
--no-optimize Disables optimization mode
--no-validation Disables possible A2L validation warnings
--gen-ast [CONFIG] Generates python file containing AST node classes
--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
--version show program's version number and exit ```
```
5 changes: 2 additions & 3 deletions a2lparser/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,14 @@
Node()

except ImportError:
print("First time initialization...")
print("No AST node classes found. Generating AST nodes...")
from a2lparser.a2l.ast.ast_generator import ASTGenerator

# Generate the AST nodes from the standard config in configs/A2L_ASAM.cfg
asam_config = A2L_CONFIGS_DIR / A2L_DEFAULT_CONFIG_NAME
ast_nodes_file = A2L_GENERATED_FILES_DIR / "a2l_ast.py"

# Generate the AST node containers
print("Generating python file containing the AST nodes...")
generator = ASTGenerator(asam_config.as_posix(), ast_nodes_file.as_posix())
generator.generate()
print(f"Generated file at: {ast_nodes_file.as_posix()}")
print(f"Generated AST nodes file at: {ast_nodes_file.as_posix()}")
14 changes: 9 additions & 5 deletions a2lparser/a2l/a2l_yacc.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from a2lparser.a2l.rules.rules_sections import RulesSections
from a2lparser.a2l.rules.rules_datatypes import RulesDatatypes
from a2lparser.a2l.rules.rules_sections_errorhandlers import RulesSectionsErrorhandlers
from a2lparser.a2l.parsing_exception import ParsingException
from a2lparser.a2lparser_exception import A2LParserException
from a2lparser.a2l.ast.abstract_syntax_tree import AbstractSyntaxTree
import a2lparser.gen.a2l_ast as ASTNodes

Expand Down Expand Up @@ -82,19 +82,23 @@ def __init__(
self.debug = debug
self.a2l_sections_list = []

def generate_ast(self, content: str) -> AbstractSyntaxTree:
def generate_ast(self, content: str, show_progressbar: bool = True) -> AbstractSyntaxTree:
"""
Generates an AbstractSyntaxTree from an input string.
"""
content_lines = content.count("\n")
with alive_bar(content_lines) as progressbar:
self.a2l_lex.progressbar = progressbar

if show_progressbar:
with alive_bar(content_lines) as progressbar:
self.a2l_lex.progressbar = progressbar
ast = self.a2l_yacc.parse(input=content, lexer=self.a2l_lex, debug=self.debug)
else:
ast = self.a2l_yacc.parse(input=content, lexer=self.a2l_lex, debug=self.debug)

if hasattr(ast, "node") and ast.node is not None:
return AbstractSyntaxTree(ast.node)

raise ParsingException("Unable to parse given input. Generated AST is empty!")
raise A2LParserException("Unable to parse given input. Generated AST is empty!")

##################################################
# General Parsing rules and starting point. #
Expand Down
2 changes: 1 addition & 1 deletion a2lparser/a2l/lex/lexer_regex.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ class LexerRegex:
hex_digits = "[0-9a-fA-F]+"
bin_prefix = "[+-]?0[bB]"
integer_suffix_opt = r"(([uU]ll)|([uU]LL)|(ll[uU]?)|(LL[uU]?)|([uU][lL])|([lL][uU]?)|[uU])?"
decimal_constant = f"([+-]?0{integer_suffix_opt})|([+-]?[1-9][0-9]*{integer_suffix_opt})"
decimal_constant = f"[+-]?([0-9]+){integer_suffix_opt}"
hex_prefix = "[+-]?0[xX]"
hex_constant = hex_prefix + hex_digits + integer_suffix_opt
exponent_part = r"""([eE][-+]?[0-9]+)"""
Expand Down
156 changes: 0 additions & 156 deletions a2lparser/a2l/parser.py

This file was deleted.

9 changes: 8 additions & 1 deletion a2lparser/a2l/rules/rules_sections.py
Original file line number Diff line number Diff line change
Expand Up @@ -1342,7 +1342,8 @@ def p_if_data_opt_block(self, p):
"""
if_data_opt : if_data_block
"""
p[0] = p[1]
if p[1]:
p[0] = p[1]

def p_if_data_opt_list(self, p):
"""
Expand All @@ -1363,6 +1364,12 @@ def p_if_data_block(self, p):
data_params = [x for x in p[2] if not isinstance(x, ASTNodes.If_Data_Block)]
p[0] = ASTNodes.If_Data_Block(Name=p[1], DataParams=data_params, If_Data_Block=if_data_block)

def p_if_data_block_empty(self, p):
"""
if_data_block : if_data_block_begin if_data_block_end
"""
p[0] = ASTNodes.If_Data_Block(Name=p[1])

def p_if_data_block_begin(self, p):
"""
if_data_block_begin : BEGIN ident
Expand Down
Loading

0 comments on commit 855884b

Please sign in to comment.