Liang

.gitignore

@@ -30,3 +30,4 @@
 *.exe
 *.out
 *.app
+build.sh

IREE-test-case-archive/README.md

@@ -0,0 +1,36 @@
+# MLIR-DLModels
+
+This is a repository of deep neural network models that are compiled with MLIR using dialects in different abstraction levels. Currently, the original models are from [IREE](https://github.com/google/iree) end-to-end test cases.
+## Commit ID
+The model MLIRs are lowered and verified using tools in MLIR, MHLO, and IREE projects. Here are the commit IDs of the project repos.
+- [IREE](https://github.com/google/iree): 794bbf4c23e790d9f897d7b6eebd0522e7c4af47
+- [LLVM](https://github.com/llvm/llvm-project): 0fbe3f3f486e01448121f7931a4ca29fac1504ab
+- [MHLO](https://github.com/tensorflow/mlir-hlo): e4d50a80ec92031c21a216e2855803cad1b7269a
+
+## File Descriptions
+
+### Directories
+- **original-iree-model**: the same models in IREE's end to end tests. Verified using command:
+```
+iree-run-mlir --iree-input-type=mhlo --iree-hal-target-backends=dylib-llvm-aot
+```
+- **mhlo-dialect-only**: replace IREE specific operations (Util) with arith dialect.
+- **lowered-linalg-dialect**: the Linalg dialect verision that is lowered from MHLO. Command:
+```
+mlir-hlo-opt -mhlo-test-unfuse-batch-norm -cse -hlo-legalize-to-linalg
+```
+- **trueth-llvm-dialect**: the LLVM dialect verision that is lowered from Linalg.
+
+Lowering command:
+```
+mlir-opt -convert-tensor-to-linalg -linalg-bufferize -arith-bufferize  -tensor-bufferize -func-bufferize -buffer-deallocation -convert-linalg-to-loops -convert-scf-to-cf  -convert-linalg-to-llvm -lower-affine -convert-scf-to-cf --convert-memref-to-llvm -convert-func-to-llvm -reconcile-unrealized-casts --arith-expand --convert-arith-to-llvm --convert-math-to-llvm -finalizing-bufferize
+```
+Verify command:
+```
+mlir-cpu-runner -e main -entry-point-result=void    -shared-libs=/path/to/libmlir_c_runner_utils.so,/path/to/libmlir_runner_utils.so
+```
+### Files
+
+- **IREE-swap.py**: swaps IREE util.global, util.address, and util.load operations to arith constant operation.
+
+- **makefile**: records the commands and flags to reproduce the lowering and verification processes. 

README.md

@@ -1,36 +1,46 @@
-# MLIR-DLModels
+# Python-based MLIR test case generation tool
 
-This is a repository of deep neural network models that are compiled with MLIR using dialects in different abstraction levels. Currently, the original models are from [IREE](https://github.com/google/iree) end-to-end test cases.
-## Commit ID
-The model MLIRs are lowered and verified using tools in MLIR, MHLO, and IREE projects. Here are the commit IDs of the project repos.
-- [IREE](https://github.com/google/iree): 794bbf4c23e790d9f897d7b6eebd0522e7c4af47
-- [LLVM](https://github.com/llvm/llvm-project): 0fbe3f3f486e01448121f7931a4ca29fac1504ab
-- [MHLO](https://github.com/tensorflow/mlir-hlo): e4d50a80ec92031c21a216e2855803cad1b7269a
+This is a tool for generation MLIR test cases
+## The function of this tool
+- Inputs: a directory of MLIR function files, a json file specifies the inputs and outputs to these functions 
+- Outputs: a directory of these functions that are added the main function with inputs, and Filecheck flags for verification. The verification scripts are also provided 
 
-## File Descriptions
+## Directories
+- **src**: the Source Files of the tool
+- **test**: The generated MLIR test cases 
+## Source Files
 
-### Directories
-- **original-iree-model**: the same models in IREE's end to end tests. Verified using command:
-```
-iree-run-mlir --iree-input-type=mhlo --iree-hal-target-backends=dylib-llvm-aot
-```
-- **mhlo-dialect-only**: replace IREE specific operations (Util) with arith dialect.
-- **lowered-linalg-dialect**: the Linalg dialect verision that is lowered from MHLO. Command:
-```
-mlir-hlo-opt -mhlo-test-unfuse-batch-norm -cse -hlo-legalize-to-linalg
-```
-- **trueth-llvm-dialect**: the LLVM dialect verision that is lowered from Linalg.
+- useful source files for the tool
+    - **config.txt**: config the paths that will be used by this tool
+    - **data.json**: a json file specifies the inputs and ouputs of the mlir functions
+    - **automate.py**: the script to automatically generate result milr files in a folder
+    - **mlir-file-modify.py**: the main script to add input variable, main function, and FileCheck flags
+    - **parser.py**: the script to parse config.txt
+    - **verification-cpu.py**: the script for validating the output mlir file on CPU
+    - **verification-gpu.py**: the script for validating the output mlir file on GPU
+
+- recording the scripts for dealing the PlaidML test cases
+    - **recording/extract-mlir-plaidML.py**: record the script to extract plaidML test cases
+    - **recording/LastTest.log**: record the input and output trace of plaidML gtest log
+    - **recording/test-env.cc**: record the changes to plaidML to print input and output logs
+
+- sampe script to generate data.json from input and output data log trace like in **LastTest.log** 
+    - **sample-scripts/json-data-gen.py**: a sample script to generate data.json from plaidML c++ source file
+    - **sample-scripts/json-gen-from-gtest-log.py**: a sample script to generate data.json from plaidML plaidML gtest log file
+
+## How to Run it
+
+- before you run
+    - change the file path in config.txt according to your environment
+- test case generation
 
-Lowering command:
-```
-mlir-opt -convert-tensor-to-linalg -linalg-bufferize -arith-bufferize  -tensor-bufferize -func-bufferize -buffer-deallocation -convert-linalg-to-loops -convert-scf-to-cf  -convert-linalg-to-llvm -lower-affine -convert-scf-to-cf --convert-memref-to-llvm -convert-func-to-llvm -reconcile-unrealized-casts --arith-expand --convert-arith-to-llvm --convert-math-to-llvm -finalizing-bufferize
-```
-Verify command:
 ```
-mlir-cpu-runner -e main -entry-point-result=void    -shared-libs=/path/to/libmlir_c_runner_utils.so,/path/to/libmlir_runner_utils.so
+python automate.py
 ```
-### Files
 
-- **IREE-swap.py**: swaps IREE util.global, util.address, and util.load operations to arith constant operation.
+- verification
 
-- **makefile**: records the commands and flags to reproduce the lowering and verification processes. 
+```
+python verification-cpu.py
+python verification-gpu.py
+```