update readme
This commit is contained in:
NiccoloN
@@ -154,108 +154,10 @@ This writes PIM artifacts under `/tmp/raptor/pim/`.
|
||||
|
||||
## Validation
|
||||
|
||||
Functional validation lives in `validation/`. It compiles ONNX models, builds a
|
||||
native ONNX-MLIR reference runner, generates random inputs, runs Raptor, runs
|
||||
the Rust functional simulator, and compares outputs.
|
||||
|
||||
Python dependencies used by the validation scripts are `numpy`, `onnx`, and
|
||||
`colorama`. The simulator requires the Rust toolchain.
|
||||
|
||||
Per-operation validation from the repository root:
|
||||
|
||||
```bash
|
||||
.venv/bin/python validation/validate.py \
|
||||
--raptor-path build_release/Release/bin/onnx-mlir \
|
||||
--onnx-include-dir onnx-mlir/include \
|
||||
--operations-dir validation/operations \
|
||||
--verbose \
|
||||
--raptor-extra-arg=--pim-detect-communication-deadlock \
|
||||
--raptor-extra-arg=--pim-export-spatial-dataflow=none
|
||||
```
|
||||
|
||||
Validate one network or a subset by pointing `--operations-dir` at any directory
|
||||
containing `.onnx` files:
|
||||
|
||||
```bash
|
||||
.venv/bin/python validation/validate.py \
|
||||
--raptor-path build_release/Release/bin/onnx-mlir \
|
||||
--onnx-include-dir onnx-mlir/include \
|
||||
--operations-dir validation/networks/yolo11n/depth_04 \
|
||||
--verbose \
|
||||
--raptor-extra-arg=--pim-detect-communication-deadlock \
|
||||
--raptor-extra-arg=--pim-export-spatial-dataflow=none
|
||||
```
|
||||
|
||||
Useful validation options:
|
||||
- `--simulator-dir <path>` - override the auto-detected functional
|
||||
`backend-simulators/pim/pim-simulator` path.
|
||||
- `--pimcomp-config <arch-a|arch-b|arch-c>` - select the checked-in
|
||||
non-functional simulation profile. Arch-A is the default; the hardware
|
||||
argument defaults match it (`168` cores, `96` crossbars/core, `128x128`
|
||||
crossbars).
|
||||
Selecting another profile also requires its matching hardware arguments.
|
||||
- `--non-functional-simulator-build-dir <path>` - override the auto-detected
|
||||
non-functional simulator build directory.
|
||||
- `--skip-non-functional-simulation` - skip non-functional simulation. If
|
||||
explicit core/crossbar arguments do not match the selected profile, validation
|
||||
reports the mismatch and skips only non-functional simulation.
|
||||
- `--threshold <float>` - maximum allowed per-element output difference.
|
||||
- `--seed <int>` - RNG seed for generated inputs.
|
||||
- `--command-timeout-seconds <float>` - timeout for compiler, runner, and
|
||||
simulator subprocesses.
|
||||
- `--verbose` - print subprocess logs and average PIM pass timings.
|
||||
- `--clean` - remove generated validation artifacts and exit.
|
||||
|
||||
Each validation run writes artifacts in the model workspace, for example under
|
||||
`validation/operations/gemm/small/`:
|
||||
- `inputs/` - generated input CSV files.
|
||||
- `outputs/` - native ONNX-MLIR reference outputs.
|
||||
- `raptor/` - compiler artifacts, including `*.onnx.mlir`, dialect dumps under
|
||||
`dialects/`, reports under `reports/`, and final PIM artifacts under `pim/`.
|
||||
- `runner/` - generated reference runner source, build tree, and shared library.
|
||||
- `simulation/out.bin` - raw functional simulation output used for comparison.
|
||||
|
||||
By default, validation also runs non-functional simulation. Per-model `Latency`
|
||||
is shown in the result table alongside `Power`, with measured, failed, and
|
||||
skipped counts plus total latency in the summary. Use these validation results
|
||||
for performance comparisons; the simulation backend does not need to be invoked
|
||||
separately. The selected pre-generated configs and meshes live under
|
||||
`validation/pimsim_configs/pimcomp/`.
|
||||
|
||||
The compiler currently dumps dialect snapshots such as `spatial0.mlir`,
|
||||
`spatial1_graph.mlir`, `spatial2_trivial_merged.mlir`,
|
||||
`spatial3_scheduled_no_comm.mlir`, `spatial4_scheduled.mlir`, `pim0.mlir`, `pim1_buff.mlir`,
|
||||
`pim2_folded.mlir`, and `pim3_memory_planned.mlir` when an output directory is
|
||||
available.
|
||||
|
||||
To rerun the functional simulator manually with tracing after validation has
|
||||
produced a `raptor/pim/` directory:
|
||||
|
||||
```bash
|
||||
cd backend-simulators/pim/pim-simulator
|
||||
cargo run --no-default-features --features tracing --release \
|
||||
--package pim-simulator --bin pim-simulator -- \
|
||||
-f /path/to/workspace/raptor/pim \
|
||||
-o /path/to/workspace/simulation/out.bin \
|
||||
-d <addr0>,<size0>,<addr1>,<size1>,...
|
||||
```
|
||||
|
||||
With `--features tracing`, the simulator writes per-core traces as
|
||||
`TraceCore0`, `TraceCore1`, ... next to `out.bin`. The validator normally
|
||||
computes the `-d` ranges from `raptor/pim/config.json` and model output shapes.
|
||||
|
||||
Available validation networks under `validation/networks/`: `vgg16`,
|
||||
`yolo11n`, `yolo11nv2`.
|
||||
|
||||
Available operation suites under `validation/operations/`: `add`, `concat`,
|
||||
`conv`, `div`, `gather`, `gemm`, `gemv`, `matmul`, `mul`, `pool`,
|
||||
`reduce_mean`, `relu`, `reshape`, `resize`, `sigmoid`, `softmax`, `split`.
|
||||
|
||||
Generated operation tests can be regenerated with:
|
||||
|
||||
```bash
|
||||
.venv/bin/python validation/operations/gen_tests.py
|
||||
```
|
||||
Functional validation compiles ONNX models, compares native ONNX-MLIR and PIM
|
||||
simulator outputs, and optionally reports latency and power. See
|
||||
[`validation/README.md`](validation/README.md) for prerequisites, usage,
|
||||
options, artifacts, and results.
|
||||
|
||||
## Build
|
||||
|
||||
|
||||
Reference in New Issue
Block a user