finally fast googlenet with correct latency artifacts for fair comparison
Validate Operations / validate-operations (push) Has been cancelled

This commit is contained in:
NiccoloN
2026-07-29 18:20:44 +02:00
parent 060a21172e
commit 1b4f070bef
74 changed files with 2773 additions and 1311 deletions
+7 -7
View File
@@ -8,7 +8,7 @@ model it can:
3. compile PIM artifacts with Raptor;
4. run the reference implementation and functional PIM simulator;
5. compare their outputs;
6. run `pimsim-nn` to report latency and power.
6. run `pimsim-nn` to report latency, power, and energy.
Run the script from the repository root with the repository Python environment.
@@ -65,7 +65,7 @@ The script discovers them recursively.
The PIMCOMP paper-model suite has a one-command Arch-A comparison:
```bash
.venv/bin/python validation/tools/run_pimcomp_paper_latency.py
.venv/bin/python validation/tools/pimcomp/run_pimcomp_paper_latency.py
```
The runner verifies PIMCOMP's population-200, 1000-iteration GA settings,
@@ -134,7 +134,7 @@ count with `-j` or `--jobs`:
| `--simulator-dir PATH` | Functional `pim-simulator` crate directory. Defaults to the in-tree simulator. |
| `--non-functional-simulator-build-dir PATH` | `pimsim-nn` build directory. Defaults to the in-tree build. |
| `--pimcomp-config {arch-a,arch-b,arch-c}` | Non-functional hardware/timing profile. Defaults to `arch-a`. |
| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency and power measurement. |
| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency, power, and energy measurement. |
| `--threshold FLOAT` | Absolute output-comparison tolerance. Defaults to `1e-3`. |
| `--relative-threshold FLOAT` | Relative output-comparison tolerance. Defaults to `1e-5`. |
| `--seed INT` | Seed for generated inputs. Defaults to `0`. |
@@ -165,17 +165,17 @@ prints the incompatible values; functional validation still runs.
The checked-in profiles are under
`validation/pimsim_configs/pimcomp/<profile>/latency_config.json`.
Use `--skip-non-functional-simulation` when latency and power are not required.
Use `--skip-non-functional-simulation` when latency, power, and energy are not required.
The summary reports non-functional results as measured, failed, unsupported, or
skipped.
Overall PASS/FAIL is determined by compilation and functional output
comparison. A non-functional simulation failure remains visible as `ERROR` in
the latency and power columns but does not change a functional PASS.
the latency, power, and energy columns but does not change a functional PASS.
`pimsim-nn` does not currently implement the `vsoftmax` instruction. When its
explicit unsupported-op diagnostic is encountered, Softmax validations retain
their functional PASS and show `UNSUPPORTED` in both non-functional columns.
their functional PASS and show `UNSUPPORTED` in the non-functional columns.
Other `pimsim-nn` failures remain `ERROR`.
## Generated artifacts
@@ -186,7 +186,7 @@ Artifacts are written beside each model:
|---|---|
| `inputs/` | Generated input CSV files. |
| `outputs/` | ONNX-MLIR reference output CSV files. |
| `raptor/` | Exported MLIR, dialect snapshots, reports, and final `pim/` artifacts. |
| `raptor/` | Exported MLIR, dialect snapshots, reports, final FP32 `pim/` artifacts, and the int8-equivalent `pimsim_nn/` latency view. |
| `runner/` | Generated reference runner source, build tree, and shared library. |
| `simulation/` | Functional simulator outputs used for numerical comparison. |
| `pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |