more complete pimcomp comparison scripts
Validate Operations / validate-operations (push) Waiting to run

update pimsim-nn submodule
This commit is contained in:
NiccoloN
2026-08-06 21:49:54 +02:00
parent 4acd3b0c81
commit 2e76164aed
20 changed files with 1626 additions and 548 deletions
+19 -15
View File
@@ -63,22 +63,24 @@ that directory while retaining the terminal table.
## Raptor vs PIMCOMP comparison
The PIMCOMP paper-model suite has a one-command Arch-A comparison:
The PIMCOMP paper-model suite has a one-command multi-architecture comparison:
```bash
.venv/bin/python validation/tools/pimcomp/run_pimcomp_paper_latency.py
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py
```
The runner verifies PIMCOMP's population-200, 1000-iteration GA settings,
builds Raptor and the existing `third_party/PIMCOMP-NN/build` tree, and compares
the four paper models one at a time. Each PIMCOMP GA run evaluates candidates
in parallel; set `OMP_NUM_THREADS` to control its worker count. Use
the four paper models in parallel. Set `--jobs` to control comparison workers;
the runner leaves `OMP_NUM_THREADS` at its environment default. Use
`--models vgg8` for one model or `--dry-run` to print the commands.
Generated artifacts are stored beside each model and ignored by Git. The
comparison reuses the same model-level `inputs/`, `outputs/`, `runner/`,
`raptor/`, and `simulation/` paths as regular validation. PIMCOMP-only
artifacts and `comparison_report.{md,json}` live under `pimcomp/`. See
comparison reuses model-level `common/inputs/`, `common/outputs/`,
`common/runner/` across architectures, modes, and pipelines. Pimsim configs and
network meshes are referenced directly from
`validation/pimsim_configs/pimcomp/`. Architecture- and pipeline-specific `raptor/`, `simulation/`, and
PIMCOMP artifacts stay under each comparison directory. See
[`networks/pimcomp_models/README.md`](networks/pimcomp_models/README.md) for
profiles, model provenance, limitations, and remote execution.
@@ -164,7 +166,9 @@ The selected PIMCOMP profile must match `--core-count`, `--crossbar-count`, and
prints the incompatible values; functional validation still runs.
The checked-in profiles are under
`validation/pimsim_configs/pimcomp/<profile>/latency_config.json`.
`validation/pimsim_configs/pimcomp/<profile>/`; latency uses
`latency_config.json`, while throughput uses
`throughput_config_<time>ms.json` and the mesh beside it.
Use `--skip-non-functional-simulation` when latency, power, and energy are not required.
The summary reports non-functional results as measured, failed, unsupported, or
@@ -185,14 +189,14 @@ Artifacts are written beside each model:
| Path | Contents |
|---|---|
| `inputs/` | Generated input CSV files. |
| `outputs/` | ONNX-MLIR reference output CSV files. |
| `raptor/` | Exported MLIR, dialect snapshots, reports, final FP32 `pim/` artifacts, and the int8-equivalent `pimsim_nn/` latency view. |
| `runner/` | Generated reference runner source, build tree, and shared library. |
| `simulation/` | Functional simulator outputs used for numerical comparison. |
| `pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
| `common/inputs/` | Shared generated input CSV files. |
| `common/outputs/` | Shared ONNX-MLIR reference output CSV files. |
| `common/runner/` | Shared reference runner source, build tree, and library. |
| `<arch>/<mode>[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and PIM artifacts. |
| `<arch>/<mode>[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. |
| `<arch>/<mode>[/pipelineN]/pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
The `raptor/` directory may include `spatial0.mlir`,
Each comparison's `raptor/` directory may include `spatial0.mlir`,
`spatial1_graph.mlir`, `spatial2_trivial_merged.mlir`,
`spatial3_scheduled_no_comm.mlir`, `spatial4_scheduled.mlir`, `pim0.mlir`,
`pim1_buff.mlir`, `pim2_folded.mlir`, and `pim3_memory_planned.mlir`.