more complete pimcomp comparison scripts
Validate Operations / validate-operations (push) Waiting to run
Validate Operations / validate-operations (push) Waiting to run
update pimsim-nn submodule
This commit is contained in:
+19
-15
@@ -63,22 +63,24 @@ that directory while retaining the terminal table.
|
||||
|
||||
## Raptor vs PIMCOMP comparison
|
||||
|
||||
The PIMCOMP paper-model suite has a one-command Arch-A comparison:
|
||||
The PIMCOMP paper-model suite has a one-command multi-architecture comparison:
|
||||
|
||||
```bash
|
||||
.venv/bin/python validation/tools/pimcomp/run_pimcomp_paper_latency.py
|
||||
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py
|
||||
```
|
||||
|
||||
The runner verifies PIMCOMP's population-200, 1000-iteration GA settings,
|
||||
builds Raptor and the existing `third_party/PIMCOMP-NN/build` tree, and compares
|
||||
the four paper models one at a time. Each PIMCOMP GA run evaluates candidates
|
||||
in parallel; set `OMP_NUM_THREADS` to control its worker count. Use
|
||||
the four paper models in parallel. Set `--jobs` to control comparison workers;
|
||||
the runner leaves `OMP_NUM_THREADS` at its environment default. Use
|
||||
`--models vgg8` for one model or `--dry-run` to print the commands.
|
||||
|
||||
Generated artifacts are stored beside each model and ignored by Git. The
|
||||
comparison reuses the same model-level `inputs/`, `outputs/`, `runner/`,
|
||||
`raptor/`, and `simulation/` paths as regular validation. PIMCOMP-only
|
||||
artifacts and `comparison_report.{md,json}` live under `pimcomp/`. See
|
||||
comparison reuses model-level `common/inputs/`, `common/outputs/`,
|
||||
`common/runner/` across architectures, modes, and pipelines. Pimsim configs and
|
||||
network meshes are referenced directly from
|
||||
`validation/pimsim_configs/pimcomp/`. Architecture- and pipeline-specific `raptor/`, `simulation/`, and
|
||||
PIMCOMP artifacts stay under each comparison directory. See
|
||||
[`networks/pimcomp_models/README.md`](networks/pimcomp_models/README.md) for
|
||||
profiles, model provenance, limitations, and remote execution.
|
||||
|
||||
@@ -164,7 +166,9 @@ The selected PIMCOMP profile must match `--core-count`, `--crossbar-count`, and
|
||||
prints the incompatible values; functional validation still runs.
|
||||
|
||||
The checked-in profiles are under
|
||||
`validation/pimsim_configs/pimcomp/<profile>/latency_config.json`.
|
||||
`validation/pimsim_configs/pimcomp/<profile>/`; latency uses
|
||||
`latency_config.json`, while throughput uses
|
||||
`throughput_config_<time>ms.json` and the mesh beside it.
|
||||
|
||||
Use `--skip-non-functional-simulation` when latency, power, and energy are not required.
|
||||
The summary reports non-functional results as measured, failed, unsupported, or
|
||||
@@ -185,14 +189,14 @@ Artifacts are written beside each model:
|
||||
|
||||
| Path | Contents |
|
||||
|---|---|
|
||||
| `inputs/` | Generated input CSV files. |
|
||||
| `outputs/` | ONNX-MLIR reference output CSV files. |
|
||||
| `raptor/` | Exported MLIR, dialect snapshots, reports, final FP32 `pim/` artifacts, and the int8-equivalent `pimsim_nn/` latency view. |
|
||||
| `runner/` | Generated reference runner source, build tree, and shared library. |
|
||||
| `simulation/` | Functional simulator outputs used for numerical comparison. |
|
||||
| `pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
|
||||
| `common/inputs/` | Shared generated input CSV files. |
|
||||
| `common/outputs/` | Shared ONNX-MLIR reference output CSV files. |
|
||||
| `common/runner/` | Shared reference runner source, build tree, and library. |
|
||||
| `<arch>/<mode>[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and PIM artifacts. |
|
||||
| `<arch>/<mode>[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. |
|
||||
| `<arch>/<mode>[/pipelineN]/pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
|
||||
|
||||
The `raptor/` directory may include `spatial0.mlir`,
|
||||
Each comparison's `raptor/` directory may include `spatial0.mlir`,
|
||||
`spatial1_graph.mlir`, `spatial2_trivial_merged.mlir`,
|
||||
`spatial3_scheduled_no_comm.mlir`, `spatial4_scheduled.mlir`, `pim0.mlir`,
|
||||
`pim1_buff.mlir`, `pim2_folded.mlir`, and `pim3_memory_planned.mlir`.
|
||||
|
||||
Reference in New Issue
Block a user