# Pimcomp model comparison

These scripts compare Raptor with Pimcomp on the models in
`validation/networks/pimcomp_models`.

`run_pimcomp_models.py` runs the supported model suite across the configured
architectures and simulation modes. `compare_raptor_pimcomp_model.py` is the
lower-level one-model comparison used by the suite runner.

## Suite runner

Run all five models, Arch-A and Arch-B, latency, and throughput:

```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
```

### Options

| Option | Description and default |
|---|---|
| `-h`, `--help` | Show help and exit. |
| `--out-dir PATH` | Suite root; artifacts are placed below `<out-dir>/<model>/artifacts/`. Default: `validation/networks/pimcomp_models`. |
| `--common-dir PATH` | Shared root for per-model reference artifacts. Default: `<out-dir>/<model>/artifacts/common`. |
| `--models MODEL [...]` | Models to run: `vgg8`, `resnet18`, `resnet34`, `googlenet`, `yolo11n`. Default: all five; pass a subset to select specific models. |
| `--archs ARCH [...]` | Pimcomp hardware profiles to run. Default: `arch-a arch-b`. |
| `--mode {latency,throughput} [...]` | Simulation modes. Default: both. |
| `--only {raptor,pimcomp}` | Re-run only one compiler side and reuse the other side's existing report. Default: run both sides. |
| `--raptor-only` | Run Raptor without compiling, validating, or simulating Pimcomp. Default: off. |
| `--pipeline {1,2,4,8}` | Select one Raptor pipeline. Default: latency pipeline 1 and throughput pipelines 2, 4, and 8. |
| `--pimsim-time-ms INT` | Throughput convergence deadline. Default: `1000`. |
| `--batch-size INT` | Functional throughput batch size. Default: `128`. |
| `--timeout-seconds FLOAT` | Per-stage timeout; `0` means unlimited. Default: `0`. |
| `-j INT`, `--jobs INT` | Parallel comparison workers. Default: `4`. |
| `--clean` | Remove generated comparison artifacts and summaries, then exit. Default: off. |
| `--ablation-variant NAME` | Put generated artifacts below `<mode>[/pipelineN]/ablation/NAME` and write the summary below `ablation/NAME`. Default: none. |
| `--dry-run` | Print commands without modifying files. Default: off. |
| `--no-fast` | Disable fast throughput convergence. Default: off. |
| `--raptor-extra-arg=ARG` | Extra Raptor compiler argument; repeat for multiple arguments. Default: none. |

Arguments beginning with `--` must use the equals form when passed through:

```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py \
  --models vgg8 \
  --raptor-extra-arg=--pim-disable-synchronization
```

The runner writes `results_comparison.csv` under the selected result root. It reuses
shared model inputs, outputs, and reference runners, and reuses Pimcomp
artifacts between pipelines in the same model/architecture/mode group. With
`--raptor-only`, only Raptor results are generated and Pimcomp is not invoked.
All generated files are kept below each model's `artifacts/` directory;
`--clean` also removes any `.lock` files left by an interrupted reference
generation.

## One-model comparator

`compare_raptor_pimcomp_model.py` compares one ONNX model. Its required
arguments are `--model PATH` and `--out-dir PATH`; use `--help` for the full
lower-level interface. The suite runner supplies the model, hardware profile,
simulation mode, and reuse paths automatically.