# Pimcomp comparison models

Models for comparing Raptor with
[Pimcomp](https://arxiv.org/pdf/2411.09159): the paper's four networks plus YOLO11n.
Use the [comparison runner](../../tools/pim/pimcomp/compare/README.md) for commands,
options, artifact locations, and plots.
The [reproduction reference](../../../docs/evaluation/pimcomp-reproducibility.md) records exact supported/ignored
operations, standalone commands, configuration layout, and GA/OpenMP controls.

## Models and provenance

| Directory | Model / input | Source |
|---|---|---|
| `resnet18/` | ResNet-18 v1, `1x3x224x224` | [ONNX Model Zoo resnet18-v1-7](https://huggingface.co/onnxmodelzoo/resnet18-v1-7); symlink to `../../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1, `1x3x224x224` | [ONNX Model Zoo resnet34-v1-7](https://huggingface.co/onnxmodelzoo/resnet34-v1-7); symbolic batch fixed to 1. |
| `googlenet/` | GoogLeNet, `1x3x224x224` | [ONNX Model Zoo googlenet-12](https://huggingface.co/onnxmodelzoo/googlenet-12), with only its two LRN nodes bypassed. |
| `vgg8/` | VGG-8 reconstruction, `1x1x28x28` | Six convolution and two fully connected layers with deterministic random weights. |
| `yolo11n/` | YOLO11n detection, `1x3x640x640` | [Ultralytics YOLO11n](https://docs.ultralytics.com/models/yolo11/); symlink to `../../yolo11n/depth_51/yolo11n_depth_51.onnx`. |

The paper's checkpoints were not published. The three Model Zoo networks
produce frontend graphs matching Pimcomp's bundled JSON graphs. VGG-8 is
suitable for compiler/simulator comparison, not paper-accuracy reproduction.
The paper's MNIST description for ResNet-18 differs from the released graph's
ImageNet input shape.

The current GoogLeNet comparison uses `googlenet/googlenet-12-no-lrn.onnx`.
Only its two LRN nodes are bypassed because Raptor cannot lower LRN to PIM.
Its terminal Softmax, output `prob_1`, weights, and other operations are retained.
The native reference uses this same graph. VGG8 and both ResNet graphs have no
removed operators; ResNet-34 fixes the symbolic batch size to one.

The current YOLO comparison uses `yolo11n/yolo11n.onnx`,
including both Softmax operations. Its native reference uses the same full model.
The checked-in ONNX graph uses opset 18. The original exporter version and checkpoint hash are not recorded.
The functional simulator executes Softmax. Pimsim-nn skips `vsoftmax` before
fetch, and Raptor's PIM queue estimator omits the same instruction cost.
Timing and energy exclude Softmax work; they do not model hardware Softmax support.
Comparison artifacts remain under this suite's `yolo11n/artifacts/` directory.

Model SHA-256 checksums:

```text
788088b908e233d924c7c26b997e89ee861290c7bc56783a306e8201d79aac8f  resnet18/resnet18-v1-7.onnx
c3231061d081bdd47884137b02134f85142752a39e87263c529cd14ed242b096  resnet34/resnet34-v1-7.onnx
c99c507058eaf41de8723408fdda7db8325cb57f0a89f2ee07a716d6e963e14e  googlenet/googlenet-12.onnx
c960c195953423388733cd0e60712cf425ca1f8917e264f5cf98e028752c61cf  googlenet/googlenet-12-no-lrn.onnx
396cdea21e5e7d02c3f26f14d22ef20975171702493f5c5e79b8e0d896e541ef  vgg8/vgg8-mnist-reconstructed.onnx
5e7bd85dc60b252674f8d7e1ad24f2317d00dfb8c524826536698462de04620d  yolo11n/yolo11n.onnx
```

## Hardware profiles

Canonical configs and meshes are in
[`validation/pimsim_configs/pimcomp/`](../../pimsim_configs/pimcomp).
They encode Table V's resources:

| Profile | Cores | Crossbars/core | Crossbar | Mesh |
|---|---:|---:|---|---|
| Arch-A | 168 | 96 | `128x128` | `12x14` |
| Arch-B | 138 | 128 | `128x128` | `6x23` |
| Arch-C | 64 (16 chips × 4) | 8 | `512x1024` | Flattened `8x8` |

All profiles use 2-bit cells and 16 ADCs for 16-bit fixed-point weights.
Arch-A/B topology factorizations preserve core counts; the paper does not
publish placement details. Arch-C does not model chip boundaries, and Raptor's
square-crossbar option cannot match its rectangular arrays. Do not change
hardware resources to bypass this limitation.

Other timing/power parameters use Pimcomp's released defaults. Resource and
instruction comparisons are reproducible; absolute paper power/energy values
are not.

## Build and run

From the repository root, after the [Raptor build](../../../README.md#build):

```bash
.venv/bin/python -m pip install -r requirements.txt
cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
```

The runner expects existing Pimcomp and Pimsim builds, verifies the paper's GA
settings (population 200, up to 1000 iterations), and uses isolated frontend
workspaces without changing the source models or shared Pimcomp configuration.
Default aggregate results are written beside the
[comparison runner](../../tools/pim/pimcomp/compare/results_comparison.csv);
explicit custom output roots retain their own aggregate CSVs.

## Correctness and precision limits

Pimcomp does not support YOLO11n's `Split` and `Resize`; no Pimcomp performance
number is reported for that model. Accepted frontend operations can also be
ignored by scheduling; see the authoritative
[frontend](../../../third_party/PIMCOMP-NN/frontend/frontend.py) and
[latency scheduler](../../../third_party/PIMCOMP-NN/backend/ElementPipelineSchedule.cpp).
Known [batch-scheduling and BatchNorm gaps](../../tools/pim/pimcomp/correctness/README.md)
must remain visible as failed functional validation.

Functional validation uses FP32 programs in the Rust simulator against native
ONNX-MLIR. Pimcomp's Rust export expands byte-sized storage/transfers to FP32.
For performance comparison, Pimcomp's original `SimulationInfo.gz` remains
unchanged; a separate Raptor `raptor/pimsim_nn/` view scales FP32 storage and
transfers to one byte per element with `setbw 8, 8`. These performance artifacts
are not numerically quantized programs and must not be used for correctness.

Vector `imm_len` is an element count. Addresses and transfer sizes for
`ld`, `st`, `lldi`, `lmv`, `send`, and `recv` are byte-based.