# PIMCOMP paper models

This directory contains the four networks evaluated in
[PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159):
VGG-8, ResNet-18, ResNet-34, and GoogLeNet.

See [RESULTS.md](RESULTS.md) for the current latency-only result status.

## Models and provenance

| Directory | Model | Input | Provenance |
| --- | --- | --- | --- |
| `resnet18/` | ResNet-18 v1 | `1x3x224x224` | Symlink to the complete ONNX Model Zoo model already present at `../resnetv2/depth_68/resnetv2_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as PIMCOMP's frontend does. |
| `googlenet/` | GoogLeNet | `1x3x224x224` | Unmodified [ONNX Model Zoo `googlenet-12`](https://huggingface.co/onnxmodelzoo/googlenet-12). |
| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Deterministic compiler workload with six convolution and two fully connected layers. |

`googlenet/googlenet-12-no-softmax.onnx` is a derived latency model that
exposes the original model's final FC logits (`loss3/classifier_1`) as its
output. This matches PIMCOMP's instruction stream, which records but does not
schedule the terminal `OP_SOFTMAX`. Keep `googlenet-12.onnx` for full-model
functional validation.

The PIMCOMP authors did not publish the ONNX checkpoints used by the paper.
Running PIMCOMP's frontend on the three Model Zoo files above produces JSON
graphs exactly equal to PIMCOMP-NN's bundled `resnet18.json`, `resnet34.json`,
and `googlenet.json`.

There is no VGG-8 artifact in the ONNX Model Zoo or any PIMCOMP-NN revision.
The included VGG-8 therefore has deterministic random weights and is suitable
for compiler and simulator comparison, not paper-accuracy reproduction. The
paper also says that VGG-8 and ResNet-18 were trained on MNIST, while the
published PIMCOMP graphs and ResNet Model Zoo artifacts use ImageNet shapes.

Current SHA-256 checksums:

```text
788088b908e233d924c7c26b997e89ee861290c7bc56783a306e8201d79aac8f  resnet18/resnet18-v1-7.onnx
c3231061d081bdd47884137b02134f85142752a39e87263c529cd14ed242b096  resnet34/resnet34-v1-7.onnx
c99c507058eaf41de8723408fdda7db8325cb57f0a89f2ee07a716d6e963e14e  googlenet/googlenet-12.onnx
a35bad96441efbee28699cb61d1656cca7f7281f14040cf01699c3d0cfd8b202  googlenet/googlenet-12-no-softmax.onnx
396cdea21e5e7d02c3f26f14d22ef20975171702493f5c5e79b8e0d896e541ef  vgg8/vgg8-mnist-reconstructed.onnx
```

## Paper hardware profiles

The files in `configs/` encode Table V's explicit resource parameters.

| Config | Cores | Crossbars/core | Crossbar | Cell | PIMCOMP layout |
| --- | ---: | ---: | --- | ---: | --- |
| `arch-a.json` | 168 | 96 | `128x128` | 2-bit | `12x14` |
| `arch-b.json` | 138 | 128 | `128x128` | 2-bit | `6x23` |
| `arch-c.json` | 64 (16 chips x 4) | 8 | `512x1024` | 2-bit | flattened `8x8` |

`adc_count` is 16, matching the paper's 16-bit fixed-point weight precision.
The paper does not give a two-dimensional core topology for Arch-A/B, so the
factorizations above preserve core count but cannot reproduce unpublished NoC
placement details. Released PIMCOMP-NN has no chip-count field; Arch-C is
therefore flattened to 64 cores and does not model chip boundaries.

The remaining latency and power values come from PIMCOMP-NN's released default
configuration. Consequently, instruction/resource comparisons are
reproducible, but absolute paper power and energy numbers are not.

## Build and validate the ONNX files

From the Raptor repository root:

```bash
.venv/bin/python -m pip install numpy onnx onnxruntime onnxsim colorama
cmake --build ./build_release
cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN

.venv/bin/python -c \
  'from pathlib import Path; import onnx; [onnx.checker.check_model(onnx.load(p)) for p in Path("validation/networks/pimcomp_models").glob("*/*.onnx")]'
```

Do not build either project with `ninja` directly.

## Compile with PIMCOMP

PIMCOMP-NN reads `third_party/PIMCOMP-NN/config.json` directly. Back it up,
select one paper profile, and restore it when the shell exits:

```bash
RAPTOR_ROOT=$PWD
PIMCOMP="$RAPTOR_ROOT/third_party/PIMCOMP-NN"
PAPER_MODELS="$RAPTOR_ROOT/validation/networks/pimcomp_models"
CONFIG_BACKUP=$(mktemp)
cp "$PIMCOMP/config.json" "$CONFIG_BACKUP"
trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT
cp "$PAPER_MODELS/configs/arch-a.json" "$PIMCOMP/config.json"
```

The Model Zoo files map exactly to PIMCOMP's bundled model names, so compile
them directly:

```bash
cd "$PIMCOMP/build"

# High-throughput mode; the paper evaluates batches of 128 samples.
./PIMCOMP-NN -m=resnet18 -r=balance -p=batch -o=YES -v=YES -s=YES
./PIMCOMP-NN -m=resnet34 -r=balance -p=batch -o=YES -v=YES -s=YES
./PIMCOMP-NN -m=googlenet -r=balance -p=batch -o=YES -v=YES -s=YES

# Low-latency mode; the paper uses batch size 1.
./PIMCOMP-NN -m=resnet18 -r=balance -p=element -o=YES -v=YES -s=YES
./PIMCOMP-NN -m=resnet34 -r=balance -p=element -o=YES -v=YES -s=YES
./PIMCOMP-NN -m=googlenet -r=balance -p=element -o=YES -v=YES -s=YES
```

VGG-8 first needs PIMCOMP's JSON frontend. Use a temporary ONNX copy because
the released frontend rewrites the input batch dimension in place:

```bash
cd /path/to/Raptor
cp validation/networks/pimcomp_models/vgg8/vgg8-mnist-reconstructed.onnx /tmp/vgg8-pimcomp.onnx
.venv/bin/python third_party/PIMCOMP-NN/frontend/frontend.py \
  --model_path /tmp/vgg8-pimcomp.onnx \
  --save_path third_party/PIMCOMP-NN/models/JSON/vgg8_paper_reconstructed.json

cd third_party/PIMCOMP-NN/build
./PIMCOMP-NN -m=vgg8_paper_reconstructed -r=balance -p=batch -o=YES -v=YES -s=YES
./PIMCOMP-NN -m=vgg8_paper_reconstructed -r=balance -p=element -o=YES -v=YES -s=YES
```

Repeat after selecting `arch-b.json` and `arch-c.json`. All four models were
compiled successfully in both modes with all three configs. The released
random placement code occasionally segfaults; an unchanged retry succeeded in
the observed cases.

The paper's optimizer uses a genetic algorithm with population 200 and up to
1000 iterations. Select it with `-r=GA` for optimizer studies. The released
source keeps population 200 but sets `max_iteration = 3`, so reproducing the
paper's optimization search also requires changing that value in
`backend/GeneticAlgorithm.h`. GA allocates roughly 32 GB in its fast evaluator;
use monolith below instead of reducing cores or crossbars when local RAM is
insufficient.

## Compare Raptor and PIMCOMP

The comparison driver uses one random input and one native ONNX-MLIR reference,
compiles both instruction streams, validates both through Raptor's Rust
simulator, and writes Markdown and JSON reports.

To reproduce the complete Arch-A latency experiment, use the serial experiment
runner. It creates an isolated PIMCOMP build with population 200 and 1000 GA
iterations, runs only the `element`/batch-1 latency pipeline, and invokes the
comparison driver for one model at a time:

```bash
.venv/bin/python validation/tools/run_pimcomp_paper_latency.py \
  --out-dir /tmp/raptor-pimcomp-paper-latency
```

Reports are written under `<out-dir>/<model>/comparison_report.{md,json}`.
Use `--models vgg8` to run one model, `--resume` after an interruption, or
`--dry-run` to inspect every command. The runner continues after a failed model
so all reports are produced, then returns a nonzero status if any comparison
failed.

Arch-A low-latency example:

```bash
RAPTOR_ROOT=$PWD
PIMCOMP="$RAPTOR_ROOT/third_party/PIMCOMP-NN"
CONFIG_BACKUP=$(mktemp)
cp "$PIMCOMP/config.json" "$CONFIG_BACKUP"
trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT
cp "$RAPTOR_ROOT/validation/networks/pimcomp_models/configs/arch-a.json" "$PIMCOMP/config.json"

"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/compare_raptor_pimcomp.py" \
  --model "$RAPTOR_ROOT/validation/networks/pimcomp_models/vgg8/vgg8-mnist-reconstructed.onnx" \
  --out-dir /tmp/compare-vgg8-arch-a-ll \
  --core-count 168 \
  --crossbar-count 96 \
  --crossbar-size 128 \
  --mesh-rows 12 \
  --mesh-cols 14 \
  --pimsim-mode latency \
  --pimcomp-pipeline element \
  --fail-on-error
```

For Arch-A high throughput, use `--pimsim-mode throughput
--pimcomp-pipeline batch`. For Arch-B, use 138 cores, 128 crossbars, a
`6x23` mesh, and `configs/arch-b.json`.

If only semantic and instruction comparison is required, add
`--skip-pimsim-nn`. This exact VGG-8 Arch-A LL smoke test passed both semantic
validations with maximum output differences below `5e-10`.

Current Raptor status:

- VGG-8, ResNet-18, fixed-batch ResNet-34, and GoogLeNet compile on Arch-A.
- Use `googlenet-12-no-softmax.onnx` for the paper-matched latency comparison.
  The original model's final `vsoftmax` is supported by Raptor's functional
  simulator but not by `pimsim-nn`; PIMCOMP does not schedule that operation.
- Raptor currently accepts one square `--crossbar-size`; Arch-C's rectangular
  `512x1024` arrays can therefore be compiled by PIMCOMP but not compared
  exactly with Raptor.

Do not change the hardware profile to bypass either limitation; that would no
longer be a paper-matched comparison.

## Monolith fallback

The local `monolith` SSH alias points to the high-memory host. Copy only this
suite and the comparison driver; `-L` materializes the ResNet-18 symlink because
the canonical `resnetv2/depth_68` file may not exist remotely:

```bash
REMOTE_REPO=/home/gmagnani/Project/Raptor
rsync -azL validation/networks/pimcomp_models/ \
  "monolith:$REMOTE_REPO/validation/networks/pimcomp_models/"
rsync -az validation/tools/compare_raptor_pimcomp.py \
  "monolith:$REMOTE_REPO/validation/tools/compare_raptor_pimcomp.py"
rsync -az validation/tools/run_pimcomp_paper_latency.py \
  "monolith:$REMOTE_REPO/validation/tools/run_pimcomp_paper_latency.py"
rsync -az --exclude=.git --exclude=build --exclude=output \
  third_party/PIMCOMP-NN/ \
  "monolith:$REMOTE_REPO/third_party/PIMCOMP-NN/"
```

Then use the same commands over SSH:

```bash
ssh monolith
cd /home/gmagnani/Project/Raptor

# One-time setup if the repository virtual environment is absent.
python3 -m venv .venv
.venv/bin/python -m pip install numpy onnx onnxruntime onnxsim colorama

# Run every latency comparison serially.
.venv/bin/python validation/tools/run_pimcomp_paper_latency.py \
  --out-dir /tmp/raptor-pimcomp-paper-latency
```

Copy reports back without transferring large compiler artifacts:

```bash
rsync -az --include='*/' --include='comparison_report.*' --exclude='*' \
  monolith:/tmp/raptor-pimcomp-paper-latency/ \
  /tmp/raptor-pimcomp-paper-latency/
```