nnicolosi/Raptor
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

even faster on pimcomp models
Validate Operations / validate-operations (push) Has been cancelled
Details

Browse Source
This commit is contained in:
NiccoloN
2026-07-31 21:15:28 +02:00
parent 9ca1a0ed9f
commit f4a3b012cc
49 changed files with 1923 additions and 583 deletions
Download Patch File Download Diff File Expand all files Collapse all files
validation/networks/pimcomp_models/README.md
+80 -13
View File
@@ -1,26 +1,85 @@
# PIMCOMP paper models
# PIMCOMP comparison models
This directory contains the four networks evaluated in
[PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159):
VGG-8, ResNet-18, ResNet-34, and GoogLeNet.
VGG-8, ResNet-18, ResNet-34, and GoogLeNet. It also contains YOLO11n as an
additional compiler comparison model.
See the runner-generated [results.csv](results.csv) for the current latency
and energy results.
## Models and provenance
| Directory | Model | Input | Provenance |
|--------------|----------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `resnet18/` | ResNet-18 v1 | `1x3x224x224` | Symlink to the complete [ONNX Model Zoo `resnet18-v1-7`](https://huggingface.co/onnxmodelzoo/resnet18-v1-7) model already present at `../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as PIMCOMP's frontend does. |
| `googlenet/` | GoogLeNet | `1x3x224x224` | Unmodified [ONNX Model Zoo `googlenet-12`](https://huggingface.co/onnxmodelzoo/googlenet-12). |
| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [PIMCOMP VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. |
| Directory | Model | Input | Provenance |
|--------------|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `resnet18/` | ResNet-18 v1 | `1x3x224x224` | Symlink to the complete [ONNX Model Zoo `resnet18-v1-7`](https://huggingface.co/onnxmodelzoo/resnet18-v1-7) model already present at `../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as PIMCOMP's frontend does. |
| `googlenet/` | GoogLeNet | `1x3x224x224` | Unmodified [ONNX Model Zoo `googlenet-12`](https://huggingface.co/onnxmodelzoo/googlenet-12). |
| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [PIMCOMP VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. |
| `yolo11n/` | YOLO11n detection | `1x3x640x640` | Derived from the canonical local model at `../yolo11n/depth_51/yolo11n_depth_51.onnx`, exported from [Ultralytics YOLO11n](https://github.com/ultralytics/ultralytics/blob/main/docs/en/models/yolo11.md). |
`googlenet/googlenet-12-latency.onnx` is the explicit common latency model.
It exposes the original model's final FC logits (`loss3/classifier_1`) and
removes its two LRN nodes and terminal Softmax so the comparison covers only
operations scheduled by PIMCOMP. Keep `googlenet-12.onnx` as the unmodified
source model.
`googlenet/googlenet-12-latency.onnx` is the explicit pimsim-nn-ready GoogLeNet model.
It removes the two LRN nodes and terminal Softmax from the original model,
so that the comparison covers only operations scheduled by PIMCOMP and supported by pimsim-nn.
`yolo11n/yolo11n-latency.onnx` is the explicit pimsim-nn-ready YOLO11n model.
It removes the Softmax nodes from the original model,
so that the compiled artifact can be simulated in pimsim-nn.
## Unsupported and ignored operations
PIMCOMP's frontend accepts exactly these ONNX operations:
```text
Add, AveragePool, BatchNormalization, Clip, Concat, Conv, Dropout, Flatten,
Gather, Gemm, GlobalAveragePool, LRN, MatMul, MaxPool, Mul, Pad, Relu, Reshape,
Shape, Sigmoid, Softmax, Squeeze, Sub, Sum, Tanh, Transpose, Unsqueeze
```
`Constant` is consumed as frontend metadata rather than emitted as a PIMCOMP
node. Every other ONNX operation is unsupported: the frontend prints
`operation: <type> not considered` and stops at the first occurrence. Thus the
complete unsupported set is the complement of the allowlist above for the
model's ONNX opset. In particular, YOLO11n contains unsupported `Split` and
`Resize` nodes.
PIMCOMP's low-latency scheduler, hierarchy mapper, and genetic algorithm use
this complete explicit no-consider set:
```text
Input, BatchNormalization, Clip, Dropout, Flatten, LRN, MatMul, Reshape,
Softmax, Squeeze, Transpose
```
Those nodes do not receive scheduled latency instructions when they remain in
the backend graph. Before that point the frontend may fuse BatchNormalization
and activation nodes into Conv/Gemm, convert a
Reshape-Transpose-Reshape channel-shuffle pattern to `OP_SHUFFLE`, remove a
specific Shape-Gather-Unsqueeze-Concat shape chain, and merge Pad into its
consumer. These transformations do not make an otherwise standalone ignored
operation timed.
`pimsim-nn` consumes PIM ISA instructions. It supports every named opcode
in the shared serialized range except `vsoftmax` (opcode 21),
which is rejected explicitly in both JSON and binary input. It
silently ignores no opcode; unknown names and numbers are errors.
These boundaries explain the dedicated artifacts:
- GoogLeNet's two LRN nodes and terminal Softmax perform real computation but
are ignored by PIMCOMP, so the common latency artifact removes them. Its
inference Dropout and shape-only Reshape can remain without adding compute.
- YOLO11n's latency artifact bypasses exactly its two Softmax nodes so it can
run in `pimsim-nn`. Every other node, including MatMul, Transpose, and the
final detection-decoding tail, remains present and timed by Raptor. No
PIMCOMP latency is reported because its frontend stops at `Split` and also
lacks `Resize`; compiling that prefix would not represent YOLO11n.
The authoritative lists are in
[`frontend.py`](../../../third_party/PIMCOMP-NN/frontend/frontend.py),
[`ElementPipelineSchedule.cpp`](../../../third_party/PIMCOMP-NN/backend/ElementPipelineSchedule.cpp),
[`ISA.h`](../../../backend-simulators/pim/pimsim-nn/src/isa/ISA.h), and
[`Instruction.cpp`](../../../backend-simulators/pim/pimsim-nn/src/isa/Instruction.cpp).
The PIMCOMP authors did not publish the ONNX checkpoints used by the paper.
Running PIMCOMP's frontend on the three Model Zoo files above produces JSON
@@ -41,6 +100,7 @@ c3231061d081bdd47884137b02134f85142752a39e87263c529cd14ed242b096 resnet34/resne
c99c507058eaf41de8723408fdda7db8325cb57f0a89f2ee07a716d6e963e14e googlenet/googlenet-12.onnx
a26f9e33901c573e60c34a3f0abbb4744fff83e4e0f21b18fc66e20395e72982 googlenet/googlenet-12-latency.onnx
396cdea21e5e7d02c3f26f14d22ef20975171702493f5c5e79b8e0d896e541ef vgg8/vgg8-mnist-reconstructed.onnx
229f3975af8933d39aee8d9031d969bff074b69c33e304a78abb35ff0c5f445f yolo11n/yolo11n-latency.onnx
```
## Paper hardware profiles
@@ -190,6 +250,13 @@ RAPTOR_ROOT=$PWD
--pimcomp-pipeline element
```
Use the same command with
`yolo11n/yolo11n-latency.onnx` to probe YOLO11n. Released PIMCOMP-NN cannot
compile it: the frontend stops at `/model.2/Split`, and it also has no mapping
for YOLO11n's two nearest-neighbor `Resize` nodes. Treating the emitted prefix
as YOLO11n would produce a misleading latency, so no PIMCOMP number is
reported for this model.
For Arch-A high throughput, use `--pimsim-mode throughput
--pimcomp-pipeline batch`. For Arch-B, use 138 cores, 128 crossbars, a
`6x23` mesh, and
validation/networks/pimcomp_models/results.csv
+4 -4
View File
@@ -1,5 +1,5 @@
model,raptor_latency_ms,pimcomp_latency_ms,raptor_energy_pj,pimcomp_energy_pj,faster_compiler,speedup
vgg8,2.463417,7.985074,649524564.120001,1597904071.120000,raptor,3.24
resnet18,31.449811,58.853733,9335506856.119984,13983168748.119974,raptor,1.87
resnet34,58.071522,91.607980,17027195415.679953,22722922369.680016,raptor,1.58
googlenet,24.247113,62.923463,8019605936.239990,14547526780.240000,raptor,2.60
vgg8,1.465778,7.985074,477298145.040001,1597904071.120000,raptor,5.45
resnet18,28.099952,58.853733,8781611766.119984,13983168748.119974,raptor,2.09
resnet34,45.781486,91.607980,14962940227.679951,22722922369.680016,raptor,2.00
googlenet,13.371204,62.923463,6117835798.919991,14547526780.240000,raptor,4.71
1 model raptor_latency_ms pimcomp_latency_ms raptor_energy_pj pimcomp_energy_pj faster_compiler speedup
2 vgg8 2.463417 1.465778 7.985074 649524564.120001 477298145.040001 1597904071.120000 raptor 3.24 5.45
3 resnet18 31.449811 28.099952 58.853733 9335506856.119984 8781611766.119984 13983168748.119974 raptor 1.87 2.09
4 resnet34 58.071522 45.781486 91.607980 17027195415.679953 14962940227.679951 22722922369.680016 raptor 1.58 2.00
5 googlenet 24.247113 13.371204 62.923463 8019605936.239990 6117835798.919991 14547526780.240000 raptor 2.60 4.71
validation/networks/pimcomp_models/yolo11n/yolo11n-latency.onnx
BIN
View File
Binary file not shown.