Python script for compare

2026-07-24 12:47:37 +02:00
parent e7611be8e1
commit f3a4e19f7c
14 changed files with 684 additions and 21 deletions
@@ -0,0 +1,33 @@
+# Raptor vs PIMCOMP latency results
+
+## GoogLeNet, Arch-A, low latency
+
+Measured with `googlenet-12-no-softmax.onnx`, batch 1, Raptor's best current
+schedule, and PIMCOMP's GA/element artifacts. Both instruction streams were
+simulated by the same `pimsim-nn` build using the complete Arch-A timing and
+precision configuration.
+
+| Compiler | Latency (ms) | Instructions | Sends | Receives | MVMUL |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Raptor | 1132.458315 | 77,717,248 | 29,115 | 29,115 | 157,158 |
+| PIMCOMP | 41.790450 | 3,398,070 | 110,334 | 110,334 | 113,639 |
+
+PIMCOMP is 27.10x faster in this latency simulation.
+
+Semantic validation did not pass the comparison driver's strict default
+tolerance: the maximum logit differences from the native ONNX reference were
+`0.01995039` for Raptor and `7.768404` for PIMCOMP. Treat these as performance
+results, not as a correctness-equivalent comparison.
+
+No throughput experiment was run.
+
+## Reproduce
+
+```bash
+.venv/bin/python validation/tools/run_pimcomp_paper_latency.py \
+  --out-dir /tmp/raptor-pimcomp-paper-latency \
+  --models googlenet
+```
+
+See [README.md](README.md) for model provenance, limitations, and monolith
+instructions.