nnicolosi/Raptor
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

add non-functional validation
Validate Operations / validate-operations (push) Has been cancelled
Details

Browse Source 
refactors
This commit is contained in:
NiccoloN
2026-07-27 10:54:59 +02:00
parent 4964e889da
commit 620e381cfb
30 changed files with 677 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files
validation/networks/pimcomp_models/README.md
+34 -20
View File
@@ -44,13 +44,18 @@ a35bad96441efbee28699cb61d1656cca7f7281f14040cf01699c3d0cfd8b202 googlenet/goog
## Paper hardware profiles
The files in `configs/` encode Table V's explicit resource parameters.
The files in
[`../../pimsim_configs/pimcomp/`](../../pimsim_configs/pimcomp/)
encode Table V's explicit resource parameters.
Each profile subdirectory contains pre-generated latency and throughput
`pimsim-nn` configs plus its matching mesh; comparison and validation select
these checked-in artifacts without generating configs at runtime.
| Config | Cores | Crossbars/core | Crossbar | Cell | PIMCOMP layout |
| --- | ---: | ---: | --- | ---: | --- |
| `arch-a.json` | 168 | 96 | `128x128` | 2-bit | `12x14` |
| `arch-b.json` | 138 | 128 | `128x128` | 2-bit | `6x23` |
| `arch-c.json` | 64 (16 chips x 4) | 8 | `512x1024` | 2-bit | flattened `8x8` |
| `arch-a/latency_config.json` | 168 | 96 | `128x128` | 2-bit | `12x14` |
| `arch-b/latency_config.json` | 138 | 128 | `128x128` | 2-bit | `6x23` |
| `arch-c/latency_config.json` | 64 (16 chips x 4) | 8 | `512x1024` | 2-bit | flattened `8x8` |
`adc_count` is 16, matching the paper's 16-bit fixed-point weight precision.
The paper does not give a two-dimensional core topology for Arch-A/B, so the
@@ -67,7 +72,7 @@ reproducible, but absolute paper power and energy numbers are not.
From the Raptor repository root:
```bash
.venv/bin/python -m pip install numpy onnx onnxruntime onnxsim colorama
.venv/bin/python -m pip install -r requirements.txt
cmake --build ./build_release
cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN
@@ -85,11 +90,11 @@ select one paper profile, and restore it when the shell exits:
```bash
RAPTOR_ROOT=$PWD
PIMCOMP="$RAPTOR_ROOT/third_party/PIMCOMP-NN"
PAPER_MODELS="$RAPTOR_ROOT/validation/networks/pimcomp_models"
PIMCOMP_CONFIGS="$RAPTOR_ROOT/validation/pimsim_configs/pimcomp"
CONFIG_BACKUP=$(mktemp)
cp "$PIMCOMP/config.json" "$CONFIG_BACKUP"
trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT
cp "$PAPER_MODELS/configs/arch-a.json" "$PIMCOMP/config.json"
cp "$PIMCOMP_CONFIGS/arch-a/latency_config.json" "$PIMCOMP/config.json"
```
The Model Zoo files map exactly to PIMCOMP's bundled model names, so compile
@@ -124,7 +129,8 @@ cd third_party/PIMCOMP-NN/build
./PIMCOMP-NN -m=vgg8_paper_reconstructed -r=balance -p=element -o=YES -v=YES -s=YES
```
Repeat after selecting `arch-b.json` and `arch-c.json`. All four models were
Repeat after selecting `arch-b/latency_config.json` and
`arch-c/latency_config.json`. All four models were
compiled successfully in both modes with all three configs. The released
random placement code occasionally segfaults; an unchanged retry succeeded in
the observed cases.
@@ -140,8 +146,9 @@ insufficient.
## Compare Raptor and PIMCOMP
The comparison driver uses one random input and one native ONNX-MLIR reference,
compiles both instruction streams, validates both through Raptor's Rust
simulator, and writes Markdown and JSON reports.
compiles both instruction streams, runs both through `pimsim-nn`, validates
Raptor through the Rust simulator, and writes Markdown and JSON reports.
PIMCOMP Rust validation also runs when its optional exporter is available.
To reproduce the complete Arch-A latency experiment, use the serial experiment
runner. It creates an isolated PIMCOMP build with population 200 and 1000 GA
@@ -156,8 +163,7 @@ comparison driver for one model at a time:
Reports are written under `<out-dir>/<model>/comparison_report.{md,json}`.
Use `--models vgg8` to run one model, `--resume` after an interruption, or
`--dry-run` to inspect every command. The runner continues after a failed model
so all reports are produced, then returns a nonzero status if any comparison
failed.
so all reports are produced.
Arch-A low-latency example:
@@ -167,28 +173,34 @@ PIMCOMP="$RAPTOR_ROOT/third_party/PIMCOMP-NN"
CONFIG_BACKUP=$(mktemp)
cp "$PIMCOMP/config.json" "$CONFIG_BACKUP"
trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT
cp "$RAPTOR_ROOT/validation/networks/pimcomp_models/configs/arch-a.json" "$PIMCOMP/config.json"
cp "$RAPTOR_ROOT/validation/pimsim_configs/pimcomp/arch-a/latency_config.json" "$PIMCOMP/config.json"
"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/compare_raptor_pimcomp.py" \
--model "$RAPTOR_ROOT/validation/networks/pimcomp_models/vgg8/vgg8-mnist-reconstructed.onnx" \
--out-dir /tmp/compare-vgg8-arch-a-ll \
--model "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/resnet34-v1-7.onnx" \
--out-dir /tmp/compare-resnet34-arch-a-ll \
--core-count 168 \
--crossbar-count 96 \
--crossbar-size 128 \
--mesh-rows 12 \
--mesh-cols 14 \
--pimsim-mode latency \
--pimcomp-pipeline element \
--fail-on-error
--pimcomp-pipeline element
```
For Arch-A high throughput, use `--pimsim-mode throughput
--pimcomp-pipeline batch`. For Arch-B, use 138 cores, 128 crossbars, a
`6x23` mesh, and `configs/arch-b.json`.
`6x23` mesh, and
`validation/pimsim_configs/pimcomp/arch-b/latency_config.json`.
If only semantic and instruction comparison is required, add
`--skip-pimsim-nn`. This exact VGG-8 Arch-A LL smoke test passed both semantic
validations with maximum output differences below `5e-10`.
`--skip-pimsim-nn`. A VGG-8 run with the same Arch-A LL settings passed both
semantic validations with maximum output differences below `5e-10`.
The current PIMCOMP-NN submodule does not include the optional
`verification/export_to_pim_simulator.py` helper. The driver therefore records
PIMCOMP Rust semantic validation as skipped, while PIMCOMP compilation,
instruction reporting, and `pimsim-nn` latency still run. Use
`--fail-on-error` only when that semantic export helper is available.
Current Raptor status:
@@ -213,6 +225,8 @@ the canonical `resnetv2/depth_68` file may not exist remotely:
REMOTE_REPO=/home/gmagnani/Project/Raptor
rsync -azL validation/networks/pimcomp_models/ \
"monolith:$REMOTE_REPO/validation/networks/pimcomp_models/"
rsync -az validation/pimsim_configs/pimcomp/ \
"monolith:$REMOTE_REPO/validation/pimsim_configs/pimcomp/"
rsync -az validation/tools/compare_raptor_pimcomp.py \
"monolith:$REMOTE_REPO/validation/tools/compare_raptor_pimcomp.py"
rsync -az validation/tools/run_pimcomp_paper_latency.py \
validation/networks/pimcomp_models/configs/arch-a.json
-54
View File
@@ -1,54 +0,0 @@
{
"chip_config": {
"core_config": {
"period": 1,
"matrix_config": {
"xbar_array_count": 96,
"period": 1,
"pipeline_mode": true,
"dac_resolution": 1,
"dac_count": 128,
"xbar_size": [128, 128],
"cell_precision": 2,
"xbar_latency": 7,
"xbar_read_power": 22.24,
"sample_hold_latency_cycle": 1,
"adc_resolution": 8,
"adc_latency_cycle": 20,
"adc_static_power": 0.322,
"adc_dynamic_power": 1.135,
"adc_count": 16,
"shift_adder_latency_cycle": 1,
"output_buffer_latency_cycle": 1
},
"vector_width": 32,
"vector_latency_cycle": 4,
"local_memory_config": {
"data_width": 64,
"period": 1,
"write_latency_cycle": 30,
"read_latency_cycle": 20
},
"global_memory_switch_id": -10
},
"global_memory_config": {
"data_width": 32,
"period": 1,
"write_latency_cycle": 50,
"read_latency_cycle": 40
},
"network_config": {
"bus_topology": "mesh",
"bus_width": 32,
"layout": [12, 14],
"net_config_file_path": "network_mesh_168.json"
},
"core_cnt": 168,
"global_memory_switch_id": -10
},
"sim_config": {
"sim_mode": 0,
"sim_time": 200,
"report_verbose_level": 0
}
}
validation/networks/pimcomp_models/configs/arch-b.json
-54
View File
@@ -1,54 +0,0 @@
{
"chip_config": {
"core_config": {
"period": 1,
"matrix_config": {
"xbar_array_count": 128,
"period": 1,
"pipeline_mode": true,
"dac_resolution": 1,
"dac_count": 128,
"xbar_size": [128, 128],
"cell_precision": 2,
"xbar_latency": 7,
"xbar_read_power": 22.24,
"sample_hold_latency_cycle": 1,
"adc_resolution": 8,
"adc_latency_cycle": 20,
"adc_static_power": 0.322,
"adc_dynamic_power": 1.135,
"adc_count": 16,
"shift_adder_latency_cycle": 1,
"output_buffer_latency_cycle": 1
},
"vector_width": 32,
"vector_latency_cycle": 4,
"local_memory_config": {
"data_width": 64,
"period": 1,
"write_latency_cycle": 30,
"read_latency_cycle": 20
},
"global_memory_switch_id": -10
},
"global_memory_config": {
"data_width": 32,
"period": 1,
"write_latency_cycle": 50,
"read_latency_cycle": 40
},
"network_config": {
"bus_topology": "mesh",
"bus_width": 32,
"layout": [6, 23],
"net_config_file_path": "network_mesh_138.json"
},
"core_cnt": 138,
"global_memory_switch_id": -10
},
"sim_config": {
"sim_mode": 0,
"sim_time": 200,
"report_verbose_level": 0
}
}
validation/networks/pimcomp_models/configs/arch-c.json
-54
View File
@@ -1,54 +0,0 @@
{
"chip_config": {
"core_config": {
"period": 1,
"matrix_config": {
"xbar_array_count": 8,
"period": 1,
"pipeline_mode": true,
"dac_resolution": 1,
"dac_count": 128,
"xbar_size": [512, 1024],
"cell_precision": 2,
"xbar_latency": 7,
"xbar_read_power": 22.24,
"sample_hold_latency_cycle": 1,
"adc_resolution": 8,
"adc_latency_cycle": 20,
"adc_static_power": 0.322,
"adc_dynamic_power": 1.135,
"adc_count": 16,
"shift_adder_latency_cycle": 1,
"output_buffer_latency_cycle": 1
},
"vector_width": 32,
"vector_latency_cycle": 4,
"local_memory_config": {
"data_width": 64,
"period": 1,
"write_latency_cycle": 30,
"read_latency_cycle": 20
},
"global_memory_switch_id": -10
},
"global_memory_config": {
"data_width": 32,
"period": 1,
"write_latency_cycle": 50,
"read_latency_cycle": 40
},
"network_config": {
"bus_topology": "mesh",
"bus_width": 32,
"layout": [8, 8],
"net_config_file_path": "network_mesh_64.json"
},
"core_cnt": 64,
"global_memory_switch_id": -10
},
"sim_config": {
"sim_mode": 0,
"sim_time": 200,
"report_verbose_level": 0
}
}