Files
Raptor/validation/tools/pim/ablation
NiccoloN 49d6fb42cf
Validate Operations / validate-operations (push) Has been cancelled
update plots
2026-10-26 12:18:36 +01:00
..
2026-10-19 16:34:01 +02:00
2026-10-26 12:18:36 +01:00
2026-10-19 16:34:01 +02:00
2026-10-19 16:34:01 +02:00

Raptor compiler ablation

Run the synchronization/Spatial-planning ablation from the repository root:

.venv/bin/python validation/tools/pim/ablation/run_ablation.py --jobs 4

Defaults: vgg8 resnet18 resnet34 googlenet, arch-a arch-b, latency (pipeline 1), and throughput (pipeline 4). Select yolo11n or arch-c explicitly when needed. Only Raptor runs; Pimcomp validation is SKIPPED.

Variant Raptor options
baseline None
no-sync --pim-disable-synchronization
no-plan --pim-disable-spatial-planning
no-sync-no-plan Both options

Options and outputs

  • --variant NAME: repeat to select variants; baseline is always included.
  • --out-dir PATH: artifact root; defaults to validation/networks/pimcomp_models.
  • --dry-run: print commands without modifying files.

Other suite-runner options, including --models, --archs, and --no-fast, are forwarded. Supplying --mode or --pipeline replaces the default latency/pipeline-4 case set. --only is rejected because the wrapper always runs Raptor only. Ctrl+C stops the active variant's workers.

Artifacts use <out-dir>/<model>/artifacts/<arch>/<mode>[/pipelineN]/ablation/<variant>/; reference inputs, outputs, and runners are shared under artifacts/common/. Per-variant summaries use <out-dir>/ablation/<variant>/results_comparison.csv. Normal comparison results are preserved.

By default, the combined results_ablation.csv is written beside this script and printed after the run. An explicit --out-dir PATH writes PATH/results_ablation.csv, keeping independent experiments separate. Default model artifacts and per-variant summaries remain in the locations above. The aggregate records case identifiers, functional_validation, and signed percentage changes in latency, throughput, power, and energy against the matching baseline case. Positive means higher; baseline values are zero.

Disabling synchronization removes host-buffer lifetime and pipeline restart guarantees, so those variants may fail numerical validation and exit nonzero. Baseline and no-plan must pass. Percentages from failing variants are not valid speedups; deadline averages do not prove steady-state convergence.

Result data

The checked-in results_ablation.csv is an intentional result dataset for tabular reporting. Ablation experiments and aggregation remain available through the runner above. Percentages from failing variants are not valid speedups. See the comparison metric definitions.