NiccoloN
b009e1ff08
normalize names and artifact paths
3.6 KiB
Raptor compiler ablation
run_ablation.py performs the complete synchronization/Spatial-planning
ablation study on the Pimcomp model suite. By default it runs vgg8,
resnet18, resnet34, and googlenet across arch-a and arch-b, latency,
and throughput pipeline 4. arch-c and yolo11n are run only when selected
explicitly.
Latency is pipeline 1; the wrapper invokes the suite runner separately for
latency and throughput pipeline 4.
.venv/bin/python validation/tools/pim/ablation/run_ablation.py \
--jobs 4
Variants
| Variant | Raptor options |
|---|---|
baseline |
None. |
no-synchronization |
--pim-disable-synchronization |
no-spatial-planning |
--pim-disable-spatial-planning |
no-synchronization-no-spatial-planning |
Both options. |
Every variant runs Raptor only. Pimcomp is not compiled, validated, or simulated. Reference inputs and outputs are generated once under the shared common-artifact root and reused by every variant. Ctrl+C terminates the active variant and all of its worker jobs.
The percentage baseline is no-synchronization-no-spatial-planning: both
ablation features are disabled, so its values are +0.00%. Every other
variant reports the signed percentage difference of its Raptor metrics from
that reference for the same model, architecture, mode, and pipeline. Positive
values mean the metric is higher; negative values mean it is lower.
Options
| Option | Description and default |
|---|---|
-h, --help |
Show help and exit. |
--out-dir PATH |
Suite root. Default: validation/networks/pimcomp_models. Model artifacts are stored below <out-dir>/<model>/artifacts; disabled variants use <arch>/<mode>[/pipelineN]/ablation/<variant>. Variant summaries remain under <out-dir>/ablation/<variant>/. |
--models MODEL [...] |
Models to run. Default: vgg8 resnet18 resnet34 googlenet; include yolo11n explicitly when needed. |
--variant NAME |
Select a variant from the table above; repeat for multiple variants. Default: all variants. The feature-full baseline and percentage reference are added automatically when needed. |
--dry-run |
Print the suite-runner commands that would run without modifying files. Default: off. |
All options of
run_pimcomp_models.py, including
--archs, --mode, --pipeline, --no-fast, and
--raptor-extra-arg=ARG, are forwarded to each variant. --out-dir,
--variant, and --dry-run belong to this wrapper; --only is reserved for
the suite runner's comparison mode and is rejected by this wrapper, which
always invokes --raptor-only. If neither --mode nor --pipeline is
forwarded, the wrapper uses its default latency and throughput/pipeline-4
case set. Supplying either option overrides that default and is passed through
as one suite-runner invocation per variant.
The feature-full baseline variant uses the normal run_pimcomp_models.py
artifact paths and is rerun with the same selected cases and forwarded options
as the disabled variants. The three disabled variants are stored below each model's
artifacts/<arch>/<mode>[/pipelineN]/ablation/ directory. Shared reference
artifacts remain under each model's artifacts/common directory. Transient
per-variant comparison summaries are removed after aggregation. The combined
table is written to <out-dir>/results_ablation.csv; it contains only the variant, case
identifiers, and signed latency_percent, throughput_percent,
power_percent, and energy_percent columns. These are Raptor metrics;
Pimcomp metrics are omitted because the ablation invokes Raptor only.