Raptor/validation/tools/pim/ablation

Raptor compiler ablation

Run the synchronization/Spatial-planning ablation from the repository root:

.venv/bin/python validation/tools/pim/ablation/run_ablation.py --jobs 4

Defaults: vgg8 resnet18 resnet34 googlenet, arch-a arch-b, latency (pipeline 1), and throughput (pipeline 4). Select yolo11n or arch-c explicitly when needed. Only Raptor runs; Pimcomp validation is SKIPPED.

Variant	Raptor options
baseline	None; planning and synchronization enabled
no-sync	--pim-disable-synchronization; throughput only
no-plan	--pim-disable-spatial-planning; synchronization remains enabled

Latency runs compare baseline with no-plan. Throughput runs compare baseline with each disabled feature separately. No run disables both features. The baseline artifact name identifies the fully enabled program, not the denominator used for percentage gains.

Options and outputs

• --variant NAME: repeat to select variants; baseline is always included.
• --out-dir PATH: artifact root; defaults to validation/networks/pimcomp_models.
• --dry-run: print commands without modifying files.

Other suite-runner options, including --models, --archs, and --no-fast, are forwarded. Supplying --mode or --pipeline replaces the default latency/pipeline-4 case set. Mode-only selection runs all suite pipelines for that mode. Pipeline-only selection uses its matching mode. --only is rejected because the wrapper always runs Raptor only. Ctrl+C stops the active variant's workers.

Artifacts use <out-dir>/<model>/artifacts/<arch>/<mode>[/pipelineN]/ablation/<variant>/; reference inputs, outputs, and runners are shared under artifacts/common/. Per-variant summaries use <out-dir>/ablation/<variant>/results_comparison.csv. Normal comparison results are preserved.

By default, the combined results_ablation.csv is written beside this script and printed after the run. An explicit --out-dir PATH writes PATH/results_ablation.csv, keeping independent experiments separate. Default model artifacts and per-variant summaries remain in the locations above. The aggregate contains one row per comparison:

Mode	comparison	baseline_variant	Main gain
Latency	plan	no-plan	latency_gain_percent
Throughput	plan	no-plan	throughput_gain_percent
Throughput	sync	no-sync	throughput_gain_percent

Throughput gain is 100 × (enabled / disabled − 1). Latency, power, and energy gains are reductions: 100 × (1 − enabled / disabled). The corresponding columns use the suffix _gain_percent. Positive means improvement; negative means a cost. Synchronization gain can be negative. No sign is forced. Both programs' functional statuses are recorded as functional_validation and baseline_functional_validation. The latter belongs to the disabled feature.

Disabling synchronization removes host-buffer lifetime and pipeline restart guarantees, so those variants may fail numerical validation and exit nonzero. Baseline and no-plan must pass. Percentages involving failing programs remain diagnostic comparisons, not valid speedups.

When fast-mode convergence fails, positive fallback throughput remains available. Its output interval uses the full simulation window. The per-case JSON report records throughput_measurement: full_duration_fallback and fast_throughput_converged: false. This result does not prove steady-state convergence. Zero completed outputs still produce NA for per-inference metrics.

The no-sync compiler option keeps the normal selected schedule and omits SYNC/WAIT instructions during code generation. Selection costs describe the synchronized-reference program. Each backend attempt also reports a separate synchronization_ablation_cost, with zero SYNC/WAIT work and no synchronization dependencies. This diagnostic estimate is not used for selection. Its complete-state recurrence can remain unproved because cores can advance independently. Numerical validation still checks the emitted program and can fail.

Result data

The checked-in results_ablation.csv is a historical dataset until the next run regenerates it with the comparison schema above. Existing measurements are preserved. Ablation experiments and aggregation remain available through the runner above. Percentages from failing variants are not valid speedups. See the comparison metric definitions.