rework documentation

add plot generation scripts adjust validation tools add up-to-date results
2026-10-19 16:34:01 +02:00
parent 9053cc5e8b
commit 034c197260
33 changed files with 3077 additions and 1798 deletions
@@ -0,0 +1 @@
+plots/
@@ -1,62 +1,108 @@
 # Pimcomp model comparison

-These scripts compare Raptor with Pimcomp on the models in
-`validation/networks/pimcomp_models`.
-
-`run_pimcomp_models.py` runs the supported model suite across the configured
-architectures and simulation modes. `compare_raptor_pimcomp_model.py` is the
-lower-level one-model comparison used by the suite runner.
+Compare Raptor and Pimcomp on the [model suite](../../../../networks/pimcomp_models/README.md).
+Run commands from the repository root with the repository virtual environment,
+built release Raptor, Pimcomp, and `pimsim-nn`.

 ## Suite runner

-Run all five models, Arch-A and Arch-B, latency, and throughput:
-
 ```bash
 .venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
 ```

-### Options
+Defaults: all five models, Arch-A/B, latency pipeline 1, throughput pipelines
+2/4/8, four workers, and a functional throughput batch of 64.
+Use `--help` for the complete option reference.

-| Option | Description and default |
+| Option | Purpose |
 |---|---|
-| `-h`, `--help` | Show help and exit. |
-| `--out-dir PATH` | Suite root; artifacts are placed below `<out-dir>/<model>/artifacts/`. Default: `validation/networks/pimcomp_models`. |
-| `--common-dir PATH` | Shared root for per-model reference artifacts. Default: `<out-dir>/<model>/artifacts/common`. |
-| `--models MODEL [...]` | Models to run: `vgg8`, `resnet18`, `resnet34`, `googlenet`, `yolo11n`. Default: all five; pass a subset to select specific models. |
-| `--archs ARCH [...]` | Pimcomp hardware profiles to run. Default: `arch-a arch-b`. |
-| `--mode {latency,throughput} [...]` | Simulation modes. Default: both. |
-| `--only {raptor,pimcomp}` | Re-run only one compiler side and reuse the other side's existing report. Default: run both sides. |
-| `--raptor-only` | Run Raptor without compiling, validating, or simulating Pimcomp. Default: off. |
-| `--pipeline {1,2,4,8}` | Select one Raptor pipeline. Default: latency pipeline 1 and throughput pipelines 2, 4, and 8. |
-| `--pimsim-time-ms INT` | Throughput convergence deadline. Default: `1000`. |
-| `--batch-size INT` | Functional throughput batch size. Default: `128`. |
-| `--timeout-seconds FLOAT` | Per-stage timeout; `0` means unlimited. Default: `0`. |
-| `-j INT`, `--jobs INT` | Parallel comparison workers. Default: `4`. |
-| `--clean` | Remove generated comparison artifacts and summaries, then exit. Default: off. |
-| `--ablation-variant NAME` | Put generated artifacts below `<mode>[/pipelineN]/ablation/NAME` and write the summary below `ablation/NAME`. Default: none. |
-| `--dry-run` | Print commands without modifying files. Default: off. |
-| `--no-fast` | Disable fast throughput convergence. Default: off. |
-| `--raptor-extra-arg=ARG` | Extra Raptor compiler argument; repeat for multiple arguments. Default: none. |
+| `--models MODEL [...]`, `--archs ARCH [...]` | Select models and hardware profiles. |
+| `--mode {latency,throughput} [...]`, `--pipeline {1,2,4,8}` | Select experiments. |
+| `--out-dir PATH` | Suite root; default: `validation/networks/pimcomp_models`. |
+| `--common-dir PATH` | Override the shared reference-artifact root. |
+| `--only {raptor,pimcomp}` | Rerun one compiler and reuse the other's existing report. |
+| `--raptor-only` | Run Raptor without invoking Pimcomp. |
+| `--batch-size N`, `--jobs N` | Set functional throughput batch size and worker count. |
+| `--pimsim-time-ms N` | Throughput convergence deadline; default: 1000 ms. |
+| `--no-fast` | Use full-duration throughput simulation for authoritative experiments. |
+| `--timeout-seconds N` | Per-stage timeout; default: 0 (unlimited). |
+| `--raptor-extra-arg=ARG` | Forward a compiler option; repeat as needed. |
+| `--dry-run` | Print commands without modifying files. |
+| `--clean` | Remove generated comparison artifacts, summaries, and stale locks. |

-Arguments beginning with `--` must use the equals form when passed through:
+Use the equals form for forwarded flags, for example
+`--raptor-extra-arg=--pim-detect-communication-deadlock`.

-```bash
-.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py \
-  --models vgg8 \
-  --raptor-extra-arg=--pim-disable-synchronization
-```
+With no `--out-dir`, the runner writes [results_comparison.csv](results_comparison.csv)
+beside this script. An explicit `--out-dir PATH` writes
+`PATH/results_comparison.csv`; selecting a subset updates only those rows.
+Default model artifacts remain under `validation/networks/pimcomp_models`.
+Artifacts use
+`<out-dir>/<model>/artifacts/<arch>/<mode>[/pipelineN]/`, with shared references
+under `artifacts/common/` (`common-functional/` for YOLO's canonical reference)
+and `comparison_report.{md,json}` under each comparison's `pimcomp/`.
+Pimcomp runs once per model/architecture/mode; later pipelines reuse its newly
+written report, including failures without simulation artifacts. Missing or
+unchanged anchor reports trigger fresh attempts. The runner continues after failures and exits nonzero if a
+compiler, simulation, or functional comparison fails.

-The runner writes `results_comparison.csv` under the selected result root. It reuses
-shared model inputs, outputs, and reference runners, and reuses Pimcomp
-artifacts between pipelines in the same model/architecture/mode group. With
-`--raptor-only`, only Raptor results are generated and Pimcomp is not invoked.
-All generated files are kept below each model's `artifacts/` directory;
-`--clean` also removes any `.lock` files left by an interrupted reference
-generation.
+Reuse checks model/configuration paths and SHA-256 contents, effective hardware,
+mode, Pimcomp pipeline/replication, seed, batch, tolerances, and simulation flags.
+Malformed, incomplete, mismatched, or legacy reports without fingerprints fail
+with a regeneration diagnostic. `--only raptor` requires compatible reports;
+`--only pimcomp` reruns Pimcomp for each group and preserves existing Raptor
+results. Ordinary runs also rerun the group's first case, even if an old report
+exists. Raptor pipeline depth is deliberately excluded from Pimcomp identity.
+Tool/build revisions and manually edited reference artifacts are not hashed;
+rerun the relevant compiler after those change.

 ## One-model comparator

-`compare_raptor_pimcomp_model.py` compares one ONNX model. Its required
-arguments are `--model PATH` and `--out-dir PATH`; use `--help` for the full
-lower-level interface. The suite runner supplies the model, hardware profile,
-simulation mode, and reuse paths automatically.
+`compare_raptor_pimcomp_model.py` requires `--model PATH` and `--out-dir PATH`.
+Use `--help` for hardware and simulation options. The suite runner supplies
+these automatically.
+The comparator defaults to absolute/relative tolerances `1e-3`/`1e-4`;
+`validate.py` uses `1e-3`/`1e-5`.
+
+## Comparison charts
+
+```bash
+.venv/bin/python validation/tools/pim/pimcomp/compare/generate_plots.py
+```
+
+Requires Matplotlib from the root `requirements.txt`. Defaults to the supplied
+`results_comparison.csv` beside the script and writes `plots/png/` and
+`plots/svg/` there. Use `--input PATH` for a runner-generated CSV and
+`--output-dir PATH` to override the destination. The plotter requires the full
+40-case matrix: five models × Arch-A/B × latency pipeline 1 and throughput
+pipelines 2/4/8. Duplicate, missing, unexpected, or inconsistent shared Pimcomp
+configurations are rejected before plots are written.
+
+| Plot family | Meaning |
+|---|---|
+| `validation_status` | Recorded functional status for each compiler and experiment. |
+| `latency_*` | Single-inference latency, latency mode only. |
+| `throughput_*` | Inferences/s, throughput mode only. |
+| `longest_stage_latency_*` | Combined single-inference latency and throughput output interval; the latter is a stage-latency proxy. |
+| `power_*`, `energy_*` | Average power and energy per inference. |
+| `raptor_pipeline_throughput` | Raptor throughput versus stages; stage 1 is derived as `1000 / latency_ms`, stages 2/4/8 are measured. |
+
+Suffixes `_absolute` and `_yolo` show measurements (`_yolo` is Raptor-only);
+`_comparison` shows ratios and `_diverging` shows equal-length bars for equal
+advantages in either direction. Ratios above 1 favor Raptor: Pimcomp/Raptor for
+latency, power, and energy; Raptor/Pimcomp for throughput. Diverging bars use
+winning factor minus one, with parity at zero.
+
+Failed validation is labeled `INVALID`; other non-PASS statuses keep their
+names, and missing measurements show `N/A`. Relative bars require both
+compilers to pass to be valid. Repeated Pimcomp batch measurements and statuses
+must agree before being collapsed. `FAIL` alone does not identify the cause.
+
+Plots consume the CSV's current contents; they do not run benchmarks or establish
+publication provenance. The checked-in dataset is intentional. See the
+[result methodology](../../../../../docs/evaluation/methodology.md#canonical-result-datasets-and-output-roots).
+
+## Metric interpretation
+
+See the [evaluation methodology](../../../../../docs/evaluation/methodology.md#metric-interpretation)
+for measurement windows, units, numerical validity, and result provenance.
@@ -4,6 +4,7 @@ from __future__ import annotations
 import argparse
 import fcntl
 import gzip
+import hashlib
 import importlib.util
 import json
 import mmap
@@ -1162,6 +1163,60 @@ def path_from_report(report: dict[str, Any], name: str) -> Path | None:
    return Path(value) if value else None


+def pimcomp_reuse_context(
+    args: argparse.Namespace,
+    model: Path,
+    functional_model: Path,
+    hardware: dict[str, int],
+) -> dict[str, Any]:
+    files = {"model": model, "functional_model": functional_model,
+             "config": args.pimcomp_config}
+    digests = {}
+    for path in set(files.values()):
+        with path.open("rb") as stream:
+            digests[path] = hashlib.file_digest(stream, "sha256").hexdigest()
+    return {
+        "version": 1,
+        "files": {name: {"path": str(path), "sha256": digests[path]}
+                  for name, path in files.items()},
+        "hardware": hardware,
+        "settings": {name: getattr(args, name) for name in (
+            "pimsim_mode", "pimsim_time_ms", "pimcomp_pipeline",
+            "pimcomp_replication", "pimcomp_model_name", "seed", "batch_size",
+            "threshold", "rtol", "no_fast", "skip_pimsim_nn",
+        )},
+    }
+
+
+def load_reused_pimcomp_report(path: Path, context: dict[str, Any] | None) -> dict[str, Any]:
+    try:
+        report = json.loads(path.read_text(encoding="utf-8"))
+        if (not isinstance(report, dict) or context is None
+            or report.get("pimcomp_reuse_context") != context
+            or report.get("pimcomp_model_source") != "original_onnx"):
+            raise ValueError("model, configuration, or experiment settings differ; legacy reports lack fingerprints")
+        for name, kind in (("pimcomp_validation", dict), ("pimcomp_performance", dict),
+                           ("pimcomp_instruction_summary", dict), ("paths", dict),
+                           ("failures", list), ("steps", list)):
+            if not isinstance(report.get(name), kind):
+                raise ValueError(f"missing or invalid {name}")
+        validation = CompareResult(**report["pimcomp_validation"])
+        if (type(validation.passed) is not bool or not isinstance(validation.max_diffs, dict)
+            or validation.status not in ("done", "failed", "skipped")):
+            raise ValueError("invalid functional validation result")
+        if not report["pimcomp_performance"]:
+            raise ValueError("missing performance result or failure status")
+        summary = report["pimcomp_instruction_summary"]
+        if not {"active_cores", "total_instructions", "op_counts"} <= summary.keys():
+            raise ValueError("incomplete instruction summary")
+        restore_side_records(report, "pimcomp", [], [])
+        for name in ("pimcomp_simulation_info", "pimcomp_exported_pim", "pimcomp_pimsim_nn", "pimsim_config"):
+            path_from_report(report, name)
+        return report
+    except (OSError, ValueError, TypeError, AttributeError, KeyError) as exc:
+        raise ValueError(f"Cannot reuse Pimcomp report {path}: {exc}. Rerun Pimcomp to regenerate it.") from exc
+
+
 def restore_side_records(
    report: dict[str, Any],
    side: str,
@@ -1579,6 +1634,11 @@ def main():
    if loaded_hardware is not None:
        hardware = loaded_hardware

+    reuse_context = None if args.raptor_only else try_stage(
+        failures, "Identify Pimcomp experiment", pimcomp_reuse_context,
+        args, model_path, functional_model_path, hardware,
+    )
+
    model_io = try_stage(
        failures,
        "Load model inputs",
@@ -1597,12 +1657,7 @@ def main():
    run_pimcomp = not args.raptor_only and not reuse_pimcomp
    if reuse_pimcomp:
        reused_pimcomp_report_path = reused_pimcomp_report_path.resolve()
-        if not reused_pimcomp_report_path.exists():
-            raise ValueError(f"Missing Pimcomp report: {reused_pimcomp_report_path}")
-        with open(reused_pimcomp_report_path, "r", encoding="utf-8") as f:
-            reused_pimcomp = json.load(f)
-        if reused_pimcomp.get("pimcomp_model_source") != "original_onnx":
-            raise ValueError("Reused Pimcomp artifacts were not generated from the original ONNX model")
+        reused_pimcomp = load_reused_pimcomp_report(reused_pimcomp_report_path, reuse_context)
        pimcomp_validation = CompareResult(**reused_pimcomp["pimcomp_validation"])
        pimcomp_perf = reused_pimcomp["pimcomp_performance"]
        pimcomp_instr = reused_pimcomp["pimcomp_instruction_summary"]
@@ -2011,6 +2066,7 @@ def main():
        "pimcomp_pipeline": args.pimcomp_pipeline,
        "pimcomp_replication": args.pimcomp_replication,
        "pimcomp_model_source": "not_run" if args.raptor_only else "original_onnx",
+        "pimcomp_reuse_context": reuse_context,
        "pimcomp_config": str(args.pimcomp_config),
        "raptor_extra_args": args.raptor_extra_arg,
        "reused_raptor_report": optional_path(args.reuse_raptor_report.resolve()) if reuse_raptor else None,
@@ -0,0 +1,514 @@
+#!/usr/bin/env python3
+"""Generate publication-ready Raptor vs Pimcomp comparison charts.
+
+Usage:
+    .venv/bin/python validation/tools/pim/pimcomp/compare/generate_plots.py
+
+Dependencies: matplotlib (no pandas or seaborn required).
+By default, reads results_comparison.csv beside this script and writes plots/
+beside it, with outputs grouped in png/ and svg/ under the output directory.
+
+YOLO models without Pimcomp measurements get separate Raptor-only charts.
+Comparisons use vertical bars for ratios, diverging advantages, and absolute
+values, with compiler colors and patterns for the Raptor pipeline stages.
+Throughput-mode latency is the output interval, a bottleneck-stage proxy,
+not end-to-end inference latency. Separate throughput plots use inferences/s.
+Energy is reported per inference; power is averaged over the simulation.
+Numerical results whose compiler did not pass functional validation have a red
+status following their value label; missing Pimcomp
+measurements appear as 'N/A'. The fixed compiler palette is shared by
+all metrics. Relative plots use Pimcomp / Raptor for latency, power and energy,
+and Raptor / Pimcomp for throughput, so >1 favors Raptor. Diverging plots put the winner's
+factor above (Raptor) or below (Pimcomp) parity, with 1x at the zero line.
+"""
+from __future__ import annotations
+
+import argparse
+import csv
+import math
+import sys
+from dataclasses import dataclass
+from itertools import groupby
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+from matplotlib.patches import Patch, Rectangle
+from matplotlib.lines import Line2D
+from matplotlib.ticker import FuncFormatter, MultipleLocator
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
+from plot_style import COLORS, HATCH_COLOR, TEXT_COLOR, configure_style, fit_value_labels, save_figure, value_label
+
+
+COMPILER_COLORS = {"raptor": COLORS[0], "pimcomp": COLORS[2]}
+STAGE_PATTERNS = {1: "", 2: "..", 4: "//", 8: "\\\\"}
+ARCHITECTURES = ("arch-a", "arch-b")
+MODELS = ("vgg8", "resnet18", "resnet34", "googlenet", "yolo11n")
+CONFIGURATIONS = (("latency", 1), ("throughput", 2), ("throughput", 4), ("throughput", 8))
+LINE_COLOR = "#A8B2C0"
+VALIDATION_STYLES = {
+    "PASS": ("#168C8B", "white", ""),
+    "FAIL": ("#D18D32", HATCH_COLOR, ""),
+    "SKIPPED": ("#E7DFF0", TEXT_COLOR, ""),
+    "NA": ("#E8EDF1", TEXT_COLOR, ""),
+}
+
+
+@dataclass(frozen=True)
+class Result:
+    arch: str
+    model: str
+    mode: str
+    pipeline: int
+    pimcomp_schedule: str
+    raptor_validation: str
+    pimcomp_validation: str
+    latency_speedup: float
+    power: float
+    energy: float
+    raptor_latency: float
+    pimcomp_latency: float
+    raptor_throughput: float
+    pimcomp_throughput: float
+    raptor_power: float
+    pimcomp_power: float
+    raptor_energy: float
+    pimcomp_energy: float
+
+    @property
+    def comparison_passed(self) -> bool:
+        return self.raptor_validation == self.pimcomp_validation == "PASS"
+
+    @property
+    def throughput_speedup(self) -> float:
+        return quotient(self.raptor_throughput, self.pimcomp_throughput)
+
+
+def positive_float(text: str | None) -> float:
+    try:
+        value = float(text)
+    except (TypeError, ValueError):
+        return math.nan
+    return value if math.isfinite(value) and value > 0 else math.nan
+
+
+def quotient(numerator: float, denominator: float) -> float:
+    if not (math.isfinite(numerator) and math.isfinite(denominator)):
+        return math.nan
+    return numerator / denominator
+
+
+def validation_status(text: str) -> str:
+    status = text.strip().upper()
+    if status not in VALIDATION_STYLES:
+        raise ValueError(f"Unknown functional-validation status: {text!r}")
+    return status
+
+
+def load_results(path: Path) -> list[Result]:
+    with path.open(newline="", encoding="utf-8-sig") as handle:
+        source = list(csv.DictReader(handle))
+    rows = []
+    expected = {(arch, model, mode, stage)
+                for arch in ARCHITECTURES for model in MODELS
+                for mode, stage in CONFIGURATIONS}
+    seen = {}
+    for line, row in enumerate(source, 2):
+        try:
+            arch, model, mode = (row[field].strip().lower() for field in ("arch", "model", "mode"))
+            stage = int(row["raptor_pipeline"])
+        except (KeyError, AttributeError, TypeError, ValueError) as exc:
+            raise ValueError(f"Invalid configuration key at CSV row {line}: {exc}") from exc
+        key = arch, model, mode, stage
+        if key not in expected:
+            raise ValueError(f"Unexpected configuration at CSV row {line}: {key}")
+        if key in seen:
+            raise ValueError(f"Duplicate configuration {key} at CSV rows {seen[key]} and {line}")
+        seen[key] = line
+        schedule = row["pimcomp_pipeline"].strip().lower()
+        if schedule != ("element" if mode == "latency" else "batch"):
+            raise ValueError(f"{key}: incompatible Pimcomp schedule {schedule!r}")
+        rows.append(Result(
+            arch=arch,
+            model=model,
+            mode=mode,
+            pipeline=stage,
+            pimcomp_schedule=schedule,
+            raptor_validation=validation_status(row["raptor_functional_validation"]),
+            pimcomp_validation=validation_status(row["pimcomp_functional_validation"]),
+            latency_speedup=quotient(positive_float(row["pimcomp_latency_ms"]),
+                                     positive_float(row["raptor_latency_ms"])),
+            power=quotient(positive_float(row["pimcomp_power_mw"]),
+                           positive_float(row["raptor_power_mw"])),
+            energy=quotient(positive_float(row["pimcomp_energy_pj"]),
+                            positive_float(row["raptor_energy_pj"])),
+            raptor_latency=positive_float(row["raptor_latency_ms"]),
+            pimcomp_latency=positive_float(row["pimcomp_latency_ms"]),
+            raptor_throughput=positive_float(row["raptor_throughput_samples_s"]),
+            pimcomp_throughput=positive_float(row["pimcomp_throughput_samples_s"]),
+            raptor_power=positive_float(row["raptor_power_mw"]),
+            pimcomp_power=positive_float(row["pimcomp_power_mw"]),
+            raptor_energy=positive_float(row["raptor_energy_pj"]),
+            pimcomp_energy=positive_float(row["pimcomp_energy_pj"]),
+        ))
+    missing = expected - seen.keys()
+    if missing:
+        raise ValueError(f"Missing {len(missing)} expected configurations. Examples: {sorted(missing)[:5]}")
+    rows.sort(key=lambda r: (ARCHITECTURES.index(r.arch), MODELS.index(r.model), r.pipeline))
+    absolute_columns(rows)
+    return rows
+
+
+def advantage_compiler(ratio: float) -> str | None:
+    if ratio > 1:
+        return "raptor"
+    if ratio < 1:
+        return "pimcomp"
+    return None
+
+
+def figure_legend(fig, results: list[Result], field: str,
+                  absolute=False, raptor_only=False):
+    compilers = ("raptor",) if raptor_only else ("raptor", "pimcomp")
+    if absolute:
+        measurements = [(row, compiler, getattr(row, f"{compiler}_{field}"))
+                        for row, compiler in absolute_columns(results, raptor_only)]
+    else:
+        measurements = [(row, None, getattr(row, field)) for row in results]
+    available = [(row, compiler, value) for row, compiler, value in measurements
+                 if math.isfinite(value)]
+    stages = {row.pipeline for row, compiler, _ in available if compiler != "pimcomp"}
+    handles = []
+    visible_compilers = {compiler if absolute else advantage_compiler(value)
+                         for _, compiler, value in available} - {None}
+    if len(visible_compilers) > 1:
+        for compiler in compilers:
+            handles.append(Patch(facecolor=COMPILER_COLORS[compiler],
+                                 label=compiler.capitalize() + ("" if absolute else " advantage")))
+    handles += [Patch(facecolor="white", edgecolor=TEXT_COLOR, hatch=STAGE_PATTERNS[depth],
+                      label=f"{depth} stage" + (" (latency)" if depth == 1 else "s"))
+                for depth in sorted(stages) if stages != {1}]
+    styles = []
+    if len(available) < len(measurements):
+        styles.append(Patch(facecolor="white", edgecolor="#B2BAC5",
+                            label="N/A: missing measurement"))
+    handles += styles
+    if handles:
+        fig.legend(handles=handles, loc="outside upper center", ncol=(len(handles) + 1) // 2,
+                   frameon=False, fontsize=9.5, handlelength=1.7, columnspacing=1.5,
+                   handletextpad=0.6, labelspacing=0.4)
+
+
+def absolute_columns(results: list[Result], raptor_only: bool = False) -> list[tuple[Result, str]]:
+    columns = []
+    for (arch, model), group in groupby(results, key=lambda row: (row.arch, row.model)):
+        rows = sorted(group, key=lambda row: row.pipeline)
+        for row in rows:
+            columns.append((row, "raptor"))
+            if row.mode == "latency" and not raptor_only:
+                columns.append((row, "pimcomp"))
+        throughput = [row for row in rows if row.mode == "throughput"]
+        if throughput and not raptor_only:
+            signatures = {
+                (row.pimcomp_schedule, row.pimcomp_validation,
+                 *(None if math.isnan(value) else value for value in (
+                     row.pimcomp_latency, row.pimcomp_throughput, row.pimcomp_power, row.pimcomp_energy)))
+                for row in throughput
+            }
+            if len(signatures) != 1:
+                raise ValueError(f"{arch}/{model}: inconsistent shared Pimcomp batch results across pipelines")
+            columns.append((throughput[0], "pimcomp"))
+    return columns
+
+
+def label_groups(ax, results: list[Result], positions: list[float] | None = None) -> None:
+    if positions is None:
+        positions = list(range(len(results)))
+    centers, labels = [], []
+    start = 0
+    for index, (model, rows) in enumerate(groupby(results, key=lambda row: row.model)):
+        end = start + len(list(rows))
+        centers.append((positions[start] + positions[end - 1]) / 2)
+        labels.append(model.upper())
+        left = ax.get_xlim()[0] if start == 0 else (positions[start - 1] + positions[start]) / 2
+        right = ax.get_xlim()[1] if end == len(results) else (positions[end - 1] + positions[end]) / 2
+        if index % 2 == 0:
+            ax.axvspan(left, right, facecolor="#F7F9FC", zorder=0)
+        if index > 0:
+            ax.axvline(left, color="#DDE3EB", linewidth=0.9)
+        start = end
+    ax.set_xticks(centers, labels, fontsize=9.5)
+
+
+def diverging_ratio(ratio: float) -> float:
+    return ratio - 1 if ratio >= 1 else 1 - 1 / ratio
+
+
+def draw_validation(results: list[Result], output_dir: Path) -> None:
+    columns = absolute_columns(results)
+    configurations = list(dict.fromkeys((row.mode, row.pipeline if compiler == "raptor" else row.pimcomp_schedule,
+                                        compiler) for row, compiler in columns))
+    positions = []
+    for index, (mode, _, _) in enumerate(configurations):
+        gap = 0.4 if index and mode != configurations[index - 1][0] else 0
+        positions.append(positions[-1] + 1 + gap if index else 0)
+    configuration_x = dict(zip(configurations, positions))
+    headers = [f"Raptor\n{stage} stages" if mode == "throughput" and compiler == "raptor"
+               else "Pimcomp\nbatch" if mode == "throughput" else compiler.capitalize()
+               for mode, stage, compiler in configurations]
+    fig, axes = plt.subplots(1, 2, figsize=(14, 3.8), layout="constrained")
+    for ax, arch in zip(axes, ARCHITECTURES):
+        ax.set_xlim(positions[0] - 0.55, positions[-1] + 0.55)
+        ax.set_ylim(len(MODELS) - 0.5, -0.5)
+        centers, modes = [], []
+        for mode, group in groupby(zip(configurations, positions), key=lambda item: item[0][0]):
+            group_positions = [x for _, x in group]
+            centers.append((group_positions[0] + group_positions[-1]) / 2)
+            modes.append(mode.capitalize())
+        ax.set_xticks(centers, modes, fontsize=10)
+        ax.set_xticks(positions, headers, minor=True)
+        ax.tick_params(axis="x", which="minor", top=True, labeltop=True,
+                       bottom=False, labelbottom=False, length=0, pad=6, labelsize=9)
+        ax.set_yticks(range(len(MODELS)), [model.upper() for model in MODELS], fontsize=9)
+        ax.tick_params(axis="both", length=0, pad=6)
+        architecture = "ARCHITECTURE A" if arch == "arch-a" else "ARCHITECTURE B"
+        ax.set_xlabel(architecture, fontsize=11, weight="bold", labelpad=8)
+        for row, compiler in columns:
+            if row.arch != arch:
+                continue
+            key = (row.mode, row.pipeline if compiler == "raptor" else row.pimcomp_schedule, compiler)
+            x, y = configuration_x[key], MODELS.index(row.model)
+            status = getattr(row, f"{compiler}_validation")
+            color, text_color, _ = VALIDATION_STYLES[status]
+            ax.add_patch(Rectangle((x - 0.5, y - 0.42), 1, 0.84,
+                                   facecolor=color, edgecolor="#657081", linewidth=0.5))
+            ax.text(x, y, "N/A" if status == "NA" else status,
+                    color=text_color, ha="center", va="center", fontsize=9)
+    save_figure(fig, "validation_status", output_dir)
+
+
+def draw_metric(results: list[Result], field: str, ylabel: str,
+                basename: str, output_dir: Path, diverging: bool = False) -> None:
+    data = [diverging_ratio(getattr(row, field)) if diverging else getattr(row, field)
+            for row in results]
+    available = [v for v in data if math.isfinite(v)]
+    upper = max(0, max(available, default=1.0))
+    lower = min(0, min(available, default=0)) if diverging else 0
+    padding = max(upper - lower, 1.0) * 0.3
+    y_max = math.ceil((upper + padding) * 2) / 2
+    y_min = math.floor((lower - padding) * 2) / 2 if diverging else 0
+    y_interval = 1.0 if y_max - y_min > 4.5 else 0.5
+
+    fig, axes = plt.subplots(1, 2, sharey=True, figsize=(14, 5.2),
+                             layout="constrained")
+    figure_legend(fig, results, field)
+
+    for ax, arch in zip(axes, ARCHITECTURES):
+        selected = [row for row in results if row.arch == arch]
+        ax.set_xlabel("ARCHITECTURE A" if arch == "arch-a" else "ARCHITECTURE B",
+                      fontsize=11, weight="bold", labelpad=8)
+        ax.set_axisbelow(True)
+        ax.set_ylim(y_min, y_max)
+        ax.set_xlim(-0.5, len(selected) - 0.5)
+        label_groups(ax, selected)
+        ax.tick_params(axis="y", labelleft=True, pad=5, length=0)
+        ax.tick_params(axis="x", labelsize=9.5, length=0, pad=6)
+        ax.yaxis.set_major_locator(MultipleLocator(y_interval))
+        if diverging:
+            ax.yaxis.set_major_formatter(FuncFormatter(lambda value, _: f"{abs(value) + 1:g}×"))
+        ax.grid(axis="y", color="#E6EAF0", linewidth=0.8)
+        ax.axhline(0 if diverging else 1, color="#536478", linewidth=1.2,
+                   linestyle="--", zorder=3)
+        ax.set_ylabel(ylabel, fontsize=10.5, labelpad=6)
+
+        for position, row in enumerate(selected):
+            ratio = getattr(row, field)
+            if not math.isfinite(ratio):
+                ax.text(position, (y_max - y_min) * 0.02, "N/A", ha="center", va="bottom",
+                        fontsize=9.5, color="#85909F", fontstyle="italic")
+                continue
+            compiler = advantage_compiler(ratio)
+            statuses = [status for status in (row.raptor_validation, row.pimcomp_validation) if status != "PASS"]
+            status = "FAIL" if "FAIL" in statuses else next(iter(statuses), "PASS")
+            pattern = STAGE_PATTERNS[row.pipeline]
+            measurement = diverging_ratio(ratio) if diverging else ratio
+            ax.bar(position, measurement, width=0.72,
+                    color=COMPILER_COLORS.get(compiler, "#657081"),
+                    hatch=pattern,
+                    edgecolor=TEXT_COLOR if pattern else "none",
+                    linewidth=0.7 if pattern else 0.0, zorder=2)
+            direction = -1 if measurement < 0 else 1
+            factor = max(ratio, 1 / ratio) if diverging else ratio
+            value_label(ax, position, measurement + direction * (y_max - y_min) * 0.012,
+                        f"{factor:.2f}×", status, direction, fontsize=9.1)
+
+    fit_value_labels(fig)
+    save_figure(fig, basename, output_dir)
+
+
+def draw_absolute(results: list[Result], field: str, ylabel: str,
+                  basename: str, output_dir: Path, scale: float = 1.0,
+                  raptor_only: bool = False) -> None:
+    columns = absolute_columns(results, raptor_only)
+    available = [getattr(row, f"{compiler}_{field}") * scale
+                 for row, compiler in columns
+                 if math.isfinite(getattr(row, f"{compiler}_{field}"))]
+    y_max = max(available, default=1.0) * 1.35
+    fig, axes = plt.subplots(1, 2, sharey=True, figsize=(14, 5.2),
+                             layout="constrained")
+    figure_legend(fig, results, field, absolute=True, raptor_only=raptor_only)
+
+    for ax, arch in zip(axes, ARCHITECTURES):
+        selected = [(row, compiler) for row, compiler in columns if row.arch == arch]
+        positions = []
+        for index, (row, _) in enumerate(selected):
+            if index == 0:
+                positions.append(0.0)
+                continue
+            previous = selected[index - 1][0]
+            gap = 0.8 if row.model != previous.model else 0.4 if row.mode != previous.mode else 0.0
+            positions.append(positions[-1] + 1.0 + gap)
+        ax.set_xlabel("ARCHITECTURE A" if arch == "arch-a" else "ARCHITECTURE B",
+                      fontsize=11, weight="bold", labelpad=8)
+        ax.set_axisbelow(True)
+        ax.set_ylim(0, y_max)
+        if positions:
+            ax.set_xlim(positions[0] - 0.9, positions[-1] + 0.9)
+        else:
+            ax.set_xlim(-0.9, 0.9)
+        label_groups(ax, [row for row, _ in selected], positions)
+        if {row.mode for row, _ in selected} == {"latency", "throughput"}:
+            centers, labels = [], []
+            for (_, mode), group in groupby(enumerate(selected), key=lambda item: (item[1][0].model, item[1][0].mode)):
+                group_positions = [positions[position] for position, _ in group]
+                centers.append((group_positions[0] + group_positions[-1]) / 2)
+                labels.append(mode.capitalize())
+            ax.set_xticks(centers, labels, minor=True)
+            ax.tick_params(axis="x", which="minor", length=0, pad=4, labelsize=8)
+        ax.set_ylabel(ylabel, fontsize=10.5)
+        if field == "throughput":
+            ax.ticklabel_format(axis="y", style="plain", useOffset=False)
+        ax.grid(axis="y", color="#E6EAF0", linewidth=0.8)
+        ax.tick_params(axis="y", length=0, pad=6)
+        ax.tick_params(axis="x", which="major", length=0,
+                       pad=22 if {row.mode for row, _ in selected} == {"latency", "throughput"} else 6)
+
+        for x, (row, compiler) in zip(positions, selected):
+            measurement = getattr(row, f"{compiler}_{field}") * scale
+            if not math.isfinite(measurement):
+                ax.text(x, y_max * 0.02, "N/A", rotation=90, ha="center",
+                        va="bottom", fontsize=8, color="#85909F")
+                continue
+            status = getattr(row, f"{compiler}_validation")
+            pattern = STAGE_PATTERNS[row.pipeline] if compiler == "raptor" else ""
+            ax.bar(x, measurement, width=1.0,
+                   color=COMPILER_COLORS[compiler],
+                   hatch=pattern,
+                   edgecolor=TEXT_COLOR if pattern else "none",
+                   linewidth=0.7 if pattern else 0.0, zorder=2)
+            label = (f"{measurement:,.2f}".rstrip("0").rstrip(".")
+                     if field == "throughput" else f"{measurement:.3g}")
+            value_label(ax, x, measurement + y_max * 0.012, label, status)
+
+    fit_value_labels(fig)
+    save_figure(fig, basename, output_dir)
+
+
+def draw_pipeline_effect(results: list[Result], output_dir: Path) -> None:
+    rows = [row for row in results if row.mode == "throughput" or row.pipeline == 1]
+    stages = sorted({row.pipeline for row in rows})
+    panels = [(model, color) for model, color in zip(MODELS, COLORS)
+              if any(row.model == model for row in rows)]
+    lookup = {(row.arch, row.model, row.pipeline): row for row in rows}
+    rates = {key: quotient(1000, row.raptor_latency) if row.mode == "latency" else row.raptor_throughput
+             for key, row in lookup.items()}
+    fig, axes = plt.subplots(len(ARCHITECTURES), len(panels), sharex=True, sharey="col",
+                             figsize=(3 * len(panels), 7.2), squeeze=False, layout="constrained")
+    for column, (model, color) in enumerate(panels):
+        for index, arch in enumerate(ARCHITECTURES):
+            ax = axes[index, column]
+            keys = [(arch, model, stage) for stage in stages]
+            values = [rates.get(key, math.nan) for key in keys]
+            valid = [key in lookup and lookup[key].raptor_validation == "PASS" for key in keys]
+            ax.plot(stages, values, color=color, linewidth=1.8)
+            for stage, value, passed in zip(stages, values, valid):
+                if math.isfinite(value):
+                    marker = ("s" if stage == 1 else "o") if passed else "x"
+                    ax.plot(stage, value, color=color, marker=marker,
+                            markersize=6, linestyle="none")
+            ax.set_xticks(stages)
+            ax.set_xlabel(model.upper(), fontsize=10.5, weight="bold", labelpad=8)
+            ax.grid(axis="y", color="#E6EAF0", linewidth=0.8)
+            ax.tick_params(axis="both", length=0, pad=6, labelbottom=True, labelleft=True)
+            ax.ticklabel_format(axis="y", style="plain", useOffset=False)
+            if column == 0:
+                ax.set_ylabel(f"{arch.replace('arch-', 'ARCHITECTURE ').upper()}\n"
+                              "Throughput (inferences/s)", fontsize=10.5)
+    for ax in axes.flat:
+        ax.set_ylim(bottom=0)
+    handles = [Line2D([], [], color="#657081", marker=marker, linestyle="none", label=label)
+               for marker, label in (("s", "Derived as 1000ms / latency_ms"),
+                                     ("o", "Throughput measurement"))]
+    if any(row.raptor_validation != "PASS" and math.isfinite(rates[key]) for key, row in lookup.items()):
+        handles.append(Line2D([], [], color="#424B58", marker="x", linestyle="none",
+                              label="Raptor validation not PASS"))
+    fig.legend(handles=handles, loc="outside upper center", ncol=len(handles), frameon=False, fontsize=9.5)
+    fig.supxlabel("Raptor pipeline stages", fontsize=11)
+    save_figure(fig, "raptor_pipeline_throughput", output_dir)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__)
+    script_dir = Path(__file__).resolve().parent
+    parser.add_argument("--input", type=Path, default=script_dir / "results_comparison.csv")
+    parser.add_argument("--output-dir", type=Path, default=script_dir / "plots")
+    args = parser.parse_args()
+    configure_style()
+    results = load_results(args.input)
+    valid = sum(row.comparison_passed for row in results)
+    print(f"Results: {len(results)} configurations, {valid} valid comparisons, "
+          f"{len(results)-valid} comparisons without two PASS statuses")
+    draw_validation(results, args.output_dir)
+    draw_pipeline_effect(results, args.output_dir)
+    yolo_only_models = {
+        model for model in MODELS if model.startswith("yolo")
+        and not any(math.isfinite(getattr(row, f"pimcomp_{field}"))
+                    for row in results if row.model == model
+                    for field in ("latency", "power", "energy"))
+    }
+    comparisons = [row for row in results if row.model not in yolo_only_models]
+    yolo_only = [row for row in results if row.model in yolo_only_models]
+    for metric, name, label, mode in [
+        ("latency_speedup", "longest_stage_latency", "Longest stage latency speedup (x)", None),
+        ("latency_speedup", "latency", "Single-inference latency speedup (x)", "latency"),
+        ("throughput_speedup", "throughput", "Throughput speedup (x)", "throughput"),
+        ("power", "power", "Average power advantage (x)", None),
+        ("energy", "energy", "Energy per inference advantage (x)", None),
+    ]:
+        selected = [row for row in comparisons if mode is None or row.mode == mode]
+        draw_metric(selected, metric, label,
+                    f"{name}_comparison", args.output_dir)
+        draw_metric(selected, metric, label,
+                    f"{name}_diverging", args.output_dir, diverging=True)
+    for metric, name, label, scale, mode in [
+        ("latency", "longest_stage_latency", "Longest stage latency (ms)", 1.0, None),
+        ("latency", "latency", "Single-inference latency (ms)", 1.0, "latency"),
+        ("power", "power", "Average power (mW)", 1.0, None),
+        ("energy", "energy", "Energy per inference (mJ)", 1e-9, None),
+        ("throughput", "throughput", "Throughput (inferences/s)", 1.0, "throughput"),
+    ]:
+        for suffix, group, raptor_only in (
+            ("absolute", comparisons, False), ("yolo", yolo_only, True)
+        ):
+            selected = [row for row in group if mode is None or row.mode == mode]
+            if not selected:
+                continue
+            draw_absolute(selected, metric, label,
+                          f"{name}_{suffix}", args.output_dir, scale,
+                          raptor_only=raptor_only)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,41 @@
+arch,model,mode,raptor_pipeline,pimcomp_pipeline,raptor_functional_validation,pimcomp_functional_validation,raptor_latency_ms,pimcomp_latency_ms,raptor_throughput_samples_s,pimcomp_throughput_samples_s,raptor_power_mw,pimcomp_power_mw,raptor_energy_pj,pimcomp_energy_pj,better_compiler,speedup
+arch-a,vgg8,latency,1,element,PASS,PASS,1.52,7.99,NA,NA,321.05,200.11,487455139.04,1597904071.12,raptor,5.26
+arch-a,vgg8,throughput,2,batch,PASS,PASS,0.91,1.04,1100.00,965.00,423.80,482.60,431626597.00,541691691.00,raptor,1.14
+arch-a,vgg8,throughput,4,batch,PASS,PASS,0.86,1.04,1160.00,965.00,432.99,482.60,416224904.20,541691691.00,raptor,1.20
+arch-a,vgg8,throughput,8,batch,PASS,PASS,1.63,1.04,615.00,965.00,308.12,482.60,616026711.90,541691691.00,pimcomp,1.57
+arch-a,resnet18,latency,1,element,PASS,FAIL,14.03,58.18,NA,NA,455.67,238.39,6394498463.12,13868866808.12,raptor,4.15
+arch-a,resnet18,throughput,2,batch,PASS,FAIL,11.73,13.46,85.20,74.30,503.11,481.60,6628073894.00,6563451004.00,raptor,1.15
+arch-a,resnet18,throughput,4,batch,PASS,FAIL,19.62,13.46,51.00,74.30,376.10,481.60,8024732847.00,6563451004.00,pimcomp,1.46
+arch-a,resnet18,throughput,8,batch,PASS,FAIL,19.92,13.46,50.20,74.30,368.24,481.60,9023016592.00,6563451004.00,pimcomp,1.48
+arch-a,resnet34,latency,1,element,PASS,FAIL,29.72,66.71,NA,NA,411.81,277.49,12238321706.68,18511425563.68,raptor,2.24
+arch-a,resnet34,throughput,2,batch,PASS,FAIL,22.26,23.58,44.90,42.40,473.84,515.68,16417303180.00,12283627410.00,raptor,1.06
+arch-a,resnet34,throughput,4,batch,PASS,FAIL,27.37,23.58,36.50,42.40,415.90,515.68,17019112320.00,12283627410.00,pimcomp,1.16
+arch-a,resnet34,throughput,8,batch,PASS,FAIL,41.04,23.58,24.40,42.40,346.15,515.68,17351854700.00,12283627410.00,pimcomp,1.74
+arch-a,googlenet,latency,1,element,PASS,PASS,9.78,62.92,NA,NA,563.88,231.19,5513234559.92,14547744283.24,raptor,6.44
+arch-a,googlenet,throughput,2,batch,PASS,FAIL,10.16,18.26,98.50,54.80,544.21,435.11,6029999329.00,8164787803.00,raptor,1.80
+arch-a,googlenet,throughput,4,batch,PASS,FAIL,11.96,18.26,83.60,54.80,473.16,435.11,6741923526.00,8164787803.00,raptor,1.53
+arch-a,googlenet,throughput,8,batch,PASS,FAIL,20.66,18.26,48.40,54.80,348.76,435.11,8421537146.00,8164787803.00,pimcomp,1.13
+arch-a,yolo11n,latency,1,element,PASS,FAIL,147.49,NA,NA,NA,290.17,NA,42797722129.00,NA,NA,NA
+arch-a,yolo11n,throughput,2,batch,PASS,FAIL,120.93,NA,8.27,NA,313.20,NA,41830757130.00,NA,NA,NA
+arch-a,yolo11n,throughput,4,batch,PASS,FAIL,92.06,NA,10.90,NA,351.03,NA,39451980450.00,NA,NA,NA
+arch-a,yolo11n,throughput,8,batch,PASS,FAIL,104.07,NA,9.61,NA,330.76,NA,44002712680.00,NA,NA,NA
+arch-b,vgg8,latency,1,element,PASS,PASS,1.53,7.15,NA,NA,292.56,173.77,448581679.04,1242814988.12,raptor,4.66
+arch-b,vgg8,throughput,2,batch,PASS,PASS,1.18,1.07,846.00,932.00,327.05,409.47,429183159.10,439288835.80,pimcomp,1.10
+arch-b,vgg8,throughput,4,batch,PASS,PASS,0.92,1.07,1090.00,932.00,383.82,409.47,464794094.50,439288835.80,raptor,1.17
+arch-b,vgg8,throughput,8,batch,PASS,PASS,1.62,1.07,616.00,932.00,276.49,409.47,524048148.70,439288835.80,pimcomp,1.51
+arch-b,resnet18,latency,1,element,PASS,FAIL,15.38,64.38,NA,NA,400.67,201.66,6160441550.12,12983432281.12,raptor,4.19
+arch-b,resnet18,throughput,2,batch,PASS,FAIL,13.08,13.43,76.40,74.40,440.79,484.80,6399942695.00,6569655551.00,raptor,1.03
+arch-b,resnet18,throughput,4,batch,PASS,FAIL,20.68,13.43,48.40,74.40,327.57,484.80,7777606323.00,6569655551.00,pimcomp,1.54
+arch-b,resnet18,throughput,8,batch,PASS,FAIL,21.17,13.43,47.20,74.40,317.64,484.80,8897039159.00,6569655551.00,pimcomp,1.58
+arch-b,resnet34,latency,1,element,PASS,FAIL,27.95,107.20,NA,NA,397.21,206.48,11103517952.68,22134174654.68,raptor,3.83
+arch-b,resnet34,throughput,2,batch,PASS,FAIL,22.28,22.85,44.90,43.80,438.24,493.33,14218683390.00,11392215660.00,raptor,1.03
+arch-b,resnet34,throughput,4,batch,PASS,FAIL,39.61,22.85,25.20,43.80,315.01,493.33,14629161660.00,11392215660.00,pimcomp,1.74
+arch-b,resnet34,throughput,8,batch,PASS,FAIL,49.34,22.85,20.30,43.80,285.76,493.33,16591020470.00,11392215660.00,pimcomp,2.16
+arch-b,googlenet,latency,1,element,PASS,PASS,11.22,37.94,NA,NA,483.27,242.04,5422799605.92,9182062336.24,raptor,3.38
+arch-b,googlenet,throughput,2,batch,PASS,FAIL,9.83,21.39,102.00,46.80,520.79,417.87,5669213312.00,9014726090.00,raptor,2.18
+arch-b,googlenet,throughput,4,batch,PASS,FAIL,14.47,21.39,69.10,46.80,401.01,417.87,6835460477.00,9014726090.00,raptor,1.48
+arch-b,googlenet,throughput,8,batch,PASS,FAIL,37.28,21.39,26.80,46.80,245.38,417.87,11453848400.00,9014726090.00,pimcomp,1.75
+arch-b,yolo11n,latency,1,element,PASS,FAIL,168.14,NA,NA,NA,245.31,NA,41248047514.00,NA,NA,NA
+arch-b,yolo11n,throughput,2,batch,PASS,FAIL,118.15,NA,8.46,NA,287.59,NA,37062981240.00,NA,NA,NA
+arch-b,yolo11n,throughput,4,batch,PASS,FAIL,94.18,NA,10.60,NA,316.45,NA,37424491250.00,NA,NA,NA
+arch-b,yolo11n,throughput,8,batch,PASS,FAIL,142.86,NA,7.00,NA,290.68,NA,41525990280.00,NA,NA,NA
@@ -4,6 +4,7 @@ from __future__ import annotations
 import argparse
 import csv
 import json
+import math
 import os
 import shlex
 import shutil
@@ -43,6 +44,7 @@ COMPARISONS = (
    ("throughput", 8, "batch"),
 )
 RESULTS_FILENAME = "results_comparison.csv"
+DEFAULT_RESULTS_PATH = Path(__file__).with_name(RESULTS_FILENAME)


@dataclass(frozen=True)
@@ -128,9 +130,9 @@ def write_results_csv(
    comparisons: tuple[tuple[str, int, str], ...] = COMPARISONS,
    ablation_variant: str | None = None,
 ) -> Path:
-    output = (root or SUITE) / RESULTS_FILENAME
+    output = DEFAULT_RESULTS_PATH if root is None else root / RESULTS_FILENAME
    if ablation_variant is not None:
-        output = output.parent / "ablation" / ablation_variant / RESULTS_FILENAME
+        output = (root or SUITE) / "ablation" / ablation_variant / RESULTS_FILENAME
    fields = (
        "arch",
        "model",
@@ -247,18 +249,26 @@ def write_results_csv(


 def performance_values(performance: dict) -> dict[str, float | None]:
+    fields = {"latency": ("latency_ms", "average_latency_ms"),
+              "throughput": ("throughput",), "power": ("average_power_mw",),
+              "energy": ("average_energy_pj",)}
+    values = dict.fromkeys(fields)
+    if performance.get("error") or performance.get("skipped") or performance.get("unsupported"):
+        return values
    parsed = parse_pimsim_nn_metrics(performance.get("raw_output", ""))
-    return {
-        "latency": (
-            performance.get("latency_ms")
-            or performance.get("average_latency_ms")
-            or parsed.get("latency_ms")
-            or parsed.get("average_latency_ms")
-        ),
-        "throughput": performance.get("throughput") or parsed.get("throughput"),
-        "power": performance.get("average_power_mw") or parsed.get("average_power_mw"),
-        "energy": performance.get("average_energy_pj") or parsed.get("average_energy_pj"),
-    }
+    for metric, names in fields.items():
+        for value in (source.get(name) for source in (performance, parsed) for name in names):
+            try:
+                number = float(value)
+            except (TypeError, ValueError):
+                continue
+            if math.isfinite(number) and number > 0:
+                values[metric] = number
+                break
+    if performance.get("output_count", parsed.get("output_count")) == 0:
+        for metric in ("latency", "throughput", "energy"):
+            values[metric] = None
+    return values


 def comparison_passed(report: dict) -> bool:
@@ -283,7 +293,9 @@ def comparison_winner(mode: str, raptor: float, pimcomp: float) -> str:


 def format_value(value: float | None) -> str:
-    return "NA" if value is None else f"{value:.2f}"
+    if value is None or not math.isfinite(value) or value <= 0:
+        return "NA"
+    return f"{value:.6g}"


 def print_stage(title: str, color: str) -> None:
@@ -352,15 +364,12 @@ def comparison_command(
    raptor_extra_args: list[str] | tuple[str, ...] = (),
    raptor_only: bool = False,
    reuse_raptor_report: Path | None = None,
-    reuse_pimcomp_dir: Path | None = None,
    reuse_pimcomp_report: Path | None = None,
 ) -> list[str]:
    time_args = ["--pimsim-time-ms", str(pimsim_time_ms)] if mode == "throughput" else []
    reuse_args = []
    if reuse_raptor_report is not None:
        reuse_args.extend(["--reuse-raptor-report", str(reuse_raptor_report)])
-    if reuse_pimcomp_dir is not None:
-        reuse_args.extend(["--reuse-pimcomp-dir", str(reuse_pimcomp_dir)])
    if reuse_pimcomp_report is not None:
        reuse_args.extend(["--reuse-pimcomp-report", str(reuse_pimcomp_report)])
    return [
@@ -430,6 +439,12 @@ def comparison_command_for(
    reuse_shared_pimcomp: bool,
 ) -> list[str]:
    report = spec.output_dir / "pimcomp/comparison_report.json"
+    reuse_pimcomp_report = None
+    if not args.raptor_only:
+        if args.only == "raptor":
+            reuse_pimcomp_report = report
+        elif reuse_shared_pimcomp:
+            reuse_pimcomp_report = spec.shared_pimcomp_dir.parent / "comparison_report.json"
    return comparison_command(
        spec.model,
        spec.functional_model,
@@ -450,20 +465,21 @@ def comparison_command_for(
            if args.only == "pimcomp"
            else None
        ),
-        reuse_pimcomp_dir=(
-            spec.shared_pimcomp_dir
-            if reuse_shared_pimcomp and args.only != "raptor" and not args.raptor_only
-            else None
-        ),
-        reuse_pimcomp_report=report if args.only == "raptor" and not args.raptor_only else None,
+        reuse_pimcomp_report=reuse_pimcomp_report,
    )


-def pimcomp_artifact_ready(output_dir: Path) -> bool:
-    return all(
-        (output_dir / name).is_file()
-        for name in ("SimulationInfo.gz", "VerificationInfo.json", "MappingResult.txt")
-    ) and (output_dir.parent / "comparison_report.json").is_file()
+def pimcomp_report_stamp(output_dir: Path) -> tuple[int, int, int] | None:
+    try:
+        stat = (output_dir.parent / "comparison_report.json").stat()
+    except FileNotFoundError:
+        return None
+    return stat.st_ino, stat.st_mtime_ns, stat.st_ctime_ns
+
+
+def pimcomp_report_ready(output_dir: Path, previous: tuple[int, int, int] | None) -> bool:
+    stamp = pimcomp_report_stamp(output_dir)
+    return stamp is not None and stamp != previous


 def run_comparison_jobs(
@@ -533,7 +549,7 @@ def main() -> int:
    parser.add_argument(
        "--out-dir",
        type=Path,
-        help="Suite root; generated artifacts go below each model's artifacts/ directory (default: validation/networks/pimcomp_models).",
+        help="Custom experiment root for artifacts and aggregate CSV. Default: model-suite artifacts and results_comparison.csv beside this script.",
    )
    parser.add_argument(
        "--common-dir",
@@ -687,11 +703,6 @@ def main() -> int:
                    required = comparison_dir / "pimcomp/comparison_report.json"
                    if not required.exists():
                        missing_reuse.append(str(required))
-                    elif args.only == "raptor":
-                        if json.loads(required.read_text(encoding="utf-8")).get(
-                            "pimcomp_model_source"
-                        ) != "original_onnx":
-                            missing_reuse.append(f"{required} (not generated from the original ONNX model)")
        if missing_reuse:
            parser.error(
                f"--only {args.only} needs existing comparison artifacts: "
@@ -713,7 +724,7 @@ def main() -> int:
        f"Comparison jobs: "
        f"{sum(len(args.models) * len(comparisons) for comparisons in comparisons_by_arch.values())}"
    )
-    print(f"Results root: {out_dir or SUITE}")
+    print(f"Artifacts root: {out_dir or SUITE}")
    print("=" * 72)

    print_stage("Prepare shared artifacts", STAGE_COLORS["Build runner"])
@@ -799,6 +810,10 @@ def main() -> int:
        print_directly = min(args.jobs, len(comparison_specs)) == 1
        try:
            with (nullcontext(None) if print_directly else TemporaryDirectory(prefix="raptor-pimcomp-")) as log_dir:
+                previous_reports = {
+                    spec.shared_pimcomp_dir: pimcomp_report_stamp(spec.shared_pimcomp_dir)
+                    for spec in anchor_specs
+                }
                anchor_failed, log_offset = run_comparison_jobs(
                    anchor_jobs,
                    args.jobs,
@@ -812,7 +827,9 @@ def main() -> int:
                        comparison_command_for(
                            spec,
                            args,
-                            reuse_shared_pimcomp=pimcomp_artifact_ready(spec.shared_pimcomp_dir),
+                            reuse_shared_pimcomp=pimcomp_report_ready(
+                                spec.shared_pimcomp_dir, previous_reports[spec.shared_pimcomp_dir]
+                            ),
                        ),
                        None,
                    )