rework documentation

add plot generation scripts
adjust validation tools
add up-to-date results

docs/evaluation/methodology.md

@@ -0,0 +1,75 @@
+# Evaluation methodology
+
+The [documentation index](../README.md) links execution guides, model provenance,
+reproduction details, and correctness experiments.
+
+## Canonical result datasets and output roots
+
+The authoritative default aggregate CSVs are:
+
+- [Comparison results](../../validation/tools/pim/pimcomp/compare/results_comparison.csv)
+- [Ablation results](../../validation/tools/pim/ablation/results_ablation.csv)
+
+They are intentionally checked in beside their scripts. Default benchmark runs
+update the corresponding canonical CSV; default plotting reads its current
+contents. Plot generation does not run benchmarks or establish publication
+provenance. Preserve the model/configuration/source/build revisions, invocation,
+reference inputs, and original per-case reports when publishing measurements.
+
+An explicit custom `--out-dir PATH` keeps aggregate CSVs under that experiment
+root. Model artifacts still default to `validation/networks/pimcomp_models`;
+changing aggregate placement does not change artifact or shared-reference paths.
+Per-variant ablation summaries remain `<artifact-root>/ablation/<variant>/results_comparison.csv`.
+Use `--input PATH` to plot a custom aggregate. Plot destinations can be selected
+with `--output-dir PATH`. Cleanup of default model artifacts preserves the two
+canonical datasets.
+
+## Functional validity and performance availability
+
+Functional PASS/FAIL and performance availability are independent. A numerically
+correct program can fail performance simulation or complete no inference before
+the deadline. Missing, failed, skipped, unsupported, nonfinite, or nonpositive
+per-inference measurements become `NA` in aggregates, not zero. With zero
+completed outputs, interval, throughput, and energy per inference are unavailable;
+measured average chip power can still be reported. Functional status is retained.
+
+Ablation percentages require positive finite values for both the variant and
+its matching baseline. Tiny positive measurements retain significant digits
+instead of being rounded to zero in the comparison CSV. An extreme valid
+decrease can round to `-100.00%`; that value alone is not evidence of failure.
+Reciprocal output interval and throughput cannot both have that decrease.
+The ablation plotter warns and displays `N/A` for such invalid paired percentages,
+while leaving the CSV and functional status unchanged.
+
+## Report reuse and implementation revisions
+
+The [comparison guide](../../validation/tools/pim/pimcomp/compare/README.md)
+describes fingerprint validation and failed-report reuse. Fingerprints cover
+model/configuration contents and experiment settings, but do not identify
+compiler/tool implementations, built binaries, or manually changed reference
+artifacts. Reuse is not proof of unchanged tool behavior: rerun the relevant
+compiler after those change, and record build/source revisions for publication.
+No revision cache is inferred from Git HEAD, which would miss dirty sources,
+external tools, and binaries built from another checkout.
+
+## Metric interpretation
+
+Metrics come from Pimsim's
+[`Chip::getSimulationReport`](../../backend-simulators/pim/pimsim-nn/src/chip/Chip.cpp)
+and [`EnergyCounter`](../../backend-simulators/pim/pimsim-nn/src/utils/EnergyCounter.cpp).
+
+- Latency mode measures time to complete one inference.
+- Throughput-mode latency is measurement duration / completed iterations,
+  equal to `1000 / throughput` before rounding. It is not end-to-end pipelined
+  inference latency; multiplying it by depth does not establish that quantity.
+- Average power is total modeled static + dynamic chip energy / elapsed time.
+- Energy per inference is total energy / completed iterations (one in latency
+  mode). Plots convert pJ to mJ with `1e-9`; no pipeline-depth or batch-size
+  division is applied.
+
+Fast mode measures throughput over the last two convergence windows but power
+and energy over the entire run up to early stop. Energy includes startup and
+work in flight, so it need not equal average power / reported throughput.
+Use `--no-fast` for full-duration measurements. Consult the
+[correctness notes](../../validation/tools/pim/pimcomp/correctness/README.md) before interpreting failed rows
+as performance comparisons.