From 034c1972607eb7f7f5352b131c85aa2c9e6db711 Mon Sep 17 00:00:00 2001 From: NiccoloN Date: Mon, 19 Oct 2026 16:34:01 +0200 Subject: [PATCH] rework documentation add plot generation scripts adjust validation tools add up-to-date results --- AGENTS.md | 5 + README.md | 379 ++----------- docs/README.md | 39 ++ docs/compiler/architecture.md | 51 ++ docs/compiler/options.md | 28 + docs/compiler/scheduling-implementation.md | 98 ++++ docs/compiler/scheduling.md | 101 ++++ docs/compiler/spatial-layouts.md | 63 +++ docs/evaluation/methodology.md | 75 +++ docs/evaluation/pimcomp-reproducibility.md | 111 ++++ requirements.txt | 1 + .../MergeComputeNodes/Scheduling/README.md | 180 +----- tools/raptor_graph_explorer/ARCHITECTURE.md | 131 +++++ tools/raptor_graph_explorer/README.md | 257 +++------ validation/README.md | 306 +++-------- validation/networks/pimcomp_models/README.md | 410 +++----------- validation/operations/INVENTORY.md | 286 ++++++++++ validation/operations/README.md | 321 +---------- validation/tests/test_pim_experiments.py | 393 +++++++++++++ validation/tests/test_pim_plots.py | 157 ++++++ validation/tools/pim/ablation/.gitignore | 1 + validation/tools/pim/ablation/README.md | 115 ++-- .../tools/pim/ablation/generate_plots.py | 156 ++++++ .../tools/pim/ablation/results_ablation.csv | 65 +++ validation/tools/pim/ablation/run_ablation.py | 24 +- .../tools/pim/pimcomp/compare/.gitignore | 1 + .../tools/pim/pimcomp/compare/README.md | 136 +++-- .../compare/compare_raptor_pimcomp_model.py | 68 ++- .../pim/pimcomp/compare/generate_plots.py | 514 ++++++++++++++++++ .../pimcomp/compare/results_comparison.csv | 41 ++ .../pim/pimcomp/compare/run_pimcomp_models.py | 89 +-- .../tools/pim/pimcomp/correctness/README.md | 136 ++--- validation/tools/pim/plot_style.py | 137 +++++ 33 files changed, 3077 insertions(+), 1798 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/compiler/architecture.md create mode 100644 docs/compiler/options.md create mode 100644 docs/compiler/scheduling-implementation.md create mode 100644 docs/compiler/scheduling.md create mode 100644 docs/compiler/spatial-layouts.md create mode 100644 docs/evaluation/methodology.md create mode 100644 docs/evaluation/pimcomp-reproducibility.md create mode 100644 tools/raptor_graph_explorer/ARCHITECTURE.md create mode 100644 validation/operations/INVENTORY.md create mode 100644 validation/tests/test_pim_experiments.py create mode 100644 validation/tests/test_pim_plots.py create mode 100644 validation/tools/pim/ablation/.gitignore create mode 100644 validation/tools/pim/ablation/generate_plots.py create mode 100644 validation/tools/pim/ablation/results_ablation.csv create mode 100644 validation/tools/pim/pimcomp/compare/.gitignore create mode 100755 validation/tools/pim/pimcomp/compare/generate_plots.py create mode 100644 validation/tools/pim/pimcomp/compare/results_comparison.csv create mode 100644 validation/tools/pim/plot_style.py diff --git a/AGENTS.md b/AGENTS.md index 1deb855..c3be3e1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -153,6 +153,11 @@ Do not refactor when: * When your changes create orphans, remove imports, variables, functions, or files made unused by your change * Every changed line should trace directly to the requested fix, a required cleanup, or a justified reuse/refactor decision +# Documentation + +* Do not create or retain agent change reports, investigation logs, debugging histories, temporary benchmark audits, or task-completion summaries in repository documentation. Report these findings in the final response instead. +* Documentation should describe the current system, its contracts, usage, and reproducible methodology. Document lasting limitations where relevant, without narrating how they were discovered or fixed. + # Diagnostics and verification * Use existing bounded diagnostic mechanisms for pass-level verification or codegen failures diff --git a/README.md b/README.md index 20c6a91..94c88cb 100644 --- a/README.md +++ b/README.md @@ -1,215 +1,64 @@ # Raptor -Raptor is a domain-specific MLIR compiler for neural networks in ONNX format, -targeting in-memory computing / processing-in-memory (PIM) architectures. It -extends ONNX-MLIR with a PIM accelerator and progressively lowers ONNX-MLIR -through custom MLIR dialects to simulator artifacts. +Raptor is an MLIR compiler for ONNX neural networks targeting processing-in-memory +(PIM) architectures. It extends ONNX-MLIR and emits per-core `.pim` binaries, +`memory.bin`, `config.json`, and weights for the simulators under +`backend-simulators/pim/`. Optional JSON instructions use `--pim-emit-json`. -The current target is the Pim simulator stack under `backend-simulators/pim`. -Raptor emits binary per-core `.pim` instruction files by default, plus -`memory.bin`, `config.json`, and weight binaries. It can also emit per-core JSON -instruction files with `--pim-emit-json`. +The target has shared host memory, PIM cores with ReRAM crossbars, explicit +inter-core communication, and no hardware branches or loops. Lowering, +scheduling, and memory planning keep repeated work compact until codegen. -## Overview +## Build -PIM architectures perform most computation directly in memory. The supported -target models a chip with: -- shared host memory, -- multiple PIM cores, -- ReRAM crossbars for vector-matrix / matrix-vector work, -- explicit communication between cores, -- no hardware branch or loop support in emitted simulator code. +Initialize submodules: -Because repeated work such as convolutions is eventually made explicit, emitted -instruction counts can grow quickly. Most compiler work therefore focuses on -lowering, scheduling, memory layout, and code-generation optimizations. - -### Targets and simulators - -- `backend-simulators/pim/pim-simulator` is the in-tree Rust functional - simulator used by validation. It reads Raptor's `pim/` artifact directory and - compares simulator output against native ONNX-MLIR execution. -- `backend-simulators/pim/pimsim-nn` contains the non-functional Pimsim - simulator used internally by validation for latency, power, and energy. - The helper scripts in `pimcomp_utils/` are for comparison with Pimcomp and - contain local paths; treat them as local utilities, not portable workflows. - -## Compilation pipeline - -The PIM sources live under `src/PIM` and tests under `test/PIM`. CMake exposes -them to ONNX-MLIR through generated shim directories under -`onnx-mlir/src/Accelerators/PIM` and `onnx-mlir/test/accelerators/PIM`. - -High-level lowering flow: - -``` -ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> Pim artifacts +```bash +git submodule update --init --recursive ``` -1. **ONNX -> Spatial** (`src/PIM/Conversion/ONNXToSpatial`). - Lowers supported ONNX ops into the `spat` dialect - (`src/PIM/Dialect/Spatial`). Conversion patterns are split by op family under - `Patterns/{Math,NN,Tensor}` and currently cover Conv, Gemm, MatMul, - elementwise Add/Mul/Div, ReduceMean, pooling, Relu, Sigmoid, Softmax, - Concat, Gather, Reshape, Resize, and Split. - The compiler-layer target adapter supplies the target-neutral - `SpatialTargetResources`. Layout-aware plan ops advertise typed alternatives - through the Spatial layout interface; the layout planner records the - selected layout and explicit materialization edges. `LowerSpatialPlans` - then pattern-lowers those selected plans. Contraction and Conv lowering - keep semantic problems, target-dependent plans, and IR materializers in - separate layers. Passes and their invariant/layout analyses live under - `Passes/Transforms` and `Passes/Analyses`. +Follow [ONNX-MLIR's LLVM/MLIR build instructions](onnx-mlir/docs/BuildOnLinuxOSX.md). +The [CI workflow](.github/workflows/validate_operations.yml) records dependency +versions and setup, including CMake and Protobuf. Rust stable is required for +the functional simulator. Python requirements include the Graph Explorer and +need Python 3.13+; create the repository environment if absent, then install: -2. **Merge, schedule, and realize Spatial communication** - (`src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes`). - `TrivialGraphComputeMerge` performs local graph merging. One - `ScheduleAndRealizeSpatial` pass then owns scheduling, intermediate - verification, communication realization, and final verification. Supporting - scheduling code lives under `MergeComputeNodes/Scheduling`. +```bash +python3.13 -m venv .venv +.venv/bin/python -m pip install -r requirements.txt +``` -3. **Spatial -> Pim** (`src/PIM/Conversion/SpatialToPim`). - Lowers Spatial operations to the `pim` dialect (`src/PIM/Dialect/Pim`), - including `pim.core`, `pim.core_batch`, communication, tensor packing, global - tensor materialization, and return-path normalization. +If compatible Protobuf libraries are unavailable locally: -4. **Bufferization** (`src/PIM/Dialect/Pim/Passes/Transforms/Bufferization`). - `PimBufferizationPreparation` establishes writable destinations without - duplicating the one-shot copy analysis, `PimOneShotBufferization` runs - MLIR's one-shot analysis, - `PimMemoryNormalization` forwards/removes redundant copies and normalizes - addressable accesses, and `PimBufferizationVerification` checks tensor - absence, contiguity, and copy address spaces. +```bash +git clone --depth 1 --branch v34.0 https://github.com/protocolbuffers/protobuf +cmake -S protobuf -B protobuf/build -G Ninja \ + -DCMAKE_BUILD_TYPE=Release -Dprotobuf_BUILD_TESTS=OFF +cmake --build protobuf/build +sudo cmake --install protobuf/build +``` -5. **Pim local-memory planning** - (`src/PIM/Dialect/Pim/Passes/Transforms/LocalMemoryPlanning`). - Computes whole-core lifetimes, reuses addresses for non-overlapping - allocations, and records the explicit plan in Pim IR. Reusable lifetime - analysis lives under `src/PIM/Dialect/Pim/Passes/Analyses`. -6. **Pim verification and code generation** (`src/PIM/Passes/PimCodegen` and - `src/PIM/Compiler`). - Verifies the memory plan and other Pim invariants, then emits `.pim` core - files, weights, and `memory.bin` / `config.json` without rerunning liveness. +Configure and build Raptor, adjusting `MLIR_DIR` to your LLVM build: -Supporting pieces: -- `src/PIM/Common` - shared IR, filesystem, diagnostics, reports, and utility - helpers. -- `src/PIM/Compiler` - Pim compiler options, planned-address materialization, binary - instruction format, artifact writing, weight emission, and codegen entry - points. -- `src/PIM/Conversion/SpatialToGraphviz` - optional Spatial graphviz conversion - pass. -- `src/PIM/Passes` - pass registration and auxiliary passes. -- `src/PIM/PimAccelerator.{cpp,hpp}` - ONNX-MLIR accelerator entry point. +```bash +cmake -S . -B build_release -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DONNX_MLIR_ACCELERATORS=PIM \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DMLIR_DIR="$PWD/onnx-mlir/llvm-project/build_release/lib/cmake/mlir" +cmake --build ./build_release +``` -## Pim compiler options +Use release builds first. For debugging, configure `build_debug` with +`-DCMAKE_BUILD_TYPE=Debug` and the matching MLIR package, then run +`cmake --build ./build_debug`. Use debug only for small cases requiring symbols. +Optional `mold` linking uses `-fuse-ld=mold` in +`CMAKE_{EXE,SHARED,MODULE}_LINKER_FLAGS`. -Pass these to `onnx-mlir` when compiling for Pim. These are all Raptor/Pim-specific -options; `onnx-mlir --help` lists the inherited ONNX-MLIR options. +Always build through `cmake --build`; invoking `ninja` directly bypasses CMake +configuration and generated shim updates. -- `--maccel=PIM` - select the Pim accelerator. Default: no Pim accelerator. -- `--EmitSpatial`, `--EmitPim`, `--EmitPimBufferized`, - `--EmitPimCodegen` - stop the Pim pipeline at the requested stage. Default: - `--EmitPimCodegen` for Pim compilation. -- `--core-count=` - required positive core count for Pim compilation. - Default: none; this option is required. -- `--crossbar-size=` - required positive crossbar width/height for Pim - compilation. Default: none; this option is required. -- `--crossbar-count=` - required positive crossbar count per core for Pim - compilation. Default: none; this option is required. -- `--pipeline=` - number of throughput pipeline stages; `1` preserves - latency scheduling. Default: `1`. -- `--pim-target-config=` - optional Pim target configuration used by the - target adapter to construct the target-neutral Spatial scheduling cost and - topology model. Resource values must match the explicit core/crossbar flags. - Default: empty; use the built-in target model. -- `--pim-memory-report=` - emit the concise combined memory report - under `reports/memory_report.txt`, or disable it. Default: `summary`. -- `--pim-only-codegen` - assume input is already bufferized Pim IR and only run - the codegen tail. Default: off. -- `--pim-disable-synchronization` - omit generated `wait` and `sync` - instructions for performance ablation. Default: off. -- `--pim-disable-spatial-planning` - select the first, trivial DenseNCHW layout - alternative for every Spatial plan operation, disabling cost-based layout - planning while leaving ONNX rewrites and graph-compute merging enabled. - Default: off. - -### Spatial layout plan variants - -Spatial plan operations advertise alternatives as an exact combination of -operand physical layouts and one result physical layout. Every plan operation -has the default `DenseNCHW -> DenseNCHW` alternative. The planner can select -the following additional variants when the operation, tensor shapes, and -target resources make them legal: - -| Physical layout or plan | Meaning and current use | -|---|---| -| `DenseNCHW` | Ordinary dense NCHW storage. This is the first alternative and the one selected by `--pim-disable-spatial-planning`. | -| `NHWCRowStrip` | Row-strip storage for NCHW logical tensors: spatial rows are processed as channel vectors. This enables row-strip lowering through compatible chains. | -| `Fragmented` | Fragmented physical input accepted by `Flatten`, which reassembles it to dense NCHW. It is not currently selected as a plan result. | -| `NCHWRowStrip` | A Spatial IR layout enum value reserved for NCHW-oriented row strips; current layout-capability implementations do not advertise it as a plan alternative. | - -The operation-specific non-trivial alternatives are: - -| Plan operation | Additional alternatives beyond dense NCHW | -|---|---| -| `Conv2D` | Dense input to row-strip output, or row-strip input to row-strip output when the target-dependent Conv lowering supports it. | -| `Flatten` | Fragmented input to dense output, or row-strip input to dense output when legal. | -| `Relu` | Row-strip input to row-strip output. | -| `SiLU` | Row-strip input to row-strip output, with a stronger intrinsic cost preference than the generic row-strip variant. | -| `ResizeNearest` | Row-strip input to row-strip output when its lowering is legal. | -| `MaxPool2D` | Dense input to row-strip output, or row-strip input to row-strip output. | -| `GlobalAveragePool` | Dense input to row-strip output, or row-strip input to row-strip output. | -| `BiasAdd` | Row-strip data input plus a dense bias input to row-strip output when the bias shape is supported. | -| `Add` | All data inputs row-strip to row-strip output. | -| `Concat` | All inputs row-strip to row-strip output. | - -Cost-based planning scores intrinsic alternative cost, operand layout -mismatches, and downstream incompatibility, then iterates in alternating -forward and reverse operation order until the bounded analysis converges. -Function results are required to remain `DenseNCHW`; explicit materialization -operations reconcile layout mismatches at boundaries. With -`--pim-disable-spatial-planning`, the pass still runs and records a valid plan, -but chooses the first dense alternative for every plan operation. Later graph -compute merging is unchanged, so elementwise operations such as `Relu` remain -separate from neighboring parallel operations and can create fan-out/fan-in -diamonds. -- `--pim-emit-json` - also emit `core_*.json` instruction files alongside - `core_*.pim`. Default: off. -- `--pim-export-spatial-dataflow=` - - control Spatial dataflow CSV reports for the graph, trivially merged graph, - scheduled, and realized snapshots under `reports/`. Default: `none`. -- `--pim-conv-lowering=` - - select the convolution lowering strategy. Default: `auto`. -- `--pim-conv-im2col-max-elements=` - maximum globally materialized im2col - elements per convolution before streaming. Default: `1048576`. -- `--pim-conv-stream-chunk-positions=` - maximum output positions per - streamed convolution chunk. Default: `1024`. -- `--pim-report-conv-lowering=` - emit a bounded convolution - lowering report. Default: `true`. -- `--pim-detect-communication-deadlock` - statically simulate expanded - send/receive ordering and reject blocking deadlocks. Default: off. -- `--pim-verify-bufferization-copy-freedom` - run the expensive official Pim - tensor-copy freedom proof before bufferization. Default: off. - -## Standard Pim hardware profile - -Raptor's standard development and YOLO validation profile is: - -| Parameter | Value | -| --- | ---: | -| Cores | 144 | -| Crossbars per core | 64 | -| Crossbar size | 128 × 128 | - -Canonical compiler flags: - -`--crossbar-count=64 --crossbar-size=128 --core-count=144` - -`--crossbar-size`, `--crossbar-count`, and `--core-count` remain mandatory and -must be passed explicitly to the compiler. - -Example: +## Compile a model ```bash ./build_release/Release/bin/onnx-mlir model.onnx -o /tmp/raptor/model \ @@ -217,131 +66,19 @@ Example: --crossbar-count=64 --crossbar-size=128 --core-count=144 ``` -This writes Pim artifacts under `/tmp/raptor/pim/`. +This writes artifacts under `/tmp/raptor/pim/`. All three hardware flags are +mandatory positive values. The example uses the development/YOLO profile; +[Pimcomp comparison profiles](validation/networks/pimcomp_models/README.md#hardware-profiles) +differ, and validation defaults to Arch-A. -## Validation +## Documentation -Functional validation compiles ONNX models, compares native ONNX-MLIR and Pim -simulator outputs, and optionally reports latency, power, and energy. See -[`validation/README.md`](validation/README.md) for prerequisites, usage, -options, artifacts, and results. +Start with the [documentation index](docs/README.md). It links the +[compiler architecture](docs/compiler/architecture.md), +[compiler options](docs/compiler/options.md), and +[evaluation methodology](docs/evaluation/methodology.md). -## Build - -Initialize submodules first: - -```bash -git submodule update --init --recursive -``` - -The project follows ONNX-MLIR's build requirements. The CI workflow documents -the currently used versions and setup: -- CMake 4.3.0 in CI, -- LLVM/MLIR checked out under `onnx-mlir/llvm-project`, -- Protobuf `v34.0`, -- Rust stable for `pim-simulator`, -- Python packages `numpy`, `onnx`, `colorama` for validation. - -### Protobuf - -Install Protobuf if your system does not already provide a compatible version: - -```bash -git clone --depth 1 --branch v34.0 https://github.com/protocolbuffers/protobuf -cmake -S protobuf -B protobuf/build -G Ninja \ - -DCMAKE_BUILD_TYPE=Release \ - -Dprotobuf_BUILD_TESTS=OFF -cmake --build protobuf/build -sudo cmake --install protobuf/build -``` - -You can then remove the temporary checkout: - -```bash -rm -rf protobuf -``` - -### MLIR - -Follow the ONNX-MLIR instructions in -`onnx-mlir/docs/BuildOnLinuxOSX.md` to build LLVM/MLIR. The local Raptor build -expects `MLIR_DIR` to point at the MLIR CMake package, for example: - -```bash -MLIR_DIR=$(pwd)/onnx-mlir/llvm-project/build_release/lib/cmake/mlir -``` - -If your LLVM build directory is named `build` instead of `build_release`, adjust -the path accordingly. - -### Raptor - -Configure a release build: - -```bash -MLIR_DIR=$(pwd)/onnx-mlir/llvm-project/build_release/lib/cmake/mlir -cmake -S . -B build_release -G Ninja \ - -DCMAKE_BUILD_TYPE=Release \ - -DONNX_MLIR_ACCELERATORS=PIM \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DMLIR_DIR=${MLIR_DIR} -``` - -Configure a debug build similarly: - -```bash -MLIR_DIR=$(pwd)/onnx-mlir/llvm-project/build_debug/lib/cmake/mlir -cmake -S . -B build_debug -G Ninja \ - -DCMAKE_BUILD_TYPE=Debug \ - -DONNX_MLIR_ACCELERATORS=PIM \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DMLIR_DIR=${MLIR_DIR} -``` - -For debug development, using `mold` can reduce link time and memory use: - -```bash -cmake -S . -B build_debug -G Ninja \ - -DCMAKE_BUILD_TYPE=Debug \ - -DONNX_MLIR_ACCELERATORS=PIM \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DMLIR_DIR=${MLIR_DIR} \ - -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=mold" \ - -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=mold" \ - -DCMAKE_MODULE_LINKER_FLAGS="-fuse-ld=mold" -``` - -Build the compiler with CMake: - -```bash -cmake --build ./build_release -cmake --build ./build_debug -``` - -Do not invoke `ninja` directly for this project; use `cmake --build` so CMake's -configuration and generated shims stay consistent. - -If a build fails because Protobuf headers are missing fixed-width integer -definitions, patch the affected Protobuf-generated files by adding -`#include `. - -## Tests - -The Rust simulator has its own tests: - -```bash -cd backend-simulators/pim/pim-simulator -cargo test -``` - -## Repository layout - -- `src/PIM/` - PIM accelerator implementation. -- `test/PIM/` - PIM C++ unit tests. -- `validation/` - functional validation scripts, ONNX operation tests, network - slices, and pimsim config generation. -- `backend-simulators/pim/pim-simulator/` - in-tree Rust functional simulator. -- `backend-simulators/pim/pimsim-nn/` - non-functional simulator submodule. -- `pimcomp_utils/` - local comparison helpers for Pimcomp. -- `.github/actions/` and `.github/workflows/validate_operations.yml` - CI setup - for MLIR/Protobuf caching, building Raptor, and validation. +[Validation commands](validation/README.md), the +[operation inventory](validation/operations/INVENTORY.md), and the +[Graph Explorer](tools/raptor_graph_explorer/README.md) remain beside their +components. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..7ae6446 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,39 @@ +# Raptor documentation + +Start with the [build and quick start](../README.md). Commands in these guides +run from the repository root unless stated otherwise. + +## Compiler + +- [Architecture, compilation pipeline, source layout, and simulator tests](compiler/architecture.md) +- [Pim compiler options and defaults](compiler/options.md) +- [Spatial physical layouts and compatibility matrix](compiler/spatial-layouts.md) +- [Scheduling ownership, costs, capacity, and complexity](compiler/scheduling.md) +- [Scheduling implementation contracts and model limitations](compiler/scheduling-implementation.md) + +## Evaluation + +- [Measurement methodology and canonical result datasets](evaluation/methodology.md) +- [Pimcomp operation support, standalone reproduction, and GA controls](evaluation/pimcomp-reproducibility.md) +- [Validation commands and defaults](../validation/README.md) +- [Pimcomp comparison and plotting](../validation/tools/pim/pimcomp/compare/README.md) +- [Raptor ablation and plotting](../validation/tools/pim/ablation/README.md) +- [Pimcomp correctness experiments](../validation/tools/pim/pimcomp/correctness/README.md) + +## Component references + +- [Operation suite](../validation/operations/README.md) and [all 177 cases](../validation/operations/INVENTORY.md) +- [Comparison models, provenance, checksums, and hardware profiles](../validation/networks/pimcomp_models/README.md) +- [Graph Explorer usage](../tools/raptor_graph_explorer/README.md) and [architecture/API](../tools/raptor_graph_explorer/ARCHITECTURE.md) + +## Normative engineering contracts + +These contracts stay in `.agents/invariants/`; general documentation links to +them rather than restating their requirements: + +- [Graph compute batch fragments](../.agents/invariants/GRAPH_COMPUTE_BATCH_INVARIANT.md) +- [Compiler performance](../.agents/invariants/PERFORMANCE_OPTIMIZATION_INVARIANT.md) +- [Pimsim performance oracle](../.agents/invariants/PIMSIM_NN_ORACLE_INVARIANT.md) +- [PIM synchronization](../.agents/invariants/PIM_SYNCHRONIZATION_INVARIANT.md) +- [Pipeline scheduling](../.agents/invariants/PIPELINE_SCHEDULING_INVARIANT.md) +- [Spatial target generality](../.agents/invariants/SPATIAL_TARGET_GENERALITY_INVARIANT.md) diff --git a/docs/compiler/architecture.md b/docs/compiler/architecture.md new file mode 100644 index 0000000..4ee4321 --- /dev/null +++ b/docs/compiler/architecture.md @@ -0,0 +1,51 @@ +# Compiler architecture and pipeline + +See the [documentation index](../README.md) for related compiler and evaluation +references. Source paths below are relative to the repository root. + +## Compilation pipeline + +```text +ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> Pim artifacts +``` + +| Stage | Responsibility and source | +|---|---| +| ONNX → Spatial | Operation lowering, target-neutral layout planning, and plan materialization in `src/PIM/Conversion/ONNXToSpatial`. | +| Spatial scheduling | Graph merging, scheduling, verification, and communication realization in `src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes`; see [scheduling ownership](scheduling.md). | +| Spatial → Pim | Cores, communication, packing, and global tensors in `src/PIM/Conversion/SpatialToPim`. | +| Bufferization | Writable destinations, one-shot analysis, memory normalization, and verification in `src/PIM/Dialect/Pim/Passes/Transforms/Bufferization`. | +| Local-memory planning | Whole-core lifetimes and address reuse in `src/PIM/Dialect/Pim/Passes/Transforms/LocalMemoryPlanning`. | +| Verification/codegen | Verify the explicit memory plan and emit artifacts in `src/PIM/Passes/PimCodegen` and `src/PIM/Compiler`. | + +Spatial plans select physical layouts and insert explicit materializations at +boundaries. Dense NCHW is the default; compatible operations can use row-strip +layouts, while Flatten can reassemble fragmented inputs. Function results +remain dense NCHW. See the [layout compatibility reference](spatial-layouts.md) +for operation alternatives, legality checks, and planner behavior. + +## Validation and tests + +[Validation instructions](../../validation/README.md) cover ONNX-MLIR reference +comparison, functional simulation, performance measurement, and artifacts. +The Rust functional simulator reads Raptor's artifact directory; +`pimsim-nn` supplies latency, power, and energy estimates. + +```bash +cd backend-simulators/pim/pim-simulator +cargo test +``` + +## Repository layout + +- `src/PIM/`: accelerator, dialects, conversions, analyses, and codegen. +- `test/PIM/`: compiler unit tests. +- `validation/`: operation/network suites and simulator configuration. +- `backend-simulators/pim/pim-simulator/`: Rust functional simulator. +- `backend-simulators/pim/pimsim-nn/`: performance simulator submodule. +- `tools/raptor_graph_explorer/`: [Spatial CSV viewer](../../tools/raptor_graph_explorer/README.md). +- `pimcomp_utils/`: comparison helpers containing local paths. +- `.github/`: CI build and validation setup. + +CMake exposes `src/PIM` and `test/PIM` through generated shims under +`onnx-mlir/src/Accelerators/PIM` and `onnx-mlir/test/accelerators/PIM`. diff --git a/docs/compiler/options.md b/docs/compiler/options.md new file mode 100644 index 0000000..8e128ba --- /dev/null +++ b/docs/compiler/options.md @@ -0,0 +1,28 @@ +# Pim compiler options + +See the [quick start](../../README.md#compile-a-model) for a complete command +and the [documentation index](../README.md) for the compilation pipeline. + +Pass these to `onnx-mlir`; `--help` also lists inherited ONNX-MLIR options. +Boolean switches default to off unless stated otherwise. + +| Option | Purpose / default | +|---|---| +| `--maccel=PIM` | Select the PIM accelerator. | +| `--EmitSpatial`, `--EmitPim`, `--EmitPimBufferized`, `--EmitPimCodegen` | Stop at a pipeline stage; default: codegen. | +| `--core-count=N`, `--crossbar-size=N`, `--crossbar-count=N` | Required hardware resources; crossbars are square. | +| `--pipeline=N` | Throughput stages; default: 1 (latency scheduling). | +| `--pim-target-config=PATH` | Target cost/topology configuration; resources must match hardware flags. Default: built-in model. | +| `--pim-memory-report=summary\|none` | Memory report at `reports/memory_report.txt`; default: `summary`. | +| `--pim-only-codegen` | Run the codegen tail on bufferized Pim IR. | +| `--pim-emit-json` | Also emit per-core JSON instructions. | +| `--pim-export-spatial-dataflow=none\|spatial1\|spatial2\|spatial3\|spatial4\|all` | Export graph, trivially merged, scheduled, and realized CSV snapshots under `reports/`; default: `none`. | +| `--pim-conv-lowering=STRATEGY` | `auto` (default), `legacy`, `depthwise`, `packed-im2col`, `streamed-patch`, `streamed-packed`, `output-channel-tiled`, `input-k-tiled`, or `tiled-2d`. | +| `--pim-conv-im2col-max-elements=N` | Global im2col limit before streaming; default: 1048576. | +| `--pim-conv-stream-chunk-positions=N` | Output positions per streamed chunk; default: 1024. | +| `--pim-report-conv-lowering=true\|false` | Bounded lowering report; default: true. | +| `--pim-disable-spatial-planning` | Choose each plan's first dense alternative; ONNX rewrites and graph merging remain enabled. | +| `--pim-disable-synchronization` | Omit wait/sync for ablation; removes correctness guarantees for buffer reuse and restart. | +| `--pim-detect-communication-deadlock` | Check expanded blocking send/receive ordering. | +| `--pim-verify-bufferization-copy-freedom` | Run the expensive pre-bufferization copy-freedom proof. | + diff --git a/docs/compiler/scheduling-implementation.md b/docs/compiler/scheduling-implementation.md new file mode 100644 index 0000000..3783f84 --- /dev/null +++ b/docs/compiler/scheduling-implementation.md @@ -0,0 +1,98 @@ +# Scheduling implementation notes + +The [scheduling overview](scheduling.md) maps ownership and summarizes costs/complexity. +Normative graph, pipeline, synchronization, and target contracts are linked +there; the details below describe current implementation choices. + +## Publication, order, and local materialization + +[`materializeScheduledCompute`](../../src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledComputeMaterialization.cpp) records +graph IDs for publication identity, not execution order. Class lanes retain +graph encounter order independently of physical core IDs, preserving regular +source-lane mappings. Within each core, instances sort by verified scheduler +slots. Only consecutive compatible instances form homogeneous runs; class-lane +ordering must not reorder a core's work. + +`insertLocalFragment` replaces a complete singleton insertion with a leading +unit-dimension reshape view only when: + +- source and destination have static shapes; +- destination rank is source rank + 1, with leading dimension 1 and remaining + dimensions exactly equal to the source shape; +- element type and encoding match; +- offset/size/stride lists all match destination rank; and +- every offset is statically zero, every size is the full corresponding + destination dimension, and every stride is statically one. + +This includes scalar sources. Other insertions retain `tensor.insert_slice` +semantics rather than being converted into views. + +Receive forwarding belongs to +[`PimBufferizationPass.cpp`](../../src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp), +after copy lowering and contiguity normalization. It requires a private +allocation and single-use receive/copy chain in the same block, zero source and +receive offsets, and equal sizes. The receive can sink to the copy target past +memory-effect-free operations and `memref.load` (including lane lookup loads). +Writes, synchronization, and other effectful operations are barriers. Forwarding +does not establish synchronization policy. + +## Transfer accounting and transpose costs + +[`ComputeGraph.cpp`](../../src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/ComputeGraph.cpp) uses the same cached publication map for +dependency extraction and projected transfer volume. Physical slots map back +to graph lanes; graph-range overlap is not a byte estimate. Blueprint edges +belong to this shared graph for both scheduling strategies. + +Exact projections charge selected physical fragments individually, including +each fragment's byte, endpoint-memory, and packet rounding. Conservative batch +dependencies charge a producer chunk as one aggregate: scale element count by +chunk lanes, multiply by effective bitwidth, then round to bytes once before +memory-access/flit rounding. Dividing an already rounded whole-result byte +count would lose sub-byte payloads. These two granularities are deliberate. +Spatial can analyze sub-byte storage; PIM communication lowering requires +byte-sized elements. Blueprint assembly charges per fragment. A full fragment +receive directly into assembly storage omits duplicate endpoint-memory edge +cost; later copy elimination can leave this estimate conservative. + +Execution transpose costs share `getTransposeRun` with lowering: round memory +reads/writes per run and add vector execution when source stride is not one. +The [Pimsim vector unit](../../backend-simulators/pim/pimsim-nn/src/core/component/VectorUnit.cpp) +charges vector payload bytes, not the full strided address footprint. The +compiler model follows that accounting; instruction setup/issue and contention +remain approximations, not extra inferred memory traffic. + +## Priorities, placement, and recovery limits + +[`PipelineScheduling.cpp`](../../src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.cpp) caches a per-template +adjustment between physical execution cost and nominal volume priority, scaling +it by chunk lanes. Partitioning and packing use the nominal priorities; +transposes count as bulk moves there. PEFT, final schedule evaluation, and +reported per-core demand use physical estimates. These quantities must not be +substituted for one another when interpreting results. + +Placement compares maximum per-core serial compute plus incident same-stage +transfer demand, then completion/span and compute load. Each participating core +is charged incident transfer demand as a placement proxy; those charges are not +summed twice into elapsed time. Within 1% demand difference, span breaks the tie. +Ordered packing plus dependency-tail priority biases cuts away from long +dependent groups; it is not a throughput prediction. Measured simulator periods +remain authoritative, including when a proxy improves. + +Core budgets come from stage allocation and bounded capacity recovery. Failed +packing budgets are not lower bounds. [PEFT reservations](../../src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PeftScheduler.cpp) +prefer weight reuse when distinct weights exceed half the fleet's one-copy +resident capacity; otherwise they favor communication/load. The reuse-first +constructor supplies feasibility recovery without replacing a successful +reservation, and can concentrate work. No exact combinatorial solver proves +infeasibility after heuristic failure. + +[`ResidentWeightPacking.hpp`](../../src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/ResidentWeightPacking.hpp) adds at most 17 +constructions after initial failure: one reverse traversal, then up to 16 +single-choice replays sharing 65,536 candidate checks. Scratch buffers are reused; +an extra `O(T)` failed-assignment snapshot is created only after both initial +orders fail. Weight indexing still has `O(R W)` worst-case equality work; partition +DP and latency insertion-gap searches remain superlinear as listed in the +scheduling overview. Cached PEFT center distances avoid a topology scan per task/candidate; +projection and priority analyses share/cache work rather than revisiting every +template body per chunk. Ready-transfer signature searches remain another +potential compile-time hotspot. diff --git a/docs/compiler/scheduling.md b/docs/compiler/scheduling.md new file mode 100644 index 0000000..5e102e8 --- /dev/null +++ b/docs/compiler/scheduling.md @@ -0,0 +1,101 @@ +# Spatial scheduling + +`ScheduleAndRealizeSpatial` constructs a shared `ComputeGraph`, then selects +latency PEFT or pipeline scheduling. Pipeline mode does not run PEFT. + +See [implementation notes](scheduling-implementation.md) for fragment-view eligibility, +receive forwarding, transfer rounding, cost-model boundaries, and recovery limits. + +## Ownership + +| Responsibility | Owner | +|---|---| +| Instances, dependencies, execution/transfer costs, resident weights | `ComputeGraph`, `ComputeInstanceUtils` | +| Target resources and physical link costs | Compiler target adapter via `SchedulingTarget` | +| Latency priorities, reservations, placement, per-core order | `PeftScheduler` | +| Capacity-driven workload splitting | `PipelineWorkloadPreparation` | +| Groups, stage partitioning, core budgets | `buildPipelineGroups`, `partitionPipelineGroups`, `assignPipelineStages` | +| Topology clustering and stage-local packing | `PipelineCoreLayout`, `packPipelineStages`, `IndexedResidentWeights` | +| Timing evaluation and publication equivalence | `evaluateSchedule`, `populateScheduleEquivalenceClasses` | +| Selected execution order and homogeneous runs | Scheduled compute planning/materialization | +| Routing, forwarding, blocking communication order | Deferred transfer planning and communication scheduling | +| READY/FREE counts, registers, overwrite/restart boundaries | Deferred boundary planning/realization | + +Both strategies produce `MergeScheduleResult`: core assignments, contiguous +per-core slots, estimated starts, last-task markers, equivalence classes, and +cost summaries. Pipeline results also assign each core a stage. Materialization +uses verified slots for execution order and merges only consecutive compatible +instances. Graph/class lane identity does not imply core identity or execution +order. + +Dependencies and transfer costs share `getGraphBatchPublicationMap`. +Statically proven physical-slot projections support many-to-many edges; +unproven projections retain complete producer dependencies, including region +captures. Malformed publication metadata fails graph construction. + +Normative contracts: +[graph fragments](../../.agents/invariants/GRAPH_COMPUTE_BATCH_INVARIANT.md), +[pipeline stages](../../.agents/invariants/PIPELINE_SCHEDULING_INVARIANT.md), +[target boundary](../../.agents/invariants/SPATIAL_TARGET_GENERALITY_INVARIANT.md), +[synchronization](../../.agents/invariants/PIM_SYNCHRONIZATION_INVARIANT.md), +and [performance](../../.agents/invariants/PERFORMANCE_OPTIMIZATION_INVARIANT.md). + +## Costs and interpretation + +Execution and fixed transfer costs use integer nanoseconds, converted from +target cycles. Bytes, elements, flits, weight slots, and lanes remain resource +quantities. Memory work is rounded at its actual fragment/run granularity; +runtime-free views cost zero. Transposes charge memory traffic and strided +vector work; concatenation charges per-input blocks per outer row. Direct +receives into assembly storage avoid duplicate endpoint-memory charges. +Sub-byte storage is analyzable in Spatial but rejected by byte-addressed PIM +communication lowering. + +The evaluator separates serial work, per-core work, dependency critical path, +summed transfer work, single-inference completion, and stage spans. Dependency +paths exclude core-order constraints; completion includes them. Stage spans +exclude cross-stage routing, restart/host synchronization, and contention. +Simulator latency and measured throughput remain the performance gates. + +Pipeline partitioning uses ordered-packing/dependency-tail priorities and +nominal movement volumes. PEFT and final evaluation use physical cost estimates. +Placement ranks maximum per-core compute/incident-transfer demand, then stage +completion and compute load; within 1% demand, stage span breaks the tie. +These are placement heuristics, not a steady-state timing model. + +## Capacity and recovery + +Distinct-weight union / per-core capacity is a lower bound; an oversized task +is infeasible. Ordered pack counts estimate budgets, and failed heuristic +packing does not prove infeasibility. Exhausted budgets restart from lower +bounds and constructive packing establishes feasibility. + +Latency reservations try load/communication-aware placement, then +largest-weight-first and shared reuse-first best fit on failure. Pipeline +packing compares arrival/load-aware candidates and uses constructive recovery. +The shared constructor retries in reverse order, then at most 16 single-choice +deviations within 65,536 core-candidate checks. Successful placements are +preserved. Exhausted search produces a bounded diagnostic, not a claim of +proved hardware infeasibility. + +## Complexity and limits + +Let `T` be tasks, `E` dependencies, `P` cores, `G` groups, `S` stages, +`R` weight references, `W` distinct weights, and `C` modeled work. + +| Analysis | Time / storage | +|---|---| +| Schedule evaluation | `O((T + E) log T + P)` / `O(T + E + P)` | +| Weight indexing | Up to `O(R W)` equality comparisons | +| Weight lower bounds | `O(R + W)` / `O(W)` bits | +| Constructive best fit | `O(T log T + P R + P W)` / `O(P W)` bits plus `O(T + P)`; bounded retries only on failure | +| Partition DP | Up to `O(S G² ((T + R) log C + W))` plus dependency traversal / `O(S G²)` cache | +| Topology clustering / cached PEFT distances | `O(P²)` time | +| Concat cost extraction | `O(A D)` for `A` inputs of rank `D`; constant auxiliary space | +| Cached pipeline priorities | `O(T + D)` time / `O(T + G)` space for `D` template-body operations | +| Exact projections | `O(Q + L)` time/temporary space for `Q` slot visits and `L` producer lanes; publication maps cached | +| Conservative producer tracing | Linear in the input projection DAG, including captures | + +Latency insertion-gap searches can remain quadratic. Issue overhead, +eliminated assembly copies, contention, and pipeline synchronization remain +cost approximations. Changes require matched execution measurements. diff --git a/docs/compiler/spatial-layouts.md b/docs/compiler/spatial-layouts.md new file mode 100644 index 0000000..4ea5d4f --- /dev/null +++ b/docs/compiler/spatial-layouts.md @@ -0,0 +1,63 @@ +# Spatial layout capabilities + +This reference describes the alternatives advertised by +[`SpatialLayoutCapabilities.cpp`](../../src/PIM/Conversion/ONNXToSpatial/Passes/Analyses/SpatialLayoutCapabilities.cpp), +through `SpatialLayoutCapabilityInterface`. Each alternative specifies all +operand layouts and one result layout; it is not a global promise that every +shape or target can use that layout. + +## Physical layouts + +Logical tensor coordinates remain NCHW. The physical enum is defined in +[`Spatial.td`](../../src/PIM/Dialect/Spatial/Spatial.td). + +| Layout | Contract and current use | +|---|---| +| `DenseNCHW` | Ordinary dense storage; first/default alternative for every layout-aware plan. | +| `NHWCRowStrip` | Physical row strips of channel vectors for logically NCHW tensors; compatible operations can preserve this representation through a chain. | +| `Fragmented` | Existing fragmented input accepted by Flatten for dense reconstruction; no current capability advertises it as a result alternative. | +| `NCHWRowStrip` | Reserved NCHW-oriented row-strip enum; no current capability advertises it as an alternative. | + +## Compatibility matrix + +Every operation below also advertises all-dense operands → `DenseNCHW`. +Here `R` means `NHWCRowStrip`, `D` means `DenseNCHW`; Conv's other operands +retain the layouts supplied to its interface on the `R` path. + +| Plan operation | Additional operand → result alternatives | Interface gate | +|---|---|---| +| Conv2D | Dense input → R; R input → R | `canLowerConvPlanToRowStrip`; `canConsumeAndProduceRowStrip`, respectively. | +| Flatten | Fragmented → D; R → D | Fragmented input already present; `canLowerFlattenFromRowStrip` for R. | +| Relu | R → R | Input already R. | +| SiLU | R → R | Input already R; intrinsic cost −3 instead of the generic R cost −2. | +| ResizeNearest | R → R | Input already R and `canLowerResizeNearestPlanToRowStrip`. | +| MaxPool2D | D → R; R → R | `canLowerMaxPoolPlanToRowStrip`. | +| GlobalAveragePool | D → R; R → R | `canLowerGlobalAveragePoolPlanToRowStrip`. | +| BiasAdd | (R data, D bias) → R | Ranked result, R data input, and `isSupportedBiasAddValue` for the bias shape. | +| Add | (R, R) → R | Both data operands already R. | +| Concat | All R inputs → R | Nonempty operand list, all already R. | + +The named predicates live in the [plan-lowering implementation](../../src/PIM/Conversion/ONNXToSpatial/Passes/Transforms) +and [bias helper](../../src/PIM/Conversion/ONNXToSpatial/Common/BiasAddUtils.hpp). They check operation geometry and +target resources; the matrix does not replace those legality checks. + +## Selection and materialization + +[`SpatialLayoutPlanningPass.cpp`](../../src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp) +starts every plan at its dense alternative. It scores intrinsic cost, operand +layout mismatches, and downstream inability to accept the candidate layout. +Strictly cheaper alternatives replace the current selection; ties preserve it. +Traversal alternates forward/reverse order for at most `2 × plan_count + 1` +rounds. Nonconvergence fails the pass with a diagnostic. + +Results returned from the function must remain dense. The pass records selected +layouts and inserts explicit `spat.materialize_layout` operations where selected +producer/consumer contracts disagree, including non-layout-aware consumers of +non-dense values. It verifies exact operand/result compatibility before +[`LowerSpatialPlans`](../../src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp) lowers plans +and materializations. + +`--pim-disable-spatial-planning` skips cost selection, choosing alternative 0 +for every plan. The planning/materialization/verification pass still runs. +ONNX rewrites and later graph merging are unchanged; disabling layout planning +does not request fusion or removal of separate elementwise graph nodes. diff --git a/docs/evaluation/methodology.md b/docs/evaluation/methodology.md new file mode 100644 index 0000000..a4c9c60 --- /dev/null +++ b/docs/evaluation/methodology.md @@ -0,0 +1,75 @@ +# Evaluation methodology + +The [documentation index](../README.md) links execution guides, model provenance, +reproduction details, and correctness experiments. + +## Canonical result datasets and output roots + +The authoritative default aggregate CSVs are: + +- [Comparison results](../../validation/tools/pim/pimcomp/compare/results_comparison.csv) +- [Ablation results](../../validation/tools/pim/ablation/results_ablation.csv) + +They are intentionally checked in beside their scripts. Default benchmark runs +update the corresponding canonical CSV; default plotting reads its current +contents. Plot generation does not run benchmarks or establish publication +provenance. Preserve the model/configuration/source/build revisions, invocation, +reference inputs, and original per-case reports when publishing measurements. + +An explicit custom `--out-dir PATH` keeps aggregate CSVs under that experiment +root. Model artifacts still default to `validation/networks/pimcomp_models`; +changing aggregate placement does not change artifact or shared-reference paths. +Per-variant ablation summaries remain `/ablation//results_comparison.csv`. +Use `--input PATH` to plot a custom aggregate. Plot destinations can be selected +with `--output-dir PATH`. Cleanup of default model artifacts preserves the two +canonical datasets. + +## Functional validity and performance availability + +Functional PASS/FAIL and performance availability are independent. A numerically +correct program can fail performance simulation or complete no inference before +the deadline. Missing, failed, skipped, unsupported, nonfinite, or nonpositive +per-inference measurements become `NA` in aggregates, not zero. With zero +completed outputs, interval, throughput, and energy per inference are unavailable; +measured average chip power can still be reported. Functional status is retained. + +Ablation percentages require positive finite values for both the variant and +its matching baseline. Tiny positive measurements retain significant digits +instead of being rounded to zero in the comparison CSV. An extreme valid +decrease can round to `-100.00%`; that value alone is not evidence of failure. +Reciprocal output interval and throughput cannot both have that decrease. +The ablation plotter warns and displays `N/A` for such invalid paired percentages, +while leaving the CSV and functional status unchanged. + +## Report reuse and implementation revisions + +The [comparison guide](../../validation/tools/pim/pimcomp/compare/README.md) +describes fingerprint validation and failed-report reuse. Fingerprints cover +model/configuration contents and experiment settings, but do not identify +compiler/tool implementations, built binaries, or manually changed reference +artifacts. Reuse is not proof of unchanged tool behavior: rerun the relevant +compiler after those change, and record build/source revisions for publication. +No revision cache is inferred from Git HEAD, which would miss dirty sources, +external tools, and binaries built from another checkout. + +## Metric interpretation + +Metrics come from Pimsim's +[`Chip::getSimulationReport`](../../backend-simulators/pim/pimsim-nn/src/chip/Chip.cpp) +and [`EnergyCounter`](../../backend-simulators/pim/pimsim-nn/src/utils/EnergyCounter.cpp). + +- Latency mode measures time to complete one inference. +- Throughput-mode latency is measurement duration / completed iterations, + equal to `1000 / throughput` before rounding. It is not end-to-end pipelined + inference latency; multiplying it by depth does not establish that quantity. +- Average power is total modeled static + dynamic chip energy / elapsed time. +- Energy per inference is total energy / completed iterations (one in latency + mode). Plots convert pJ to mJ with `1e-9`; no pipeline-depth or batch-size + division is applied. + +Fast mode measures throughput over the last two convergence windows but power +and energy over the entire run up to early stop. Energy includes startup and +work in flight, so it need not equal average power / reported throughput. +Use `--no-fast` for full-duration measurements. Consult the +[correctness notes](../../validation/tools/pim/pimcomp/correctness/README.md) before interpreting failed rows +as performance comparisons. diff --git a/docs/evaluation/pimcomp-reproducibility.md b/docs/evaluation/pimcomp-reproducibility.md new file mode 100644 index 0000000..a70d77f --- /dev/null +++ b/docs/evaluation/pimcomp-reproducibility.md @@ -0,0 +1,111 @@ +# Pimcomp reproduction reference + +The [model README](../../validation/networks/pimcomp_models/README.md) owns provenance, checksums, hardware profiles, and +numeric-artifact limitations. The [comparison guide](../../validation/tools/pim/pimcomp/compare/README.md) +owns suite commands, reports, reuse rules, and metric interpretation. + +## Frontend acceptance versus scheduled computation + +The current [frontend mapping](../../third_party/PIMCOMP-NN/frontend/frontend.py) +accepts exactly these ONNX node types: + +```text +Add, AveragePool, BatchNormalization, Clip, Concat, Conv, Dropout, Flatten, +Gather, Gemm, GlobalAveragePool, LRN, MatMul, MaxPool, Mul, Pad, Relu, Reshape, +Shape, Sigmoid, Softmax, Squeeze, Sub, Sum, Tanh, Transpose, Unsqueeze +``` + +`Constant` is consumed as metadata rather than emitted as a backend node. +Other types cause `operation: not considered` and stop conversion at +the first occurrence. In particular, YOLO11n's Split and Resize are unsupported; +the emitted prefix cannot represent a complete inference. + +The latency scheduler's explicit no-consider set, also used by hierarchy +mapping and GA post-operation accounting, is: + +```text +Input, BatchNormalization, Clip, Dropout, Flatten, LRN, MatMul, Reshape, +Softmax, Squeeze, Transpose +``` + +See [ElementPipelineSchedule.cpp](../../third_party/PIMCOMP-NN/backend/ElementPipelineSchedule.cpp), +[HierarchyMapping.cpp](../../third_party/PIMCOMP-NN/backend/HierarchyMapping.cpp), +and [GeneticAlgorithm.cpp](../../third_party/PIMCOMP-NN/backend/GeneticAlgorithm.cpp). +These accepted nodes receive no standalone latency instructions while remaining +in the backend graph. Acceptance therefore does not establish timed computation. + +Before scheduling, the frontend may fuse BatchNorm/Relu/Tanh/Sigmoid into +Conv/Gemm, turn Reshape–Transpose–Reshape into channel shuffle, remove a specific +Shape–Gather–Unsqueeze–Concat shape chain, and merge Pad into its consumer. +Those transformations differ from both frontend rejection and leaving a node +unscheduled. In particular, the [BatchNorm correctness gap](../../validation/tools/pim/pimcomp/correctness/README.md#resnet-batchnorm-gap) +shows why a fusion label is not proof of numeric equivalence. + +Pimsim accepts ISA instructions rather than ONNX operators. Its +[instruction decoder](../../backend-simulators/pim/pimsim-nn/src/isa/Instruction.cpp) +rejects `vsoftmax` (opcode 21) and unknown opcodes; unsupported operations must +not be approximated in the oracle. This is why the dedicated GoogLeNet artifact +removes LRN/Softmax and the YOLO artifact removes Softmax. YOLO retains its other +operations, including MatMul/Transpose and detection decoding, for Raptor. + +## Standalone frontend/backend reproduction + +Build Pimcomp with CMake and an OpenMP-capable C++ compiler: + +```bash +cmake -S third_party/PIMCOMP-NN -B third_party/PIMCOMP-NN/build +cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN +``` + +The backend resolves `../config.json`, `../models/JSON/.json`, and +`../output/` relative to its working directory, not its executable. The frontend +rewrites its input ONNX batch dimension in place. Use an isolated workspace and +a model copy, as the comparison driver does, to preserve source/config files. +From the Raptor root, after installing the [frontend dependencies](../../third_party/PIMCOMP-NN/README.md): + +```bash +RAPTOR_ROOT=$PWD +PIMCOMP="$RAPTOR_ROOT/third_party/PIMCOMP-NN" +PIMCOMP_RUN=$(mktemp -d) +mkdir -p "$PIMCOMP_RUN/models/JSON" "$PIMCOMP_RUN/output" "$PIMCOMP_RUN/work" +cp validation/pimsim_configs/pimcomp/arch-a/latency_config.json "$PIMCOMP_RUN/config.json" +cp validation/networks/pimcomp_models/vgg8/vgg8-mnist-reconstructed.onnx "$PIMCOMP_RUN/vgg8.onnx" +.venv/bin/python "$PIMCOMP/frontend/frontend.py" \ + --model_path "$PIMCOMP_RUN/vgg8.onnx" \ + --save_path "$PIMCOMP_RUN/models/JSON/vgg8.json" +( + cd "$PIMCOMP_RUN/work" + OMP_NUM_THREADS=1 "$PIMCOMP/build/PIMCOMP-NN" \ + -m=vgg8 -r=GA -p=element -o=YES -v=YES -s=YES +) +``` + +Outputs remain under `$PIMCOMP_RUN/output`. `-v=YES` emits verification data; +`-s=YES` emits `SimulationInfo.gz`; `-o=YES` saves the evaluation report. +Use `-p=batch` for throughput scheduling, with the matching throughput config +when running Pimsim. The paper's batch experiment used 128 samples; the current +comparison runner's functional batch default is 64. Element mode represents one +inference. Select Arch-B/C by copying that profile's config; Arch-C's rectangular +crossbars remain unsupported by Raptor. + +For the bundled Model Zoo graphs, copy `models/JSON/resnet18.json`, +`resnet34.json`, or `googlenet.json` from Pimcomp to the workspace and change +`-m` accordingly. That reproduces bundled graph scheduling; use the suite +runner to regenerate graphs from the exact checked-in performance ONNX files. + +## Genetic algorithm and reproducibility limits + +[`GeneticAlgorithm.h`](../../third_party/PIMCOMP-NN/backend/GeneticAlgorithm.h) +sets population 200 and at most 1000 iterations. `-r=GA` selects it; standalone +backend default replication is `balance`, whereas the suite explicitly uses GA. +GA initialization calls `srand(1)`. Fitness evaluation uses OpenMP with static +work distribution; `OMP_NUM_THREADS` bounds its parallelism (use 1 for serial +reproduction), otherwise the OpenMP runtime chooses the thread count. + +The fixed GA seed does not make every backend stage deterministic: +`ElementPipelineSchedule.cpp` separately calls `srand(time(NULL))`. Keep model +hashes, source/build revisions, profile, mode, and thread count with experiment +results. Compare correctness +against the original ONNX reference, including the known batch and BatchNorm +failures; neither omitted operators nor prefilled reference intermediates repair +those programs. diff --git a/requirements.txt b/requirements.txt index 0bd9401..55afd4f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,5 @@ colorama>=0.4.6,<1 +matplotlib>=3.8,<4 numpy>=1.26.4,<3 onnx>=1.17,<2 onnxsim>=0.6.5,<1 diff --git a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/README.md b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/README.md index 00aea04..9bbf175 100644 --- a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/README.md +++ b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/README.md @@ -1,177 +1,9 @@ # Spatial scheduling -`ScheduleAndRealizeSpatial` selects one scheduling strategy after constructing -the shared `ComputeGraph`. Pipeline scheduling does not run, require, or repair -a latency PEFT placement. +The [scheduling overview](../../../../../../../../docs/compiler/scheduling.md) +documents ownership, costs, capacity recovery, complexity, and links to the +normative invariants. See [implementation details](../../../../../../../../docs/compiler/scheduling-implementation.md) +for fragment views, receive forwarding, transfer rounding, and placement models. -## Ownership and flow - -| Decision | Owner | -| --- | --- | -| Compute instances, dependency extraction, execution/transfer costs, resident-weight identities | `ComputeGraph` and `ComputeInstanceUtils` | -| Target configuration and physical link costs | Compiler target adapter, exposed through `SchedulingTarget` | -| Latency priorities, reservations, physical assignment, insertion gaps, per-core order | `PeftScheduler` | -| Capacity-driven additive workload splitting | `PipelineWorkloadPreparation` | -| Operation groups, stage adjacency and input constraints | `buildPipelineGroups` | -| Stage partitioning | `partitionPipelineGroups` | -| Stage core budgets | `assignPipelineStages` | -| Physical topology clustering | `PipelineCoreLayout` | -| Stage-local placement and resident-weight packing | `packPipelineStages` and `IndexedResidentWeights` | -| Schedule timing evaluation | `evaluateSchedule` | -| Publication-compatible core equivalence classes | `populateScheduleEquivalenceClasses` | -| Emission of the selected core order and homogeneous runs | Scheduled compute planning/materialization | -| Routing, local forwarding and transfer families | Deferred transfer planning | -| Deadlock-free blocking communication order | Deferred communication scheduling | -| READY/FREE counts, registers, overwrite boundaries and restart synchronization | Deferred boundary planning | -| Explicit communication and synchronization operations | Deferred boundary realization | - -Both strategies produce `MergeScheduleResult`: physical core assignment, -contiguous per-core execution slots, estimated starts, last-task markers, -equivalence classes and cost summaries. Pipeline schedules also provide the -stage of every physical core. Graph identity identifies publications; it does -not order execution. Materialization sorts only by the verified scheduler slots -and combines only consecutive compatible graph instances. -Class lanes keep graph encounter order independently of physical core IDs, -preserving regular source-lane mappings without changing any core's execution -order. Complete singleton fragment publications use reshape views only with -matching static shapes, zero offsets, complete sizes and unit strides, including -scalar fragments. Other insertions retain their slice semantics. Receive forwarding -runs after copy normalization and can cross lookup-table loads, while writes -and synchronization remain barriers. - -Physical fragment slots are resolved through `getGraphBatchPublicationMap`. -Direct statically evaluable batch projections can have many-to-many edges. -Unproven projections retain the complete producer dependency. Blueprint -dependencies belong to the shared graph, rather than a separate pipeline-only -dependency graph. -Sources reached through intermediate projections retain conservative producer -dependencies, including region captures. Graph construction reports malformed -publication metadata through normal pass failure. -Projected communication costs count the selected physical fragments through -the same publication map; graph-range overlap is not a transfer-volume estimate. -Conservative costs scale payload bytes before rounding memory/packet costs, so -small fragments retain endpoint and packet overhead. Conservative producer-chunk -payloads scale elements/bits before byte rounding, preserving one aggregate -chunk estimate; projected publication costs retain their per-fragment granularity. -Spatial sub-byte types reach analysis, while the current PIM byte-addressed -communication lowering rejects non-byte-sized element storage. - -## Cost dimensions - -Execution times and fixed transfer costs use integer nanoseconds. Hardware -cycles are converted using the corresponding target period. Bytes, elements, -flits, resident-weight slots and lane counts remain resource quantities. -Blueprint reconstruction charges memory work separately for each fragment; -small projected slices are not assumed to require a single copy. Full receives -that can write directly into assembly storage do not charge endpoint memory work -again on their edges. Later copy elimination can make this assembly estimate -conservative. Blueprint reconstruction uses target memory read/write costs; element counts -and cycles are not added to nanoseconds. Runtime-free views cost zero. -Runtime transposes charge target memory read/write traffic rather than generic -nested-operation fallback cost. Their shared lowering/run analysis charges rounded -memory accesses per run and vector execution for strided runs. The matched oracle -charges vector payload bytes, not the full strided address footprint. Instruction -setup/issue time and contention remain approximation boundaries. -Concatenation charges each input block separately -for each outer row, including memory-width rounding. Axis, size and target-period -scaling have focused coverage. - -The evaluator distinguishes total serial work, maximum per-core work, -dependency critical path, summed edge transfer latency, single-inference -completion and individual stage spans. The dependency path excludes core-order -constraints; completion includes them. Summed communication is a work metric, -not elapsed time. Stage spans exclude cross-stage routing, restart/host-buffer -synchronization and network contention. They are scheduling proxies, not an -authoritative prediction of steady-state throughput. Simulator latency and -measured throughput period remain the performance gates. - -Partitioning retains its successful ordered-packing plus dependency-tail -priority. This deliberately biases cuts away from long dependent groups and -is not reported as predicted execution time. Pipeline partitioning and packing -use nominal volume priorities: transpose traffic is weighted as a bulk movement, -preserving the measured heuristic without pretending that individual runs are -free. The per-operation priority adjustment is cached and scaled by chunk lanes. -PEFT and final schedule evaluation always use physical execution estimates. -Pipeline candidate placement compares -maximum per-core serial compute/incident-transfer demand, then stage completion -and compute load. Incident transfer latency is charged to each participating -core as a placement proxy; it is not added twice to predicted elapsed time. -This preserves useful parallelism that an arrival-time-only ranking can lose. -The final reported compute/incident-transfer demand uses physical execution -estimates, separately from the priority scores used to select the placement. -When serial-demand estimates are within 1%, stage span breaks the near tie; -this is a heuristic resolution rule, not a pipeline timing model. Core budgets -come from stage allocation and capacity recovery. A proposed additional one-core -load refinement was removed after a matched throughput regression; improving -the placement proxy did not establish a better execution period. - -## Capacity and recovery - -The distinct-weight union divided by per-core capacity is a provable lower -bound; an oversized task is individually infeasible. Ordered pack counts are -heuristic core-budget estimates for unrestricted placement. Their failure does -not prove hardware infeasibility. When these estimates exhaust the fleet, core -allocation restarts from the lower bounds and actual packing establishes -feasibility. - -Latency reservations prioritize physical communication/load, or reuse under -high weight pressure. If graph-order reservations fail, a largest-weight-first -retry retains load-aware placement before using the shared reuse-first best-fit -constructor. The constructor establishes feasibility; its concentrated assignment -does not replace a successful load-aware reservation. Pipeline packing compares arrival-aware and load-aware placements; -ordered packing and the same shared constructor supply constructive recovery -assignments. Failed search budgets guide bounded core reallocation, but are not -resource lower bounds. Diagnostics distinguish a proved capacity limit from an -unsuccessful heuristic search. No exact combinatorial solver is used. -The shared constructor retries once in reverse order after greedy failure. -Only if both orders fail, it replays at most16 single-choice deviations with a -shared budget of65536 core-candidate checks, reusing its scratch buffers. -Successful existing assignments are preserved; exhausted repair is not proof of -infeasibility. PEFT propagates reservation failures to the scheduling pass for -a bounded diagnostic and normal pass failure. - -## Complexity - -Let `T` be tasks, `E` dependencies, `P` processors, `G` operation groups, `S` -stages, `R` resident-weight references, `W` distinct weights and `C` total modeled -work. - -- Schedule evaluation takes `O((T + E) log T + P)` time with deterministic ready - ordering, and `O(T + E + P)` space. -- Resident-weight indexing retains `O(R W)` worst-case equality comparisons. - Lower bounds take `O(R + W)` time and `O(W)` bits. Constructive best fit takes - `O(T log T + P R + P W)` time, with `O(P W)` bits and `O(T + P)` additional - storage. It runs on failure paths rather than replacing successful placement. - Recovery adds at most17 constructions with the same complexity (one reverse - traversal plus16 single-choice replays); repair candidate checks are also - capped at65536. Scratch storage is reused, with one extra `O(T)` failed-assignment - snapshot only after both original attempts fail. -- The partition DP remains superlinear: up to - `O(S G² ((T + R) log C + W))` time for ordered segment estimates, plus group - dependency traversal, and `O(S G²)` cache space. This bounded existing search - is retained rather than replaced with global placement optimization. -- Physical topology clustering and cached PEFT center distances take `O(P²)` - time. Center lookups are constant time instead of repeating a full physical - topology scan for each task/candidate processor. -- Concatenation cost extraction takes `O(A D)` time and constant auxiliary space - for `A` inputs of rank `D`; it does not expand the outer rows or copy blocks. -- Graph execution costs are scaled by chunk lane counts. Pipeline priority - analysis adds two cost traversals per operation template and caches the - adjustment, taking `O(T + D)` time and `O(T + G)` space for `D` total template - body operations. It does not revisit template bodies for every chunk. -- Exact projection analysis visits each selected physical slot once per - consumer chunk and input use, with a producer-lane bitset for deduplication. - If `L` is producer lanes and `Q` is projected slot visits, it takes - `O(Q + L)` time and `O(L + Q)` temporary space; publication maps are cached - across all chunks. Cost extraction shares this traversal with dependencies. -- Conservative producer tracing visits each value/projection once per input, - including captured region operands, with linear time and space in that input's - projection DAG. Direct physical-batch edges retain their existing fast path. -- Pipeline mode removes a complete PEFT run, including its reservation and - insertion-gap searches. Latency mode retains those algorithms and their - existing potentially quadratic gap traversal. - -Remaining approximation boundaries include operation issue overhead, eliminated -assembly copies, communication contention and pipeline synchronization. Weight -indexing and ready-transfer signature searches remain possible compiler -hotspots. Changes to these require matched execution measurements. +Implementation sources remain in this directory; project-wide compiler +navigation starts at [docs/README.md](../../../../../../../../docs/README.md). diff --git a/tools/raptor_graph_explorer/ARCHITECTURE.md b/tools/raptor_graph_explorer/ARCHITECTURE.md new file mode 100644 index 0000000..ee38ff4 --- /dev/null +++ b/tools/raptor_graph_explorer/ARCHITECTURE.md @@ -0,0 +1,131 @@ +# Graph Explorer architecture and API + +The [README](README.md) covers installation, input headers, commands, and tests. +This reference describes the contracts maintained by the implementation. + +## Ingestion and identity + +[`discovery.py`](raptor_graph_explorer/discovery.py) recursively pairs reports +by their full filename prefix, including ZIP inputs, and rejects unsafe archive +paths. That prefix is `report_id`; unrelated reports never share endpoints. + +[`ingest.py`](raptor_graph_explorer/ingest.py) streams CSVs in batches of 2,000 +rows through temporary SQLite staging tables. Node IDs, operation IDs, lanes, +cores, and channels remain separate fields. Known node prefixes are `gc`, +`gcb`, `sc`, `scb`; unknown ones retain `unknown:`. Explicit operation +and lane columns are authoritative. Blank operation IDs, and blank batch lanes, +may use a parseable source ID with a warning. Explicit edge lanes override +endpoint lanes, with contradictory values diagnosed. Missing numbers stay +SQL NULL; malformed rows are diagnosed and omitted, never coerced to zero. +Extra columns survive in `attributes_json`. + +Duplicate node IDs retain the first source row and produce diagnostics. Bulk +joins resolve edges within their report; unknown endpoints are omitted with +bounded examples. Staging tables are dropped even on error. The ingestion +connection closes before indexes, aggregates, motifs, and diagnostics are +built. Secondary indexes are created after raw insertion. CSVs without extra +columns reuse an empty JSON representation. + +Raw edge IDs use report ID plus original CSV row number; aggregate IDs use +report ID and grouping keys. Input filesystem paths and browser layout do not +define identity. The manifest stores versions and counts, not duplicate raw +rows or machine-specific input paths. Database schema compatibility is checked +before serving; incompatible outputs must be rebuilt. + +## Aggregation and exact lane mappings + +Operation nodes group by `(report_id, op_id)`, retaining kind counts, lane/core +cardinalities and ranges, and SSA summaries. Operation edges group by the +source/target operation pair, retaining counts, weights, tensor types, channels, +and exact mapping classification. Core aggregation has a distinct `unassigned` +group for missing cores. Node-kind aggregation is a small diagnostic view. + +[`mappings.py`](raptor_graph_explorer/mappings.py) uses all normalized lane +pairs, in the following precedence order: + +| Class | Meaning | +|---|---| +| `scalar_to_scalar` | Both lanes absent for every pair. | +| `scalar_to_batch` / `batch_to_scalar` | One side entirely absent, the other entirely present; range, cardinality, and contiguity recorded. | +| `identity` | One-to-one relation with equal lanes. | +| `constant_offset` | One-to-one with one nonzero target-minus-source offset. | +| `permutation` | Remaining one-to-one relation with equal cardinalities. | +| `broadcast` / `fan_in` | One source to many targets / many sources to one target. | +| `stencil` | Multiple sources share exactly the same target-offset set, containing 2–64 offsets. | +| `irregular` | Complete lane data matching none of the preceding forms. | +| `unknown` | Empty or partially missing relation outside the scalar/batch cases. | + +SQL computes degrees/cardinalities; stencil comparison streams one operation +pair at a time. No sampling determines classification. Mapping rows remain +paginated; matrices show exact sparse points up to the configured threshold, +then bounded density bins. + +## Display projection and layout + +[`projection.py`](raptor_graph_explorer/projection.py) rebuilds the entire +display from the current expanded-operation set. Expanding replaces an +operation aggregate with its raw nodes, including isolated nodes. An aggregate +edge survives only while both endpoint operations are collapsed. Otherwise its +raw edges replace it: each endpoint is raw if its operation is expanded and +aggregate if collapsed. The complete set determines endpoints, so expansion +order cannot leave stale endpoints or duplicate aggregate/raw representations. +Raw view expands every operation, preserving each valid source edge once. + +Operation ranks use the complete operation graph, including isolated nodes. +Collapsed operations occupy one row; expanded lanes use compact rows in numeric +order, followed by deterministic lane-less rows. Sparse lane numbers leave no +empty rows, and nodes sharing a lane get centered deterministic horizontal +offsets. Operation/raw views use stable server coordinates restored by Reset +layout. Core/node-kind views use ELK. Views flow top-to-bottom. + +The [viewer](raptor_graph_explorer/static/index.html) first requests an +operation aggregate before applying the report's view selection. Raw selections +use raw details rather than aggregate-only endpoints. Operation labels use the +first SSA name, falling back to source identity; label collision handling and +bounded node sizes keep dense views usable. Dependencies are pinned in that +file (Graphology, Sigma.js, ELK); first browser use requires CDN access. + +## Structural motifs + +[`motifs.py`](raptor_graph_explorer/motifs.py) loads only the operation aggregate +into NetworkX. It condenses SCCs into a DAG, attaches synthetic source/sink +nodes, and computes dominators and reverse post-dominators. Branching entries +are paired with their immediate post-dominator; regions retain nodes reachable +before that exit which can reach it, rejecting external entrances/exits. +One shortest representative path per branch avoids enumerating all paths. + +`exact_diamond` has two direct one-node branches; `rhomboid` has disjoint +branch interiors; `split_rejoin` covers other valid single-entry/single-exit +regions. Identical memberships are deduplicated; nested motifs remain. +SCC members map back to operation IDs, with deterministic representative +operations for multi-operation SCC endpoints/paths and `contains_cycle` markers. +Screen geometry never determines motif membership. + +## Read-only API + +All routes below use GET; see [`api.py`](raptor_graph_explorer/api.py). +Responses include `Cache-Control: no-store` to avoid mixing viewer revisions. + +| Route | Result / relevant query parameters | +|---|---| +| `/api/manifest` | Tool/schema versions and report counts. | +| `/api/reports` | Available reports. | +| `/api/reports/{report_id}/summary` | Counts, mapping classes, diagnostics. | +| `/api/reports/{report_id}/graph/{level}` | Complete `operation`, `core`, or `node_kind` aggregate. | +| `/api/reports/{report_id}/display-graph` | Deterministic projection; `view=operation` or `raw`, repeated `expanded_op_id`, `expand_all`. | +| `/api/reports/{report_id}/motifs` | Structural motifs. | +| `/api/aggregate-nodes/{aggregate_id}` | Aggregate details; operation SSA names capped at 100. | +| `/api/aggregate-edges/{aggregate_edge_id}` | Aggregate edge details. | +| `/api/aggregate-edges/{aggregate_edge_id}/mapping` | Exact raw lane rows; `limit`, `offset`. Operation edges only. | +| `/api/aggregate-edges/{aggregate_edge_id}/matrix` | Exact sparse points or density bins. Operation edges only. | +| `/api/reports/{report_id}/raw-nodes` | Raw node page; `limit`, `offset`. | +| `/api/reports/{report_id}/raw-edges` | Raw edge page; `limit`, `offset`. | +| `/api/reports/{report_id}/subgraph` | `aggregate_id`, `direction=in/out/both`, `depth`, optional `expand_lanes`. | + +`create_app` defaults: page size 100 with a maximum of 500; expansion cap 5,000; +exact matrix threshold 2,000 points; density grid 64×64. Subgraph depth defaults +to 1 and caps at 5. Oversized raw/expanded projections return HTTP 400 without a +partial graph. The projection cap counts raw nodes plus incident raw edges; +collapsed aggregate nodes/edges are additional. Aggregate endpoints have no +pagination or expansion cap, so very large operation graphs remain a scaling +limit. Raw CSV graphs are never loaded wholesale into NetworkX. diff --git a/tools/raptor_graph_explorer/README.md b/tools/raptor_graph_explorer/README.md index 9f98a96..eeded00 100644 --- a/tools/raptor_graph_explorer/README.md +++ b/tools/raptor_graph_explorer/README.md @@ -1,228 +1,117 @@ # Raptor Graph Explorer -Raptor Graph Explorer is an external Python 3.13 tool that preprocesses Raptor's existing Spatial dataflow CSV reports into SQLite and serves an interactive browser viewer. It does not modify Raptor, compiler passes, MLIR, or CSV generation. The CSV files remain the source of truth. +Python 3.13+ tool for exploring Raptor Spatial dataflow CSVs in a browser. +It streams reports into SQLite, aggregates operations/cores/node kinds, and +provides bounded raw-lane inspection. Compiler output remains unchanged. -Raw lane graphs can contain hundreds of thousands or millions of rows. Sending those rows to a browser makes layout, rendering, and even JSON serialization impractical. The explorer therefore loads an operation aggregate by default and retrieves raw rows only through bounded pages or a capped, explicitly requested lane expansion. +## Install and run -## Architecture - -```text -CSV pairs / directories / ZIP archives - -> recursive discovery and safe ZIP extraction - -> streaming CSV validation and SQLite staging - -> normalized raw_nodes and raw_edges - -> operation, core, and node-kind aggregates - -> exact lane mappings - -> SCC condensation and structural split/rejoin motifs - -> graph.sqlite + manifest.json - -> read-only FastAPI API - -> Sigma.js WebGL viewer with stable operation/lane layout -``` - -Node identifiers, operation IDs, lanes, cores, and channels are stored separately. In particular, graph lanes, physical fragment slots, scheduled lanes, physical cores, and communication channels are not treated as interchangeable identities. - -## Installation - -From the Raptor repository root: +From the repository root, create a Python 3.13+ environment if `.venv` does not +exist: ```bash python3.13 -m venv .venv -source .venv/bin/activate -python3.13 -m pip install -e tools/raptor_graph_explorer[dev] ``` -The installed command is `raptor-graph-explorer`. The equivalent module entry point is `python3.13 -m raptor_graph_explorer`. - -## Commands - -Build a persistent output: +For a new or existing environment, verify the version before installing. Select +a compatible environment if this check fails; do not reuse an older Python: ```bash -raptor-graph-explorer build \ - validation/networks/vgg16/depth_07/raptor/reports \ - --output /tmp/vgg_graph +.venv/bin/python -c 'import sys; assert sys.version_info >= (3, 13), "Python 3.13+ required"' +.venv/bin/python -m pip install -e 'tools/raptor_graph_explorer[dev]' +.venv/bin/raptor-graph-explorer build /path/to/raptor/reports --output /tmp/raptor_graph +.venv/bin/raptor-graph-explorer serve /tmp/raptor_graph --port 8765 ``` -Use repeated inputs or pass several paths after `build`. Inputs can be report directories, directories containing report subdirectories, ZIP archives, or explicit `.nodes.csv` / `.edges.csv` files. `--strict` rejects any data-integrity diagnostic. Without it, valid pairs and unambiguous rows are retained. `--overwrite` replaces an existing output. +Open `http://127.0.0.1:8765`. The server is read-only, has no authentication, +and is intended for local use. Browser dependencies load from pinned CDNs and +require network access on first use; preprocessing and inspection work offline. -Serve an existing output: +Other commands: ```bash -raptor-graph-explorer serve /tmp/vgg_graph --port 8765 +# Build and serve a directory, CSV pair, or ZIP archive. +.venv/bin/raptor-graph-explorer run /path/to/reports.zip --output /tmp/raptor_graph +# Print counts, mappings, motifs, and diagnostics. +.venv/bin/raptor-graph-explorer inspect /tmp/raptor_graph ``` -Build and immediately serve: +`build` and `run` accept multiple input paths and discover reports recursively. +`--overwrite` replaces an existing output. `build --strict` rejects +data-integrity errors; otherwise valid pairs and unambiguous rows are retained. +Outputs are `graph.sqlite` and `manifest.json`. Rebuild outputs if the tool +reports an unsupported schema version. -```bash -raptor-graph-explorer run \ - /mnt/data/conv_relu_conv.zip \ - --output /tmp/conv_relu_conv_graph \ - --overwrite -``` +See [architecture and API guarantees](ARCHITECTURE.md) for identity, expansion, +mapping/motif analysis, ingestion constraints, and the endpoint reference. -Print counts, mapping classes, SCCs, motifs, and diagnostics: +## Input contract -```bash -raptor-graph-explorer inspect /tmp/vgg_graph -``` - -The server defaults to `127.0.0.1:8765`. It has no authentication and is intended for local read-only use. - -## Supported CSV reports - -Files are paired using the complete prefix before `.nodes.csv` and `.edges.csv`. That prefix is the stable `report_id`; `spatial3_scheduled_no_comm.nodes.csv` therefore pairs only with `spatial3_scheduled_no_comm.edges.csv`. - -Node columns: +Pair files by the complete prefix before `.nodes.csv` and `.edges.csv`. +Required headers: ```csv Id,op_id,lane,core,ssa_name ``` -Edge columns: - ```csv Source,Target,Weight,Type,stage,source_lane,target_lane,channel_id ``` -Known node prefixes are `gc`, `gcb`, `sc`, and `scb`. Unknown prefixes are retained as `unknown:`. Explicit CSV `op_id` and lane columns are authoritative; a blank value may fall back to a parseable node ID and produces a diagnostic. Explicit edge lanes take precedence over endpoint lanes. Missing numeric values remain SQL `NULL`; malformed numeric values are never changed to zero. Additional CSV columns are preserved in `attributes_json`. +Explicit operation IDs and lanes take precedence over parsed node IDs. Missing +numeric values remain missing; malformed values produce diagnostics. Extra +columns are preserved. Edges resolve only within their report; unknown +endpoints are diagnosed and omitted. Unsafe ZIP paths are rejected. -Edges are resolved only against nodes from the same report. Unknown endpoints omit that edge and produce a bounded diagnostic. ZIP paths are checked before extraction and traversal entries are rejected. +Graph lanes, physical fragment slots, scheduled lanes, cores, and channels are +distinct identities. Spatial3 reports contain explicit communication only; +isolated nodes are valid and no dependencies are inferred from SSA text. -## Output +## Views and analysis -```text -OUTPUT_DIRECTORY/ -├── graph.sqlite -└── manifest.json -``` +- **Operation:** aggregates by operation and supports lane expansion, exact + mapping details, and structural motifs. +- **Core / Node kind:** summarize placement and operation kinds. +- **Raw instances:** displays source nodes and valid edges, including isolated + nodes; spatial4 reports select this view automatically. -`graph.sqlite` contains reports, normalized raw nodes and edges, aggregate nodes and edges, motifs, motif membership, diagnostics, and focused indexes. `manifest.json` contains schema/tool versions and compact per-report counts. It never duplicates raw graph rows or stores machine-specific input paths. +The viewer first loads an operation aggregate. Search, filters, expansion, +selection details, and layout controls support focused inspection. Raw and +expanded graphs cap raw nodes plus incident raw edges at 5,000; exceeding it returns an +error rather than a partial graph. Use an aggregate view or increase +`create_app(..., expansion_cap=...)` when the machine can handle it. -Schema version 1 is checked whenever an output is served. Unsupported or missing versions fail at startup with a clear error. +Lane mappings use all normalized pairs, without sampling. Classes include +scalar/batch relations, identity, constant offset, permutation, broadcast, +fan-in, stencil, irregular, and unknown. Small matrices show exact sparse +points; large ones use a bounded 64×64 density view. -## Aggregate views +Motifs identify structural single-entry/single-exit split/rejoin regions on the +operation graph's SCC condensation. `exact_diamond`, `rhomboid`, and +`split_rejoin` describe progressively broader forms; a diamond-shaped drawing +alone is not evidence of a motif. Cycles remain marked. -### Operation +## API and limits -Nodes group by `(report_id, op_id)` and retain node-kind counts, instance count, lane and core cardinalities/ranges, and an SSA summary. Edges group by `(report_id, source_op_id, target_op_id)` and retain raw count, weight statistics, tensor histogram, distinct channels, and exact mapping classification. This is the default view and the only graph used for motif analysis. +Read-only FastAPI endpoints expose manifests, report summaries, aggregate +graphs/details, mappings, matrices, motifs, and raw pages. See +[`api.py`](raptor_graph_explorer/api.py) for routes and `create_app` limits. +Raw pages default to 100 rows and cap at 500; subgraph depth caps at five. +Aggregate graph endpoints return the complete aggregate, so very large +operation counts can still be expensive. -### Core - -Assigned nodes group by `(report_id, core)`. Missing cores use a separate `unassigned` group, never core zero. Edges retain raw count, total weight, channel count, operation-pair count, and tensor histogram. - -### Node kind - -Nodes group by `(report_id, node_kind)`. This deliberately small diagnostic view shows how graph/scalar/batch/scheduled kinds connect; it does not replace operation aggregation. - -### Raw instances - -Raw instances displays every `raw_nodes` row exactly once and every valid `raw_edges` row exactly once. Nodes are queried independently from edges, so degree-zero nodes remain visible. A spatial4 report starts in this view automatically; Operation, Core, and Node kind remain selectable. - -Spatial3 CSVs record explicit communication only and may legitimately contain many isolated nodes. The explorer never fabricates dependencies from semicolon-separated SSA names or other node text. - -## Exact lane mappings - -Classifications use all normalized source/target lane pairs for an operation edge. They are evaluated in this order: - -- `scalar_to_scalar`: every pair has two absent lanes. -- `scalar_to_batch`: every source lane is absent and every target lane is present; target range, cardinality, and contiguity are recorded. -- `batch_to_scalar`: the corresponding source-lane case. -- `identity`: a one-to-one relation with equal source and target lanes. -- `constant_offset`: a one-to-one relation with one nonzero `target - source` offset. -- `permutation`: a one-to-one relation that is neither identity nor constant offset. -- `broadcast`: one distinct source lane maps to multiple target lanes. -- `fan_in`: multiple source lanes map to one distinct target lane. -- `stencil`: more than one source has exactly the same bounded target-offset set; the complete sorted set is recorded. -- `irregular`: complete lane data matches none of the exact forms. -- `unknown`: lane data is partially missing outside the scalar/batch cases. - -No sampling is used. SQL computes cardinalities and degrees; stencil offsets stream one operation edge at a time. Raw mapping rows remain paginated, and the matrix endpoint returns exact sparse points only below its threshold. Larger relations become a bounded 64×64 density matrix. - -## Structural motifs - -Motifs are not inferred from rendered geometry. For each operation graph the tool: - -1. computes strongly connected components and condenses them into a DAG; -2. attaches synthetic source and sink nodes; -3. computes dominators and reverse-graph post-dominators; -4. considers entries with at least two outgoing branches and their immediate post-dominator exit; -5. retains nodes reachable before the exit that can also reach the exit; -6. rejects unexpected entrances to, or exits from, internal nodes; -7. records one shortest representative path per branch without enumerating all paths; -8. maps condensed components back to operation aggregate IDs. - -`exact_diamond` is a two-branch direct reconvergence with one internal condensed node per branch. `rhomboid` has clean disjoint branch interiors. `split_rejoin` is the general valid single-entry/single-exit region. Identical memberships are deduplicated while meaningful nested motifs are preserved. SCC membership marks `contains_cycle`; cycles are never analyzed as if the raw operation graph were a DAG. - -## Viewer and API - -The viewer initially fetches an aggregate operation graph, so a previously selected raw view cannot make a new report exceed the display safety cap before the first render. Browser responses disable caching so HTML and JavaScript from different tool revisions cannot be mixed. Sigma renders the graph with WebGL. Controls cover report/view selection, text and tensor search, mapping filters, self edges, relayout, fitting, and motif selection. Edges use a fixed width. Raw nodes and edges have their own selection details and never call aggregate-only detail or mapping endpoints. Nodes without an SSA name fall back to their source identity. Mapping panels remain available for operation aggregate edges. - -Operation expansion is a deterministic projection of the source graph. Expand selected, Expand all operations, Collapse selected operation, and Collapse all rebuild the complete display from the current expanded-operation set. An expanded operation's raw nodes replace its aggregate node, including isolated raw nodes. An aggregate edge is retained only when both endpoint operations are collapsed; otherwise its raw edges replace it with endpoints calculated from the complete expansion set. Expansion order therefore cannot leave stale endpoints or simultaneous aggregate/raw representations. - -Operation ranks are calculated from the complete operation graph, including isolated operations, and every view flows top-to-bottom. A collapsed operation reserves one displayed row. Expanded nodes use compact rows for distinct lanes in numeric order, followed by deterministic lane-less rows, so sparse lane numbers create no empty geometric space. Raw nodes sharing a lane receive centered deterministic horizontal offsets. Operation and raw views use stable server coordinates, and `Reset layout` restores them; core and node-kind views use ELK, and `Rerun layout` runs it again. Operation labels use the first SSA name; Sigma's collision grid thins normal labels, and a label is skipped when its measured screen rectangle intersects another visible node. Selected or hovered nodes keep the original white bubble with black text. Selecting a motif frames its member nodes on the first click. Aggregate-node sizes remain bounded. - -Browser dependencies are pinned in one place, `static/index.html`: - -- Graphology 0.25.4; -- Sigma.js 2.4.0; -- ELK.js 0.11.1. - -They load from CDNs, so the browser viewer needs network access on first use. Python preprocessing, inspection, and API tests do not. - -Read-only endpoints include: - -```text -/api/manifest -/api/reports -/api/reports/{report_id}/summary -/api/reports/{report_id}/graph/{level} -/api/reports/{report_id}/display-graph -/api/reports/{report_id}/motifs -/api/aggregate-nodes/{aggregate_id} -/api/aggregate-edges/{aggregate_edge_id} -/api/aggregate-edges/{aggregate_edge_id}/mapping -/api/aggregate-edges/{aggregate_edge_id}/matrix -/api/reports/{report_id}/raw-nodes -/api/reports/{report_id}/raw-edges -/api/reports/{report_id}/subgraph -``` - -Raw pages default to 100 and cannot exceed 500. Subgraph depth cannot exceed five. Raw and expanded display graphs default to a combined 5,000-node/edge safety cap. The API returns no partial graph when it is exceeded and tells the user to select an aggregate view or increase the configured `expansion_cap`. These limits are configurable when embedding `create_app`. - -## Performance behavior - -- CSV readers stream rows and insert them in bounded batches. -- Temporary SQLite staging tables are function-scoped and dropped immediately; bulk joins resolve endpoints without per-edge node queries. -- CSVs without additional columns use a constant empty JSON representation. -- Ingestion closes its write connection before derived indexes, aggregation, motifs, and diagnostics reopen the database. -- Secondary indexes are created after raw insertion and focus on serving and aggregation queries. -- Raw edges are never retained as a Python object graph or loaded into NetworkX. -- Only aggregate operation nodes/edges enter NetworkX. -- Mapping statistics are grouped in SQL; exact stencil comparison streams one operation pair. -- API raw pages, display projections, expansions, and matrices have hard limits. -- Stable IDs derive only from report IDs, grouping keys, and original CSV row numbers. - -The normal suite contains a generated multi-lane smoke report. Set `RAPTOR_GRAPH_SLOW=1` to run the opt-in case with 100 operations, 1,000 lanes, and 100,000 raw edges: +## Tests and troubleshooting ```bash -RAPTOR_GRAPH_SLOW=1 python3.13 -m pytest tools/raptor_graph_explorer/tests/test_aggregation.py -m slow +.venv/bin/python -m pytest tools/raptor_graph_explorer/tests +RAPTOR_GRAPH_SLOW=1 .venv/bin/python -m pytest \ + tools/raptor_graph_explorer/tests/test_aggregation.py -m slow ``` -## Current limitations +The opt-in test generates 100 operations, 1,000 lanes, and 100,000 raw edges. -- The first browser load requires access to the pinned CDNs; offline vendoring is not included. -- Aggregate graph endpoints return the complete selected aggregate level. Raw graphs remain bounded, but an unusually huge operation aggregate may eventually need aggregate pagination. -- Motif representative paths operate at SCC-condensation granularity and choose a deterministic representative operation when an entry, exit, or branch SCC contains multiple operations. -- The API is local and read-only; authentication, write APIs, and a graph database server are intentionally out of scope. - -## Troubleshooting - -- **No complete report pairs:** check that both files share the exact prefix and end in `.nodes.csv` / `.edges.csv`. -- **Strict build failed:** run without `--strict`, then inspect the diagnostics panel or `diagnostics` table for bounded examples. -- **Output already exists:** pass `--overwrite` only when replacement is intended. -- **Unsupported schema:** rebuild the output with the installed tool version; old databases are not silently reinterpreted. -- **Raw/expanded display returns 400:** use Operation, Core, or Node kind view, or embed the app with a larger `expansion_cap` when the local machine can safely handle it. -- **Viewer is blank but APIs work:** verify browser access to the three pinned CDN URLs and inspect the browser console. -- **No motifs:** this is a valid result; motifs require a structural single-entry/single-exit split and rejoin, not a diamond-like screen shape. - -No Raptor build or compiler modification is required. +- **No complete pairs:** check matching prefixes and both CSV suffixes. +- **Strict build failed:** inspect diagnostics from a build without `--strict`. +- **Raw/expanded display returns 400:** use aggregation or adjust the cap. +- **Blank viewer with working APIs:** check browser CDN access and console. +- **No motifs:** valid when no structural split/rejoin region exists. diff --git a/validation/README.md b/validation/README.md index 1d5e612..0f0a5f8 100644 --- a/validation/README.md +++ b/validation/README.md @@ -1,265 +1,133 @@ # Raptor validation -`validate.py` validates every ONNX model below a selected directory. For each -model it can: - -1. compile an ONNX-MLIR reference library and runner; -2. generate deterministic random inputs; -3. compile Pim artifacts with Raptor; -4. run the reference implementation and functional Pim simulator; -5. compare their outputs; -6. run Pimsim to report latency, throughput, power, and energy. - -Run the script from the repository root with the repository Python environment. +`validate.py` discovers ONNX models recursively, compiles a native ONNX-MLIR +reference and Raptor artifacts, compares functional simulator outputs, and +optionally measures Pimsim latency, throughput, power, and energy. +Run from the repository root using `.venv`. ## Prerequisites -- A built Raptor compiler, normally - `build_release/Release/bin/onnx-mlir`. -- ONNX-MLIR runtime headers, normally `onnx-mlir/include`. -- The `numpy`, `onnx`, and `colorama` packages installed in `.venv`. -- The Rust toolchain used by the functional simulator. -- The Rust functional simulator under - `backend-simulators/pim/pim-simulator`, unless overridden. -- A built `pimsim-nn` under - `backend-simulators/pim/pimsim-nn/build`, unless non-functional simulation is - skipped or its path is overridden. +- Built release Raptor and ONNX-MLIR runtime headers. +- `numpy`, `onnx`, and `colorama` installed in the repository environment. +- Rust toolchain and `backend-simulators/pim/pim-simulator`. +- Built `backend-simulators/pim/pimsim-nn/build`, unless performance + simulation is skipped. -## Basic usage +See the [build guide](../README.md#build). -Validate the complete operation suite: +## Usage ```bash .venv/bin/python validation/validate.py \ --raptor-path build_release/Release/bin/onnx-mlir \ --onnx-include-dir onnx-mlir/include \ --operations-dir validation/operations \ - --verbose \ - --raptor-extra-arg=--pim-detect-communication-deadlock \ - --raptor-extra-arg=--pim-export-spatial-dataflow=none + --raptor-extra-arg=--pim-detect-communication-deadlock ``` -Validate one operation category or case: +Run the complete operations suite outside the sandbox so parallel workers can +start. For a focused case, change `--operations-dir` to +`validation/operations/gemm/small` or a network directory such as +`validation/networks/yolo11n/depth_04`. -```bash -.venv/bin/python validation/validate.py \ - --raptor-path build_release/Release/bin/onnx-mlir \ - --onnx-include-dir onnx-mlir/include \ - --operations-dir validation/operations/gemm/small -``` +Defaults run latency (one input) and throughput (pipeline 4, four distinct +inputs). Both reuse reference artifacts, and every throughput output is +compared with its own reference. Models run with four workers by default. +Results go to `validation_results.csv` in the selected model tree. -Validate a network or network slice: - -```bash -.venv/bin/python validation/validate.py \ - --raptor-path build_release/Release/bin/onnx-mlir \ - --onnx-include-dir onnx-mlir/include \ - --operations-dir validation/networks/yolo11n/depth_04 -``` - -`--operations-dir` may point to any directory tree containing `.onnx` files. -The script discovers them recursively and writes `validation_results.csv` in -that directory while retaining separate latency and throughput terminal tables. - -## Pim validation tools - -- [Pimcomp model suite](networks/pimcomp_models/README.md) -- [Pimcomp model comparison tools](tools/pim/pimcomp/compare/README.md) -- [Pimcomp correctness study](tools/pim/pimcomp/correctness/README.md) -- [Raptor compiler ablation study](tools/pim/ablation/README.md) - -## Validation modes - -The default mode runs latency and throughput in one validation job. Latency -uses one input, while throughput uses `--pipeline=4` with four distinct inputs. -Both modes reuse the generated input batch, native runner, and reference -outputs, and every throughput output is compared with its own reference. - -Use `--compile-only` to build the reference runner and Pim artifacts without -executing either implementation: - -```bash -.venv/bin/python validation/validate.py \ - --raptor-path build_release/Release/bin/onnx-mlir \ - --onnx-include-dir onnx-mlir/include \ - --operations-dir validation/operations/gemm/small \ - --compile-only -``` - -Use `--run-only` to reuse those artifacts and perform input generation, -reference execution, simulation, and comparison: - -```bash -.venv/bin/python validation/validate.py \ - --raptor-path build_release/Release/bin/onnx-mlir \ - --onnx-include-dir onnx-mlir/include \ - --operations-dir validation/operations/gemm/small \ - --run-only -``` - -`--compile-only` and `--run-only` are mutually exclusive. Run-only mode fails -with a diagnostic if its required compiled artifacts are missing. - -## Parallel execution and output - -Models run in parallel using all available CPUs by default. Set the worker -count with `-j` or `--jobs`: - -```bash -.venv/bin/python validation/validate.py \ - --raptor-path build_release/Release/bin/onnx-mlir \ - --onnx-include-dir onnx-mlir/include \ - --operations-dir validation/operations \ - --jobs 8 -``` +Append `--compile-only` to build artifacts without execution, then +`--run-only` to reuse them for input generation, execution, and comparison. +These flags are mutually exclusive; run-only fails if compiled artifacts are +missing. `--clean` removes generated artifacts and stale locks, and requires +neither compiler nor include paths. ## Options -| Option | Description | +Use `--help` for the full command reference. + +| Option | Purpose / default | |---|---| -| `-h`, `--help` | Print command help and exit. | -| `--raptor-path PATH` | Raptor compiler binary. Required unless `--clean` is used. | -| `--onnx-include-dir PATH` | ONNX-MLIR runtime include directory. Required unless `--clean` is used. | -| `--operations-dir PATH` | Directory tree containing models. Defaults to `validation/operations`. | -| `--simulator-dir PATH` | Functional `pim-simulator` crate directory. Defaults to the in-tree simulator. | -| `--non-functional-simulator-build-dir PATH` | Pimsim build directory. Defaults to the in-tree build. | -| `--pimcomp-config {arch-a,arch-b,arch-c}` | Non-functional hardware/timing profile. Defaults to `arch-a`. | -| `--skip-non-functional-simulation` | Skip Pimsim latency, throughput, power, and energy measurement. | -| `--no-fast` | Disable fast throughput convergence for authoritative full-duration Pimsim experiments. | -| `--threshold FLOAT` | Absolute output-comparison tolerance. Defaults to `1e-3`. | -| `--relative-threshold FLOAT` | Relative output-comparison tolerance. Defaults to `1e-5`. | -| `--seed INT` | Seed for generated inputs. Defaults to `0`. | -| `--crossbar-size INT` | Crossbar dimensions passed to Raptor. Defaults to the Arch-A value, `128`. | -| `--crossbar-count INT` | Crossbars per core passed to Raptor. Defaults to the Arch-A value, `96`. | -| `--core-count INT` | Pim core count passed to Raptor. Defaults to the Arch-A value, `168`. | -| `--raptor-extra-arg=ARG` | Additional Raptor compiler argument. Repeat for multiple arguments. | -| `--command-timeout-seconds FLOAT` | Timeout for each compiler, runner, and simulator subprocess. Defaults to `1000000.0`. | -| `-j INT`, `--jobs INT` | Parallel validation workers. Defaults to all available CPUs and must be at least one. | -| `--clean` | Remove generated validation artifacts and exit. | -| `--compile-only` | Compile reference and Pim artifacts without execution or comparison. | -| `--run-only` | Reuse compiled artifacts and perform execution, simulation, and comparison. | -| `--verbose` | Print passing per-stage and subprocess logs, plus average Pim pass timings. | +| `--raptor-path PATH` | Compiler binary; required except for cleanup. | +| `--onnx-include-dir PATH` | Runtime headers; required except for cleanup. | +| `--operations-dir PATH` | Model tree; default: `validation/operations`. | +| `--simulator-dir PATH` | Override the in-tree functional simulator crate. | +| `--non-functional-simulator-build-dir PATH` | Override the in-tree Pimsim build. | +| `--pimcomp-config {arch-a,arch-b,arch-c}` | Performance profile; default: Arch-A. | +| `--core-count N`, `--crossbar-count N`, `--crossbar-size N` | Raptor resources; defaults: Arch-A (168, 96, 128). | +| `--skip-non-functional-simulation` | Skip performance measurement. | +| `--no-fast` | Use full-duration throughput measurement. | +| `--threshold FLOAT`, `--relative-threshold FLOAT` | Output tolerances; defaults: `1e-3`, `1e-5`. | +| `--seed N` | Input RNG seed; default: 0. | +| `--raptor-extra-arg=ARG` | Forward compiler arguments; repeat as needed. | +| `--command-timeout-seconds FLOAT` | Subprocess timeout; default: 1000000 seconds. | +| `-j N`, `--jobs N` | Parallel workers; default: 4. | +| `--compile-only`, `--run-only`, `--clean` | Compile, reuse, or remove artifacts. | +| `--verbose` | Include passing subprocess logs and average Pim pass timings. | -Arguments beginning with `--` that are passed through to Raptor should use the -equals form: +Forwarded options beginning with `--` require the equals form, for example +`--raptor-extra-arg=--pim-detect-communication-deadlock`. -```bash ---raptor-extra-arg=--pim-detect-communication-deadlock -``` +## Performance and result interpretation -## Hardware profiles and non-functional simulation +The selected [hardware profile](networks/pimcomp_models/README.md#hardware-profiles) +must match the core/crossbar flags. A mismatch disables performance simulation +and prints the incompatible values; functional validation still runs. +Canonical configs and meshes live under `validation/pimsim_configs/pimcomp/`. -The selected Pimcomp profile must match `--core-count`, `--crossbar-count`, and -`--crossbar-size`. A mismatch disables only non-functional simulation and -prints the incompatible values; functional validation still runs. +Throughput defaults to `pimsim-nn --fast` with a 1000 ms convergence deadline. +It compares consecutive two-round windows at 1% tolerance and falls back to +full-duration results if convergence fails. Use `--no-fast` for authoritative +experiments. See [metric definitions](../docs/evaluation/methodology.md#metric-interpretation) +for measurement windows, power, and energy interpretation. -The checked-in profiles are under -`validation/pimsim_configs/pimcomp//`; latency uses -`latency_config.json`, while throughput uses -`throughput_config_