diff --git a/README.md b/README.md index 66350ee..20c6a91 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ targeting in-memory computing / processing-in-memory (PIM) architectures. It extends ONNX-MLIR with a PIM accelerator and progressively lowers ONNX-MLIR through custom MLIR dialects to simulator artifacts. -The current target is the PIM simulator stack under `backend-simulators/pim`. +The current target is the Pim simulator stack under `backend-simulators/pim`. Raptor emits binary per-core `.pim` instruction files by default, plus `memory.bin`, `config.json`, and weight binaries. It can also emit per-core JSON instruction files with `--pim-emit-json`. @@ -29,9 +29,9 @@ lowering, scheduling, memory layout, and code-generation optimizations. - `backend-simulators/pim/pim-simulator` is the in-tree Rust functional simulator used by validation. It reads Raptor's `pim/` artifact directory and compares simulator output against native ONNX-MLIR execution. -- `backend-simulators/pim/pimsim-nn` is the non-functional simulator submodule - used internally by validation for latency, power, and energy. - The helper scripts in `pimcomp_utils/` are for comparison with PIMCOMP-NN and +- `backend-simulators/pim/pimsim-nn` contains the non-functional Pimsim + simulator used internally by validation for latency, power, and energy. + The helper scripts in `pimcomp_utils/` are for comparison with Pimcomp and contain local paths; treat them as local utilities, not portable workflows. ## Compilation pipeline @@ -43,7 +43,7 @@ them to ONNX-MLIR through generated shim directories under High-level lowering flow: ``` -ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> PIM artifacts +ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> Pim artifacts ``` 1. **ONNX -> Spatial** (`src/PIM/Conversion/ONNXToSpatial`). @@ -81,20 +81,20 @@ ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> PIM artifacts addressable accesses, and `PimBufferizationVerification` checks tensor absence, contiguity, and copy address spaces. -5. **PIM local-memory planning** +5. **Pim local-memory planning** (`src/PIM/Dialect/Pim/Passes/Transforms/LocalMemoryPlanning`). Computes whole-core lifetimes, reuses addresses for non-overlapping - allocations, and records the explicit plan in PIM IR. Reusable lifetime + allocations, and records the explicit plan in Pim IR. Reusable lifetime analysis lives under `src/PIM/Dialect/Pim/Passes/Analyses`. -6. **PIM verification and code generation** (`src/PIM/Passes/PimCodegen` and +6. **Pim verification and code generation** (`src/PIM/Passes/PimCodegen` and `src/PIM/Compiler`). - Verifies the memory plan and other PIM invariants, then emits `.pim` core + Verifies the memory plan and other Pim invariants, then emits `.pim` core files, weights, and `memory.bin` / `config.json` without rerunning liveness. Supporting pieces: - `src/PIM/Common` - shared IR, filesystem, diagnostics, reports, and utility helpers. -- `src/PIM/Compiler` - PIM compiler options, planned-address materialization, binary +- `src/PIM/Compiler` - Pim compiler options, planned-address materialization, binary instruction format, artifact writing, weight emission, and codegen entry points. - `src/PIM/Conversion/SpatialToGraphviz` - optional Spatial graphviz conversion @@ -102,42 +102,97 @@ Supporting pieces: - `src/PIM/Passes` - pass registration and auxiliary passes. - `src/PIM/PimAccelerator.{cpp,hpp}` - ONNX-MLIR accelerator entry point. -## PIM compiler options +## Pim compiler options -Pass these to `onnx-mlir` when compiling for PIM. These are all Raptor/PIM-specific +Pass these to `onnx-mlir` when compiling for Pim. These are all Raptor/Pim-specific options; `onnx-mlir --help` lists the inherited ONNX-MLIR options. -- `--maccel=PIM` - select the PIM accelerator. +- `--maccel=PIM` - select the Pim accelerator. Default: no Pim accelerator. - `--EmitSpatial`, `--EmitPim`, `--EmitPimBufferized`, - `--EmitPimCodegen` - stop the PIM pipeline at the requested stage. The PIM - default is `--EmitPimCodegen`. -- `--core-count=` - required positive core count for PIM compilation. -- `--pipeline=` - throughput pipeline stages. `1` preserves latency - scheduling; values above `1` require `core-count` to be divisible by `M`. -- `--crossbar-size=` - crossbar width/height. Default in code is `128`. -- `--crossbar-count=` - crossbars per core. Default in code is `64`. -- `--pim-target-config=` - optional PIM target configuration used by the + `--EmitPimCodegen` - stop the Pim pipeline at the requested stage. Default: + `--EmitPimCodegen` for Pim compilation. +- `--core-count=` - required positive core count for Pim compilation. + Default: none; this option is required. +- `--crossbar-size=` - required positive crossbar width/height for Pim + compilation. Default: none; this option is required. +- `--crossbar-count=` - required positive crossbar count per core for Pim + compilation. Default: none; this option is required. +- `--pipeline=` - number of throughput pipeline stages; `1` preserves + latency scheduling. Default: `1`. +- `--pim-target-config=` - optional Pim target configuration used by the target adapter to construct the target-neutral Spatial scheduling cost and topology model. Resource values must match the explicit core/crossbar flags. + Default: empty; use the built-in target model. - `--pim-memory-report=` - emit the concise combined memory report - under `reports/memory_report.txt`, or disable it. Default is `summary`. -- `--pim-only-codegen` - assume input is already bufferized PIM IR and only run - the codegen tail. + under `reports/memory_report.txt`, or disable it. Default: `summary`. +- `--pim-only-codegen` - assume input is already bufferized Pim IR and only run + the codegen tail. Default: off. +- `--pim-disable-synchronization` - omit generated `wait` and `sync` + instructions for performance ablation. Default: off. +- `--pim-disable-spatial-planning` - select the first, trivial DenseNCHW layout + alternative for every Spatial plan operation, disabling cost-based layout + planning while leaving ONNX rewrites and graph-compute merging enabled. + Default: off. + +### Spatial layout plan variants + +Spatial plan operations advertise alternatives as an exact combination of +operand physical layouts and one result physical layout. Every plan operation +has the default `DenseNCHW -> DenseNCHW` alternative. The planner can select +the following additional variants when the operation, tensor shapes, and +target resources make them legal: + +| Physical layout or plan | Meaning and current use | +|---|---| +| `DenseNCHW` | Ordinary dense NCHW storage. This is the first alternative and the one selected by `--pim-disable-spatial-planning`. | +| `NHWCRowStrip` | Row-strip storage for NCHW logical tensors: spatial rows are processed as channel vectors. This enables row-strip lowering through compatible chains. | +| `Fragmented` | Fragmented physical input accepted by `Flatten`, which reassembles it to dense NCHW. It is not currently selected as a plan result. | +| `NCHWRowStrip` | A Spatial IR layout enum value reserved for NCHW-oriented row strips; current layout-capability implementations do not advertise it as a plan alternative. | + +The operation-specific non-trivial alternatives are: + +| Plan operation | Additional alternatives beyond dense NCHW | +|---|---| +| `Conv2D` | Dense input to row-strip output, or row-strip input to row-strip output when the target-dependent Conv lowering supports it. | +| `Flatten` | Fragmented input to dense output, or row-strip input to dense output when legal. | +| `Relu` | Row-strip input to row-strip output. | +| `SiLU` | Row-strip input to row-strip output, with a stronger intrinsic cost preference than the generic row-strip variant. | +| `ResizeNearest` | Row-strip input to row-strip output when its lowering is legal. | +| `MaxPool2D` | Dense input to row-strip output, or row-strip input to row-strip output. | +| `GlobalAveragePool` | Dense input to row-strip output, or row-strip input to row-strip output. | +| `BiasAdd` | Row-strip data input plus a dense bias input to row-strip output when the bias shape is supported. | +| `Add` | All data inputs row-strip to row-strip output. | +| `Concat` | All inputs row-strip to row-strip output. | + +Cost-based planning scores intrinsic alternative cost, operand layout +mismatches, and downstream incompatibility, then iterates in alternating +forward and reverse operation order until the bounded analysis converges. +Function results are required to remain `DenseNCHW`; explicit materialization +operations reconcile layout mismatches at boundaries. With +`--pim-disable-spatial-planning`, the pass still runs and records a valid plan, +but chooses the first dense alternative for every plan operation. Later graph +compute merging is unchanged, so elementwise operations such as `Relu` remain +separate from neighboring parallel operations and can create fan-out/fan-in +diamonds. - `--pim-emit-json` - also emit `core_*.json` instruction files alongside - `core_*.pim`. + `core_*.pim`. Default: off. - `--pim-export-spatial-dataflow=` - control Spatial dataflow CSV reports for the graph, trivially merged graph, - scheduled, and realized snapshots under `reports/`. Default is `none`. + scheduled, and realized snapshots under `reports/`. Default: `none`. - `--pim-conv-lowering=` - - select the convolution lowering strategy. Default is `auto`. + select the convolution lowering strategy. Default: `auto`. - `--pim-conv-im2col-max-elements=` - maximum globally materialized im2col - elements per convolution before streaming. Default is `1048576`. + elements per convolution before streaming. Default: `1048576`. - `--pim-conv-stream-chunk-positions=` - maximum output positions per - streamed convolution chunk. Default is `1024`. + streamed convolution chunk. Default: `1024`. +- `--pim-report-conv-lowering=` - emit a bounded convolution + lowering report. Default: `true`. - `--pim-detect-communication-deadlock` - statically simulate expanded - send/receive ordering and reject blocking deadlocks. Default is off. + send/receive ordering and reject blocking deadlocks. Default: off. +- `--pim-verify-bufferization-copy-freedom` - run the expensive official Pim + tensor-copy freedom proof before bufferization. Default: off. -## Standard PIM hardware profile +## Standard Pim hardware profile Raptor's standard development and YOLO validation profile is: @@ -151,7 +206,8 @@ Canonical compiler flags: `--crossbar-count=64 --crossbar-size=128 --core-count=144` -`--core-count` remains mandatory and must be passed explicitly to the compiler. +`--crossbar-size`, `--crossbar-count`, and `--core-count` remain mandatory and +must be passed explicitly to the compiler. Example: @@ -161,11 +217,11 @@ Example: --crossbar-count=64 --crossbar-size=128 --core-count=144 ``` -This writes PIM artifacts under `/tmp/raptor/pim/`. +This writes Pim artifacts under `/tmp/raptor/pim/`. ## Validation -Functional validation compiles ONNX models, compares native ONNX-MLIR and PIM +Functional validation compiles ONNX models, compares native ONNX-MLIR and Pim simulator outputs, and optionally reports latency, power, and energy. See [`validation/README.md`](validation/README.md) for prerequisites, usage, options, artifacts, and results. @@ -278,7 +334,7 @@ cd backend-simulators/pim/pim-simulator cargo test ``` -## Repository Layout +## Repository layout - `src/PIM/` - PIM accelerator implementation. - `test/PIM/` - PIM C++ unit tests. @@ -286,6 +342,6 @@ cargo test slices, and pimsim config generation. - `backend-simulators/pim/pim-simulator/` - in-tree Rust functional simulator. - `backend-simulators/pim/pimsim-nn/` - non-functional simulator submodule. -- `pimcomp_utils/` - local comparison helpers for PIMCOMP-NN. +- `pimcomp_utils/` - local comparison helpers for Pimcomp. - `.github/actions/` and `.github/workflows/validate_operations.yml` - CI setup for MLIR/Protobuf caching, building Raptor, and validation. diff --git a/backend-simulators/pim/pim-simulator/src/bin/pim-simulator/main.rs b/backend-simulators/pim/pim-simulator/src/bin/pim-simulator/main.rs index 5358af1..38b538e 100644 --- a/backend-simulators/pim/pim-simulator/src/bin/pim-simulator/main.rs +++ b/backend-simulators/pim/pim-simulator/src/bin/pim-simulator/main.rs @@ -54,9 +54,9 @@ struct Args { #[arg(long)] batch_size: Option, - /// Input binary for one iteration; repeat once per batch entry - #[arg(long = "input")] - inputs: Vec, + /// Directory containing input_*.bin files, one per batch entry + #[arg(long = "input-dir")] + input_dir: PathBuf, /// Optional directory for per-iteration output dumps #[arg(long)] @@ -278,16 +278,9 @@ fn input_regions(config: &Value) -> Result> { } fn retrieve_inputs(args: &Args, batch_size: u32) -> Result>> { - if args.inputs.len() != batch_size as usize { - bail!( - "batch size {batch_size} requires {} inputs, got {}", - batch_size, - args.inputs.len() - ); - } - args.inputs - .iter() - .map(|path| fs::read(path).with_context(|| format!("Failed to read input file: {path:?}"))) + (0..batch_size) + .map(|index| args.input_dir.join(format!("input_{index}.bin"))) + .map(|path| fs::read(&path).with_context(|| format!("Failed to read input file: {path:?}"))) .collect() } diff --git a/backend-simulators/pim/pim-simulator/src/lib/binary_to_instruction/mod.rs b/backend-simulators/pim/pim-simulator/src/lib/binary_to_instruction/mod.rs index 61e0edd..e54c218 100644 --- a/backend-simulators/pim/pim-simulator/src/lib/binary_to_instruction/mod.rs +++ b/backend-simulators/pim/pim-simulator/src/lib/binary_to_instruction/mod.rs @@ -80,19 +80,19 @@ fn read_i32_le(bytes: &[u8], offset: usize) -> i32 { fn parse_binary_records(bytes: &[u8]) -> Result> { ensure!(bytes.len() >= HEADER_SIZE, "binary core file too small"); - ensure!(&bytes[0..4] == MAGIC, "invalid PIM binary magic"); + ensure!(&bytes[0..4] == MAGIC, "invalid Pim binary magic"); let version = read_u32_le(bytes, 4); ensure!( version == VERSION, - "unsupported PIM binary version {version}" + "unsupported Pim binary version {version}" ); let instruction_count = read_u32_le(bytes, 8) as usize; let expected_len = HEADER_SIZE + instruction_count * RECORD_SIZE; ensure!( bytes.len() == expected_len, - "PIM binary size mismatch: expected {expected_len} bytes, got {}", + "Pim binary size mismatch: expected {expected_len} bytes, got {}", bytes.len() ); @@ -335,7 +335,7 @@ fn append_record( .set_offset_select_value(generic1, 0); inst_builder.make_inst(sync, inst_data_builder.build()); } - _ => bail!("unsupported PIM binary opcode {opcode}"), + _ => bail!("unsupported Pim binary opcode {opcode}"), } Ok(()) } diff --git a/backend-simulators/pim/pim-simulator/src/lib/pimcore.rs b/backend-simulators/pim/pim-simulator/src/lib/pimcore.rs index 17da145..83a83ce 100644 --- a/backend-simulators/pim/pim-simulator/src/lib/pimcore.rs +++ b/backend-simulators/pim/pim-simulator/src/lib/pimcore.rs @@ -802,7 +802,7 @@ impl<'a> Executable<'a> { pub fn dump(&self) { let core_instructions = &self.core_instructions; for (i, core_instruction) in core_instructions.iter().enumerate() { - eprintln!("INST OF CORE {}:", i); + eprintln!("Instructions for core {}:", i); for inst in &core_instruction.instructions { inst.dump(); } diff --git a/src/PIM/Common/IR/AddressAnalysis.hpp b/src/PIM/Common/IR/AddressAnalysis.hpp index 7a2240b..feb99d2 100644 --- a/src/PIM/Common/IR/AddressAnalysis.hpp +++ b/src/PIM/Common/IR/AddressAnalysis.hpp @@ -19,7 +19,7 @@ struct ResolvedContiguousAddress { }; /// Records compile-time facts used when interpreting address arithmetic and -/// loop-carried aliases inside PIM regions. +/// loop-carried aliases inside Pim regions. struct StaticValueKnowledge { llvm::DenseMap indexValues; llvm::DenseMap aliases; diff --git a/src/PIM/Common/IR/CoreBlockUtils.cpp b/src/PIM/Common/IR/CoreBlockUtils.cpp index 814b197..d9d0b37 100644 --- a/src/PIM/Common/IR/CoreBlockUtils.cpp +++ b/src/PIM/Common/IR/CoreBlockUtils.cpp @@ -85,12 +85,12 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block, auto step = resolveIndexValue(forOp.getStep(), knowledge); if (failed(lower) || failed(upper) || failed(step) || (mode == CoreWalkMode::ExecuteCommunication && *step <= 0)) { - forOp.emitOpError() << "requires statically evaluable scf.for bounds for PIM " << purpose; + forOp.emitOpError() << "requires statically evaluable scf.for bounds for Pim " << purpose; hasFailure = true; continue; } if (*step <= 0) { - forOp.emitOpError("requires positive scf.for step for PIM verification"); + forOp.emitOpError("requires positive scf.for step for Pim verification"); hasFailure = true; continue; } @@ -126,7 +126,7 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block, if (auto ifOp = mlir::dyn_cast(op)) { auto condition = resolveIndexValue(ifOp.getCondition(), knowledge); if (failed(condition)) { - ifOp.emitOpError() << "requires statically evaluable scf.if condition for PIM " << purpose; + ifOp.emitOpError() << "requires statically evaluable scf.if condition for Pim " << purpose; hasFailure = true; continue; } @@ -147,7 +147,7 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block, if (auto switchOp = mlir::dyn_cast(op)) { auto selector = resolveIndexValue(switchOp.getArg(), knowledge); if (failed(selector)) { - switchOp.emitOpError() << "requires a statically evaluable scf.index_switch selector for PIM " << purpose; + switchOp.emitOpError() << "requires a statically evaluable scf.index_switch selector for Pim " << purpose; hasFailure = true; continue; } diff --git a/src/PIM/Common/IR/CoreBlockUtils.hpp b/src/PIM/Common/IR/CoreBlockUtils.hpp index d41828c..c288367 100644 --- a/src/PIM/Common/IR/CoreBlockUtils.hpp +++ b/src/PIM/Common/IR/CoreBlockUtils.hpp @@ -14,7 +14,7 @@ namespace onnx_mlir { using PimCoreCommunicationPlan = llvm::DenseMap>; /// Returns true for ops in a `pim.core` body that only participate in static -/// address or index computation and therefore do not emit PIM instructions. +/// address or index computation and therefore do not emit Pim instructions. bool isCoreStaticAddressOp(mlir::Operation* op); /// Walks a `pim.core` body's communication stream, statically unrolling diff --git a/src/PIM/Common/IR/EntryPointUtils.cpp b/src/PIM/Common/IR/EntryPointUtils.cpp index 88fd10c..5cc0dc1 100644 --- a/src/PIM/Common/IR/EntryPointUtils.cpp +++ b/src/PIM/Common/IR/EntryPointUtils.cpp @@ -9,7 +9,7 @@ llvm::FailureOr getPimEntryFunc(mlir::ModuleOp moduleOp) { llvm::SmallVector entryPoints(moduleOp.getOps()); if (entryPoints.size() > 1) { - moduleOp.emitError("PIM pipeline requires a single ONNX entry point, but found ") << entryPoints.size(); + moduleOp.emitError("Pim pipeline requires a single ONNX entry point, but found ") << entryPoints.size(); return mlir::failure(); } if (!entryPoints.empty()) { @@ -38,7 +38,7 @@ llvm::FailureOr getPimEntryFunc(mlir::ModuleOp moduleOp) { if (nonExternalFuncs.size() == 1) return nonExternalFuncs.front(); - moduleOp.emitError("could not resolve a unique PIM entry function"); + moduleOp.emitError("could not resolve a unique Pim entry function"); return mlir::failure(); } diff --git a/src/PIM/Common/IR/EntryPointUtils.hpp b/src/PIM/Common/IR/EntryPointUtils.hpp index 907c1a7..0bbf612 100644 --- a/src/PIM/Common/IR/EntryPointUtils.hpp +++ b/src/PIM/Common/IR/EntryPointUtils.hpp @@ -5,7 +5,7 @@ namespace onnx_mlir { -/// Resolves the function the PIM pipeline should treat as its entry point. +/// Resolves the function the Pim pipeline should treat as its entry point. /// Prefers ONNX entry-point metadata, then `main_graph`, then the only /// non-external function if the module is otherwise unambiguous. llvm::FailureOr getPimEntryFunc(mlir::ModuleOp moduleOp); diff --git a/src/PIM/Common/IR/WeightUtils.hpp b/src/PIM/Common/IR/WeightUtils.hpp index 4628d6a..3a16b6f 100644 --- a/src/PIM/Common/IR/WeightUtils.hpp +++ b/src/PIM/Common/IR/WeightUtils.hpp @@ -32,7 +32,7 @@ struct ResolvedWeightView { bool hasWeightAlways(mlir::Operation* op); /// Tags an op as producing a value that should stay materialized as a reusable -/// weight across later PIM lowering/codegen stages. +/// weight across later Pim lowering/codegen stages. void markWeightAlways(mlir::Operation* op); bool isSpatialMvmVmmWeightUse(mlir::OpOperand& use); diff --git a/src/PIM/Common/Support/CheckedArithmetic.cpp b/src/PIM/Common/Support/CheckedArithmetic.cpp index 77a8769..5ca07c2 100644 --- a/src/PIM/Common/Support/CheckedArithmetic.cpp +++ b/src/PIM/Common/Support/CheckedArithmetic.cpp @@ -11,7 +11,7 @@ namespace onnx_mlir::pim { namespace { static void emitCrashMessage(llvm::StringRef fieldName, llvm::StringRef message) { - llvm::errs() << "PIM " << fieldName << " " << message << "\n"; + llvm::errs() << "Pim " << fieldName << " " << message << "\n"; } template @@ -65,7 +65,7 @@ InFlightDiagnostic emitCheckedArithmeticError(Operation* anchor, llvm::StringRef } InFlightDiagnostic emitCheckedArithmeticError(Location loc, llvm::StringRef fieldName, llvm::StringRef message) { - return emitError(loc) << "PIM " << fieldName << " " << message; + return emitError(loc) << "Pim " << fieldName << " " << message; } FailureOr checkedI32(int64_t value, Operation* anchor, llvm::StringRef fieldName) { @@ -174,7 +174,7 @@ FailureOr getCheckedShapedTypeSizeInBytes(ShapedType type, Location lo int32_t checkedI32OrCrash(int64_t value, llvm::StringRef fieldName) { if (value < std::numeric_limits::min() || value > std::numeric_limits::max()) { emitCrashMessage(fieldName, "is outside representable range"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return static_cast(value); } @@ -182,7 +182,7 @@ int32_t checkedI32OrCrash(int64_t value, llvm::StringRef fieldName) { int32_t checkedI32OrCrash(uint64_t value, llvm::StringRef fieldName) { if (value > static_cast(std::numeric_limits::max())) { emitCrashMessage(fieldName, "is outside representable range"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return static_cast(value); } @@ -190,7 +190,7 @@ int32_t checkedI32OrCrash(uint64_t value, llvm::StringRef fieldName) { uint8_t checkedU8OrCrash(uint64_t value, llvm::StringRef fieldName) { if (value > static_cast(std::numeric_limits::max())) { emitCrashMessage(fieldName, "is outside representable range"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return static_cast(value); } @@ -198,7 +198,7 @@ uint8_t checkedU8OrCrash(uint64_t value, llvm::StringRef fieldName) { size_t checkedSizeOrCrash(int64_t value, llvm::StringRef fieldName) { if (value < 0) { emitCrashMessage(fieldName, "is outside representable range"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return static_cast(value); } @@ -206,7 +206,7 @@ size_t checkedSizeOrCrash(int64_t value, llvm::StringRef fieldName) { size_t checkedAddOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) { if (rhs > std::numeric_limits::max() - lhs) { emitCrashMessage(fieldName, "addition overflow"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return lhs + rhs; } @@ -214,7 +214,7 @@ size_t checkedAddOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) { size_t checkedMulOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) { if (lhs != 0 && rhs > std::numeric_limits::max() / lhs) { emitCrashMessage(fieldName, "multiplication overflow"); - llvm_unreachable("PIM checked arithmetic failure"); + llvm_unreachable("Pim checked arithmetic failure"); } return lhs * rhs; } diff --git a/src/PIM/Common/Support/FileSystemUtils.hpp b/src/PIM/Common/Support/FileSystemUtils.hpp index 34386d0..64f76db 100644 --- a/src/PIM/Common/Support/FileSystemUtils.hpp +++ b/src/PIM/Common/Support/FileSystemUtils.hpp @@ -4,7 +4,7 @@ namespace onnx_mlir { -/// Returns the directory that should hold PIM artifacts/debug dumps for the +/// Returns the directory that should hold Pim artifacts/debug dumps for the /// current compiler invocation. std::string getOutputDir(); diff --git a/src/PIM/Compiler/PimBinaryFormat.hpp b/src/PIM/Compiler/PimBinaryFormat.hpp index 26673b6..bff1209 100644 --- a/src/PIM/Compiler/PimBinaryFormat.hpp +++ b/src/PIM/Compiler/PimBinaryFormat.hpp @@ -171,19 +171,19 @@ inline Opcode opcodeFromString(llvm::StringRef opName) { for (auto [index, name] : llvm::enumerate(kOpcodeNames)) if (opName == name) return static_cast(index); - llvm_unreachable("Unsupported PIM binary opcode"); + llvm_unreachable("Unsupported Pim binary opcode"); } inline llvm::StringRef opcodeToString(Opcode opcode) { size_t index = static_cast(opcode); - assert(index < kOpcodeNames.size() && "Unsupported PIM binary opcode"); + assert(index < kOpcodeNames.size() && "Unsupported Pim binary opcode"); return kOpcodeNames[index]; } inline InstructionRecord makeInstructionRecord(const llvm::json::Object& instruction) { InstructionRecord record; std::optional opName = instruction.getString("op"); - assert(opName && "Missing op field in PIM instruction"); + assert(opName && "Missing op field in Pim instruction"); record.opcode = opcodeFromString(*opName); const auto& format = kInstructionJsonFormats[static_cast(record.opcode)]; if (format.rd) diff --git a/src/PIM/Compiler/PimCodeGen.cpp b/src/PIM/Compiler/PimCodeGen.cpp index 002dd4e..ec21da1 100644 --- a/src/PIM/Compiler/PimCodeGen.cpp +++ b/src/PIM/Compiler/PimCodeGen.cpp @@ -125,7 +125,7 @@ static bool isZeroSplatGlobal(mlir::Value value) { return false; } -// PIM instruction immediates are serialized as signed int32_t fields today +// Pim instruction immediates are serialized as signed int32_t fields today // (`sldi` goes through checkedI32OrCrash), so local addresses must stay within // the non-negative int32_t range. static FailureOr checkedAlignTo(size_t value, size_t alignment, Operation* anchor, StringRef fieldName) { @@ -141,7 +141,7 @@ static void printMemoryOverflowDiagnostic(const MemoryValueKey& key, size_t requestedSize, size_t currentFirstAvailableAddress, size_t alignedEndAddress) { - llvm::errs() << "PIM local memory allocation overflow\n"; + llvm::errs() << "Pim local memory allocation overflow\n"; llvm::errs() << "Requested allocation size: " << requestedSize << " bytes\n"; llvm::errs() << "Current firstAvailableAddress: " << currentFirstAvailableAddress << "\n"; llvm::errs() << "Aligned end address: " << alignedEndAddress << "\n"; @@ -187,7 +187,7 @@ size_t PimMemory::allocateAddress(size_t size, const MemoryValueKey& key) { size, firstAvailableAddress, succeeded(checkedAlignedEnd) ? *checkedAlignedEnd : kPimLocalMemoryAddressLimit); - llvm_unreachable("PIM local memory allocation overflow"); + llvm_unreachable("Pim local memory allocation overflow"); } firstAvailableAddress = *checkedAlignedEnd; return address; @@ -276,7 +276,7 @@ void PimMemory::allocateCore(const CompiledCoreMemoryPlan& plan, std::optionalbyteOffset, "resolved PIM byte offset"); - return pim::checkedAddOrCrash(iter->second.address, byteOffset, "resolved PIM address"); + size_t byteOffset = pim::checkedSizeOrCrash(resolvedAddress->byteOffset, "resolved Pim byte offset"); + return pim::checkedAddOrCrash(iter->second.address, byteOffset, "resolved Pim address"); } llvm::FailureOr PimAcceleratorMemory::getIndexValue(mlir::Value value, @@ -706,6 +706,8 @@ void PimCodeGen::codeGenSendOp(pim::PimSendOp sendOp, const StaticValueKnowledge void PimCodeGen::codeGenWaitOp( pim::PimWaitOp waitOp, const StaticValueKnowledge& knowledge) const { + if (pimDisableSynchronization) + return; auto eventRegister = indexOf(waitOp.getEventRegister(), knowledge); auto waitValue = indexOf(waitOp.getWaitValue(), knowledge); assert(succeeded(eventRegister) && succeeded(waitValue) @@ -722,6 +724,8 @@ void PimCodeGen::codeGenWaitOp( void PimCodeGen::codeGenSyncOp( pim::PimSyncOp syncOp, const StaticValueKnowledge& knowledge) const { + if (pimDisableSynchronization) + return; auto targetCoreId = indexOf(syncOp.getTargetCoreId(), knowledge); auto eventRegister = indexOf(syncOp.getEventRegister(), knowledge); assert(succeeded(targetCoreId) && succeeded(eventRegister) @@ -958,7 +962,7 @@ static LogicalResult executeCompiledCorePlan( auto step = node.step.evaluate(knowledge); auto forOp = cast(node.op); if (failed(lowerBound) || failed(upperBound) || failed(step) || *step <= 0) { - forOp.emitOpError("requires statically evaluable scf.for bounds for PIM codegen"); + forOp.emitOpError("requires statically evaluable scf.for bounds for Pim codegen"); return failure(); } @@ -984,7 +988,7 @@ static LogicalResult executeCompiledCorePlan( auto condition = node.condition.evaluate(knowledge); auto ifOp = cast(node.op); if (failed(condition)) { - ifOp.emitOpError("requires statically evaluable scf.if condition for PIM codegen"); + ifOp.emitOpError("requires statically evaluable scf.if condition for Pim codegen"); return failure(); } @@ -998,7 +1002,7 @@ static LogicalResult executeCompiledCorePlan( auto selector = node.condition.evaluate(knowledge); auto switchOp = cast(node.op); if (failed(selector)) { - switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for PIM codegen"); + switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for Pim codegen"); return failure(); } const llvm::SmallVectorImpl* selectedBody = node.defaultBody.get(); @@ -1184,12 +1188,12 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std:: } auto getCompiledProgram = [&](Operation* op) { auto it = compiledPrograms.find(op); - assert(it != compiledPrograms.end() && "missing compiled PIM core program"); + assert(it != compiledPrograms.end() && "missing compiled Pim core program"); return it->second.get(); }; auto getMemoryPlan = [&](Operation* op) { auto it = memoryPlans.find(op); - assert(it != memoryPlans.end() && "missing PIM core memory plan"); + assert(it != memoryPlans.end() && "missing Pim core memory plan"); return it->second.get(); }; @@ -1267,7 +1271,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std:: if (failed(weightView)) { std::string message; llvm::raw_string_ostream os(message); - os << "requires a statically resolvable dense global weight view during PIM codegen; weight=" + os << "requires a statically resolvable dense global weight view during Pim codegen; weight=" << vmmOp.getWeight() << " type=" << vmmOp.getWeight().getType(); result.recordDiagnostic(vmmOp, os.str()); return failure(); @@ -1275,7 +1279,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std:: if (weightView->shape.size() != 2) { std::string message; llvm::raw_string_ostream os(message); - os << "requires a rank-2 matrix weight view during PIM codegen; resolved shape=["; + os << "requires a rank-2 matrix weight view during Pim codegen; resolved shape=["; llvm::interleaveComma(weightView->shape, os); os << "] weight=" << vmmOp.getWeight() << " type=" << vmmOp.getWeight().getType(); result.recordDiagnostic(vmmOp, os.str()); @@ -1387,7 +1391,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std:: } if (diagnostics.hasFailure()) diagnostics.emitSuppressedSummary(summaryAnchor ? summaryAnchor : moduleOp.getOperation(), - "PIM codegen diagnostic(s)"); + "Pim codegen diagnostic(s)"); for (size_t jobIndex = 0; jobIndex < jobs.size(); ++jobIndex) if (jobResults[jobIndex].status != CompilerSuccess) @@ -1453,7 +1457,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std:: if (!batchPerCoreRow) batchPerCoreRow = result.reportRow; else if (!(*batchPerCoreRow == result.reportRow)) - llvm_unreachable("one PIM core batch produced inconsistent per-core memory reports"); + llvm_unreachable("one Pim core batch produced inconsistent per-core memory reports"); } uint64_t batchReportId = jobs[group.front()].batchReportId.value_or(0); diff --git a/src/PIM/Compiler/PimCompilerOptions.cpp b/src/PIM/Compiler/PimCompilerOptions.cpp index aaf67c7..1c03f3c 100644 --- a/src/PIM/Compiler/PimCompilerOptions.cpp +++ b/src/PIM/Compiler/PimCompilerOptions.cpp @@ -9,25 +9,25 @@ namespace onnx_mlir { llvm::cl::opt pimEmissionTarget( - llvm::cl::desc("[Optional] Choose PIM-related target to emit (once selected it will cancel the other targets):"), - llvm::cl::values(clEnumVal(EmitSpatial, "Lower model to spatial IR")), - llvm::cl::values(clEnumVal(EmitPim, "Lower model to PIM IR")), - llvm::cl::values(clEnumVal(EmitPimBufferized, "Lower model to PIM IR and bufferize it")), - llvm::cl::values(clEnumVal(EmitPimCodegen, "Lower model to PIM IR and generate code for PIM")), + llvm::cl::desc("[Optional] Choose Pim-related target to emit (once selected it will cancel the other targets):"), + llvm::cl::values(clEnumVal(EmitSpatial, "Lower model to Spatial IR")), + llvm::cl::values(clEnumVal(EmitPim, "Lower model to Pim IR")), + llvm::cl::values(clEnumVal(EmitPimBufferized, "Lower model to Pim IR and bufferize it")), + llvm::cl::values(clEnumVal(EmitPimCodegen, "Lower model to Pim IR and generate code for Pim")), llvm::cl::init(EmitPimCodegen), llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt pimMemoryReport( "pim-memory-report", - llvm::cl::desc("Emit a human-readable PIM memory planning report"), - llvm::cl::values(clEnumValN(PimMemoryReportNone, "none", "Do not emit any PIM memory planning report")), - llvm::cl::values(clEnumValN(PimMemoryReportSummary, "summary", "Emit a concise PIM memory summary")), + llvm::cl::desc("Emit a human-readable Pim memory planning report"), + llvm::cl::values(clEnumValN(PimMemoryReportNone, "none", "Do not emit any Pim memory planning report")), + llvm::cl::values(clEnumValN(PimMemoryReportSummary, "summary", "Emit a concise Pim memory summary")), llvm::cl::init(PimMemoryReportSummary), llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt pimConvLowering( "pim-conv-lowering", - llvm::cl::desc("Convolution lowering strategy for PIM"), + llvm::cl::desc("Convolution lowering strategy for Pim"), llvm::cl::values(clEnumValN(PimConvLoweringAuto, "auto", "Select the Conv lowering strategy automatically")), llvm::cl::values(clEnumValN(PimConvLoweringLegacy, "legacy", "Use the legacy explicit-im2col Conv lowering")), llvm::cl::values(clEnumValN(PimConvLoweringDepthwise, "depthwise", "Force the depthwise-specialized Conv lowering")), @@ -55,20 +55,20 @@ llvm::cl::opt pimExportSpatialDataflow( llvm::cl::desc("Emit Gephi-importable CSV dataflow reports for Spatial pipeline snapshots"), llvm::cl::values(clEnumValN(SpatialDataflowExportNone, "none", "Do not emit Spatial dataflow CSV reports")), llvm::cl::values( - clEnumValN(SpatialDataflowExportSpatial1, "spatial1", "Emit spatial1 graph dataflow CSV reports")), + clEnumValN(SpatialDataflowExportSpatial1, "spatial1", "Emit Spatial1 graph dataflow CSV reports")), llvm::cl::values( - clEnumValN(SpatialDataflowExportSpatial2, "spatial2", "Emit spatial2 trivially merged graph dataflow CSV reports")), + clEnumValN(SpatialDataflowExportSpatial2, "spatial2", "Emit Spatial2 trivially merged graph dataflow CSV reports")), llvm::cl::values( - clEnumValN(SpatialDataflowExportSpatial3, "spatial3", "Emit spatial3 scheduled dataflow CSV reports")), + clEnumValN(SpatialDataflowExportSpatial3, "spatial3", "Emit Spatial3 scheduled dataflow CSV reports")), llvm::cl::values( - clEnumValN(SpatialDataflowExportSpatial4, "spatial4", "Emit spatial4 realized dataflow CSV reports")), + clEnumValN(SpatialDataflowExportSpatial4, "spatial4", "Emit Spatial4 realized dataflow CSV reports")), llvm::cl::values(clEnumValN(SpatialDataflowExportAll, "all", "Emit all Spatial dataflow CSV reports")), llvm::cl::init(SpatialDataflowExportNone), llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt pimOnlyCodegen("pim-only-codegen", - llvm::cl::desc("Only generate code for PIM (assume input is already in bufferized PIM IR)"), + llvm::cl::desc("Only generate code for Pim (assume input is already in bufferized Pim IR)"), llvm::cl::init(false), llvm::cl::cat(OnnxMlirOptions)); @@ -96,21 +96,37 @@ llvm::cl::opt pimEmitJson("pim-emit-json", llvm::cl::opt pimDetectCommunicationDeadlock( "pim-detect-communication-deadlock", - llvm::cl::desc("Expensively simulate the statically expanded PIM send/receive order at verification time and fail if a blocking communication deadlock is found"), + llvm::cl::desc("Expensively simulate the statically expanded Pim send/receive order at verification time and fail if a blocking communication deadlock is found"), llvm::cl::init(false), llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt pimVerifyBufferizationCopyFreedom( "pim-verify-bufferization-copy-freedom", - llvm::cl::desc("Run the expensive official PIM tensor-copy freedom proof before bufferization"), + llvm::cl::desc("Run the expensive official Pim tensor-copy freedom proof before bufferization"), + llvm::cl::init(false), + llvm::cl::cat(OnnxMlirOptions)); + +llvm::cl::opt pimDisableSynchronization( + "pim-disable-synchronization", + llvm::cl::desc("Omit Pim wait/sync instructions from generated code for performance ablation"), + llvm::cl::init(false), + llvm::cl::cat(OnnxMlirOptions)); + +llvm::cl::opt pimDisableSpatialPlanning( + "pim-disable-spatial-planning", + llvm::cl::desc("Select the trivial Spatial layout plan for performance ablation"), llvm::cl::init(false), llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt - crossbarSize("crossbar-size", llvm::cl::desc("Width and height of a single crossbar"), llvm::cl::init(128)); + crossbarSize("crossbar-size", + llvm::cl::desc("Width and height of a single crossbar (required for Pim compilation)"), + llvm::cl::init(0)); llvm::cl::opt - crossbarCountInCore("crossbar-count", llvm::cl::desc("Number of crossbars in each core"), llvm::cl::init(64)); + crossbarCountInCore("crossbar-count", + llvm::cl::desc("Number of crossbars in each core (required for Pim compilation)"), + llvm::cl::init(0)); llvm::cl::opt pipelineStages( "pipeline", @@ -119,32 +135,35 @@ llvm::cl::opt pipelineStages( llvm::cl::cat(OnnxMlirOptions)); llvm::cl::opt coresCount("core-count", - llvm::cl::desc("Number of cores in the chip. Required for PIM compilation."), + llvm::cl::desc("Number of cores in the chip. Required for Pim compilation."), llvm::cl::init(-1)); llvm::cl::opt pimTargetConfig( "pim-target-config", - llvm::cl::desc("PIM target configuration used to construct the Spatial scheduling cost model"), + llvm::cl::desc("Pim target configuration used to construct the Spatial scheduling cost model"), llvm::cl::init(""), llvm::cl::cat(OnnxMlirOptions)); -bool hasExplicitPimCoreCount() { return coresCount.getNumOccurrences() != 0; } - -void verifyExplicitPimCoreCount() { - if (!hasExplicitPimCoreCount()) - llvm::report_fatal_error("PIM compilation requires an explicit --core-count="); +void verifyPimCompilerOptions() { + if (coresCount.getNumOccurrences() == 0) + llvm::report_fatal_error("Pim compilation requires an explicit --core-count="); if (coresCount.getValue() <= 0) - llvm::report_fatal_error("PIM compilation requires --core-count to be a positive integer"); -} - -void verifyPimPipelineStages() { + llvm::report_fatal_error("Pim compilation requires --core-count to be a positive integer"); + if (crossbarSize.getNumOccurrences() == 0) + llvm::report_fatal_error("Pim compilation requires an explicit --crossbar-size="); + if (crossbarSize.getValue() == 0) + llvm::report_fatal_error("Pim compilation requires --crossbar-size to be a positive integer"); + if (crossbarCountInCore.getNumOccurrences() == 0) + llvm::report_fatal_error("Pim compilation requires an explicit --crossbar-count="); + if (crossbarCountInCore.getValue() == 0) + llvm::report_fatal_error("Pim compilation requires --crossbar-count to be a positive integer"); if (pipelineStages.getValue() == 0) - llvm::report_fatal_error("PIM compilation requires --pipeline to be positive"); + llvm::report_fatal_error("Pim compilation requires --pipeline to be positive"); if (static_cast(coresCount.getValue()) < pipelineStages.getValue()) - llvm::report_fatal_error("PIM compilation requires --pipeline not to exceed --core-count"); + llvm::report_fatal_error("Pim compilation requires --pipeline not to exceed --core-count"); if (crossbarCountInCore.getValue() > std::numeric_limits::max() / pipelineStages.getValue()) - llvm::report_fatal_error("PIM compilation --crossbar-count * --pipeline overflows"); + llvm::report_fatal_error("Pim compilation --crossbar-count * --pipeline overflows"); } } // namespace onnx_mlir diff --git a/src/PIM/Compiler/PimCompilerOptions.hpp b/src/PIM/Compiler/PimCompilerOptions.hpp index e6cff19..4b02afa 100644 --- a/src/PIM/Compiler/PimCompilerOptions.hpp +++ b/src/PIM/Compiler/PimCompilerOptions.hpp @@ -59,6 +59,8 @@ extern llvm::cl::opt pimEmitJson; extern llvm::cl::opt pimReportConvLowering; extern llvm::cl::opt pimDetectCommunicationDeadlock; extern llvm::cl::opt pimVerifyBufferizationCopyFreedom; +extern llvm::cl::opt pimDisableSynchronization; +extern llvm::cl::opt pimDisableSpatialPlanning; extern llvm::cl::opt crossbarSize; extern llvm::cl::opt crossbarCountInCore; @@ -68,8 +70,6 @@ extern llvm::cl::opt pimTargetConfig; extern llvm::cl::opt pimConvIm2colMaxElements; extern llvm::cl::opt pimConvStreamChunkPositions; -bool hasExplicitPimCoreCount(); -void verifyExplicitPimCoreCount(); -void verifyPimPipelineStages(); +void verifyPimCompilerOptions(); } // namespace onnx_mlir diff --git a/src/PIM/Compiler/PimCompilerUtils.cpp b/src/PIM/Compiler/PimCompilerUtils.cpp index f95b8ff..5452f52 100644 --- a/src/PIM/Compiler/PimCompilerUtils.cpp +++ b/src/PIM/Compiler/PimCompilerUtils.cpp @@ -97,7 +97,7 @@ spatial::ConvLoweringStrategy getSpatialConvLoweringStrategy(PimConvLoweringType case PimConvLoweringInputKTiled: return spatial::ConvLoweringStrategy::InputKTiled; case PimConvLoweringTiled2D: return spatial::ConvLoweringStrategy::Tiled2D; } - llvm_unreachable("unknown PIM Conv lowering strategy"); + llvm_unreachable("unknown Pim Conv lowering strategy"); } spatial::SpatialDataflowExportStage getPimSpatialDataflowExportStage( @@ -110,7 +110,7 @@ spatial::SpatialDataflowExportStage getPimSpatialDataflowExportStage( case SpatialDataflowExportSpatial4: return spatial::SpatialDataflowExportStage::Spatial4; case SpatialDataflowExportAll: return spatial::SpatialDataflowExportStage::All; } - llvm_unreachable("unknown PIM Spatial dataflow export stage"); + llvm_unreachable("unknown Pim Spatial dataflow export stage"); } spatial::SpatialTargetResources getPimSpatialTargetResources(const spatial::SchedulingTarget& target) { @@ -120,7 +120,7 @@ spatial::SpatialTargetResources getPimSpatialTargetResources(const spatial::Sche resources.processorCount = target.processorCount; resources.vectorWidth = target.vectorWidth; if (failed(resources.verify())) - llvm::report_fatal_error("PIM target resources are incomplete"); + llvm::report_fatal_error("Pim target resources are incomplete"); return resources; } @@ -138,7 +138,7 @@ const llvm::json::Object& requireObject(const llvm::json::Object& object, llvm::StringRef path) { const llvm::json::Object* nested = object.getObject(key); if (!nested) - llvm::report_fatal_error("PIM target config is missing object '" + path + "." + key + "'"); + llvm::report_fatal_error("Pim target config is missing object '" + path + "." + key + "'"); return *nested; } @@ -151,7 +151,7 @@ Cost getConfigCost(const llvm::json::Object& object, return fallback; if (!std::isfinite(*number) || *number < 0.0 || (!allowZero && *number == 0.0) || *number > static_cast(std::numeric_limits::max())) - llvm::report_fatal_error("PIM target config field '" + key + "' must be a valid positive number"); + llvm::report_fatal_error("Pim target config field '" + key + "' must be a valid positive number"); return static_cast(std::ceil(*number)); } @@ -159,11 +159,11 @@ std::pair getConfigPair(const llvm::json::Object& object, llvm::StringRef key) { const llvm::json::Array* values = object.getArray(key); if (!values || values->size() != 2) - llvm::report_fatal_error("PIM target config field '" + key + "' must contain two integers"); + llvm::report_fatal_error("Pim target config field '" + key + "' must contain two integers"); std::optional first = (*values)[0].getAsInteger(); std::optional second = (*values)[1].getAsInteger(); if (!first || !second || *first <= 0 || *second <= 0) - llvm::report_fatal_error("PIM target config field '" + key + "' must contain two positive integers"); + llvm::report_fatal_error("Pim target config field '" + key + "' must contain two positive integers"); return {static_cast(*first), static_cast(*second)}; } @@ -174,7 +174,7 @@ void loadPimInterProcessorLatencies( network.getString("net_config_file_path"); if (!filename) llvm::report_fatal_error( - "PIM target config is missing network latency file path"); + "Pim target config is missing network latency file path"); llvm::SmallString<256> networkPath(*filename); if (!llvm::sys::path::is_absolute(networkPath)) { @@ -187,19 +187,19 @@ void loadPimInterProcessorLatencies( auto buffer = llvm::MemoryBuffer::getFile(networkPath); if (!buffer) llvm::report_fatal_error( - llvm::Twine("failed to read PIM network config '") + llvm::Twine("failed to read Pim network config '") + networkPath + "': " + buffer.getError().message()); auto parsed = llvm::json::parse(buffer.get()->getBuffer()); if (!parsed) llvm::report_fatal_error( - llvm::Twine("failed to parse PIM network config '") + llvm::Twine("failed to parse Pim network config '") + networkPath + "': " + llvm::toString(parsed.takeError())); const llvm::json::Object* root = parsed->getAsObject(); const llvm::json::Object* latencies = root ? root->getObject("latency") : nullptr; if (!latencies) llvm::report_fatal_error( - "PIM network config is missing its latency matrix"); + "Pim network config is missing its latency matrix"); target.interProcessorLatencyNs.assign( target.processorCount * target.processorCount, 0); @@ -210,7 +210,7 @@ void loadPimInterProcessorLatencies( const llvm::json::Object* row = latencies->getObject(sourceKey); if (!row) llvm::report_fatal_error( - llvm::Twine("PIM network config is missing latency row ") + llvm::Twine("Pim network config is missing latency row ") + sourceKey); for (size_t destination = 0; destination < target.processorCount; ++destination) { @@ -220,7 +220,7 @@ void loadPimInterProcessorLatencies( std::optional latency = row->getNumber(destinationKey); if (!latency || !std::isfinite(*latency) || *latency <= 0.0) llvm::report_fatal_error( - llvm::Twine("PIM network config is missing latency ") + llvm::Twine("Pim network config is missing latency ") + sourceKey + " -> " + destinationKey); Cost roundedLatency = static_cast(std::ceil(*latency)); target.interProcessorLatencyNs[ @@ -244,17 +244,17 @@ spatial::SchedulingTarget getPimSchedulingTarget() { auto buffer = llvm::MemoryBuffer::getFile(pimTargetConfig); if (!buffer) llvm::report_fatal_error( - llvm::Twine("failed to read PIM target config '") + llvm::Twine("failed to read Pim target config '") + pimTargetConfig.getValue() + "': " + buffer.getError().message()); auto parsed = llvm::json::parse(buffer.get()->getBuffer()); if (!parsed) llvm::report_fatal_error( - llvm::Twine("failed to parse PIM target config '") + llvm::Twine("failed to parse Pim target config '") + pimTargetConfig.getValue() + "': " + llvm::toString(parsed.takeError())); const llvm::json::Object* root = parsed->getAsObject(); if (!root) - llvm::report_fatal_error("PIM target config must contain a JSON object"); + llvm::report_fatal_error("Pim target config must contain a JSON object"); const llvm::json::Object& chip = requireObject(*root, "chip_config", "root"); const llvm::json::Object& core = requireObject(chip, "core_config", "chip_config"); @@ -267,7 +267,7 @@ spatial::SchedulingTarget getPimSchedulingTarget() { std::optional coreCount = chip.getInteger("core_cnt"); if (!coreCount || *coreCount <= 0) - llvm::report_fatal_error("PIM target config field 'core_cnt' must be a positive integer"); + llvm::report_fatal_error("Pim target config field 'core_cnt' must be a positive integer"); target.processorCount = static_cast(*coreCount); target.residentWeightCapacity = getConfigCost(matrix, "xbar_array_count", target.residentWeightCapacity); @@ -278,7 +278,7 @@ spatial::SchedulingTarget getPimSchedulingTarget() { || target.residentWeightCapacity != crossbarCountInCore.getValue() || target.matrixRows != crossbarSize.getValue() || target.matrixColumns != crossbarSize.getValue()) - llvm::report_fatal_error("PIM target config resources do not match --core-count, " + llvm::report_fatal_error("Pim target config resources do not match --core-count, " "--crossbar-count, and --crossbar-size"); loadPimInterProcessorLatencies(target, network); @@ -331,8 +331,7 @@ void addPassesPim(OwningOpRef& module, PassManager& pm, EmissionTargetType& emissionTarget, std::string outputNameNoExt) { - verifyExplicitPimCoreCount(); - verifyPimPipelineStages(); + verifyPimCompilerOptions(); spatial::SchedulingTarget schedulingTarget = getPimSchedulingTarget(); spatial::SpatialTargetResources targetResources = getPimSpatialTargetResources(schedulingTarget); @@ -352,7 +351,8 @@ void addPassesPim(OwningOpRef& module, spatial::SpatialDataflowExportStage exportStage = getPimSpatialDataflowExportStage(pimExportSpatialDataflow.getValue()); pm.addPass(createONNXToSpatialPass(targetResources, planningOptions)); - pm.addPass(createSpatialLayoutPlanningPass(targetResources)); + pm.addPass(createSpatialLayoutPlanningPass( + targetResources, pimDisableSpatialPlanning.getValue())); pm.addPass(createLowerSpatialPlansPass(targetResources, planningOptions, exportStage)); pm.addPass(createTrivialGraphComputeMergePass( schedulingTarget.residentWeightCapacity, exportStage)); diff --git a/src/PIM/Compiler/PimCoreProgram.cpp b/src/PIM/Compiler/PimCoreProgram.cpp index 3dde637..b9ef297 100644 --- a/src/PIM/Compiler/PimCoreProgram.cpp +++ b/src/PIM/Compiler/PimCoreProgram.cpp @@ -46,7 +46,7 @@ static LogicalResult compileCoreEmissionPlan(Block& block, SmallVectorImpl(op)) { auto condition = compileIndexExpr(ifOp.getCondition()); if (failed(condition)) { - ifOp.emitOpError("requires statically evaluable scf.if condition for PIM codegen"); + ifOp.emitOpError("requires statically evaluable scf.if condition for Pim codegen"); return failure(); } CompiledCoreNode node; @@ -82,7 +82,7 @@ static LogicalResult compileCoreEmissionPlan(Block& block, SmallVectorImpl(op)) { auto selector = compileIndexExpr(switchOp.getArg()); if (failed(selector)) { - switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for PIM codegen"); + switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for Pim codegen"); return failure(); } CompiledCoreNode node; diff --git a/src/PIM/Conversion/ONNXToSpatial/Common/ComputeRegionBuilder.hpp b/src/PIM/Conversion/ONNXToSpatial/Common/ComputeRegionBuilder.hpp index 0132655..4ef4977 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Common/ComputeRegionBuilder.hpp +++ b/src/PIM/Conversion/ONNXToSpatial/Common/ComputeRegionBuilder.hpp @@ -249,7 +249,7 @@ auto createEmptySpatGraphComputeBatch(RewriterT& rewriter, if (laneCount <= 0 || laneCount > std::numeric_limits::max()) return mlir::FailureOr(mlir::failure()); - auto laneCountAttr = pim::getCheckedI32Attr(rewriter, loc, laneCount, "spatial compute_batch lane count"); + auto laneCountAttr = pim::getCheckedI32Attr(rewriter, loc, laneCount, "Spatial compute_batch lane count"); if (mlir::failed(laneCountAttr)) return mlir::FailureOr(mlir::failure()); diff --git a/src/PIM/Conversion/ONNXToSpatial/Common/ShapeTilingUtils.hpp b/src/PIM/Conversion/ONNXToSpatial/Common/ShapeTilingUtils.hpp index aba0c7d..14d60d0 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Common/ShapeTilingUtils.hpp +++ b/src/PIM/Conversion/ONNXToSpatial/Common/ShapeTilingUtils.hpp @@ -25,7 +25,7 @@ llvm::SmallVector sliceVector(const mlir::Value& vectorToSlice, mlir::Location loc); /// Partitions one logical vector into per-core crossbar-sized slices using the -/// current PIM target geometry. +/// current Pim target geometry. llvm::DenseMap> sliceVectorPerCrossbarPerCore( const mlir::Value& vectorToSlice, mlir::PatternRewriter& rewriter, diff --git a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp index 89ed83d..18c6b44 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp +++ b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp @@ -46,7 +46,7 @@ struct LowerSpatialPlansPass final } auto entryFunc = getPimEntryFunc(moduleOp); if (failed(entryFunc)) { - moduleOp.emitError("failed to locate the PIM entry function during LowerSpatialPlans"); + moduleOp.emitError("failed to locate the Pim entry function during LowerSpatialPlans"); signalPassFailure(); return; } diff --git a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/ONNXToSpatialPass.cpp b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/ONNXToSpatialPass.cpp index 41eb330..9760f00 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/ONNXToSpatialPass.cpp +++ b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/ONNXToSpatialPass.cpp @@ -158,7 +158,7 @@ void ONNXToSpatialPass::runOnOperation() { auto entryFunc = getPimEntryFunc(moduleOp); if (failed(entryFunc)) { - moduleOp.emitError("failed to locate the PIM entry function during ONNX-to-Spatial lowering"); + moduleOp.emitError("failed to locate the Pim entry function during ONNX-to-Spatial lowering"); signalPassFailure(); return; } @@ -245,7 +245,7 @@ void ONNXToSpatialPass::runOnOperation() { RewritePatternSet postPatterns(ctx); populatePostPatterns(postPatterns, ctx); if (failed(applyPartialConversion(*entryFunc, postTarget, std::move(postPatterns)))) { - moduleOp.emitError("failed to normalize weight-like Spatial compute operands before Spatial-to-PIM lowering"); + moduleOp.emitError("failed to normalize weight-like Spatial compute operands before Spatial-to-Pim lowering"); signalPassFailure(); return; } diff --git a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp index 7099344..90fecea 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp +++ b/src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp @@ -42,8 +42,9 @@ static SmallVector getOperandLayouts( class SpatialLayoutAnalysis { public: SpatialLayoutAnalysis(func::FuncOp funcOp, - const spatial::SpatialTargetResources& target) - : funcOp(funcOp), target(target) {} + const spatial::SpatialTargetResources& target, + bool selectTrivialPlan) + : funcOp(funcOp), target(target), selectTrivialPlan(selectTrivialPlan) {} FailureOr run() { SpatialLayoutSelection selection; @@ -56,6 +57,9 @@ public: selection.selectedAlternative[&op] = 0; } + if (selectTrivialPlan) + return selection; + const size_t maxRounds = 2 * planOps.size() + 1; for (size_t round = 0; round < maxRounds; ++round) { bool changed = false; @@ -168,6 +172,7 @@ private: func::FuncOp funcOp; const spatial::SpatialTargetResources& target; + bool selectTrivialPlan; }; static LogicalResult materializeMismatchedUses( @@ -251,8 +256,9 @@ struct SpatialLayoutPlanningPass final } SpatialLayoutPlanningPass() = default; - explicit SpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target) - : target(target), hasTarget(true) {} + SpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target, + bool selectTrivialPlan) + : target(target), selectTrivialPlan(selectTrivialPlan), hasTarget(true) {} void runOnOperation() override { ModuleOp moduleOp = getOperation(); @@ -263,13 +269,13 @@ struct SpatialLayoutPlanningPass final } auto entryFunc = getPimEntryFunc(moduleOp); if (failed(entryFunc)) { - moduleOp.emitError("failed to locate the PIM entry function during Spatial layout planning"); + moduleOp.emitError("failed to locate the Pim entry function during Spatial layout planning"); signalPassFailure(); return; } func::FuncOp funcOp = *entryFunc; - SpatialLayoutAnalysis analysis(funcOp, target); + SpatialLayoutAnalysis analysis(funcOp, target, selectTrivialPlan); FailureOr selection = analysis.run(); if (failed(selection)) { signalPassFailure(); @@ -301,6 +307,7 @@ struct SpatialLayoutPlanningPass final } spatial::SpatialTargetResources target; + bool selectTrivialPlan = false; bool hasTarget = false; }; @@ -311,8 +318,8 @@ std::unique_ptr createSpatialLayoutPlanningPass() { } std::unique_ptr createSpatialLayoutPlanningPass( - const spatial::SpatialTargetResources& target) { - return std::make_unique(target); + const spatial::SpatialTargetResources& target, bool selectTrivialPlan) { + return std::make_unique(target, selectTrivialPlan); } } // namespace onnx_mlir diff --git a/src/PIM/Conversion/ONNXToSpatial/Patterns/Math/Conv.cpp b/src/PIM/Conversion/ONNXToSpatial/Patterns/Math/Conv.cpp index 39677aa..56a2564 100644 --- a/src/PIM/Conversion/ONNXToSpatial/Patterns/Math/Conv.cpp +++ b/src/PIM/Conversion/ONNXToSpatial/Patterns/Math/Conv.cpp @@ -199,7 +199,7 @@ static bool writeConvLoweringReport(const ConvLoweringReportEntry& entry, return false; } - reportFile << "# PIM Conv Lowering Report (bounded to 512 rows)\n\n"; + reportFile << "# Pim conv lowering report (bounded to 512 rows)\n\n"; reportFile << "## Plan selection\n"; writeConvReportTableHeader(reportFile, "Selector"); bool realizationSectionStarted = false; diff --git a/src/PIM/Conversion/SpatialToPim/BatchCoreLoweringPatterns.cpp b/src/PIM/Conversion/SpatialToPim/BatchCoreLoweringPatterns.cpp index 83e9384..d1cac41 100644 --- a/src/PIM/Conversion/SpatialToPim/BatchCoreLoweringPatterns.cpp +++ b/src/PIM/Conversion/SpatialToPim/BatchCoreLoweringPatterns.cpp @@ -307,7 +307,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeBatchOp(spatial::SpatSchedul "resultful compute_batch lowering currently requires a spat.in_parallel terminator"); } - auto coreIds = getRequiredScheduledBatchCoreIds(computeBatchOp, "spatial compute_batch core id"); + auto coreIds = getRequiredScheduledBatchCoreIds(computeBatchOp, "Spatial compute_batch core id"); if (failed(coreIds)) return failure(); SmallVector batchWeights(computeBatchOp.getWeights().begin(), computeBatchOp.getWeights().end()); @@ -317,7 +317,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeBatchOp(spatial::SpatSchedul rewriter.setInsertionPointAfter(computeBatchOp); auto laneCountAttr = pim::getCheckedI32Attr( - rewriter, computeBatchOp, static_cast(computeBatchOp.getLaneCount()), "pim core_batch lane count"); + rewriter, computeBatchOp, static_cast(computeBatchOp.getLaneCount()), "Pim core_batch lane count"); if (failed(laneCountAttr)) return failure(); auto coreBatchOp = diff --git a/src/PIM/Conversion/SpatialToPim/CoreLoweringPatterns.cpp b/src/PIM/Conversion/SpatialToPim/CoreLoweringPatterns.cpp index 02dad5b..2fda833 100644 --- a/src/PIM/Conversion/SpatialToPim/CoreLoweringPatterns.cpp +++ b/src/PIM/Conversion/SpatialToPim/CoreLoweringPatterns.cpp @@ -410,7 +410,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeOp(spatial::SpatScheduledCom continue; } - return computeOp.emitOpError("has an unsupported remaining result use during Spatial-to-PIM lowering"); + return computeOp.emitOpError("has an unsupported remaining result use during Spatial-to-Pim lowering"); } rewriter.setInsertionPoint(yieldOp); @@ -420,7 +420,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeOp(spatial::SpatScheduledCom if (!computeOp.getWeights().empty()) computeWeights.append(computeOp.getWeights().begin(), computeOp.getWeights().end()); rewriter.setInsertionPointAfter(computeOp); - auto checkedCoreId = getRequiredScheduledCoreId(computeOp, "spatial compute core id"); + auto checkedCoreId = getRequiredScheduledCoreId(computeOp, "Spatial compute core id"); if (failed(checkedCoreId)) return failure(); auto coreIdAttr = pim::getCheckedI32Attr(rewriter, computeOp, static_cast(*checkedCoreId), "pim core id"); diff --git a/src/PIM/Conversion/SpatialToPim/ReturnPathNormalization.cpp b/src/PIM/Conversion/SpatialToPim/ReturnPathNormalization.cpp index 9f802f3..7de9cc2 100644 --- a/src/PIM/Conversion/SpatialToPim/ReturnPathNormalization.cpp +++ b/src/PIM/Conversion/SpatialToPim/ReturnPathNormalization.cpp @@ -734,7 +734,7 @@ raptor::SpatialToPimPass::ReturnPathLoweringResult raptor::SpatialToPimPass::low auto storedType = dyn_cast(storedValue.getType()); if (!storedType) { producerOp->emitOpError( - "has an unsupported non-ranked concat-return helper yield during Spatial-to-PIM lowering"); + "has an unsupported non-ranked concat-return helper yield during Spatial-to-Pim lowering"); return ReturnPathLoweringResult::Failure; } rewriter.setInsertionPointAfterValue(storedValue); @@ -748,7 +748,7 @@ raptor::SpatialToPimPass::ReturnPathLoweringResult raptor::SpatialToPimPass::low SmallVector destinationIndices; if (failed(mapIndicesThroughHelperChain( sourceIndices, concatReturnUse->concatShape, concatReturnUse->helperChain, destinationIndices))) { - producerOp->emitOpError("has an unsupported concat-return helper chain during Spatial-to-PIM lowering"); + producerOp->emitOpError("has an unsupported concat-return helper chain during Spatial-to-Pim lowering"); return ReturnPathLoweringResult::Failure; } diff --git a/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.cpp b/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.cpp index 70db2d9..01ab547 100644 --- a/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.cpp +++ b/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.cpp @@ -88,7 +88,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() { operationsToRemove.clear(); ModuleOp moduleOp = getOperation(); if (!hasTarget || failed(targetResources.verify())) { - moduleOp.emitError("Spatial-to-PIM lowering requires valid injected target resources"); + moduleOp.emitError("Spatial-to-Pim lowering requires valid injected target resources"); signalPassFailure(); return; } @@ -96,7 +96,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() { auto entryFunc = getPimEntryFunc(moduleOp); if (failed(entryFunc)) { - moduleOp.emitError("failed to locate the PIM entry function during Spatial-to-PIM lowering"); + moduleOp.emitError("failed to locate the Pim entry function during Spatial-to-Pim lowering"); signalPassFailure(); return; } @@ -135,7 +135,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() { RewritePatternSet initialPatterns(ctx); populateInitialPatterns(initialPatterns); if (failed(applyPartialConversion(moduleOp, target, std::move(initialPatterns)))) { - moduleOp.emitError("failed to lower required Spatial ops to the initial PIM form"); + moduleOp.emitError("failed to lower required Spatial ops to the initial Pim form"); signalPassFailure(); return; } @@ -153,7 +153,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() { auto returnOp = cast(funcOp.front().getTerminator()); addReturnOutputBuffers(returnOp, rewriter); if (failed(allocateAndInitializeCoreLocalVariables(funcOp, rewriter))) { - funcOp.emitOpError("failed to allocate or initialize core-local tensors during Spatial-to-PIM lowering"); + funcOp.emitOpError("failed to allocate or initialize core-local tensors during Spatial-to-Pim lowering"); signalPassFailure(); return; } @@ -285,7 +285,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() { RewritePatternSet communicationPatterns(ctx); populateChannelLoweringPatterns(communicationPatterns); if (failed(applyFullConversion(funcOp, communicationTarget, std::move(communicationPatterns)))) { - funcOp.emitOpError("failed to lower Spatial communication ops to PIM communication ops"); + funcOp.emitOpError("failed to lower Spatial communication ops to Pim communication ops"); signalPassFailure(); return; } diff --git a/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.hpp b/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.hpp index cfa1118..a09a399 100644 --- a/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.hpp +++ b/src/PIM/Conversion/SpatialToPim/SpatialToPimPass.hpp @@ -26,7 +26,7 @@ namespace raptor { struct SpatialToPimPass : mlir::PassWrapper> { MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(SpatialToPimPass) llvm::StringRef getArgument() const override { return "convert-spatial-to-pim"; } - llvm::StringRef getDescription() const override { return "Lower Spatial ops to PIM-ready format"; } + llvm::StringRef getDescription() const override { return "Lower Spatial ops to Pim-ready format"; } SpatialToPimPass() = default; explicit SpatialToPimPass(const spatial::SpatialTargetResources& target) diff --git a/src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp b/src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp index 33c3c40..ac16b52 100644 --- a/src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp +++ b/src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp @@ -402,7 +402,7 @@ static LogicalResult verifyPimCoresNeedNoTensorCopies( bufferization::BufferizationState state; if (failed(bufferization::insertTensorCopies(*clone, options, state))) { - moduleOp.emitError("official one-shot analysis failed while verifying PIM core copy freedom"); + moduleOp.emitError("official one-shot analysis failed while verifying Pim core copy freedom"); return failure(); } @@ -415,10 +415,10 @@ static LogicalResult verifyPimCoresNeedNoTensorCopies( Operation* requiredBy = alloc->getUsers().empty() ? alloc.getOperation() : *alloc->getUsers().begin(); diagnostics.report(requiredBy, [](Operation* op) { - op->emitOpError("official one-shot bufferization requires a tensor copy inside a PIM core"); + op->emitOpError("official one-shot bufferization requires a tensor copy inside a Pim core"); }); }); - diagnostics.emitSuppressedSummary(moduleOp, "required PIM core tensor copies"); + diagnostics.emitSuppressedSummary(moduleOp, "required Pim core tensor copies"); return success(!diagnostics.hasFailure()); } @@ -440,7 +440,7 @@ static LogicalResult runOneShotPimBufferization( bufferization::BufferizationState state; if (failed(bufferization::insertTensorCopies(moduleOp, hostOptions, state)) || failed(bufferization::bufferizeModuleOp(moduleOp, options, state))) { - moduleOp.emitError("Failed to bufferize PIM and Spatial ops"); + moduleOp.emitError("Failed to bufferize Pim and Spatial ops"); return failure(); } return success(); @@ -478,7 +478,7 @@ static LogicalResult verifyContiguousRuntimeOperands(ModuleOp moduleOp) { if (succeeded(resolveContiguousAddress(operand, knowledge)) || succeeded(compileContiguousAddressExpr(operand))) return; op.emitOpError() << "operand #" << operandIndex - << " is not backed by contiguous addressable storage after PIM bufferization"; + << " is not backed by contiguous addressable storage after Pim bufferization"; hasFailure = true; }; @@ -552,7 +552,7 @@ static LogicalResult verifyContiguousRuntimeOperands(ModuleOp moduleOp) { }); if (hasFailure) { - moduleOp.emitError("PIM bufferization must fully normalize executable runtime operand contiguity before codegen"); + moduleOp.emitError("Pim bufferization must fully normalize executable runtime operand contiguity before codegen"); return failure(); } return success(); @@ -589,7 +589,7 @@ static LogicalResult verifyPimCopyAddressSpaces(ModuleOp moduleOp) { }); if (failureCount != 0) moduleOp.emitError() << "found " << failureCount - << " PIM copy address-space violation(s); the first is reported above"; + << " Pim copy address-space violation(s); the first is reported above"; return success(failureCount == 0); } @@ -673,7 +673,7 @@ static LogicalResult normalizePimMemory(ModuleOp moduleOp, func::FuncOp funcOp) GreedyRewriteConfig contiguityConfig; contiguityConfig.enableFolding(false); if (failed(applyPatternsGreedily(moduleOp, std::move(contiguityPatterns), contiguityConfig))) { - moduleOp.emitError("failed to normalize PIM copy contiguity during bufferization"); + moduleOp.emitError("failed to normalize Pim copy contiguity during bufferization"); return failure(); } annotateWeightsMemrefs(moduleOp, funcOp); @@ -684,7 +684,7 @@ static LogicalResult normalizePimMemory(ModuleOp moduleOp, func::FuncOp funcOp) static FailureOr requirePimEntryFunc(ModuleOp moduleOp, StringRef phase) { auto entryFunc = getPimEntryFunc(moduleOp); if (failed(entryFunc)) { - moduleOp.emitError("failed to locate the PIM entry function during ") << phase; + moduleOp.emitError("failed to locate the Pim entry function during ") << phase; return failure(); } return *entryFunc; @@ -701,12 +701,12 @@ struct PimBufferizationPreparationPass StringRef getArgument() const override { return "pim-bufferization-preparation"; } StringRef getDescription() const override { - return "Prepare writable tensor destinations for PIM one-shot bufferization."; + return "Prepare writable tensor destinations for Pim one-shot bufferization."; } void runOnOperation() final { ModuleOp moduleOp = getOperation(); - auto funcOp = requirePimEntryFunc(moduleOp, "PIM bufferization preparation"); + auto funcOp = requirePimEntryFunc(moduleOp, "Pim bufferization preparation"); if (failed(funcOp)) { signalPassFailure(); return; @@ -725,7 +725,7 @@ struct PimOneShotBufferizationPass StringRef getArgument() const override { return "pim-one-shot-bufferization"; } StringRef getDescription() const override { - return "Run one-shot bufferization for PIM and Spatial tensors."; + return "Run one-shot bufferization for Pim and Spatial tensors."; } void runOnOperation() final { @@ -740,12 +740,12 @@ struct PimMemoryNormalizationPass StringRef getArgument() const override { return "pim-memory-normalization"; } StringRef getDescription() const override { - return "Normalize PIM memory copies and verify addressable operands."; + return "Normalize Pim memory copies and verify addressable operands."; } void runOnOperation() final { ModuleOp moduleOp = getOperation(); - auto funcOp = requirePimEntryFunc(moduleOp, "PIM memory normalization"); + auto funcOp = requirePimEntryFunc(moduleOp, "Pim memory normalization"); if (failed(funcOp)) { signalPassFailure(); return; @@ -761,20 +761,20 @@ static LogicalResult verifyNoTensorValues(ModuleOp moduleOp) { if (failureCount >= 8) return; if (op->getDialect()->getNamespace() == "tensor") { - op->emitOpError("tensor operation remains after PIM bufferization"); + op->emitOpError("tensor operation remains after Pim bufferization"); ++failureCount; return; } for (Value value : op->getOperands()) { if (isa(value.getType())) { - op->emitOpError("tensor operand remains after PIM bufferization"); + op->emitOpError("tensor operand remains after Pim bufferization"); ++failureCount; return; } } for (Value value : op->getResults()) { if (isa(value.getType())) { - op->emitOpError("tensor result remains after PIM bufferization"); + op->emitOpError("tensor result remains after Pim bufferization"); ++failureCount; return; } @@ -782,7 +782,7 @@ static LogicalResult verifyNoTensorValues(ModuleOp moduleOp) { }); if (failureCount != 0) moduleOp.emitError() << "found " << failureCount - << " tensor value(s) after PIM bufferization" + << " tensor value(s) after Pim bufferization" << (failureCount == 8 ? " (first 8 reported)" : ""); return success(failureCount == 0); } @@ -793,7 +793,7 @@ struct PimBufferizationVerificationPass StringRef getArgument() const override { return "pim-bufferization-verification"; } StringRef getDescription() const override { - return "Verify tensor elimination, contiguity, and PIM copy address spaces."; + return "Verify tensor elimination, contiguity, and Pim copy address spaces."; } void runOnOperation() final { diff --git a/src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/HostConstantFoldingPass.cpp b/src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/HostConstantFoldingPass.cpp index ef553ff..b51024a 100644 --- a/src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/HostConstantFoldingPass.cpp +++ b/src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/HostConstantFoldingPass.cpp @@ -16,7 +16,7 @@ struct HostConstantFoldingPass : PassWrapper { } }; -// Converts PIM copies from dense globals into direct folded globals before codegen. +// Converts Pim copies from dense globals into direct folded globals before codegen. struct FoldConstantMemCpPattern final : OpRewritePattern { using OpRewritePattern::OpRewritePattern; diff --git a/src/PIM/Dialect/Pim/Passes/Transforms/InstructionSelection/InstructionSelectionPass.cpp b/src/PIM/Dialect/Pim/Passes/Transforms/InstructionSelection/InstructionSelectionPass.cpp index 206fe81..b62e3bf 100644 --- a/src/PIM/Dialect/Pim/Passes/Transforms/InstructionSelection/InstructionSelectionPass.cpp +++ b/src/PIM/Dialect/Pim/Passes/Transforms/InstructionSelection/InstructionSelectionPass.cpp @@ -40,7 +40,7 @@ struct LowerTransposePattern final : OpRewritePattern { auto sourceType = dyn_cast(op.getInput().getType()); auto targetType = dyn_cast(op.getOutputBuffer().getType()); if (!sourceType || !targetType || !sourceType.hasStaticShape() || !targetType.hasStaticShape()) - return op.emitOpError("requires static memref operands before PIM instruction selection"); + return op.emitOpError("requires static memref operands before Pim instruction selection"); ArrayRef sourceShape = sourceType.getShape(); size_t rank = sourceShape.size(); @@ -147,7 +147,7 @@ struct InstructionSelectionPass : PassWrapper buildCoreMemoryPlan(Operation* coreLikeOp) { plan.intervals = std::move(*intervals); auto placements = planLocalMemoryPlacements(plan.intervals, kPimLocalMemoryAddressLimit); if (failed(placements)) { - coreLikeOp->emitError("PIM local-memory plan exceeds the signed int32 address range"); + coreLikeOp->emitError("Pim local-memory plan exceeds the signed int32 address range"); return failure(); } plan.placements = std::move(*placements); for (const LocalMemoryPlacement& placement : plan.placements) { auto end = alignedEnd(placement.address, placement.size, kPimLocalMemoryAddressLimit); if (failed(end)) { - coreLikeOp->emitError("PIM local-memory plan has invalid address arithmetic"); + coreLikeOp->emitError("Pim local-memory plan has invalid address arithmetic"); return failure(); } plan.arenaSize = std::max(plan.arenaSize, *end); diff --git a/src/PIM/Dialect/Pim/Passes/Transforms/Verification/VerificationPass.cpp b/src/PIM/Dialect/Pim/Passes/Transforms/Verification/VerificationPass.cpp index 4db708c..125b3b1 100644 --- a/src/PIM/Dialect/Pim/Passes/Transforms/Verification/VerificationPass.cpp +++ b/src/PIM/Dialect/Pim/Passes/Transforms/Verification/VerificationPass.cpp @@ -117,7 +117,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag for (StringRef name : kRemovedLocalMemoryPlanAttrNames) if (coreLikeOp->hasAttr(name)) { diagnostics.report(coreLikeOp, [name](Operation* op) { - op->emitError() << "contains removed PIM local-memory planning attribute '" << name << "'"; + op->emitError() << "contains removed Pim local-memory planning attribute '" << name << "'"; }); hasFailure = true; } @@ -137,7 +137,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag auto analyzed = pim::analyzeLocalMemoryLifetimes(coreLikeOp); if (failed(analyzed)) { diagnostics.report(coreLikeOp, [](Operation* op) { - op->emitError("cannot analyze PIM local-memory lifetimes for plan verification"); + op->emitError("cannot analyze Pim local-memory lifetimes for plan verification"); }); return failure(); } @@ -156,7 +156,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag for (StringRef name : kRemovedLocalMemoryPlanAttrNames) if (allocation->hasAttr(name)) { diagnostics.report(allocation, [name](Operation* op) { - op->emitOpError() << "contains removed PIM local-memory planning attribute '" << name << "'"; + op->emitOpError() << "contains removed Pim local-memory planning attribute '" << name << "'"; }); hasFailure = true; } @@ -171,7 +171,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag uint64_t address = static_cast(addressAttr.getInt()); if (address % 4 != 0 || address > arenaSize || interval.size > arenaSize - address) { diagnostics.report(allocation, [&](Operation* op) { - op->emitOpError() << "has invalid PIM local-memory range [" << address << ", " + op->emitOpError() << "has invalid Pim local-memory range [" << address << ", " << (address <= arenaSize && interval.size <= arenaSize - address ? address + interval.size : arenaSize) @@ -221,7 +221,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag memref::AllocOp otherAllocation = other.allocation; diagnostics.report(allocation, [&](Operation*) { auto diagnostic = allocation.emitOpError() - << "PIM local-memory plan assigns simultaneously live allocations to overlapping ranges; first range [" + << "Pim local-memory plan assigns simultaneously live allocations to overlapping ranges; first range [" << conflicting->first << ", " << conflicting->first + other.size << "), second range [" << address << ", " << address + interval.size << "), live positions overlap at [" << std::max(interval.start, other.start) << ", " << std::min(interval.end, other.end) << "]"; @@ -471,7 +471,7 @@ static LogicalResult appendCoreCommunicationEvents(Block& block, auto targetCoreId = resolveIndexValue(sendOp.getTargetCoreId(), knowledge); if (failed(targetCoreId)) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("cannot statically resolve send target core for PIM communication deadlock check"); + illegalOp->emitOpError("cannot statically resolve send target core for Pim communication deadlock check"); }); return failure(); } @@ -490,7 +490,7 @@ static LogicalResult appendCoreCommunicationEvents(Block& block, if (failed(sourceCoreId)) { diagnostics.report(&op, [](Operation* illegalOp) { illegalOp->emitOpError( - "cannot statically resolve receive source core for PIM communication deadlock check"); + "cannot statically resolve receive source core for Pim communication deadlock check"); }); return failure(); } @@ -530,7 +530,7 @@ static void printCommunicationWindow(llvm::raw_ostream& os, static void printCommunicationDeadlockReport(const DenseMap& coreEvents, const DenseMap& programCounters, ArrayRef cycle) { - llvm::errs() << "\n=== PIM static communication deadlock report ===\n"; + llvm::errs() << "\n=== Pim static communication deadlock report ===\n"; llvm::errs() << "wait cycle:"; for (int64_t coreId : cycle) llvm::errs() << " " << coreId; @@ -565,7 +565,7 @@ static void printCommunicationDeadlockReport(const DenseMapsecond); } - llvm::errs() << "=== end PIM static communication deadlock report ===\n\n"; + llvm::errs() << "=== end Pim static communication deadlock report ===\n\n"; } static void emitCommunicationDeadlockCycle(ModuleOp moduleOp, @@ -576,8 +576,8 @@ static void emitCommunicationDeadlockCycle(ModuleOp moduleOp, auto diagnostic = moduleOp.emitError() - << "PIM communication deadlock check found a blocking send/receive cycle while statically simulating the " - "expanded per-core communication streams; see the PIM static communication deadlock report above"; + << "Pim communication deadlock check found a blocking send/receive cycle while statically simulating the " + "expanded per-core communication streams; see the Pim static communication deadlock report above"; for (int64_t coreId : cycle) { auto eventsIt = coreEvents.find(coreId); @@ -728,7 +728,7 @@ static LogicalResult verifyNoStaticCommunicationDeadlock(ModuleOp moduleOp, auto diagnostic = moduleOp.emitError() - << "PIM communication deadlock check stalled without finding a closed wait cycle; this usually means a " + << "Pim communication deadlock check stalled without finding a closed wait cycle; this usually means a " "send/receive peer is missing or ordered after a finished core"; for (const auto& [coreId, events] : coreEvents) { size_t pc = programCounters[coreId]; @@ -746,7 +746,7 @@ struct VerificationPass : PassWrapper> StringRef getArgument() const override { return "verify-pim-pass"; } StringRef getDescription() const override { - return "Verify that bufferized PIM IR contains only explicit host/device transfers"; + return "Verify that bufferized Pim IR contains only explicit host/device transfers"; } VerificationPass() {} @@ -763,7 +763,7 @@ struct VerificationPass : PassWrapper> pim::CappedDiagnosticReporter diagnostics; if (!hasTarget || failed(targetResources.verify())) { - moduleOp.emitError("PIM codegen verification requires valid injected target resources"); + moduleOp.emitError("Pim codegen verification requires valid injected target resources"); signalPassFailure(); return; } @@ -792,7 +792,7 @@ struct VerificationPass : PassWrapper> return; diagnostics.report(op, [](Operation* illegalOp) { - illegalOp->emitError("illegal Spatial operation reached PIM codegen verification"); + illegalOp->emitError("illegal Spatial operation reached Pim codegen verification"); }); }); @@ -833,7 +833,7 @@ struct VerificationPass : PassWrapper> if (!isAddressOnlyHostOp(&op)) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("illegal host-side runtime op remains after PIM bufferization; " + illegalOp->emitOpError("illegal host-side runtime op remains after Pim bufferization; " "fold it to constants or lower it into pim.core"); }); continue; @@ -849,7 +849,7 @@ struct VerificationPass : PassWrapper> if (diagnostics.hasFailure()) { diagnostics.emitSuppressedSummary(moduleOp, "verification failures"); - moduleOp.emitError("PIM codegen verification failed; see diagnostics above"); + moduleOp.emitError("Pim codegen verification failed; see diagnostics above"); hasFailure = true; } @@ -928,7 +928,7 @@ private: bool hasFailure = false; if (!isSupportedCoreInstructionOp(&op)) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("unsupported executable op reached PIM codegen verification"); + illegalOp->emitOpError("unsupported executable op reached Pim codegen verification"); }); hasFailure = true; } @@ -990,7 +990,7 @@ private: if (failed(resolveIndexValue(storeOp.getHostTargetOffset(), knowledge)) || failed(resolveIndexValue(storeOp.getDeviceSourceOffset(), knowledge))) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen"); + illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen"); }); hasFailure = true; } @@ -1006,7 +1006,7 @@ private: if (failed(resolveIndexValue(loadOp.getDeviceTargetOffset(), knowledge)) || failed(resolveIndexValue(loadOp.getHostSourceOffset(), knowledge))) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen"); + illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen"); }); hasFailure = true; } @@ -1022,7 +1022,7 @@ private: if (failed(resolveIndexValue(copyOp.getTargetOffset(), knowledge)) || failed(resolveIndexValue(copyOp.getSourceOffset(), knowledge))) { diagnostics.report(&op, [](Operation* illegalOp) { - illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen"); + illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen"); }); hasFailure = true; } @@ -1032,7 +1032,7 @@ private: && failed(resolveIndexValue(receiveOp.getOutputOffset(), knowledge))) { diagnostics.report(&op, [](Operation* illegalOp) { illegalOp->emitOpError( - "output offset must be statically evaluable for PIM codegen"); + "output offset must be statically evaluable for Pim codegen"); }); hasFailure = true; } diff --git a/src/PIM/Dialect/Pim/Pim.td b/src/PIM/Dialect/Pim/Pim.td index f0a7b9a..ad0e55b 100644 --- a/src/PIM/Dialect/Pim/Pim.td +++ b/src/PIM/Dialect/Pim/Pim.td @@ -11,7 +11,7 @@ include "mlir/Dialect/Bufferization/IR/BufferViewFlowOpInterface.td" def PimDialect : Dialect { let name = "pim"; - let summary = "A low-level dialect for the PIM coprocessors on ReRAM crossbars"; + let summary = "A low-level dialect for the Pim coprocessors on ReRAM crossbars"; let cppNamespace = "::onnx_mlir::pim"; } @@ -27,7 +27,7 @@ def PimTensor : def PimCoreOp : PimOp<"core", [SingleBlock, DeclareOpInterfaceMethods]> { - let summary = "Execute a block on a PIM core"; + let summary = "Execute a block on a Pim core"; let regions = (region SizedRegion<1>:$body); diff --git a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledSpatialPasses.cpp b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledSpatialPasses.cpp index 8d2b0c1..4a95e58 100644 --- a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledSpatialPasses.cpp +++ b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledSpatialPasses.cpp @@ -28,7 +28,7 @@ static bool hasValidTarget(const SchedulingTarget& target) { static FailureOr requireEntry(ModuleOp moduleOp) { auto entry = getPimEntryFunc(moduleOp); if (failed(entry)) { - moduleOp.emitError("failed to locate the PIM entry function during Spatial scheduling and realization"); + moduleOp.emitError("failed to locate the Pim entry function during Spatial scheduling and realization"); return failure(); } return *entry; diff --git a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.cpp b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.cpp index ab035cf..1639001 100644 --- a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.cpp +++ b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.cpp @@ -841,7 +841,7 @@ static FailureOr assignPipelineStages( std::string &error) { if (graph.nodes.empty()) return PipelineStageAssignment { - {}, std::vector(layout.getStageCount(), 1)}; + {}, std::vector(layout.getStageSizes())}; std::vector tasksByOrder(graph.nodes.size()); std::iota(tasksByOrder.begin(), tasksByOrder.end(), 0); llvm::sort(tasksByOrder, [&](size_t lhs, size_t rhs) { diff --git a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/SpatialDataflowCsvExporter.cpp b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/SpatialDataflowCsvExporter.cpp index 5f2964c..660876f 100644 --- a/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/SpatialDataflowCsvExporter.cpp +++ b/src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/SpatialDataflowCsvExporter.cpp @@ -193,7 +193,7 @@ FailureOr buildTopLevelOpInfo(Operation& op, bool isScheduled, s if constexpr (std::is_same_v) { if (auto compute = dyn_cast(&op)) { - auto coreId = getOptionalScheduledCoreId(compute, "spatial dataflow export core id"); + auto coreId = getOptionalScheduledCoreId(compute, "Spatial dataflow export core id"); if (failed(coreId)) return failure(); if (*coreId) @@ -207,7 +207,7 @@ FailureOr buildTopLevelOpInfo(Operation& op, bool isScheduled, s template FailureOr> getBatchLaneCoreIds(BatchOpTy batch) { if constexpr (std::is_same_v) { - auto coreIds = getOptionalScheduledBatchCoreIds(batch, "spatial dataflow export core ids"); + auto coreIds = getOptionalScheduledBatchCoreIds(batch, "Spatial dataflow export core ids"); if (failed(coreIds)) return failure(); if (!*coreIds) diff --git a/src/PIM/Dialect/Spatial/Spatial.td b/src/PIM/Dialect/Spatial/Spatial.td index 81bd576..b0b1242 100644 --- a/src/PIM/Dialect/Spatial/Spatial.td +++ b/src/PIM/Dialect/Spatial/Spatial.td @@ -13,7 +13,7 @@ include "mlir/Interfaces/SideEffectInterfaces.td" def SpatialDialect : Dialect { let name = "spat"; - let summary = "Dialect designed for deep learning computation in a spatial architecture"; + let summary = "Dialect designed for deep learning computation in a Spatial architecture"; let cppNamespace = "::onnx_mlir::spatial"; let useDefaultAttributePrinterParser = 0; let extraClassDeclaration = [{ diff --git a/src/PIM/Passes/PIMPasses.h b/src/PIM/Passes/PIMPasses.h index 4a788bf..bac7ae2 100644 --- a/src/PIM/Passes/PIMPasses.h +++ b/src/PIM/Passes/PIMPasses.h @@ -25,7 +25,8 @@ std::unique_ptr createONNXToSpatialPass( const spatial::SpatialTargetResources& target, const ONNXToSpatialPlanningOptions& options); std::unique_ptr createSpatialLayoutPlanningPass(); -std::unique_ptr createSpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target); +std::unique_ptr createSpatialLayoutPlanningPass( + const spatial::SpatialTargetResources& target, bool selectTrivialPlan = false); std::unique_ptr createLowerSpatialPlansPass(); std::unique_ptr createLowerSpatialPlansPass( const spatial::SpatialTargetResources& target, diff --git a/src/PIM/Passes/PimCodegen/EmitPimCodePass.cpp b/src/PIM/Passes/PimCodegen/EmitPimCodePass.cpp index a150f3c..4669db1 100644 --- a/src/PIM/Passes/PimCodegen/EmitPimCodePass.cpp +++ b/src/PIM/Passes/PimCodegen/EmitPimCodePass.cpp @@ -12,7 +12,7 @@ namespace { struct EmitPimCodePass : PassWrapper> { MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(EmitPimCodePass); StringRef getArgument() const override { return "emit-pim-code-pass"; } - StringRef getDescription() const override { return "Emit PIM simulator code artifacts"; } + StringRef getDescription() const override { return "Emit Pim simulator code artifacts"; } EmitPimCodePass() {} EmitPimCodePass(const EmitPimCodePass& pass) {} @@ -25,7 +25,7 @@ struct EmitPimCodePass : PassWrapper> { int compiler_error_code = compileToPimCode(moduleOp, pimDir); if (compiler_error_code != CompilerSuccess) { - moduleOp.emitError() << "failed to emit PIM simulator code artifacts; compiler error code " + moduleOp.emitError() << "failed to emit Pim simulator code artifacts; compiler error code " << compiler_error_code; signalPassFailure(); } diff --git a/test/PIM/SpatialSchedulingTargetTest.cpp b/test/PIM/SpatialSchedulingTargetTest.cpp index c6c937c..06f6166 100644 --- a/test/PIM/SpatialSchedulingTargetTest.cpp +++ b/test/PIM/SpatialSchedulingTargetTest.cpp @@ -130,6 +130,14 @@ int main() { 3, 3, 3, 0, }; std::string pipelineError; + ComputeGraph emptyGraph; + MergeScheduleResult emptySchedule; + emptySchedule.processorCount = 2; + assert(mlir::succeeded(applyPipelineScheduling( + emptyGraph, emptySchedule, 2, physical, pipelineError))); + assert(emptySchedule.processorCount == physical.processorCount); + assert(emptySchedule.processorStages == std::vector({0, 0, 1, 1})); + MergeScheduleResult pipelineSchedule = logicalSchedule; assert(mlir::succeeded(applyPipelineScheduling( graph, pipelineSchedule, 2, physical, pipelineError))); diff --git a/tools/analyze_spatial_ir_cardinality.py b/tools/analyze_spatial_ir_cardinality.py index 90abdef..0976308 100644 --- a/tools/analyze_spatial_ir_cardinality.py +++ b/tools/analyze_spatial_ir_cardinality.py @@ -238,7 +238,7 @@ def print_report(path: Path, counts: Counter, groups: dict[tuple[str, str], Chai def main() -> None: - parser = argparse.ArgumentParser(description="Analyze repeated Spatial/PIM tensor IR cardinality patterns.") + parser = argparse.ArgumentParser(description="Analyze repeated Spatial/Pim tensor IR cardinality patterns.") parser.add_argument("paths", nargs="+", help="MLIR files to analyze.") parser.add_argument("--limit", type=int, default=12, help="Maximum number of hot chains to print per file.") args = parser.parse_args() diff --git a/validation/.gitignore b/validation/.gitignore index 2f72deb..359b5cd 100644 --- a/validation/.gitignore +++ b/validation/.gitignore @@ -1,23 +1,12 @@ -operations/**/inputs -operations/**/outputs -operations/**/raptor -operations/**/runner -operations/**/simulation +operations/**/artifacts operations/**/*.csv !operations/validation_results.csv -networks/**/inputs -networks/**/outputs -networks/**/raptor -networks/**/raptor_functional -networks/**/pimcomp -networks/**/runner -networks/**/simulation -networks/**/real_image_val +networks/**/artifacts networks/**/*.png networks/**/*.jpg networks/**/*.csv !networks/validation_results.csv !networks/full_net/validation_results.csv -!networks/pimcomp_models/validation_results.csv -!networks/pimcomp_models/results.csv +!networks/pimcomp_models/results_comparison.csv +!networks/pimcomp_models/results_ablation.csv diff --git a/validation/README.md b/validation/README.md index d13c003..1d5e612 100644 --- a/validation/README.md +++ b/validation/README.md @@ -1,14 +1,14 @@ -# Raptor Validation +# Raptor validation `validate.py` validates every ONNX model below a selected directory. For each model it can: 1. compile an ONNX-MLIR reference library and runner; 2. generate deterministic random inputs; -3. compile PIM artifacts with Raptor; -4. run the reference implementation and functional PIM simulator; +3. compile Pim artifacts with Raptor; +4. run the reference implementation and functional Pim simulator; 5. compare their outputs; -6. run `pimsim-nn` to report latency, throughput, power, and energy. +6. run Pimsim to report latency, throughput, power, and energy. Run the script from the repository root with the repository Python environment. @@ -61,29 +61,12 @@ Validate a network or network slice: The script discovers them recursively and writes `validation_results.csv` in that directory while retaining separate latency and throughput terminal tables. -## Raptor vs PIMCOMP comparison +## Pim validation tools -The PIMCOMP paper-model suite has a one-command multi-architecture comparison: - -```bash -.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py -``` - -The runner verifies PIMCOMP's population-200, 1000-iteration GA settings, -expects Raptor and the existing `third_party/PIMCOMP-NN/build` tree to already -be built, and compares the configured paper models in parallel. Set `--jobs` to -control comparison workers; -the runner leaves `OMP_NUM_THREADS` at its environment default. Use -`--models vgg8` for one model or `--dry-run` to print the commands. - -Generated artifacts are stored beside each model and ignored by Git. The -comparison reuses model-level `common/inputs/`, `common/outputs/`, -`common/runner/` across architectures, modes, and pipelines. Pimsim configs and -network meshes are referenced directly from -`validation/pimsim_configs/pimcomp/`. Architecture- and pipeline-specific `raptor/`, `simulation/`, and -PIMCOMP artifacts stay under each comparison directory. See -[`networks/pimcomp_models/README.md`](networks/pimcomp_models/README.md) for -profiles, model provenance, limitations, and remote execution. +- [Pimcomp model suite](networks/pimcomp_models/README.md) +- [Pimcomp model comparison tools](tools/pim/pimcomp/compare/README.md) +- [Pimcomp correctness study](tools/pim/pimcomp/correctness/README.md) +- [Raptor compiler ablation study](tools/pim/ablation/README.md) ## Validation modes @@ -92,7 +75,7 @@ uses one input, while throughput uses `--pipeline=4` with four distinct inputs. Both modes reuse the generated input batch, native runner, and reference outputs, and every throughput output is compared with its own reference. -Use `--compile-only` to build the reference runner and PIM artifacts without +Use `--compile-only` to build the reference runner and Pim artifacts without executing either implementation: ```bash @@ -139,23 +122,23 @@ count with `-j` or `--jobs`: | `--onnx-include-dir PATH` | ONNX-MLIR runtime include directory. Required unless `--clean` is used. | | `--operations-dir PATH` | Directory tree containing models. Defaults to `validation/operations`. | | `--simulator-dir PATH` | Functional `pim-simulator` crate directory. Defaults to the in-tree simulator. | -| `--non-functional-simulator-build-dir PATH` | `pimsim-nn` build directory. Defaults to the in-tree build. | +| `--non-functional-simulator-build-dir PATH` | Pimsim build directory. Defaults to the in-tree build. | | `--pimcomp-config {arch-a,arch-b,arch-c}` | Non-functional hardware/timing profile. Defaults to `arch-a`. | -| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency, throughput, power, and energy measurement. | -| `--no-fast` | Disable fast throughput convergence for authoritative full-duration `pimsim-nn` experiments. | +| `--skip-non-functional-simulation` | Skip Pimsim latency, throughput, power, and energy measurement. | +| `--no-fast` | Disable fast throughput convergence for authoritative full-duration Pimsim experiments. | | `--threshold FLOAT` | Absolute output-comparison tolerance. Defaults to `1e-3`. | | `--relative-threshold FLOAT` | Relative output-comparison tolerance. Defaults to `1e-5`. | | `--seed INT` | Seed for generated inputs. Defaults to `0`. | | `--crossbar-size INT` | Crossbar dimensions passed to Raptor. Defaults to the Arch-A value, `128`. | | `--crossbar-count INT` | Crossbars per core passed to Raptor. Defaults to the Arch-A value, `96`. | -| `--core-count INT` | PIM core count passed to Raptor. Defaults to the Arch-A value, `168`. | +| `--core-count INT` | Pim core count passed to Raptor. Defaults to the Arch-A value, `168`. | | `--raptor-extra-arg=ARG` | Additional Raptor compiler argument. Repeat for multiple arguments. | | `--command-timeout-seconds FLOAT` | Timeout for each compiler, runner, and simulator subprocess. Defaults to `1000000.0`. | | `-j INT`, `--jobs INT` | Parallel validation workers. Defaults to all available CPUs and must be at least one. | | `--clean` | Remove generated validation artifacts and exit. | -| `--compile-only` | Compile reference and PIM artifacts without execution or comparison. | +| `--compile-only` | Compile reference and Pim artifacts without execution or comparison. | | `--run-only` | Reuse compiled artifacts and perform execution, simulation, and comparison. | -| `--verbose` | Print passing per-stage and subprocess logs, plus average PIM pass timings. | +| `--verbose` | Print passing per-stage and subprocess logs, plus average Pim pass timings. | Arguments beginning with `--` that are passed through to Raptor should use the equals form: @@ -166,7 +149,7 @@ equals form: ## Hardware profiles and non-functional simulation -The selected PIMCOMP profile must match `--core-count`, `--crossbar-count`, and +The selected Pimcomp profile must match `--core-count`, `--crossbar-count`, and `--crossbar-size`. A mismatch disables only non-functional simulation and prints the incompatible values; functional validation still runs. @@ -189,27 +172,30 @@ comparison. A non-functional simulation failure remains visible as `ERROR` in the corresponding latency, throughput, power, or energy columns but does not change a functional PASS. -`pimsim-nn` does not currently implement the `vsoftmax` instruction. When its +Pimsim does not currently implement the `vsoftmax` instruction. When its explicit unsupported-op diagnostic is encountered, Softmax validations retain their functional PASS and show `UNSUPPORTED` in the non-functional columns. -Other `pimsim-nn` failures remain `ERROR`. +Other Pimsim failures remain `ERROR`. ## Generated artifacts -Artifacts are written beside each model: +Generated files are grouped below an `artifacts/` directory beside each model +or operation case. This keeps checked-in ONNX files and generated trees +separate and lets `--clean` remove the complete workspace, including stale +validation lock files: | Path | Contents | |---|---| -| `inputs.csv` | Generated inputs, one batch entry per line. | -| `inputs/`, `outputs/`, `runner/` | Inputs, reference outputs, and the runner shared by latency and throughput validation. | -| `raptor/pim/`, `simulation/latency/` | Latency PIM artifacts and functional simulator outputs. | -| `raptor/throughput/pim/`, `simulation/throughput/` | Pipeline-4, batch-4 throughput PIM artifacts and functional simulator outputs. | -| `common/inputs/` | Shared generated input CSV files. | -| `common/outputs/` | Shared ONNX-MLIR reference output CSV files. | -| `common/runner/` | Shared reference runner source, build tree, and library. | -| `/[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and PIM artifacts. | -| `/[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. | -| `/[/pipelineN]/pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. | +| `artifacts/inputs.csv` | Generated inputs, one batch entry per line. | +| `artifacts/inputs/`, `artifacts/outputs/`, `artifacts/runner/` | Inputs, reference outputs, and the runner shared by latency and throughput validation. | +| `artifacts/raptor/pim/`, `artifacts/simulation/latency/` | Latency Pim artifacts and functional simulator outputs. | +| `artifacts/raptor/throughput/pim/`, `artifacts/simulation/throughput/` | Pipeline-4, batch-4 throughput Pim artifacts and functional simulator outputs. | +| `artifacts/common/inputs/` | Shared generated input CSV files. | +| `artifacts/common/outputs/` | Shared ONNX-MLIR reference output CSV files. | +| `artifacts/common/runner/` | Shared reference runner source, build tree, and library. | +| `artifacts//[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and Pim artifacts. | +| `artifacts//[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. | +| `artifacts//[/pipelineN]/pimcomp/` | Pimcomp graph, instruction, simulator, and comparison-report artifacts. | Each comparison's `raptor/` directory may include `spatial0.mlir`, `spatial1_graph.mlir`, `spatial2_trivial_merged.mlir`, @@ -240,22 +226,23 @@ The generated operation inventory is documented in ## Manual functional simulator tracing -After validation has produced a `raptor/pim/` directory, rerun the functional +After validation has produced an `artifacts/raptor/pim/` directory, rerun the functional simulator with tracing from its crate directory: ```bash cd backend-simulators/pim/pim-simulator cargo run --no-default-features --features tracing --release \ --package pim-simulator --bin pim-simulator -- \ - -f /path/to/workspace/raptor/pim \ - -o /path/to/workspace/simulation/out.bin \ + -f /path/to/workspace/artifacts/raptor/pim \ + -o /path/to/workspace/artifacts/simulation/out.bin \ -d ,,,,... \ --mode latency \ - --input /path/to/workspace/simulation/inputs/input_0.bin + --batch-size 1 \ + --input-dir /path/to/workspace/artifacts/simulation/inputs ``` -Throughput mode additionally requires `--batch-size N` and exactly `N` -`--input` arguments. Each input binary concatenates the model tensors in graph +Throughput mode additionally requires `--batch-size N` and at least `N` +`input_.bin` files in `--input-dir`. Each input binary concatenates the model tensors in graph input order. The comparison validator also writes one native reference and one `simulation/*_iterations/output_*.bin` dump per batch entry, and checks every entry rather than only the final output. @@ -268,7 +255,7 @@ validator normally derives the `-d` address and byte ranges from The final table reports latency and throughput functional pass/fail state plus non-functional latency, throughput, power, and energy. The summary includes pass/fail totals, non-functional simulation -counts, total measured latency, and average PIM pass timings when `--verbose` +counts, total measured latency, and average Pim pass timings when `--verbose` is enabled. - Exit status `0`: all discovered models passed, or cleanup completed. diff --git a/validation/networks/pimcomp_models/README.md b/validation/networks/pimcomp_models/README.md index 15428d8..9f53f14 100644 --- a/validation/networks/pimcomp_models/README.md +++ b/validation/networks/pimcomp_models/README.md @@ -1,38 +1,38 @@ -# PIMCOMP comparison models +# Pimcomp comparison models This directory contains the four networks evaluated in -[PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159): +[Pimcomp: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159): VGG-8, ResNet-18, ResNet-34, and GoogLeNet. It also contains YOLO11n as an additional compiler comparison model. -See the runner-generated [results.csv](results.csv) for the current comparison +See the runner-generated [results_comparison.csv](results_comparison.csv) for the current comparison results. Rows are retained separately for each model, architecture, mode, and pipeline. It records separate `PASS`/`FAIL` functional-validation fields for -the Raptor and PIMCOMP artifacts; rows without a generated report contain `NA`. -Running the runner with `--arch arch-b` or `--arch arch-c` appends those -architecture rows without replacing the existing `arch-a` entries. +the Raptor and Pimcomp artifacts; rows without a generated report contain `NA`. +Use `--archs` to select the architecture rows to generate; existing rows for +other architectures remain unchanged. ## Models and provenance | Directory | Model | Input | Provenance | |--------------|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `resnet18/` | ResNet-18 v1 | `1x3x224x224` | Symlink to the complete [ONNX Model Zoo `resnet18-v1-7`](https://huggingface.co/onnxmodelzoo/resnet18-v1-7) model already present at `../resnet18/depth_68/resnet18_depth_68.onnx`. | -| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as PIMCOMP's frontend does. | +| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as Pimcomp's frontend does. | | `googlenet/` | GoogLeNet | `1x3x224x224` | Unmodified [ONNX Model Zoo `googlenet-12`](https://huggingface.co/onnxmodelzoo/googlenet-12). | -| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [PIMCOMP VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. | +| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [Pimcomp VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. | | `yolo11n/` | YOLO11n detection | `1x3x640x640` | Derived from the canonical local model at `../yolo11n/depth_51/yolo11n_depth_51.onnx`, exported from [Ultralytics YOLO11n](https://github.com/ultralytics/ultralytics/blob/main/docs/en/models/yolo11.md). | -`googlenet/googlenet-12-pimsim-nn.onnx` is the explicit pimsim-nn-ready GoogLeNet model. +`googlenet/googlenet-12-pimsim-nn.onnx` is the explicit Pimsim-ready GoogLeNet model. It removes the two LRN nodes and terminal Softmax from the original model, -so that the comparison covers only operations scheduled by PIMCOMP and supported by pimsim-nn. +so that the comparison covers only operations scheduled by Pimcomp and supported by Pimsim. -`yolo11n/yolo11n-pimsim-nn.onnx` is the explicit pimsim-nn-ready YOLO11n model. +`yolo11n/yolo11n-pimsim-nn.onnx` is the explicit Pimsim-ready YOLO11n model. It removes the Softmax nodes from the original model, -so that the compiled artifact can be simulated in pimsim-nn. +so that the compiled artifact can be simulated in Pimsim. ## Unsupported and ignored operations -PIMCOMP's frontend accepts exactly these ONNX operations: +Pimcomp's frontend accepts exactly these ONNX operations: ```text Add, AveragePool, BatchNormalization, Clip, Concat, Conv, Dropout, Flatten, @@ -40,14 +40,14 @@ Gather, Gemm, GlobalAveragePool, LRN, MatMul, MaxPool, Mul, Pad, Relu, Reshape, Shape, Sigmoid, Softmax, Squeeze, Sub, Sum, Tanh, Transpose, Unsqueeze ``` -`Constant` is consumed as frontend metadata rather than emitted as a PIMCOMP +`Constant` is consumed as frontend metadata rather than emitted as a Pimcomp node. Every other ONNX operation is unsupported: the frontend prints `operation: not considered` and stops at the first occurrence. Thus the complete unsupported set is the complement of the allowlist above for the model's ONNX opset. In particular, YOLO11n contains unsupported `Split` and `Resize` nodes. -PIMCOMP's low-latency scheduler, hierarchy mapper, and genetic algorithm use +Pimcomp's low-latency scheduler, hierarchy mapper, and genetic algorithm use this complete explicit no-consider set: ```text @@ -63,7 +63,7 @@ specific Shape-Gather-Unsqueeze-Concat shape chain, and merge Pad into its consumer. These transformations do not make an otherwise standalone ignored operation timed. -`pimsim-nn` consumes PIM ISA instructions. It supports every named opcode +Pimsim consumes Pim ISA instructions. It supports every named opcode in the shared serialized range except `vsoftmax` (opcode 21), which is rejected explicitly in both JSON and binary input. It silently ignores no opcode; unknown names and numbers are errors. @@ -71,12 +71,12 @@ silently ignores no opcode; unknown names and numbers are errors. These boundaries explain the dedicated artifacts: - GoogLeNet's two LRN nodes and terminal Softmax perform real computation but - are ignored by PIMCOMP, so the common latency artifact removes them. Its + are ignored by Pimcomp, so the common latency artifact removes them. Its inference Dropout and shape-only Reshape can remain without adding compute. - YOLO11n's latency artifact bypasses exactly its two Softmax nodes so it can - run in `pimsim-nn`. Every other node, including MatMul, Transpose, and the + run in Pimsim. Every other node, including MatMul, Transpose, and the final detection-decoding tail, remains present and timed by Raptor. No - PIMCOMP latency is reported because its frontend stops at `Split` and also + Pimcomp latency is reported because its frontend stops at `Split` and also lacks `Resize`; compiling that prefix would not represent YOLO11n. The authoritative lists are in @@ -85,16 +85,16 @@ The authoritative lists are in [`ISA.h`](../../../backend-simulators/pim/pimsim-nn/src/isa/ISA.h), and [`Instruction.cpp`](../../../backend-simulators/pim/pimsim-nn/src/isa/Instruction.cpp). -The PIMCOMP authors did not publish the ONNX checkpoints used by the paper. -Running PIMCOMP's frontend on the three Model Zoo files above produces JSON -graphs exactly equal to PIMCOMP-NN's bundled `resnet18.json`, `resnet34.json`, +The Pimcomp authors did not publish the ONNX checkpoints used by the paper. +Running Pimcomp's frontend on the three Model Zoo files above produces JSON +graphs exactly equal to Pimcomp's bundled `resnet18.json`, `resnet34.json`, and `googlenet.json`. -There is no VGG-8 artifact in the ONNX Model Zoo or any PIMCOMP-NN revision. +There is no VGG-8 artifact in the ONNX Model Zoo or any Pimcomp revision. The included VGG-8 therefore has deterministic random weights and is suitable for compiler and simulator comparison, not paper-accuracy reproduction. The paper also says that VGG-8 and ResNet-18 were trained on MNIST, while the -published PIMCOMP graphs and ResNet Model Zoo artifacts use ImageNet shapes. +published Pimcomp graphs and ResNet Model Zoo artifacts use ImageNet shapes. Current SHA-256 checksums: @@ -113,10 +113,10 @@ The files in [`../../pimsim_configs/pimcomp/`](../../pimsim_configs/pimcomp/) encode Table V's explicit resource parameters. Each profile subdirectory contains pre-generated latency and throughput -`pimsim-nn` configs plus its matching mesh; comparison and validation reference +Pimsim configs plus its matching mesh; comparison and validation reference these canonical artifacts directly. -| Config | Cores | Crossbars/core | Crossbar | Cell | PIMCOMP layout | +| Config | Cores | Crossbars/core | Crossbar | Cell | Pimcomp layout | |------------------------------|------------------:|---------------:|------------|------:|-----------------| | `arch-a/latency_config.json` | 168 | 96 | `128x128` | 2-bit | `12x14` | | `arch-b/latency_config.json` | 138 | 128 | `128x128` | 2-bit | `6x23` | @@ -125,10 +125,10 @@ these canonical artifacts directly. `adc_count` is 16, matching the paper's 16-bit fixed-point weight precision. The paper does not give a two-dimensional core topology for Arch-A/B, so the factorizations above preserve core count but cannot reproduce unpublished NoC -placement details. Released PIMCOMP-NN has no chip-count field; Arch-C is +placement details. Released Pimcomp has no chip-count field; Arch-C is therefore flattened to 64 cores and does not model chip boundaries. -The remaining latency and power values come from PIMCOMP-NN's released default +The remaining latency and power values come from Pimcomp's released default configuration. Consequently, instruction/resource comparisons are reproducible, but absolute paper power and energy numbers are not. @@ -147,9 +147,9 @@ cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN Do not build either project with `ninja` directly. -## Compile with PIMCOMP +## Compile with Pimcomp -PIMCOMP-NN reads `third_party/PIMCOMP-NN/config.json` directly. Back it up, +Pimcomp reads `third_party/PIMCOMP-NN/config.json` directly. Back it up, select one paper profile, and restore it when the shell exits: ```bash @@ -162,7 +162,7 @@ trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT cp "$PIMCOMP_CONFIGS/arch-a/latency_config.json" "$PIMCOMP/config.json" ``` -The Model Zoo files map exactly to PIMCOMP's bundled model names, so compile +The Model Zoo files map exactly to Pimcomp's bundled model names, so compile them directly: ```bash @@ -179,7 +179,7 @@ cd "$PIMCOMP/build" ./PIMCOMP-NN -m=googlenet -r=balance -p=element -o=YES -v=YES -s=YES ``` -VGG-8 first needs PIMCOMP's JSON frontend. Use a temporary ONNX copy because +VGG-8 first needs Pimcomp's JSON frontend. Use a temporary ONNX copy because the released frontend rewrites the input batch dimension in place: ```bash @@ -201,62 +201,66 @@ random placement code occasionally segfaults; an unchanged retry succeeded in the observed cases. The paper's optimizer uses a genetic algorithm with population 200 and up to -1000 iterations. The checked-out PIMCOMP submodule already has both paper +1000 iterations. The checked-out Pimcomp submodule already has both paper settings in `backend/GeneticAlgorithm.h`; select them with `-r=GA`. Fitness evaluation uses OpenMP and bounded bandwidth timelines. Set `OMP_NUM_THREADS` to control its parallelism; otherwise OpenMP uses the available CPUs. The GA uses the fixed seed `1`, so repeated serial and parallel runs are reproducible. -## Compare Raptor and PIMCOMP +## Compare Raptor and Pimcomp The comparison driver uses one random input and one native ONNX-MLIR reference, -compiles both instruction streams, runs both through `pimsim-nn`, runs +compiles both instruction streams, runs both through Pimsim, runs functional validation through `pim-simulator`, and writes Markdown and JSON reports. -To reproduce all configured architectures and both latency/throughput modes, -use the model-by-model runner. It verifies the paper GA settings, expects Raptor +To reproduce the default `arch-a`/`arch-b` architectures and both +latency/throughput modes, use the model-by-model runner. Use `--archs` to +specify a different architecture set. It verifies the paper GA settings, expects Raptor and the existing `third_party/PIMCOMP-NN/build` tree to already be built, then -runs the comparisons in parallel and regenerates `results.csv` from the JSON +runs the comparisons in parallel and regenerates `results_comparison.csv` from the JSON reports: ```bash -.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py +.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py ``` -Use `--arch arch-a --mode latency` for only the Arch-A latency experiment. +Use `--archs arch-a --mode latency` for only the Arch-A latency experiment, or +`--archs arch-a arch-b arch-c` to run all three architectures. -Each model directory has a shared ignored `common/` directory containing +Each model directory has a shared ignored `artifacts/common/` directory containing `inputs/`, `outputs/`, and the native `runner/`. -Pimsim-nn configs and network meshes remain canonical under +Pimsim configs and network meshes remain canonical under `validation/pimsim_configs/pimcomp/` and are referenced in place. -Architecture- and pipeline-specific `raptor/`, `simulation/`, and PIMCOMP -artifacts remain under each comparison directory; PIMCOMP outputs are prepared +Architecture- and pipeline-specific `raptor/`, `simulation/`, and Pimcomp +artifacts remain under each `artifacts//[/pipelineN]/` comparison +directory; ablation variants use +`artifacts//[/pipelineN]/ablation//`. Pimcomp outputs are prepared once per model/architecture/mode and linked into the other pipeline directories; `comparison_report.{md,json}` live under its `pimcomp/`. The frontend regenerates -one isolated `models/JSON/` graph because PIMCOMP requires that relative +one isolated `models/JSON/` graph because Pimcomp requires that relative layout; it is removed after a successful backend run and the shared submodule model directory is never modified. The original ONNX model is passed to the frontend unchanged. -PIMCOMP's source tree and build directory remain unchanged at runtime. Use +Pimcomp's source tree and build directory remain unchanged at runtime. Use `--models vgg8` to run one model, `--mode throughput` to select one mode, `--pipeline 4` to select one throughput pipeline, `--only raptor` or `--only pimcomp` to reuse the other compiler's existing artifacts, `--dry-run` to inspect every command, or `--out-dir PATH` to keep results outside `validation/`. Use `--clean` to remove generated comparison artifacts and -summaries. Selecting a subset replaces only those comparison rows and -recomputes the aggregate `results.csv`; missing shared inputs, outputs, or the +summaries, including stale reference lock files. Selecting a subset replaces only those comparison rows and +recomputes the aggregate `results_comparison.csv`; missing shared inputs, outputs, or the reference runner are generated even for an isolated run. Use `--jobs 4` to cap parallel comparisons. The per-stage timeout is unlimited by default; pass a positive `--timeout-seconds` value to impose one. Throughput comparisons default to `pimsim-nn --fast` with a 1000 ms convergence deadline. Add `--no-fast` for authoritative full-duration runs. -PIMCOMP receives the original ONNX model, and its frontend applies native +Pimcomp receives the original ONNX model, and its frontend applies native BatchNormalization fusion when the graph matches its supported Conv/Gemm pattern. The runner continues after a failed model so all reports are produced. -The known PIMCOMP batch-scheduling correctness issue and a reproducible +The known Pimcomp batch-scheduling correctness issue and a reproducible reference-intermediate prefill experiment are documented in [`validation/tools/pim/pimcomp/correctness/README.md`](../../tools/pim/pimcomp/correctness/README.md). @@ -265,9 +269,9 @@ Arch-A low-latency example: ```bash RAPTOR_ROOT=$PWD -"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py" \ +"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py" \ --model "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/resnet34-v1-7.onnx" \ - --out-dir "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/arch-a/latency" \ + --out-dir "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/artifacts/arch-a/latency" \ --pimcomp-config "$RAPTOR_ROOT/validation/pimsim_configs/pimcomp/arch-a/latency_config.json" \ --core-count 168 \ --crossbar-count 96 \ @@ -279,10 +283,10 @@ RAPTOR_ROOT=$PWD ``` Use the same command with -`yolo11n/yolo11n-pimsim-nn.onnx` to probe YOLO11n. Released PIMCOMP-NN cannot +`yolo11n/yolo11n-pimsim-nn.onnx` to probe YOLO11n. Released Pimcomp cannot compile it: the frontend stops at `/model.2/Split`, and it also has no mapping for YOLO11n's two nearest-neighbor `Resize` nodes. Treating the emitted prefix -as YOLO11n would produce a misleading latency, so no PIMCOMP number is +as YOLO11n would produce a misleading latency, so no Pimcomp number is reported for this model. For Arch-A high throughput, use `--pimsim-mode throughput @@ -303,18 +307,18 @@ generated report. The functional and non-functional simulators intentionally consume different artifacts: -- Raptor and PIMCOMP are validated against the native ONNX-MLIR reference as +- Raptor and Pimcomp are validated against the native ONNX-MLIR reference as FP32 programs in the Rust simulator. Raptor's emitted program is already - FP32. The PIMCOMP-to-Rust export expands its element-addressed storage and + FP32. The Pimcomp-to-Rust export expands its element-addressed storage and byte-sized transfers to FP32, emits `setbw 32, 32`, and keeps vector `imm_len` fields as element counts. -- PIMCOMP's original `SimulationInfo.gz` is copied unchanged for `pimsim-nn`. - PIMCOMP hardcodes `setbw 8, 8` and one byte per element without performing +- Pimcomp's original `SimulationInfo.gz` is copied unchanged for Pimsim. + Pimcomp hardcodes `setbw 8, 8` and one byte per element without performing numerical quantization; this artifact is used only for latency estimation. - Raptor's original FP32 artifact remains unchanged for functional validation. A separate `raptor/pimsim_nn/` view uses `setbw 8, 8` and scales its byte-addressed storage and transfer sizes from four bytes to one byte per - element. Vector `imm_len` fields remain element counts. Like PIMCOMP's + element. Vector `imm_len` fields remain element counts. Like Pimcomp's artifact, this view is not numerically valid and is used only for a fair non-functional comparison. @@ -326,10 +330,10 @@ Current Raptor status: - VGG-8, ResNet-18, fixed-batch ResNet-34, and GoogLeNet compile on Arch-A. - Use `googlenet-12-pimsim-nn.onnx` for the paper-matched latency comparison. - It removes the two LRN nodes and terminal softmax that PIMCOMP does not + It removes the two LRN nodes and terminal softmax that Pimcomp does not schedule. - Raptor currently accepts one square `--crossbar-size`; Arch-C's rectangular - `512x1024` arrays can therefore be compiled by PIMCOMP but not compared + `512x1024` arrays can therefore be compiled by Pimcomp but not compared exactly with Raptor. Do not change the hardware profile to bypass either limitation; that would no @@ -365,7 +369,7 @@ python3 -m venv .venv .venv/bin/python -m pip install numpy onnx onnxruntime colorama # Run every configured comparison in parallel. -.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py +.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py ``` Copy reports back without transferring large compiler artifacts: diff --git a/validation/networks/pimcomp_models/results.csv b/validation/networks/pimcomp_models/results.csv deleted file mode 100644 index a3d1668..0000000 --- a/validation/networks/pimcomp_models/results.csv +++ /dev/null @@ -1,61 +0,0 @@ -arch,model,mode,raptor_pipeline,pimcomp_pipeline,raptor_functional_validation,pimcomp_functional_validation,raptor_latency_ms,pimcomp_latency_ms,raptor_throughput_samples_s,pimcomp_throughput_samples_s,raptor_power_mw,pimcomp_power_mw,raptor_energy_pj,pimcomp_energy_pj,better_compiler,speedup -arch-a,vgg8,latency,1,element,PASS,PASS,1.47,7.99,NA,NA,325.67,200.11,477232782.04,1597904071.12,raptor,5.45 -arch-a,vgg8,throughput,2,batch,PASS,PASS,0.94,1.04,1060.00,965.00,377.54,482.60,474325940.70,541691691.00,raptor,1.10 -arch-a,vgg8,throughput,4,batch,PASS,PASS,1.19,1.04,839.00,965.00,310.32,482.60,500148858.20,541691691.00,pimcomp,1.15 -arch-a,vgg8,throughput,8,batch,PASS,PASS,1.52,1.04,658.00,965.00,279.37,482.60,554348483.90,541691691.00,pimcomp,1.47 -arch-a,resnet18,latency,1,element,PASS,FAIL,15.65,58.18,NA,NA,425.87,238.39,6666000553.12,13869146503.12,raptor,3.72 -arch-a,resnet18,throughput,2,batch,PASS,FAIL,30.55,13.46,32.70,74.30,295.90,481.60,10218790630.00,6562535800.00,pimcomp,2.27 -arch-a,resnet18,throughput,4,batch,PASS,FAIL,26.60,13.46,37.60,74.30,313.14,481.60,10993189350.00,6562535800.00,pimcomp,1.98 -arch-a,resnet18,throughput,8,batch,PASS,FAIL,35.51,13.46,28.20,74.30,275.95,481.60,12844245960.00,6562535800.00,pimcomp,2.63 -arch-a,resnet34,latency,1,element,PASS,FAIL,32.37,66.71,NA,NA,391.82,277.48,12684227802.68,18511540483.68,raptor,2.06 -arch-a,resnet34,throughput,2,batch,FAIL,FAIL,NA,23.58,NA,42.40,NA,515.66,NA,12277699310.00,NA,NA -arch-a,resnet34,throughput,4,batch,FAIL,FAIL,NA,23.58,NA,42.40,NA,515.66,NA,12277699310.00,NA,NA -arch-a,resnet34,throughput,8,batch,PASS,FAIL,115.11,23.58,8.69,42.40,229.85,515.66,29881854960.00,12277699310.00,pimcomp,4.88 -arch-a,googlenet,latency,1,element,PASS,PASS,13.03,62.92,NA,NA,465.11,231.19,6060646341.92,14547510894.24,raptor,4.83 -arch-a,googlenet,throughput,2,batch,PASS,FAIL,24.01,18.27,41.70,54.70,328.17,435.11,8529823770.00,8164469406.00,pimcomp,1.31 -arch-a,googlenet,throughput,4,batch,PASS,FAIL,27.01,18.27,37.00,54.70,305.68,435.11,10498372450.00,8164469406.00,pimcomp,1.48 -arch-a,googlenet,throughput,8,batch,PASS,FAIL,40.03,18.27,25.00,54.70,268.61,435.11,15290418970.00,8164469406.00,pimcomp,2.19 -arch-a,yolo11n,latency,1,element,PASS,FAIL,267.73,NA,NA,NA,235.73,NA,63111612279.00,NA,NA,NA -arch-a,yolo11n,throughput,2,batch,PASS,FAIL,500.00,NA,2.00,NA,227.74,NA,113867559100.00,NA,NA,NA -arch-a,yolo11n,throughput,4,batch,PASS,FAIL,500.00,NA,2.00,NA,233.16,NA,116581237400.00,NA,NA,NA -arch-a,yolo11n,throughput,8,batch,PASS,FAIL,500.00,NA,2.00,NA,253.59,NA,126797224900.00,NA,NA,NA -arch-b,vgg8,latency,1,element,PASS,PASS,1.44,7.15,NA,NA,304.72,173.77,438329594.04,1242814988.12,raptor,4.97 -arch-b,vgg8,throughput,2,batch,PASS,PASS,1.56,1.07,640.00,932.00,287.50,409.47,478592086.00,439288835.80,pimcomp,1.46 -arch-b,vgg8,throughput,4,batch,PASS,PASS,1.14,1.07,874.00,932.00,323.79,409.47,457617420.30,439288835.80,pimcomp,1.07 -arch-b,vgg8,throughput,8,batch,PASS,PASS,1.48,1.07,676.00,932.00,282.34,409.47,437422264.40,439288835.80,pimcomp,1.38 -arch-b,resnet18,latency,1,element,PASS,FAIL,17.98,64.38,NA,NA,362.63,201.66,6520518736.12,12983607699.12,raptor,3.58 -arch-b,resnet18,throughput,2,batch,PASS,FAIL,33.84,13.43,29.60,74.40,253.32,484.80,9423636948.00,6569655551.00,pimcomp,2.51 -arch-b,resnet18,throughput,4,batch,PASS,FAIL,26.14,13.43,38.30,74.40,286.86,484.80,9493311384.00,6569655551.00,pimcomp,1.94 -arch-b,resnet18,throughput,8,batch,PASS,FAIL,28.29,13.43,35.30,74.40,272.66,484.80,11314980340.00,6569655551.00,pimcomp,2.11 -arch-b,resnet34,latency,1,element,PASS,FAIL,49.51,107.20,NA,NA,284.72,206.47,14097763205.68,22134333948.68,raptor,2.17 -arch-b,resnet34,throughput,2,batch,PASS,FAIL,93.13,22.80,10.70,43.90,215.38,493.39,21278771500.00,11381213070.00,pimcomp,4.10 -arch-b,resnet34,throughput,4,batch,PASS,FAIL,67.55,22.80,14.80,43.90,237.97,493.39,19346594270.00,11381213070.00,pimcomp,2.97 -arch-b,resnet34,throughput,8,batch,PASS,FAIL,115.01,22.80,8.70,43.90,197.27,493.39,26678372930.00,11381213070.00,pimcomp,5.05 -arch-b,googlenet,latency,1,element,PASS,PASS,16.08,37.94,NA,NA,378.99,242.04,6095729659.92,9181933205.24,raptor,2.36 -arch-b,googlenet,throughput,2,batch,PASS,FAIL,28.37,21.39,35.20,46.70,267.15,417.83,8684222064.00,9014655684.00,pimcomp,1.33 -arch-b,googlenet,throughput,4,batch,PASS,FAIL,26.16,21.39,38.20,46.70,280.21,417.83,9300527914.00,9014655684.00,pimcomp,1.22 -arch-b,googlenet,throughput,8,batch,PASS,FAIL,46.45,21.39,21.50,46.70,237.73,417.83,14078049200.00,9014655684.00,pimcomp,2.17 -arch-b,yolo11n,latency,1,element,PASS,FAIL,316.70,NA,NA,NA,195.43,NA,61892225295.00,NA,NA,NA -arch-b,yolo11n,throughput,2,batch,PASS,FAIL,333.33,NA,3.00,NA,214.01,NA,71336401290.00,NA,NA,NA -arch-b,yolo11n,throughput,4,batch,PASS,FAIL,333.33,NA,3.00,NA,216.24,NA,72079194110.00,NA,NA,NA -arch-b,yolo11n,throughput,8,batch,PASS,FAIL,500.00,NA,2.00,NA,218.42,NA,109211119900.00,NA,NA,NA -arch-c,vgg8,latency,1,element,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,vgg8,throughput,2,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,vgg8,throughput,4,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,vgg8,throughput,8,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet18,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet18,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet18,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet18,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet34,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet34,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet34,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,resnet34,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,googlenet,latency,1,element,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,googlenet,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,googlenet,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,googlenet,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,yolo11n,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,yolo11n,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,yolo11n,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA -arch-c,yolo11n,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA diff --git a/validation/networks/pimcomp_models/validation_results.csv b/validation/networks/pimcomp_models/validation_results.csv deleted file mode 100644 index 2a5ea3e..0000000 --- a/validation/networks/pimcomp_models/validation_results.csv +++ /dev/null @@ -1,6 +0,0 @@ -Operation,Result,Compile,Host mem,Cores mem,Cores,Xbars,Latency,Power,Energy -vgg8-mnist-reconstructed,PASS,1.009 s,1.37 MiB,3.14 MiB,141,761,1.465778 ms,325.627854 mW,477298145.040001 pJ -resnet18-v1-7,PASS,11.548 s,9.89 MiB,40.24 MiB,168,7676,28.099952 ms,312.513408 mW,8781611766.119984 pJ -resnet34-v1-7,PASS,28.495 s,9.90 MiB,48.89 MiB,168,15292,45.781486 ms,326.833870 mW,14962940227.679951 pJ -googlenet-12-pimsim-nn,PASS,6.573 s,10.74 MiB,22.41 MiB,168,7176,13.371204 ms,457.538139 mW,6117835798.919991 pJ -yolo11n-pimsim-nn,FAIL,58.572 s,82.55 MiB,185.68 MiB,168,6484,885.264931 ms,189.218985 mW,167508931321.001465 pJ diff --git a/validation/operations/README.md b/validation/operations/README.md index 4997d63..3663578 100644 --- a/validation/operations/README.md +++ b/validation/operations/README.md @@ -1,7 +1,7 @@ -# Operation Validation Suite +# Operation validation suite This directory contains the ONNX models used by `validation/validate.py` to -validate individual operations through compilation, PIM simulation, and +validate individual operations through compilation, Pim simulation, and comparison with the ONNX-MLIR reference runtime. ## Naming diff --git a/validation/operations/validation_results.csv b/validation/operations/validation_results.csv deleted file mode 100644 index ab2d740..0000000 --- a/validation/operations/validation_results.csv +++ /dev/null @@ -1,178 +0,0 @@ -Operation,Arch,Result (l),Result (t),Compile (l),Host mem (l),Cores mem (l),Cores (l),Xbars (l),Latency (l),Power (l),Energy (l),Compile (t),Host mem (t),Cores mem (t),Cores (t),Xbars (t),Avg latency (t),Throughput (t),Avg power (t),Avg energy (t) -add/after_gemm,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it -add/basic,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -add/broadcast_row,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -add/channel_broadcast_1024,arch-a,PASS,PASS,0.052 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.036 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it -add/leading_dimension_broadcast,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -concat/channel_axis,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.16 mW,35718.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,2200000.00 samples/s,0.00 ms,2.16 mW,934.67 pJ/it -concat/negative_axis,arch-a,PASS,PASS,0.046 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.09 mW,81450.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,961000.00 samples/s,0.00 ms,2.09 mW,2108.00 pJ/it -concat/three_inputs_channel_axis,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.15 mW,50328.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1560000.00 samples/s,0.00 ms,2.15 mW,1332.67 pJ/it -conv/batch_2,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,2,2,0.01 ms,82.62 mW,1131451.48 pJ,0.044 s,0.00 MiB,0.01 MiB,4,2,147000.00 samples/s,0.01 ms,92.90 mW,669920.48 pJ/it -conv/batch_4_pointwise,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,116.08 mW,456420.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,69.62 mW,291187.04 pJ/it -conv/depthwise_1024_channels,arch-a,PASS,PASS,0.082 s,0.19 MiB,0.38 MiB,129,128,0.22 ms,178.45 mW,39393966.72 pJ,0.117 s,0.38 MiB,0.46 MiB,45,118,4930.00 samples/s,0.20 ms,144.34 mW,33005602.67 pJ/it -conv/depthwise_grouped,arch-a,PASS,PASS,0.055 s,0.01 MiB,0.00 MiB,5,4,0.01 ms,107.78 mW,671878.96 pJ,0.042 s,0.01 MiB,0.00 MiB,7,4,418000.00 samples/s,0.00 ms,168.64 mW,479471.12 pJ/it -conv/dilated_3x3,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,118.77 mW,1034819.16 pJ,0.057 s,0.01 MiB,0.01 MiB,12,9,104000.00 samples/s,0.01 ms,155.40 mW,1574973.57 pJ/it -conv/dynamic,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,92.28 mW,169336.00 pJ,0.044 s,0.00 MiB,0.00 MiB,6,0,778000.00 samples/s,0.00 ms,86.62 mW,124092.33 pJ/it -conv/explicit_padding,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.02 MiB,17,16,0.01 ms,145.34 mW,1454397.84 pJ,0.048 s,0.01 MiB,0.02 MiB,19,16,179000.00 samples/s,0.01 ms,201.80 mW,1234440.20 pJ/it -conv/grouped_many_groups,arch-a,PASS,PASS,0.474 s,0.05 MiB,0.09 MiB,65,64,0.18 ms,142.21 mW,25867112.36 pJ,0.476 s,0.08 MiB,0.73 MiB,87,64,3660.00 samples/s,0.27 ms,97.46 mW,30277556.57 pJ/it -conv/grouped_two_groups,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,101.46 mW,543914.48 pJ,0.047 s,0.00 MiB,0.01 MiB,9,2,378000.00 samples/s,0.00 ms,105.40 mW,353700.73 pJ/it -conv/huge_pointwise_1024,arch-a,PASS,PASS,0.151 s,0.01 MiB,0.11 MiB,73,64,0.02 ms,249.58 mW,3895759.36 pJ,0.163 s,0.08 MiB,0.10 MiB,52,64,19200.00 samples/s,0.05 ms,163.09 mW,8722111.37 pJ/it -conv/huge_pointwise_1024_dynamic,arch-a,PASS,PASS,0.080 s,8.04 MiB,12.61 MiB,168,0,2.63 ms,169.52 mW,445489032.00 pJ,0.224 s,12.75 MiB,6.82 MiB,45,0,188.00 samples/s,5.31 ms,134.20 mW,746263826.70 pJ/it -conv/input_224_7x7_stride2,arch-a,PASS,PASS,0.759 s,24.14 MiB,61.87 MiB,168,169,38.41 ms,185.26 mW,7116544212.12 pJ,1.060 s,51.25 MiB,67.85 MiB,45,84,24.70 samples/s,40.56 ms,148.32 mW,6561548886.00 pJ/it -conv/kernel_2x2,arch-a,PASS,PASS,0.061 s,0.00 MiB,0.00 MiB,1,1,0.00 ms,83.83 mW,360568.24 pJ,0.046 s,0.00 MiB,0.00 MiB,3,1,356000.00 samples/s,0.00 ms,93.82 mW,298919.07 pJ/it -conv/kernel_3x3,arch-a,PASS,PASS,0.061 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,123.80 mW,889640.16 pJ,0.046 s,0.01 MiB,0.01 MiB,12,9,271000.00 samples/s,0.00 ms,195.49 mW,787478.85 pJ/it -conv/kernel_equals_input_spatial,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,89.61 mW,415689.48 pJ,0.041 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,104.42 mW,305239.85 pJ/it -conv/large_input_channels_1x1,arch-a,PASS,PASS,0.085 s,0.01 MiB,0.02 MiB,9,8,0.01 ms,117.84 mW,900569.92 pJ,0.081 s,0.02 MiB,0.02 MiB,10,8,135000.00 samples/s,0.01 ms,113.25 mW,910683.04 pJ/it -conv/large_output_channels_1x1,arch-a,PASS,PASS,0.095 s,0.01 MiB,0.02 MiB,17,8,0.01 ms,128.44 mW,1139415.92 pJ,0.078 s,0.02 MiB,0.02 MiB,18,8,93600.00 samples/s,0.01 ms,151.83 mW,1755770.60 pJ/it -conv/large_spatial,arch-a,PASS,PASS,0.061 s,0.01 MiB,0.04 MiB,37,36,0.02 ms,172.07 mW,2928344.64 pJ,0.061 s,0.01 MiB,0.04 MiB,39,36,84200.00 samples/s,0.01 ms,208.30 mW,2603711.62 pJ/it -conv/multi_channel,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,105.68 mW,685040.72 pJ,0.050 s,0.00 MiB,0.00 MiB,4,3,140000.00 samples/s,0.01 ms,54.54 mW,393755.39 pJ/it -conv/non_square_kernel_1x3,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,99.35 mW,679752.48 pJ,0.044 s,0.00 MiB,0.00 MiB,3,2,140000.00 samples/s,0.01 ms,51.01 mW,367557.81 pJ/it -conv/non_square_kernel_3x1,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,95.89 mW,1292976.48 pJ,0.045 s,0.00 MiB,0.00 MiB,3,2,72200.00 samples/s,0.01 ms,47.79 mW,664683.81 pJ/it -conv/non_uniform_stride,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,104.05 mW,790874.72 pJ,0.044 s,0.00 MiB,0.00 MiB,4,3,126000.00 samples/s,0.01 ms,53.56 mW,429978.05 pJ/it -conv/output_channel_grouping_minimal,arch-a,PASS,PASS,0.080 s,0.10 MiB,0.34 MiB,131,128,0.26 ms,170.73 mW,44125916.72 pJ,0.148 s,0.28 MiB,0.96 MiB,87,84,2630.00 samples/s,0.38 ms,136.74 mW,58542917.01 pJ/it -conv/pointwise_1x1,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,1,0.01 ms,80.24 mW,987244.24 pJ,0.041 s,0.00 MiB,0.00 MiB,3,1,131000.00 samples/s,0.01 ms,89.08 mW,719413.57 pJ/it -conv/pointwise_tiled_chain,arch-a,PASS,PASS,0.642 s,0.01 MiB,0.04 MiB,20,80,0.04 ms,153.90 mW,6443957.20 pJ,0.620 s,0.06 MiB,0.08 MiB,22,80,16400.00 samples/s,0.06 ms,186.69 mW,12410041.37 pJ/it -conv/real_asymmetric_padding,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.03 MiB,29,28,0.01 ms,153.67 mW,2221606.72 pJ,0.055 s,0.01 MiB,0.03 MiB,31,28,105000.00 samples/s,0.01 ms,204.35 mW,2058223.49 pJ/it -conv/relu_conv_store,arch-a,PASS,PASS,0.085 s,0.16 MiB,0.67 MiB,168,184,0.56 ms,183.08 mW,103057723.80 pJ,0.209 s,0.37 MiB,0.58 MiB,60,58,1500.00 samples/s,0.67 ms,146.73 mW,101291048.70 pJ/it -conv/same_lower_3x3,arch-a,PASS,PASS,0.060 s,0.01 MiB,0.02 MiB,26,25,0.01 ms,166.15 mW,2215009.00 pJ,0.054 s,0.01 MiB,0.03 MiB,28,25,119000.00 samples/s,0.01 ms,204.48 mW,1837219.40 pJ/it -conv/same_padding_3x3,arch-a,PASS,PASS,0.058 s,0.01 MiB,0.02 MiB,26,25,0.01 ms,166.15 mW,2215009.00 pJ,0.056 s,0.01 MiB,0.03 MiB,28,25,119000.00 samples/s,0.01 ms,204.48 mW,1837219.40 pJ/it -conv/strategy_depthwise_16,arch-a,PASS,PASS,0.093 s,0.06 MiB,0.35 MiB,168,168,0.34 ms,197.94 mW,66331479.08 pJ,0.257 s,0.20 MiB,0.17 MiB,45,84,2650.00 samples/s,0.38 ms,153.71 mW,60322816.49 pJ/it -conv/strategy_input_k_tiled,arch-a,PASS,PASS,0.082 s,0.08 MiB,0.27 MiB,109,108,0.35 ms,170.81 mW,60422605.92 pJ,0.100 s,0.22 MiB,0.28 MiB,45,90,4250.00 samples/s,0.24 ms,143.52 mW,36750587.75 pJ/it -conv/strategy_output_channel_tiled,arch-a,PASS,PASS,0.073 s,0.03 MiB,0.16 MiB,74,72,0.09 ms,155.74 mW,14244739.28 pJ,0.110 s,0.13 MiB,0.22 MiB,81,72,9220.00 samples/s,0.11 ms,145.73 mW,18577796.14 pJ/it -conv/strategy_streamed_packed,arch-a,PASS,PASS,0.155 s,3.34 MiB,7.89 MiB,168,168,9.35 ms,179.86 mW,1682364509.56 pJ,0.357 s,5.32 MiB,7.79 MiB,43,42,113.00 samples/s,8.87 ms,61.68 mW,558465684.90 pJ/it -conv/strategy_streamed_patch,arch-a,PASS,PASS,0.115 s,0.34 MiB,1.32 MiB,168,168,1.90 ms,181.91 mW,346476645.64 pJ,0.299 s,0.82 MiB,1.21 MiB,43,42,501.00 samples/s,1.99 ms,62.44 mW,125148059.00 pJ/it -conv/strategy_tiled_2d,arch-a,PASS,PASS,0.111 s,0.11 MiB,0.44 MiB,168,168,0.42 ms,182.13 mW,75690907.84 pJ,0.219 s,0.36 MiB,0.46 MiB,47,144,4070.00 samples/s,0.25 ms,155.26 mW,43400768.51 pJ/it -conv/stride_2,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.00 MiB,5,4,0.01 ms,110.78 mW,580154.96 pJ,0.043 s,0.01 MiB,0.00 MiB,7,4,476000.00 samples/s,0.00 ms,175.93 mW,424103.12 pJ/it -conv/with_bias_3x3,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,4,3,0.01 ms,104.16 mW,776220.72 pJ,0.046 s,0.00 MiB,0.01 MiB,4,3,133000.00 samples/s,0.01 ms,53.85 mW,416288.72 pJ/it -conv/with_constant,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,1,0.01 ms,81.74 mW,541270.24 pJ,0.043 s,0.00 MiB,0.00 MiB,4,1,231000.00 samples/s,0.00 ms,133.09 mW,655848.91 pJ/it -conv/without_kernel_shape_attr,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,123.80 mW,889640.16 pJ,0.047 s,0.01 MiB,0.01 MiB,12,9,271000.00 samples/s,0.00 ms,195.49 mW,787478.85 pJ/it -conv/yolo11n_depthwise_head,arch-a,PASS,PASS,1.709 s,8.66 MiB,34.24 MiB,168,255,42.70 ms,200.52 mW,8562452412.00 pJ,2.373 s,27.15 MiB,20.31 MiB,87,214,20.60 samples/s,48.55 ms,157.12 mW,8007922821.00 pJ/it -conv/yolo11n_heavy,arch-a,PASS,PASS,0.519 s,4.82 MiB,19.10 MiB,161,800,8.53 ms,350.87 mW,2994683017.00 pJ,1.007 s,11.06 MiB,13.87 MiB,85,420,79.10 samples/s,12.64 ms,218.17 mW,2857451109.00 pJ/it -conv/yolo11n_stem,arch-a,PASS,PASS,0.893 s,12.86 MiB,37.59 MiB,168,488,14.24 ms,301.23 mW,4289558246.00 pJ,1.374 s,25.19 MiB,21.24 MiB,85,126,44.60 samples/s,22.42 ms,176.88 mW,4028789243.00 pJ/it -div/after_gemm,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it -div/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -div/channel_broadcast_1024,arch-a,PASS,PASS,0.049 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.039 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it -div/leading_dimension_broadcast,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -div/runtime_scalar_rhs,arch-a,PASS,PASS,0.053 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.040 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it -div/scalar_constant,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -gather/3d_input_axis1,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.08 mW,45990.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,1700000.00 samples/s,0.00 ms,2.08 mW,1174.67 pJ/it -gather/axis0_matrix_indices,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.07 mW,54414.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,1440000.00 samples/s,0.00 ms,2.07 mW,1390.67 pJ/it -gather/axis1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.06 mW,62526.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,1250000.00 samples/s,0.00 ms,2.06 mW,1598.67 pJ/it -gather/negative_axis,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.03 mW,112134.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,697000.00 samples/s,0.00 ms,2.03 mW,2870.67 pJ/it -gather/negative_indices,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.13 mW,29376.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,2670000.00 samples/s,0.00 ms,2.12 mW,748.67 pJ/it -gemm/alpha_beta,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.27 mW,784908.96 pJ,0.050 s,0.01 MiB,0.01 MiB,6,4,212000.00 samples/s,0.00 ms,107.78 mW,574589.95 pJ/it -gemm/bias_rank2_broadcast,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,105.98 mW,749484.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,229000.00 samples/s,0.00 ms,109.50 mW,540708.12 pJ/it -gemm/dynamic,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.48 mW,221475.00 pJ,0.044 s,0.00 MiB,0.00 MiB,5,0,489000.00 samples/s,0.00 ms,44.34 mW,93754.67 pJ/it -gemm/dynamic_alpha,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.42 mW,298198.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,464000.00 samples/s,0.00 ms,44.39 mW,105291.67 pJ/it -gemm/dynamic_beta,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.32 mW,397230.00 pJ,0.042 s,0.00 MiB,0.00 MiB,5,0,247000.00 samples/s,0.00 ms,20.21 mW,81901.75 pJ/it -gemm/dynamic_bias,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.45 mW,243703.00 pJ,0.046 s,0.00 MiB,0.00 MiB,5,0,422000.00 samples/s,0.00 ms,20.28 mW,48009.75 pJ/it -gemm/dynamic_bias_alpha_beta,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,5,0,0.01 ms,91.28 mW,513811.00 pJ,0.043 s,0.00 MiB,0.00 MiB,5,0,188000.00 samples/s,0.01 ms,20.20 mW,107673.75 pJ/it -gemm/dynamic_transpose_b,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.38 mW,118883.00 pJ,0.040 s,0.00 MiB,0.00 MiB,5,0,803000.00 samples/s,0.00 ms,44.55 mW,58232.00 pJ/it -gemm/huge_1024,arch-a,PASS,PASS,0.149 s,0.01 MiB,0.10 MiB,73,64,0.02 ms,215.07 mW,3767010.36 pJ,0.160 s,0.05 MiB,0.09 MiB,51,64,26400.00 samples/s,0.04 ms,135.13 mW,5670462.59 pJ/it -gemm/large,arch-a,PASS,PASS,0.058 s,0.02 MiB,0.03 MiB,17,16,0.01 ms,140.15 mW,1573768.84 pJ,0.050 s,0.03 MiB,0.03 MiB,17,16,78500.00 samples/s,0.01 ms,79.60 mW,1008067.12 pJ/it -gemm/large_k_small_n,arch-a,PASS,PASS,0.088 s,0.01 MiB,0.01 MiB,9,8,0.00 ms,133.53 mW,633217.92 pJ,0.082 s,0.01 MiB,0.01 MiB,9,8,182000.00 samples/s,0.01 ms,84.31 mW,476982.66 pJ/it -gemm/non_square,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,118.96 mW,419565.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,71.16 mW,302182.45 pJ/it -gemm/scalar_bias,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,105.98 mW,749484.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,229000.00 samples/s,0.00 ms,109.50 mW,540708.12 pJ/it -gemm/small,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,90.14 mW,398436.48 pJ,0.043 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,105.00 mW,297577.19 pJ/it -gemm/small_k_large_n,arch-a,PASS,PASS,0.090 s,0.01 MiB,0.02 MiB,17,8,0.01 ms,131.01 mW,1043061.92 pJ,0.082 s,0.02 MiB,0.02 MiB,18,8,92700.00 samples/s,0.01 ms,152.38 mW,1742739.72 pJ/it -gemm/square_weights,arch-a,PASS,PASS,0.073 s,0.03 MiB,0.08 MiB,42,40,0.02 ms,151.77 mW,3284393.60 pJ,0.075 s,0.07 MiB,0.09 MiB,44,40,25700.00 samples/s,0.04 ms,155.80 mW,6344327.95 pJ/it -gemm/transpose_a,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.14 mW,628868.96 pJ,0.047 s,0.01 MiB,0.01 MiB,6,4,290000.00 samples/s,0.00 ms,115.23 mW,456854.62 pJ/it -gemm/transpose_a_and_b,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.14 mW,628868.96 pJ,0.044 s,0.01 MiB,0.01 MiB,6,4,290000.00 samples/s,0.00 ms,115.23 mW,456854.62 pJ/it -gemm/transpose_b,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,118.96 mW,419565.96 pJ,0.045 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,71.16 mW,302182.45 pJ/it -gemm/transpose_b_with_bias,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,110.55 mW,557818.96 pJ,0.046 s,0.01 MiB,0.01 MiB,5,4,218000.00 samples/s,0.00 ms,67.31 mW,333372.79 pJ/it -gemm/with_bias,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,108.77 mW,604966.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,203000.00 samples/s,0.00 ms,66.54 mW,341027.79 pJ/it -gemv/all_constant,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,0,0,0.00 ms,2.00 mW,0.00 pJ,0.039 s,0.00 MiB,0.00 MiB,0,0,0.00 samples/s,0.00 ms,2.00 mW,0.00 pJ/it -gemv/constant_weight,arch-a,PASS,PASS,0.065 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,111.15 mW,573535.96 pJ,0.051 s,0.01 MiB,0.01 MiB,8,4,263000.00 samples/s,0.00 ms,154.40 mW,641346.96 pJ/it -gemv/non_uniform_bias,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.051 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it -gemv/scalar_bias,arch-a,PASS,PASS,0.061 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.052 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it -gemv/uniform_bias,arch-a,PASS,PASS,0.066 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.049 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it -matmul/basic,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,90.14 mW,398436.48 pJ,0.040 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,105.00 mW,297577.19 pJ/it -matmul/batched_3d,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.59 mW,646972.96 pJ,0.048 s,0.01 MiB,0.01 MiB,6,4,275000.00 samples/s,0.00 ms,114.15 mW,469999.95 pJ/it -matmul/batched_3d_dynamic,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,92.19 mW,167975.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,758000.00 samples/s,0.00 ms,18.49 mW,24624.50 pJ/it -matmul/batched_left_constant,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.02 MiB,9,8,0.01 ms,114.39 mW,1009105.92 pJ,0.059 s,0.01 MiB,0.02 MiB,11,8,149000.00 samples/s,0.01 ms,159.00 mW,1186296.28 pJ/it -matmul/batched_lhs_broadcast,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.39 mW,621440.96 pJ,0.042 s,0.01 MiB,0.01 MiB,6,4,276000.00 samples/s,0.00 ms,114.66 mW,463736.29 pJ/it -matmul/batched_rhs_broadcast,arch-a,PASS,PASS,0.058 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.59 mW,646972.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,275000.00 samples/s,0.00 ms,114.15 mW,469999.95 pJ/it -matmul/dynamic,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.42 mW,148195.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,660000.00 samples/s,0.00 ms,44.46 mW,70471.33 pJ/it -matmul/huge_1024,arch-a,PASS,PASS,0.146 s,0.01 MiB,0.10 MiB,73,64,0.02 ms,215.07 mW,3767010.36 pJ,0.162 s,0.05 MiB,0.09 MiB,51,64,26400.00 samples/s,0.04 ms,135.13 mW,5670462.59 pJ/it -matmul/left_constant,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.86 mW,637168.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,297000.00 samples/s,0.00 ms,115.66 mW,451626.62 pJ/it -matmul/matrix_vector,arch-a,PASS,PASS,0.098 s,0.52 MiB,0.78 MiB,168,173,0.38 ms,202.13 mW,77751476.88 pJ,0.289 s,1.04 MiB,0.84 MiB,44,171,2150.00 samples/s,0.46 ms,119.16 mW,65096859.53 pJ/it -matmul/vector_matrix,arch-a,PASS,PASS,0.089 s,0.01 MiB,0.01 MiB,9,8,0.01 ms,118.70 mW,878749.92 pJ,0.083 s,0.02 MiB,0.02 MiB,10,8,140000.00 samples/s,0.01 ms,115.31 mW,910373.16 pJ/it -matmul/yolo_attention,arch-a,PASS,PASS,0.474 s,1.02 MiB,43.44 MiB,168,0,8.15 ms,170.00 mW,1385775865.00 pJ,0.645 s,5.57 MiB,42.54 MiB,75,0,145.00 samples/s,6.89 ms,119.04 mW,932561637.00 pJ/it -mul/after_conv,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.64 mW,586955.72 pJ,0.050 s,0.00 MiB,0.00 MiB,4,3,194000.00 samples/s,0.01 ms,58.31 mW,304072.05 pJ/it -mul/after_conv_scalar_constant,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.64 mW,586955.72 pJ,0.046 s,0.00 MiB,0.00 MiB,4,3,194000.00 samples/s,0.01 ms,58.31 mW,304072.05 pJ/it -mul/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -mul/channel_broadcast_1024,arch-a,PASS,PASS,0.050 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.038 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it -mul/leading_dimension_broadcast,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -mul/scalar_constant,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -pool/avg_basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,929400.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,84200.00 samples/s,0.01 ms,2.02 mW,24013.00 pJ/it -pool/avg_ceil_mode,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.03 mW,339210.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,230000.00 samples/s,0.00 ms,2.03 mW,8786.67 pJ/it -pool/avg_explicit_padding,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.03 mW,685860.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,114000.00 samples/s,0.01 ms,2.03 mW,17745.00 pJ/it -pool/avg_include_pad,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,661116.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,118000.00 samples/s,0.01 ms,2.02 mW,17017.00 pJ/it -pool/avg_large_channels,arch-a,PASS,PASS,0.059 s,0.04 MiB,0.02 MiB,1,0,0.24 ms,78.00 mW,18397284.00 pJ,0.046 s,0.04 MiB,0.02 MiB,1,0,4250.00 samples/s,0.24 ms,2.00 mW,471380.00 pJ/it -pool/avg_non_uniform_stride,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,1129134.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,69300.00 samples/s,0.01 ms,2.02 mW,29111.00 pJ/it -pool/avg_real_asymmetric_padding,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.03 ms,78.02 mW,1959204.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,39900.00 samples/s,0.03 ms,2.02 mW,50769.00 pJ/it -pool/max_after_conv,arch-a,PASS,PASS,0.063 s,0.00 MiB,0.00 MiB,5,4,0.01 ms,99.12 mW,1209961.96 pJ,0.051 s,0.00 MiB,0.00 MiB,5,4,140000.00 samples/s,0.01 ms,58.47 mW,468396.45 pJ/it -pool/max_basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.06 mW,324744.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,241000.00 samples/s,0.00 ms,2.06 mW,8532.67 pJ/it -pool/max_ceil_mode,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.07 mW,151464.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,516000.00 samples/s,0.00 ms,2.07 mW,3972.67 pJ/it -pool/max_global_style_kernel_equals_input,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.01 mW,657534.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,119000.00 samples/s,0.01 ms,2.01 mW,16843.00 pJ/it -pool/max_non_square_kernel,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,1060572.00 pJ,0.047 s,0.00 MiB,0.00 MiB,1,0,73700.00 samples/s,0.01 ms,2.02 mW,27353.00 pJ/it -pool/max_real_asymmetric_padding,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.03 mW,814992.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,96100.00 samples/s,0.01 ms,2.03 mW,21173.00 pJ/it -pool/max_same_upper,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.04 mW,625068.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,125000.00 samples/s,0.01 ms,2.04 mW,16233.00 pJ/it -pool/max_stride2_multichannel,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.02 ms,78.02 mW,1245870.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,62800.00 samples/s,0.02 ms,2.02 mW,32117.00 pJ/it -reduce_mean/4d_spatial,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.02 mW,326814.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,239000.00 samples/s,0.00 ms,2.02 mW,8390.67 pJ/it -reduce_mean/4d_spatial_keepdims_0,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,94.35 mW,61801.00 pJ,0.041 s,0.00 MiB,0.00 MiB,4,0,1470000.00 samples/s,0.00 ms,44.66 mW,31425.33 pJ/it -reduce_mean/after_conv,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,100.79 mW,1119699.72 pJ,0.057 s,0.00 MiB,0.00 MiB,4,3,125000.00 samples/s,0.01 ms,54.14 mW,474782.17 pJ/it -reduce_mean/all_axes_keepdims_0,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.24 mW,30982.00 pJ,0.038 s,0.00 MiB,0.00 MiB,2,0,2720000.00 samples/s,0.00 ms,44.37 mW,16707.00 pJ/it -reduce_mean/all_axes_keepdims_1,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it -reduce_mean/basic,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.51 mW,34881.00 pJ,0.042 s,0.00 MiB,0.00 MiB,4,0,2600000.00 samples/s,0.00 ms,5.85 mW,2235.67 pJ/it -reduce_mean/channel_axis_nchw,arch-a,PASS,PASS,0.054 s,0.03 MiB,0.02 MiB,4,0,0.16 ms,93.60 mW,15436518.00 pJ,0.040 s,0.03 MiB,0.08 MiB,4,0,12900.00 samples/s,0.08 ms,5.00 mW,388853.50 pJ/it -reduce_mean/keepdims_0,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.40 mW,68368.00 pJ,0.043 s,0.00 MiB,0.00 MiB,5,0,1300000.00 samples/s,0.00 ms,44.76 mW,36032.00 pJ/it -reduce_mean/large_dimension_1024,arch-a,PASS,PASS,0.048 s,0.01 MiB,0.00 MiB,1,0,0.00 ms,78.02 mW,217278.00 pJ,0.040 s,0.01 MiB,0.00 MiB,1,0,359000.00 samples/s,0.00 ms,2.02 mW,5274.00 pJ/it -reduce_mean/legacy_axes_1_2_keepdims_1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.35 mW,21505.00 pJ,0.042 s,0.00 MiB,0.00 MiB,2,0,3620000.00 samples/s,0.00 ms,3.45 mW,898.00 pJ/it -reduce_mean/legacy_axis1_keepdims_0,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,9,0,0.00 ms,92.50 mW,183708.00 pJ,0.042 s,0.00 MiB,0.00 MiB,9,0,654000.00 samples/s,0.00 ms,45.89 mW,72573.50 pJ/it -reduce_mean/legacy_axis1_keepdims_1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,8,0,0.00 ms,94.56 mW,129830.00 pJ,0.049 s,0.00 MiB,0.00 MiB,8,0,1340000.00 samples/s,0.00 ms,10.15 mW,7594.50 pJ/it -reduce_mean/legacy_empty_axes_noop,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it -reduce_mean/legacy_nchw_spatial,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.01 mW,498648.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,156000.00 samples/s,0.01 ms,2.01 mW,12796.67 pJ/it -reduce_mean/legacy_negative_axis,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.52 mW,51717.00 pJ,0.041 s,0.00 MiB,0.00 MiB,6,0,1760000.00 samples/s,0.00 ms,8.07 mW,4588.50 pJ/it -reduce_mean/legacy_reduce_all_keepdims_1,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it -reduce_mean/negative_axis,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.52 mW,51717.00 pJ,0.041 s,0.00 MiB,0.00 MiB,6,0,1760000.00 samples/s,0.00 ms,8.07 mW,4588.50 pJ/it -relu/4d,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.18 mW,40734.00 pJ,0.046 s,0.00 MiB,0.00 MiB,1,0,1930000.00 samples/s,0.00 ms,2.18 mW,1014.00 pJ/it -relu/after_conv,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.89 mW,577437.72 pJ,0.048 s,0.00 MiB,0.00 MiB,4,3,191000.00 samples/s,0.01 ms,58.26 mW,304800.72 pJ/it -relu/after_gemm,arch-a,PASS,PASS,0.054 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.16 mW,790056.96 pJ,0.042 s,0.01 MiB,0.01 MiB,6,4,230000.00 samples/s,0.00 ms,109.21 mW,545663.29 pJ/it -relu/basic,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it -reshape/4d_to_2d_flatten,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.28 mW,20196.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,3910000.00 samples/s,0.00 ms,2.28 mW,488.00 pJ/it -reshape/infer_dim_minus_one,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it -reshape/same_rank,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it -reshape/zero_copies_input_dim,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it -resize/height_only,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.55 mW,64833.00 pJ,0.038 s,0.00 MiB,0.00 MiB,4,0,1880000.00 samples/s,0.00 ms,5.60 mW,2986.00 pJ/it -resize/nearest_2x,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.57 mW,109761.00 pJ,0.038 s,0.00 MiB,0.00 MiB,4,0,1450000.00 samples/s,0.00 ms,5.46 mW,3776.00 pJ/it -resize/nearest_downsample,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.45 mW,33925.00 pJ,0.039 s,0.00 MiB,0.00 MiB,2,0,2330000.00 samples/s,0.00 ms,3.28 mW,1360.50 pJ/it -resize/non_uniform_scales,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.58 mW,164037.00 pJ,0.042 s,0.00 MiB,0.00 MiB,6,0,1250000.00 samples/s,0.00 ms,7.76 mW,6207.25 pJ/it -resize/width_only,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.50 mW,53029.00 pJ,0.037 s,0.00 MiB,0.00 MiB,2,0,1700000.00 samples/s,0.00 ms,3.20 mW,1833.50 pJ/it -resize/with_sizes,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,3,0,0.00 ms,92.54 mW,73756.00 pJ,0.039 s,0.00 MiB,0.00 MiB,3,0,1700000.00 samples/s,0.00 ms,4.39 mW,2586.75 pJ/it -sigmoid/4d,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.18 mW,40734.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1930000.00 samples/s,0.00 ms,2.18 mW,1014.00 pJ/it -sigmoid/after_gemm,arch-a,PASS,PASS,0.054 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.16 mW,790056.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,230000.00 samples/s,0.00 ms,109.21 mW,545663.29 pJ/it -sigmoid/basic,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it -slice/2d_basic,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it -slice/after_conv,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.01 MiB,7,6,0.01 ms,118.19 mW,1335082.88 pJ,0.053 s,0.00 MiB,0.01 MiB,7,6,106000.00 samples/s,0.01 ms,73.96 mW,732218.55 pJ/it -slice/default_axes,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it -slice/large_channel_1024,arch-a,PASS,PASS,0.049 s,0.01 MiB,0.00 MiB,1,0,0.00 ms,78.14 mW,221304.00 pJ,0.036 s,0.01 MiB,0.00 MiB,1,0,353000.00 samples/s,0.00 ms,2.14 mW,5058.00 pJ/it -slice/nchw_spatial_crop,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.24 mW,101868.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,769000.00 samples/s,0.00 ms,2.24 mW,2851.67 pJ/it -slice/negative_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,44004.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,1790000.00 samples/s,0.00 ms,2.30 mW,1227.67 pJ/it -slice/negative_indices,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,25212.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.30 mW,675.67 pJ/it -slice/step2,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.29 mW,159876.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,490000.00 samples/s,0.00 ms,2.29 mW,4619.67 pJ/it -softmax/3d_last_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.035 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED -softmax/basic,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED -softmax/channel_axis,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.040 s,0.00 MiB,0.00 MiB,3,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED -softmax/large_dimension_1024,arch-a,PASS,PASS,0.049 s,0.01 MiB,0.01 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.01 MiB,0.01 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED -softmax/negative_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED -split/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,31554.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,2490000.00 samples/s,0.00 ms,2.30 mW,861.67 pJ/it -split/equal_three_way,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,44160.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1780000.00 samples/s,0.00 ms,2.30 mW,1231.67 pJ/it -split/negative_axis,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.29 mW,84786.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,925000.00 samples/s,0.00 ms,2.29 mW,2413.67 pJ/it -split/uneven_channel_axis_4d,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.046 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it -sub/after_gemm,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it -sub/basic,arch-a,PASS,PASS,0.046 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -sub/broadcast_row,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it -sub/channel_broadcast_1024,arch-a,PASS,PASS,0.052 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.037 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it -sub/constant_lhs_broadcast,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25188.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,656.67 pJ/it -sub/leading_dimension_broadcast,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it diff --git a/validation/raptor_validation/artifacts.py b/validation/raptor_validation/artifacts.py new file mode 100644 index 0000000..0b53a42 --- /dev/null +++ b/validation/raptor_validation/artifacts.py @@ -0,0 +1,33 @@ +from pathlib import Path + + +ARTIFACTS_DIRNAME = "artifacts" + + +def artifacts_dir(workspace_dir: str | Path) -> Path: + return Path(workspace_dir) / ARTIFACTS_DIRNAME + + +def runner_uses_library(runner_path: str | Path, library_path: str | Path) -> bool: + runner_path = Path(runner_path) + library_path = Path(library_path) + try: + return ( + runner_path.is_file() + and library_path.is_file() + and str(library_path.resolve()).encode() in runner_path.read_bytes() + ) + except OSError: + return False + + +def remove_lock_files(root: str | Path) -> int: + root = Path(root) + if not root.exists(): + return 0 + removed = 0 + for path in root.rglob("*.lock"): + if path.is_file() or path.is_symlink(): + path.unlink(missing_ok=True) + removed += 1 + return removed diff --git a/validation/raptor_validation/pimcomp_models.py b/validation/raptor_validation/pimcomp_models.py new file mode 100644 index 0000000..8dc7853 --- /dev/null +++ b/validation/raptor_validation/pimcomp_models.py @@ -0,0 +1,39 @@ +from __future__ import annotations + +import argparse +from pathlib import Path + + +REPO = Path(__file__).resolve().parents[2] +SUITE = REPO / "validation/networks/pimcomp_models" +MODELS = { + "vgg8": SUITE / "vgg8/vgg8-mnist-reconstructed.onnx", + "resnet18": SUITE / "resnet18/resnet18-v1-7.onnx", + "resnet34": SUITE / "resnet34/resnet34-v1-7.onnx", + "googlenet": SUITE / "googlenet/googlenet-12-pimsim-nn.onnx", + "yolo11n": SUITE / "yolo11n/yolo11n-pimsim-nn.onnx", +} +FUNCTIONAL_MODELS = { + **MODELS, + "yolo11n": REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx", +} +MODEL_NAMES = tuple(MODELS) +DEFAULT_MODELS = MODEL_NAMES +ABLATION_DEFAULT_MODELS = ("vgg8", "resnet18", "resnet34", "googlenet") + + +def add_models_argument( + parser: argparse.ArgumentParser, + default: tuple[str, ...] = DEFAULT_MODELS, +) -> None: + help_text = "Models to run (default: " + ", ".join(default) + ")." + if "yolo11n" not in default: + help_text += " Select yolo11n explicitly when needed." + parser.add_argument( + "--models", + nargs="+", + choices=MODEL_NAMES, + default=list(default), + metavar="MODEL", + help=help_text, + ) diff --git a/validation/raptor_validation/raptor.py b/validation/raptor_validation/raptor.py index f23a549..8c8e724 100644 --- a/validation/raptor_validation/raptor.py +++ b/validation/raptor_validation/raptor.py @@ -8,13 +8,13 @@ from .subprocess_utils import run_command_with_reporter PIM_PASS_LABELS = ( ("ONNXToSpatialPass", "ONNX to Spatial"), - ("MergeComputeNodesPass", "Merge Compute Nodes"), - ("SpatialToPimPass", "Spatial to PIM"), - ("PimBufferizationPass", "Bufferize PIM"), - ("HostConstantFoldingPass", "Fold Host Constants"), - ("PimLocalMemoryPlanningPass", "Plan Local Memory"), - ("VerificationPass", "Verify PIM"), - ("EmitPimCodePass", "Emit PIM Code"), + ("MergeComputeNodesPass", "Merge compute nodes"), + ("SpatialToPimPass", "Spatial to Pim"), + ("PimBufferizationPass", "Bufferize Pim"), + ("HostConstantFoldingPass", "Fold host constants"), + ("PimLocalMemoryPlanningPass", "Plan local memory"), + ("VerificationPass", "Verify Pim"), + ("EmitPimCodePass", "Emit Pim code"), ) PIM_PASS_LABEL_BY_SUFFIX = dict(PIM_PASS_LABELS) TIMING_LINE_RE = re.compile(r"^\s*([0-9]+\.[0-9]+)\s+\(\s*[0-9.]+%\)\s+(.+?)\s*$") diff --git a/validation/raptor_validation/validate_one.py b/validation/raptor_validation/validate_one.py index 3aa09c6..5558a52 100644 --- a/validation/raptor_validation/validate_one.py +++ b/validation/raptor_validation/validate_one.py @@ -9,6 +9,7 @@ import numpy as np from dataclasses import dataclass, field from pathlib import Path from colorama import Style, Fore +from .artifacts import artifacts_dir, runner_uses_library from .gen_network_runner import gen_network_runner from .onnx_utils import ( _ONNX_TO_NP, @@ -26,13 +27,13 @@ from .subprocess_utils import run_command_with_reporter STAGE_TITLES = ( "Compile ONNX", - "Build Runner", - "Generate Inputs", - "Run Reference", - "Compile PIM", - "Run Functional Simulation", - "Compare Outputs", - "Run Non-functional Simulation", + "Build runner", + "Generate inputs", + "Run reference", + "Compile Pim", + "Run functional simulation", + "Compare outputs", + "Run non-functional simulation", ) STAGE_COLORS = { STAGE_TITLES[0]: Fore.BLUE, @@ -46,8 +47,7 @@ STAGE_COLORS = { } STAGE_COUNT = len(STAGE_TITLES) GENERATED_DIR_NAMES = ( - "inputs", "outputs", "pimcomp", "raptor", "runner", "simulation", - "throughput_validation", + "artifacts", ) MODE_FULL = "full" @@ -58,15 +58,15 @@ MODE_STAGE_TITLES = { MODE_FULL: STAGE_TITLES, MODE_COMPILE_ONLY: ( "Compile ONNX", - "Build Runner", - "Compile PIM", + "Build runner", + "Compile Pim", ), MODE_RUN_ONLY: ( - "Generate Inputs", - "Run Reference", - "Run Functional Simulation", - "Compare Outputs", - "Run Non-functional Simulation", + "Generate inputs", + "Run reference", + "Run functional simulation", + "Compare outputs", + "Run non-functional simulation", ), } @@ -75,7 +75,7 @@ PIMSIM_FAILED = "ERROR" PIMSIM_UNSUPPORTED = "UNSUPPORTED" PIMSIM_SKIPPED = "SKIP" PIMSIM_NOT_RUN = "-" -PIMSIM_UNSUPPORTED_VSOFTMAX = "pimsim-nn does not support opcode vsoftmax" +PIMSIM_UNSUPPORTED_VSOFTMAX = "Pimsim does not support opcode vsoftmax" class PimSimUnsupportedError(RuntimeError): @@ -338,7 +338,7 @@ def run_pimsim_nn( else ("throughput", "average_latency_ms", "average_power_mw", "average_energy_pj") ) if any(name not in metrics for name in required): - raise RuntimeError(f"pimsim-nn output did not contain required {execution_mode} metrics") + raise RuntimeError(f"Pimsim output did not contain required {execution_mode} metrics") return metrics @@ -361,6 +361,11 @@ def clean_workspace_artifacts(workspace_dir, model_stem): for suffix in (".onnx.mlir", ".so", ".tmp"): remove_path(workspace_dir / f"{model_stem}{suffix}") + for path in workspace_dir.rglob("*.lock"): + if path.is_file() or path.is_symlink(): + path.unlink(missing_ok=True) + removed_paths.append(path) + return removed_paths @@ -407,32 +412,34 @@ def build_dump_ranges(config_path, outputs_descriptor): def build_pim_simulator_command( - pim_dir, output_bin_path, dump_ranges, input_paths, mode="latency", + pim_dir, output_bin_path, dump_ranges, input_dir, batch_size, mode="latency", batch_output_dir=None): if mode not in ("latency", "throughput"): raise ValueError(f"unknown simulator mode: {mode}") - if not input_paths: + if batch_size < 1: raise ValueError("simulator requires at least one input") + if input_dir is None: + raise ValueError("simulator requires an input directory") command = [ "cargo", "run", "--no-default-features", "--release", "--package", "pim-simulator", "--bin", "pim-simulator", "--", "-f", str(pim_dir), "-o", str(output_bin_path), "-d", dump_ranges, - "--mode", mode, "--batch-size", str(len(input_paths)), + "--mode", mode, "--batch-size", str(batch_size), + "--input-dir", str(input_dir), ] if batch_output_dir is not None: command += ["--batch-output-dir", str(batch_output_dir)] - for path in input_paths: - command += ["--input", str(path)] return command def run_pim_simulator( simulator_dir, pim_dir, output_bin_path, dump_ranges, reporter=None, - timeout_sec=None, input_paths=(), mode="latency", batch_output_dir=None): + timeout_sec=None, input_dir=None, batch_size=1, mode="latency", batch_output_dir=None): command = build_pim_simulator_command( pim_dir, output_bin_path, dump_ranges, - input_paths, + input_dir, + batch_size, mode=mode, batch_output_dir=batch_output_dir, ) @@ -516,7 +523,7 @@ def validate_execution( try: print_stage( reporter, model_index, model_total, model_name, - "Run Functional Simulation", name, + "Run functional simulation", name, ) write_inputs_to_memory_bin( pim_dir / "memory.bin", pim_dir / "config.json", input_batch[0]) @@ -526,13 +533,13 @@ def validate_execution( run_pim_simulator( simulator_dir, pim_dir, simulation_dir / "out.bin", dump_ranges, reporter=reporter, timeout_sec=command_timeout_seconds, - input_paths=input_paths[:batch_size], mode=name, + input_dir=input_paths[0].parent, batch_size=batch_size, mode=name, batch_output_dir=output_dir) reporter.advance() print_stage( reporter, model_index, model_total, model_name, - "Compare Outputs", name, + "Compare outputs", name, ) reporter.suspend() try: @@ -553,7 +560,7 @@ def validate_execution( print_stage( reporter, model_index, model_total, model_name, - "Run Non-functional Simulation", name, + "Run non-functional simulation", name, ) config_path = execution["pimsim_config"] if state["compiled"] and pimsim_nn_build_dir is not None and config_path is not None: @@ -580,7 +587,7 @@ def validate_execution( elif not state["compiled"]: state["pimsim_status"] = PIMSIM_NOT_RUN else: - print_info(reporter, "pimsim-nn non-functional simulation skipped") + print_info(reporter, "Pimsim non-functional simulation skipped") reporter.advance() @@ -609,12 +616,12 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, owns_reporter = reporter is None reporter = reporter or ProgressReporter(model_total, stages_per_model=len(MODE_STAGE_TITLES[mode]), verbose=verbose) - workspace_dir = network_onnx_path.parent + workspace_dir = artifacts_dir(network_onnx_path.parent) raptor_dir = workspace_dir / "raptor" runner_dir = workspace_dir / "runner" runner_build_dir = runner_dir / "build" if mode != MODE_RUN_ONLY: - clean_workspace_artifacts(workspace_dir, network_onnx_path.stem) + clean_workspace_artifacts(network_onnx_path.parent, network_onnx_path.stem) Path.mkdir(raptor_dir, parents=True, exist_ok=True) Path.mkdir(runner_build_dir, parents=True, exist_ok=True) @@ -669,7 +676,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, print_info(reporter, f"Shared library saved to {network_so_path}") reporter.advance() - print_stage(reporter, model_index, model_total, network_onnx_path.name, "Build Runner") + print_stage(reporter, model_index, model_total, network_onnx_path.name, "Build runner") gen_network_runner( network_onnx_path, network_so_path, onnx_include_dir, entry="run_main_graph", out=runner_dir / "runner.c", verbose=False) @@ -683,7 +690,10 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, report_validation_failure(reporter, "reference", "compilation", exc) else: required_paths = (network_so_path, network_mlir_path, runner_path) - reference_ready = all(path.exists() for path in required_paths) + reference_ready = ( + all(path.exists() for path in required_paths) + and runner_uses_library(runner_path, network_so_path) + ) if not reference_ready: report_validation_failure(reporter, "reference", "artifact lookup", FileNotFoundError( "run-only mode requires the compiled shared library, ONNX MLIR, and runner")) @@ -696,7 +706,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, states[name]["compiled"] = (pim_dir / "config.json").exists() if not states[name]["compiled"]: report_validation_failure(reporter, name, "artifact lookup", FileNotFoundError( - f"run-only mode requires compiled PIM artifacts at {pim_dir}")) + f"run-only mode requires compiled Pim artifacts at {pim_dir}")) else: states[name]["resource_metrics"] = collect_pim_resource_metrics(pim_dir) if name == "latency": @@ -705,7 +715,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, try: print_stage( reporter, model_index, model_total, network_onnx_path.name, - "Compile PIM", name, + "Compile Pim", name, ) root.mkdir(parents=True, exist_ok=True) started = time.perf_counter() @@ -724,7 +734,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, states[name]["resource_metrics"] = collect_pim_resource_metrics(pim_dir) if name == "latency": resource_metrics = states[name]["resource_metrics"] - print_info(reporter, f"PIM artifacts saved to {pim_dir}") + print_info(reporter, f"Pim artifacts saved to {pim_dir}") except Exception as exc: report_validation_failure(reporter, name, "compilation", exc) reporter.advance() @@ -735,7 +745,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, else: input_batch = input_paths = reference_dirs = outputs_descriptor = None try: - print_stage(reporter, model_index, model_total, network_onnx_path.name, "Generate Inputs") + print_stage(reporter, model_index, model_total, network_onnx_path.name, "Generate inputs") inputs_descriptor, outputs_descriptor = onnx_io(network_onnx_path) first_inputs, _ = gen_random_inputs(inputs_descriptor, seed=seed) input_batch = generate_input_batch( @@ -754,7 +764,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir, if not reference_ready: raise FileNotFoundError("reference runner is unavailable") - print_stage(reporter, model_index, model_total, network_onnx_path.name, "Run Reference") + print_stage(reporter, model_index, model_total, network_onnx_path.name, "Run reference") reference_dirs = [] for index, flags in enumerate(input_flags): reference_dir = workspace_dir / "outputs" / f"{index:06d}" diff --git a/validation/tools/classification_local_image_validation.py b/validation/tools/classification_local_image_validation.py index c7e84e7..f2228f8 100644 --- a/validation/tools/classification_local_image_validation.py +++ b/validation/tools/classification_local_image_validation.py @@ -16,6 +16,7 @@ if str(VALIDATION_DIR) not in sys.path: sys.path.insert(0, str(VALIDATION_DIR)) from raptor_validation.onnx_utils import _ONNX_TO_NP, onnx_io, write_inputs_binary, write_inputs_to_memory_bin +from raptor_validation.artifacts import artifacts_dir from raptor_validation.validate_one import ( MODE_COMPILE_ONLY, build_dump_ranges, @@ -75,10 +76,11 @@ def ensure_local_artifacts(args, model_path: Path): def ensure_existing_artifacts(model_dir: Path): + artifact_root = artifacts_dir(model_dir) required_paths = [ - model_dir / "runner" / "build" / "runner", - model_dir / "raptor" / "pim" / "config.json", - model_dir / "raptor" / "pim" / "memory.bin", + artifact_root / "runner" / "build" / "runner", + artifact_root / "raptor" / "pim" / "config.json", + artifact_root / "raptor" / "pim" / "memory.bin", ] missing = [str(path) for path in required_paths if not path.exists()] if missing: @@ -185,13 +187,13 @@ def draw_classification_panel(image: Image.Image, results, output_path: Path): def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray): - model_dir = model_path.parent - runner_build_dir = model_dir / "runner" / "build" + artifact_root = artifacts_dir(model_path.parent) + runner_build_dir = artifact_root / "runner" / "build" runner_path = runner_build_dir / "runner" - pim_dir = model_dir / "raptor" / "pim" - simulation_dir = model_dir / "classification_demo" / "simulation" - reference_dir = model_dir / "classification_demo" / "reference" - inputs_dir = model_dir / "classification_demo" / "inputs" + pim_dir = artifact_root / "raptor" / "pim" + simulation_dir = artifact_root / "classification_demo" / "simulation" + reference_dir = artifact_root / "classification_demo" / "reference" + inputs_dir = artifact_root / "classification_demo" / "inputs" simulation_dir.mkdir(parents=True, exist_ok=True) reference_dir.mkdir(parents=True, exist_ok=True) @@ -222,7 +224,7 @@ def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray): subprocess.run(runner_cmd, cwd=runner_build_dir, check=True) write_inputs_to_memory_bin(pim_dir / "memory.bin", pim_dir / "config.json", [tensor]) - input_bin_path = simulation_dir / "input.bin" + input_bin_path = simulation_dir / "input_0.bin" write_inputs_binary(input_bin_path, [tensor]) dump_ranges = build_dump_ranges(pim_dir / "config.json", output_descriptors) output_bin_path = simulation_dir / "out.bin" @@ -232,7 +234,8 @@ def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray): output_bin_path, dump_ranges, timeout_sec=args.command_timeout_seconds, - input_paths=[input_bin_path], + input_dir=input_bin_path.parent, + batch_size=1, ) output_index, output_name, output_dtype_code, output_shape = output_descriptors[0] diff --git a/validation/tools/conv_lowering_tiles_drawio.py b/validation/tools/conv_lowering_tiles_drawio.py index bdfe437..12dcd14 100644 --- a/validation/tools/conv_lowering_tiles_drawio.py +++ b/validation/tools/conv_lowering_tiles_drawio.py @@ -37,55 +37,55 @@ class Tile: TILES = ( - Tile("classic-reference", "REFERENCE", "Classic Conv + exact weight unfolding", + Tile("classic-reference", "Reference", "Classic Conv + exact weight unfolding", "Original OIHW weights; the two implementations choose different K orders.", "reference", "Y[p,o] = Σc,kh,kw Xpatch[p,c,kh,kw] · W[o,c,kh,kw]", "At every output position, multiply the patch by one filter and add every product."), - Tile("pimcomp-element", "PIMCOMP", "Element pipeline", + Tile("pimcomp-element", "Pimcomp", "Element pipeline", "One patch vector per input cycle; mapped weights stay fixed.", "pimcomp_element", "patchPIM[1×K] · WflatPIM[K×O] → Yp[1×O]", "Keep Wflat in the arrays; stream one patch each cycle to produce all O outputs."), - Tile("pimcomp-batch", "PIMCOMP", "Batch / replicated pipeline", + Tile("pimcomp-batch", "Pimcomp", "Batch / replicated pipeline", "Complete Wflat copies divide patches or input samples.", "pimcomp_batch", "for replica r: Yr = patchr[1×K] · WflatPIM[K×O]", "Copy all weights R times and send different patches to the copies in parallel."), - Tile("raptor-legacy-im2col", "RAPTOR", "Legacy explicit im2col", + Tile("raptor-legacy-im2col", "Raptor", "Legacy explicit im2col", "Every patch becomes one row of a global P×K matrix.", "legacy", "Y[P×O] = im2col(X)[P×K] · WflatR[K×O]", "Write every image patch as one matrix row, then multiply the two large matrices."), - Tile("raptor-packed-im2col", "RAPTOR", "Packed im2col", + Tile("raptor-packed-im2col", "Raptor", "Packed im2col", "Pack q patch rows and repeat Wflat on a block diagonal.", "packed", "packedY[1×qO] = [patch0|…|patchq−1] · diag(WflatR,…,WflatR)", "Join q patches and use diagonal weight copies so one multiply computes q independent outputs."), - Tile("raptor-streamed-patch", "RAPTOR", "Streamed patch", + Tile("raptor-streamed-patch", "Raptor", "Streamed patch", "Gather one patch into bounded scratch; avoid global im2col.", "streamed_patch", "Yp[1×O] = scratchPatchp[1×K] · WflatR[K×O]", "Gather one patch, multiply it, write its output, and reuse scratch for the next patch."), - Tile("raptor-streamed-packed", "RAPTOR", "Streamed packed", + Tile("raptor-streamed-packed", "Raptor", "Streamed packed", "Gather q patch rows in bounded scratch, then block-diagonal pack them.", "streamed_packed", "packedY = packedScratch[1×qK] · diag(WflatR×q)[qK×qO]", "Gather q patches in small scratch, join them, multiply by diagonal weights, then unpack q outputs."), - Tile("raptor-depthwise", "RAPTOR", "Depthwise special case", + Tile("raptor-depthwise", "Raptor", "Depthwise special case", "Each channel owns one row-major 3×3 kernel; channels never reduce together.", "depthwise", "Y[p,c] = Σkh,kw Xpatch[p,c,kh,kw] · W[c,kh,kw]", "For each channel separately, multiply its nine patch values by its nine weights and add."), - Tile("raptor-output-channel-tiled", "RAPTOR", "Output-channel tiled", + Tile("raptor-output-channel-tiled", "Raptor", "Output-channel tiled", "Every O tile retains all channel-major K rows and selects output columns.", "c_tiled", "Y[:,Oj] = patch[1×K] · Wflat[:,Oj][K×|Oj|]; concat j", "Reuse the full patch for each output-filter group, then join the output groups."), - Tile("raptor-input-k-tiled", "RAPTOR", "Input-K tiled", + Tile("raptor-input-k-tiled", "Raptor", "Input-K tiled", "Split matching K ranges; add their partial output vectors.", "k_tiled", "Y[1×O] = Σi patch[Ki] · Wflat[Ki,:]", "Multiply matching K slices independently, then add their partial output vectors."), - Tile("raptor-tiled-2d", "RAPTOR", "Two-dimensional tiled", + Tile("raptor-tiled-2d", "Raptor", "Two-dimensional tiled", "Partition both K rows and output-filter columns.", "tiled_2d", "Y[:,Oj] = Σi patch[Ki] · Wflat[Ki,Oj]; concat j", "Split both directions: add results down K and join results across output groups."), - Tile("raptor-row-strip", "RAPTOR", "Pixel-major row-strip", + Tile("raptor-row-strip", "Raptor", "Pixel-major row-strip", "A lane forms patches across one output row and slices K.", "row_strip", "for x: Y[r,x,:] = Σi patch[r,x,Ki] · Wflat[Ki,:]", "Move across one output row; at each x form a patch, multiply its K slices, and add."), - Tile("raptor-row-strip-c-tiled", "RAPTOR", "Row-strip + output tiling", + Tile("raptor-row-strip-c-tiled", "Raptor", "Row-strip + output tiling", "Each row lane is duplicated across disjoint output-column tiles.", "row_strip_c", "for x,j: Y[r,x,Oj] = patch[r,x,:] · Wflat[:,Oj]", "Give each output-filter group a copy of the row lane, then join their output columns."), @@ -473,7 +473,7 @@ def operation_scene(d: Drawio, parent: str, scene: str) -> None: def reference_body(d: Drawio, parent: str) -> None: add_box(d, parent, 24, 252, 712, 158, "", fill="#ffffff", stroke=PIMCOMP) add_text(d, parent, 40, 260, 680, 22, - "PIMCOMP: spatial-major, row-wise positions; C interleaved", 11, + "Pimcomp: Spatial-major, row-wise positions; C interleaved", 11, color=PIMCOMP, align="center", bold=True) order_vector(d, parent, 306, "pimcomp", weight=False, label="Input patch") order_vector(d, parent, 366, "pimcomp", weight=True, label="Matching W") @@ -483,7 +483,7 @@ def reference_body(d: Drawio, parent: str) -> None: add_box(d, parent, 24, 424, 712, 158, "", fill="#ffffff", stroke=RAPTOR) add_text(d, parent, 40, 432, 680, 22, - "RAPTOR: channel-major; each 3×3 plane is row-major", 11, + "Raptor: channel-major; each 3×3 plane is row-major", 11, color=RAPTOR, align="center", bold=True) order_vector(d, parent, 478, "raptor", weight=False, label="Input patch") order_vector(d, parent, 538, "raptor", weight=True, label="Matching W") @@ -500,22 +500,22 @@ def reference_body(d: Drawio, parent: str) -> None: def layout_reference(d: Drawio, parent: str, tile: Tile, accent: str) -> None: if tile.scene == "depthwise": layout = "independent row-major Kc=9 per channel" - elif tile.owner == "PIMCOMP": - layout = "PIMCOMP spatial-major K order" + elif tile.owner == "Pimcomp": + layout = "Pimcomp Spatial-major K order" else: - layout = "RAPTOR channel-major K order" + layout = "Raptor channel-major K order" add_box(d, parent, 24, 140, 712, 42, "", fill=PALE, stroke=accent) add_text(d, parent, 38, 146, 684, 30, - f"LAYOUT → see REFERENCE tile: {layout}", 10, + f"Layout → see Reference tile: {layout}", 10, color=accent, align="center", bold=True) def algorithm_card(d: Drawio, parent: str, tile: Tile, accent: str) -> None: add_box(d, parent, 24, 638, 712, 102, "", fill="#ffffff", stroke=accent) - add_text(d, parent, 40, 646, 90, 34, "ALGORITHM", 9, + add_text(d, parent, 40, 646, 90, 34, "Algorithm", 9, color=accent, bold=True) add_text(d, parent, 132, 644, 588, 38, tile.algorithm, 10) - add_text(d, parent, 40, 690, 90, 34, "MATH", 9, + add_text(d, parent, 40, 690, 90, 34, "Math", 9, color=accent, bold=True) add_text(d, parent, 132, 686, 588, 42, tile.formula, 10, color=accent, bold=True) @@ -524,7 +524,7 @@ def algorithm_card(d: Drawio, parent: str, tile: Tile, accent: str) -> None: def render_tile(d: Drawio, tile: Tile, index: int) -> None: col, row = index % COLS, index // COLS parent = d.group(col * (TILE + GAP), row * (TILE + GAP), tile.slug) - accent = {"REFERENCE": REFERENCE, "PIMCOMP": PIMCOMP, "RAPTOR": RAPTOR}[tile.owner] + accent = {"Reference": REFERENCE, "Pimcomp": PIMCOMP, "Raptor": RAPTOR}[tile.owner] add_box(d, parent, 0, 0, TILE, TILE, "", fill="#fbfcff", stroke=accent, stroke_width=3) add_box(d, parent, 24, 20, 106, 28, tile.owner, fill=accent, stroke=accent, diff --git a/validation/tools/pim/ablation/README.md b/validation/tools/pim/ablation/README.md new file mode 100644 index 0000000..f9fe2aa --- /dev/null +++ b/validation/tools/pim/ablation/README.md @@ -0,0 +1,66 @@ +# Raptor compiler ablation + +`run_ablation.py` performs the complete synchronization/Spatial-planning +ablation study on the Pimcomp model suite. By default it runs `vgg8`, +`resnet18`, `resnet34`, and `googlenet` across `arch-a` and `arch-b`, latency, +and throughput pipeline 4. `arch-c` and `yolo11n` are run only when selected +explicitly. +Latency is pipeline 1; the wrapper invokes the suite runner separately for +latency and throughput pipeline 4. + +```bash +.venv/bin/python validation/tools/pim/ablation/run_ablation.py \ + --jobs 4 +``` + +## Variants + +| Variant | Raptor options | +|---|---| +| `baseline` | None. | +| `no-sync` | `--pim-disable-synchronization` | +| `no-plan` | `--pim-disable-spatial-planning` | +| `no-sync-no-plan` | Both options. | + +Every variant runs Raptor only. Pimcomp is not compiled, validated, or +simulated. Reference inputs and outputs are generated once under the shared +common-artifact root and reused by every variant. Ctrl+C terminates the active +variant and all of its worker jobs. + +The percentage baseline is `baseline`, so its available values are `+0.00%`. +Every other +variant reports the signed percentage difference of its Raptor metrics from +that reference for the same model, architecture, mode, and pipeline. Positive +values mean the metric is higher; negative values mean it is lower. + +## Options + +| Option | Description and default | +|---|---| +| `-h`, `--help` | Show help and exit. | +| `--out-dir PATH` | Suite root. Default: `validation/networks/pimcomp_models`. Model artifacts are stored below `//artifacts`; disabled variants use `/[/pipelineN]/ablation/`. Variant summaries remain under `/ablation//`. | +| `--models MODEL [...]` | Models to run. Default: `vgg8 resnet18 resnet34 googlenet`; include `yolo11n` explicitly when needed. | +| `--variant NAME` | Select a variant from the table above; repeat for multiple variants. Default: all variants. The feature-full baseline and percentage reference are added automatically when needed. | +| `--dry-run` | Print the suite-runner commands that would run without modifying files. Default: off. | + +All options of +[`run_pimcomp_models.py`](../pimcomp/compare/README.md), including +`--archs`, `--mode`, `--pipeline`, `--no-fast`, and +`--raptor-extra-arg=ARG`, are forwarded to each variant. `--out-dir`, +`--variant`, and `--dry-run` belong to this wrapper; `--only` is reserved for +the suite runner's comparison mode and is rejected by this wrapper, which +always invokes `--raptor-only`. If neither `--mode` nor `--pipeline` is +forwarded, the wrapper uses its default latency and throughput/pipeline-4 +case set. Supplying either option overrides that default and is passed through +as one suite-runner invocation per variant. + +The feature-full `baseline` variant uses the normal `run_pimcomp_models.py` +artifact paths and is rerun with the same selected cases and forwarded options +as the disabled variants. The three disabled variants are stored below each model's +`artifacts//[/pipelineN]/ablation/` directory. Shared reference +artifacts remain under each model's `artifacts/common` directory. Transient +per-variant comparison summaries are removed after aggregation. The combined +table is written to `/results_ablation.csv`; it contains only the variant, case +identifiers, and signed percentage `latency`, `throughput`, `power`, and +`energy` columns. These are Raptor metrics; +Pimcomp metrics are omitted because the ablation invokes Raptor only. diff --git a/validation/tools/pim/ablation/run_ablation.py b/validation/tools/pim/ablation/run_ablation.py new file mode 100755 index 0000000..71ecf55 --- /dev/null +++ b/validation/tools/pim/ablation/run_ablation.py @@ -0,0 +1,327 @@ +#!/usr/bin/env python3 +"""Run the synchronization and Spatial-planning ablation matrix on Pimcomp models.""" + +from __future__ import annotations + +import argparse +import csv +import math +import os +import shlex +import signal +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + + +REPO = Path(__file__).resolve().parents[4] +RUNNER = REPO / "validation/tools/pim/pimcomp/compare/run_pimcomp_models.py" +DEFAULT_OUT_DIR = REPO / "validation/networks/pimcomp_models" +sys.path.insert(0, str(REPO / "validation")) + +from raptor_validation.pimcomp_models import ( + ABLATION_DEFAULT_MODELS, + add_models_argument, +) +from raptor_validation.artifacts import remove_lock_files + + +@dataclass(frozen=True) +class Variant: + name: str + raptor_args: tuple[str, ...] + + +VARIANTS = ( + Variant("baseline", ()), + Variant("no-sync", ("--pim-disable-synchronization",)), + Variant("no-plan", ("--pim-disable-spatial-planning",)), + Variant( + "no-sync-no-plan", + ("--pim-disable-synchronization", "--pim-disable-spatial-planning"), + ), +) +VARIANT_BY_NAME = {variant.name: variant for variant in VARIANTS} +COMPARISON_RESULTS_FILENAME = "results_comparison.csv" +ABLATION_RESULTS_FILENAME = "results_ablation.csv" +CASE_FIELDS = ("arch", "model", "mode", "raptor_pipeline") +PERCENTAGE_FIELDS = ( + ("raptor_latency_ms", "latency"), + ("raptor_throughput_samples_s", "throughput"), + ("raptor_power_mw", "power"), + ("raptor_energy_pj", "energy"), +) +RESULT_FIELDS = (*CASE_FIELDS, *(target for _, target in PERCENTAGE_FIELDS)) +DEFAULT_CASE_ARGUMENTS = ( + ("latency", ("--mode", "latency")), + ("throughput/pipeline4", ("--mode", "throughput", "--pipeline", "4")), +) +DEFAULT_CASE_KEYS = frozenset({("latency", "1"), ("throughput", "4")}) + + +def parse_args(argv: list[str] | None = None) -> tuple[argparse.Namespace, list[str]]: + parser = argparse.ArgumentParser( + description="Run the complete compiler ablation matrix on the Pimcomp model suite.", + allow_abbrev=False, + ) + parser.add_argument( + "--out-dir", + type=Path, + default=DEFAULT_OUT_DIR, + help="Artifact root (default: validation/networks/pimcomp_models).", + ) + add_models_argument(parser, ABLATION_DEFAULT_MODELS) + parser.add_argument( + "--variant", + choices=tuple(VARIANT_BY_NAME), + action="append", + dest="variants", + help="Run only this variant; baseline is added when needed. Repeat as needed.", + ) + parser.add_argument("--dry-run", action="store_true", help="Print runner commands without modifying files.") + args, forwarded = parser.parse_known_args(argv) + if any(option == "--only" or option.startswith("--only=") for option in forwarded): + parser.error("--only is not supported; the ablation wrapper always runs Raptor only") + return args, forwarded + + +def selected_variants(names: list[str] | None) -> list[Variant]: + requested = set(names or VARIANT_BY_NAME) + requested.add("baseline") + return [variant for variant in VARIANTS if variant.name in requested] + + +def has_option(arguments: list[str], option: str) -> bool: + return any(argument == option or argument.startswith(f"{option}=") for argument in arguments) + + +def runner_argument_sets(forwarded: list[str]) -> tuple[tuple[str, list[str]], ...]: + if has_option(forwarded, "--mode") or has_option(forwarded, "--pipeline"): + return (("requested", forwarded),) + return tuple( + (label, [*forwarded, *case_arguments]) + for label, case_arguments in DEFAULT_CASE_ARGUMENTS + ) + + +def variant_output_dir(out_dir: Path, variant: Variant) -> Path: + return out_dir if variant.name == "baseline" else out_dir / "ablation" / variant.name + + +def remove_variant_summaries(out_dir: Path) -> None: + summary_root = out_dir / "ablation" + if not summary_root.is_dir(): + return + for summary in summary_root.glob("*/results_comparison.csv"): + summary.unlink() + for variant_dir in summary_root.iterdir(): + if variant_dir.is_dir() and not any(variant_dir.iterdir()): + variant_dir.rmdir() + if not any(summary_root.iterdir()): + summary_root.rmdir() + + +def runner_command( + out_dir: Path, + variant: Variant, + models: list[str], + forwarded: list[str], + common_root: Path, + dry_run: bool, +) -> list[str]: + command = [ + sys.executable, + str(RUNNER), + "--out-dir", + str(out_dir), + "--raptor-only", + "--models", + *models, + ] + if variant.name != "baseline": + command.extend(("--ablation-variant", variant.name)) + if not has_option(forwarded, "--common-dir"): + command.extend(("--common-dir", str(common_root))) + command.extend(forwarded) + command.extend(f"--raptor-extra-arg={arg}" for arg in variant.raptor_args) + if dry_run: + command.append("--dry-run") + return command + + +def terminate_process_group(process: subprocess.Popen[bytes]) -> None: + if process.poll() is not None: + return + try: + os.killpg(process.pid, signal.SIGTERM) + except ProcessLookupError: + return + try: + process.wait(timeout=5) + except subprocess.TimeoutExpired: + try: + os.killpg(process.pid, signal.SIGKILL) + except ProcessLookupError: + return + process.wait() + + +def percentage_difference(value: str | None, reference: str | None) -> str: + try: + current = float(value) if value is not None else math.nan + baseline = float(reference) if reference is not None else math.nan + except ValueError: + return "NA" + if not math.isfinite(current) or not math.isfinite(baseline) or baseline == 0: + return "NA" + return f"{(current - baseline) / baseline * 100:+.2f}%" + + +def aggregate_results( + out_dir: Path, + variants: list[Variant], + selected_cases: frozenset[tuple[str, str]] | None = None, + selected_models: frozenset[str] | None = None, +) -> tuple[Path | None, list[str]]: + fields: list[str] | None = None + rows_by_variant: dict[str, list[dict[str, str]]] = {} + failures = [] + for variant in variants: + results_path = variant_output_dir(out_dir, variant) / COMPARISON_RESULTS_FILENAME + if not results_path.is_file(): + failures.append(f"{variant.name}: missing {results_path}") + continue + with results_path.open(newline="", encoding="utf-8") as stream: + reader = csv.DictReader(stream) + if reader.fieldnames is None: + failures.append(f"{variant.name}: empty {results_path}") + continue + if fields is None: + fields = reader.fieldnames + elif reader.fieldnames != fields: + failures.append(f"{variant.name}: inconsistent columns in {results_path}") + continue + selected_rows = rows_by_variant.setdefault(variant.name, []) + for row in reader: + if selected_models is not None and row.get("model") not in selected_models: + continue + if selected_cases is not None and ( + row.get("mode"), row.get("raptor_pipeline") + ) not in selected_cases: + continue + selected_rows.append(row) + if fields is None: + return None, failures + baseline_rows = { + tuple(row.get(field, "") for field in CASE_FIELDS): row + for row in rows_by_variant.get("baseline", []) + } + if "baseline" not in rows_by_variant: + failures.append("baseline: missing percentage reference results") + result_rows = [] + variant_order = {variant.name: index for index, variant in enumerate(VARIANTS)} + for variant in variants: + for row in rows_by_variant.get(variant.name, []): + reference = baseline_rows.get(tuple(row.get(field, "") for field in CASE_FIELDS)) + result_rows.append( + { + "variant": variant.name, + **{field: row.get(field, "") for field in CASE_FIELDS}, + **{ + target: percentage_difference( + row.get(source), reference.get(source) if reference else None + ) + for source, target in PERCENTAGE_FIELDS + }, + } + ) + result_rows.sort( + key=lambda row: ( + row["arch"], + row["model"], + row["mode"], + variant_order[row["variant"]], + int(row["raptor_pipeline"]), + ) + ) + output = out_dir / ABLATION_RESULTS_FILENAME + with output.open("w", newline="", encoding="utf-8") as stream: + writer = csv.DictWriter(stream, fieldnames=("variant", *RESULT_FIELDS), lineterminator="\n") + writer.writeheader() + writer.writerows(result_rows) + return output, failures + + +def main(argv: list[str] | None = None) -> int: + args, forwarded = parse_args(argv) + if not RUNNER.is_file(): + print(f"Missing Pimcomp runner: {RUNNER}", file=sys.stderr) + return 1 + variants = selected_variants(args.variants) + out_dir = args.out_dir.resolve() + case_sets = runner_argument_sets(forwarded) + if args.dry_run: + for variant in variants: + for _, case_forwarded in case_sets: + print( + shlex.join( + runner_command( + out_dir, + variant, + args.models, + case_forwarded, + out_dir, + True, + ) + ) + ) + return 0 + + out_dir.mkdir(parents=True, exist_ok=True) + failed = [] + current_process = None + try: + for variant in variants: + for case_label, case_forwarded in case_sets: + command = runner_command( + out_dir, variant, args.models, case_forwarded, out_dir, False + ) + print(f"[{variant.name}/{case_label}] {shlex.join(command)}") + current_process = subprocess.Popen( + command, + cwd=REPO, + start_new_session=True, + ) + returncode = current_process.wait() + current_process = None + if returncode: + failed.append(f"{variant.name}/{case_label}: runner exited with {returncode}") + except KeyboardInterrupt: + if current_process is not None: + terminate_process_group(current_process) + remove_lock_files(out_dir) + remove_variant_summaries(out_dir) + print("Interrupted; terminated the active ablation job.", file=sys.stderr) + return 130 + + remove_lock_files(out_dir) + selected_cases = DEFAULT_CASE_KEYS if len(case_sets) > 1 else None + output, aggregation_failures = aggregate_results( + out_dir, + variants, + selected_cases, + frozenset(args.models), + ) + remove_variant_summaries(out_dir) + failed.extend(aggregation_failures) + if output is not None: + print(f"Ablation results: {output}") + if failed: + print("Failed: " + "; ".join(failed), file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/validation/tools/pim/pimcomp/compare/README.md b/validation/tools/pim/pimcomp/compare/README.md new file mode 100644 index 0000000..6cb6fbb --- /dev/null +++ b/validation/tools/pim/pimcomp/compare/README.md @@ -0,0 +1,62 @@ +# Pimcomp model comparison + +These scripts compare Raptor with Pimcomp on the models in +`validation/networks/pimcomp_models`. + +`run_pimcomp_models.py` runs the supported model suite across the configured +architectures and simulation modes. `compare_raptor_pimcomp_model.py` is the +lower-level one-model comparison used by the suite runner. + +## Suite runner + +Run all five models, Arch-A and Arch-B, latency, and throughput: + +```bash +.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py +``` + +### Options + +| Option | Description and default | +|---|---| +| `-h`, `--help` | Show help and exit. | +| `--out-dir PATH` | Suite root; artifacts are placed below `//artifacts/`. Default: `validation/networks/pimcomp_models`. | +| `--common-dir PATH` | Shared root for per-model reference artifacts. Default: `//artifacts/common`. | +| `--models MODEL [...]` | Models to run: `vgg8`, `resnet18`, `resnet34`, `googlenet`, `yolo11n`. Default: all five; pass a subset to select specific models. | +| `--archs ARCH [...]` | Pimcomp hardware profiles to run. Default: `arch-a arch-b`. | +| `--mode {latency,throughput} [...]` | Simulation modes. Default: both. | +| `--only {raptor,pimcomp}` | Re-run only one compiler side and reuse the other side's existing report. Default: run both sides. | +| `--raptor-only` | Run Raptor without compiling, validating, or simulating Pimcomp. Default: off. | +| `--pipeline {1,2,4,8}` | Select one Raptor pipeline. Default: latency pipeline 1 and throughput pipelines 2, 4, and 8. | +| `--pimsim-time-ms INT` | Throughput convergence deadline. Default: `1000`. | +| `--batch-size INT` | Functional throughput batch size. Default: `128`. | +| `--timeout-seconds FLOAT` | Per-stage timeout; `0` means unlimited. Default: `0`. | +| `-j INT`, `--jobs INT` | Parallel comparison workers. Default: `4`. | +| `--clean` | Remove generated comparison artifacts and summaries, then exit. Default: off. | +| `--ablation-variant NAME` | Put generated artifacts below `[/pipelineN]/ablation/NAME` and write the summary below `ablation/NAME`. Default: none. | +| `--dry-run` | Print commands without modifying files. Default: off. | +| `--no-fast` | Disable fast throughput convergence. Default: off. | +| `--raptor-extra-arg=ARG` | Extra Raptor compiler argument; repeat for multiple arguments. Default: none. | + +Arguments beginning with `--` must use the equals form when passed through: + +```bash +.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py \ + --models vgg8 \ + --raptor-extra-arg=--pim-disable-synchronization +``` + +The runner writes `results_comparison.csv` under the selected result root. It reuses +shared model inputs, outputs, and reference runners, and reuses Pimcomp +artifacts between pipelines in the same model/architecture/mode group. With +`--raptor-only`, only Raptor results are generated and Pimcomp is not invoked. +All generated files are kept below each model's `artifacts/` directory; +`--clean` also removes any `.lock` files left by an interrupted reference +generation. + +## One-model comparator + +`compare_raptor_pimcomp_model.py` compares one ONNX model. Its required +arguments are `--model PATH` and `--out-dir PATH`; use `--help` for the full +lower-level interface. The suite runner supplies the model, hardware profile, +simulation mode, and reuse paths automatically. diff --git a/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py b/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py similarity index 86% rename from validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py rename to validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py index 021b0c7..75f98a8 100755 --- a/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py +++ b/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py @@ -2,6 +2,7 @@ from __future__ import annotations import argparse +import fcntl import gzip import importlib.util import json @@ -30,8 +31,8 @@ PIMSIM_CONFIG_DIR = VALIDATION_DIR / "pimsim_configs/pimcomp" PIMCOMP_OUTPUT_FILES = ("SimulationInfo.gz", "VerificationInfo.json", "MappingResult.txt") sys.path.insert(0, str(VALIDATION_DIR)) -from raptor_validation.gen_network_runner import gen_network_runner # noqa: E402 -from raptor_validation.onnx_utils import ( # noqa: E402 +from raptor_validation.gen_network_runner import gen_network_runner +from raptor_validation.onnx_utils import ( _ONNX_TO_NP, generate_input_batch, gen_random_inputs, @@ -41,12 +42,13 @@ from raptor_validation.onnx_utils import ( # noqa: E402 write_input_batch_csv, write_inputs_to_memory_bin, ) -from raptor_validation.raptor import compile_with_raptor # noqa: E402 -from raptor_validation.pimsim_nn import ( # noqa: E402 +from raptor_validation.raptor import compile_with_raptor +from raptor_validation.pimsim_nn import ( export_raptor_pimsim_artifact, parse_pimsim_nn_metrics, ) -from raptor_validation.validate_one import ( # noqa: E402 +from raptor_validation.artifacts import runner_uses_library +from raptor_validation.validate_one import ( STAGE_COLORS, build_pim_simulator_command, build_dump_ranges, @@ -99,7 +101,7 @@ def print_step( stage: str | None = None, ): color = STAGE_COLORS.get(stage or name, Fore.WHITE) - print("\n" + Style.BRIGHT + color + f"[{name}]" + Style.RESET_ALL) + print(Style.BRIGHT + color + f"[{name}]" + Style.RESET_ALL) if cmd is not None: print(f" cwd: {cwd or REPO}") print(f" $ {shell_join(cmd)}") @@ -134,7 +136,7 @@ def exception_message(exc: BaseException) -> str: def print_failure(name: str, exc: BaseException | str) -> None: message = exc if isinstance(exc, str) else exception_message(exc) print( - "\n" + Style.BRIGHT + Fore.RED + f"[{name} FAILED]" + Style.RESET_ALL, + Style.BRIGHT + Fore.RED + f"[{name} FAILED]" + Style.RESET_ALL, file=sys.stderr, ) for line in message.splitlines()[:20]: @@ -239,6 +241,17 @@ def reference_inputs_exist( ) +def reference_batch_dirs(root: Path, batch_size: int) -> list[Path]: + return [root / f"batch_{index:06d}" for index in range(batch_size)] + + +def reference_batch_outputs_exist( + outputs_desc: list[tuple[int, str, int, list[int]]], + reference_dirs: list[Path], +) -> bool: + return all(reference_outputs_exist(outputs_desc, reference_dir) for reference_dir in reference_dirs) + + def compare_simulator_outputs( output_bin: Path, outputs_desc: list[tuple[int, str, int, list[int]]], @@ -308,7 +321,7 @@ def select_pimsim_config(args: argparse.Namespace, hardware: dict[str, int]) -> if args.pimsim_mode == "latency" or config["sim_config"]["sim_time"] == args.pimsim_time_ms: return path raise ValueError( - f"No pre-generated {args.pimsim_mode} pimsim-nn config for {args.pimsim_time_ms} ms matches {hardware}" + f"No pre-generated {args.pimsim_mode} Pimsim config for {args.pimsim_time_ms} ms matches {hardware}" ) @@ -329,13 +342,14 @@ def compile_reference( runner_dir = work_dir / "runner" build_dir = runner_dir / "build" raptor_dir.mkdir(parents=True, exist_ok=True) + shutil.rmtree(build_dir, ignore_errors=True) build_dir.mkdir(parents=True, exist_ok=True) stem = model_path.stem onnx_ir_base = raptor_dir / stem runner_base = runner_dir / stem run_logged( - "Compile Reference ONNX IR", + "Compile reference ONNX IR", [str(args.raptor_path), str(model_path), "-o", str(onnx_ir_base), "--EmitONNXIR", "--mlir-elide-elementsattrs-if-larger=16", "--enable-conv-opt-pass=false"], cwd=REPO, @@ -344,7 +358,7 @@ def compile_reference( stage="Compile ONNX", ) run_logged( - "Compile Reference Native", + "Compile reference native", [str(args.raptor_path), "-O3", str(model_path), "-o", str(runner_base)], cwd=REPO, timeout_sec=args.timeout_seconds, @@ -353,7 +367,7 @@ def compile_reference( ) network_so = runner_base.with_suffix(".so") - print_step("Generate Runner Source", stage="Build Runner") + print_step("Generate runner source", stage="Build runner") gen_network_runner( model_path, network_so, @@ -364,15 +378,15 @@ def compile_reference( ) run_logged( - "Configure Runner", + "Configure runner", ["cmake", str(runner_dir), "-DCMAKE_BUILD_TYPE=Release", "-DCMAKE_C_FLAGS_RELEASE=-O3"], cwd=build_dir, timeout_sec=args.timeout_seconds, steps=steps, - stage="Build Runner", + stage="Build runner", ) run_logged( - "Build Runner", + "Build runner", ["cmake", "--build", ".", "-j"], cwd=build_dir, timeout_sec=args.timeout_seconds, @@ -382,6 +396,20 @@ def compile_reference( return build_dir / "runner" +def ensure_reference_runner( + args: argparse.Namespace, + model_path: Path, + work_dir: Path, + steps: list[StepRecord], +) -> Path: + runner_dir = work_dir / "runner" + runner_path = runner_dir / "build/runner" + library_path = runner_dir / f"{model_path.stem}.so" + if runner_uses_library(runner_path, library_path): + return runner_path + return compile_reference(args, model_path, work_dir, steps) + + def generate_reference_outputs( runner_path: Path, runner_build_dir: Path, @@ -390,6 +418,8 @@ def generate_reference_outputs( steps: list[StepRecord], args: argparse.Namespace, out_dir: Path, + *, + print_header: bool = True, ) -> Path: inputs_dir = out_dir / "inputs" reference_dir = out_dir / "outputs" @@ -397,11 +427,12 @@ def generate_reference_outputs( reference_dir.mkdir(parents=True, exist_ok=True) flags, _ = save_inputs_to_files(model_path, arrays_in_order, inputs_dir) run_logged( - "Run Reference", + "Run reference", [str(runner_path), *flags, "--save-csv-dir", str(reference_dir)], cwd=runner_build_dir, timeout_sec=args.timeout_seconds, steps=steps, + print_header=print_header, ) return reference_dir @@ -414,19 +445,56 @@ def generate_reference_batch_outputs( steps: list[StepRecord], args: argparse.Namespace, out_dir: Path, + *, + print_header: bool = True, ) -> list[Path]: - return [ - generate_reference_outputs( + if print_header: + print_step("Run reference") + references = [] + for index, sample in enumerate(input_batch): + references.append( + generate_reference_outputs( + runner_path, + runner_build_dir, + model_path, + sample, + steps, + args, + out_dir / f"batch_{index:06d}", + print_header=False, + ) + ) + return references + + +def prepare_reference_batch_outputs( + runner_path: Path, + runner_build_dir: Path, + model_path: Path, + input_batch: list[list[np.ndarray]], + outputs_desc: list[tuple[int, str, int, list[int]]], + steps: list[StepRecord], + args: argparse.Namespace, + out_dir: Path, + *, + print_header: bool = True, +) -> list[Path]: + reference_dirs = reference_batch_dirs(out_dir, len(input_batch)) + lock_path = out_dir.parent / ".reference.lock" + with lock_path.open("w", encoding="utf-8") as lock: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX) + if reference_batch_outputs_exist(outputs_desc, reference_dirs): + return reference_dirs + return generate_reference_batch_outputs( runner_path, runner_build_dir, model_path, - sample, + input_batch, steps, args, - out_dir / f"batch_{index:06d}", + out_dir, + print_header=print_header, ) - for index, sample in enumerate(input_batch) - ] def prepare_common_artifacts( @@ -437,10 +505,8 @@ def prepare_common_artifacts( inputs_desc, outputs_desc, arrays_in_order = load_model_inputs(model_path, args.seed) inputs_dir = common_dir / "inputs" outputs_dir = common_dir / "outputs" - runner_path = common_dir / "runner/build/runner" steps: list[StepRecord] = [] - if not runner_path.exists(): - runner_path = compile_reference(args, model_path, common_dir, steps) + runner_path = ensure_reference_runner(args, model_path, common_dir, steps) inputs_ready = reference_inputs_exist(inputs_desc, inputs_dir) outputs_ready = reference_outputs_exist(outputs_desc, outputs_dir) @@ -478,7 +544,7 @@ def compile_raptor_target( "--pim-emit-json", *args.raptor_extra_arg, ] - print_step("Compile Raptor", cmd, REPO, "Compile PIM") + print_step("Compile Raptor", cmd, REPO, "Compile Pim") start = time.perf_counter() command = shell_join(cmd) raptor_extra_args = [ @@ -503,7 +569,7 @@ def compile_raptor_target( except Exception as exc: steps.append( StepRecord( - name="Compile Raptor PIM", + name="Compile Raptor Pim", duration_sec=time.perf_counter() - start, command=command, status="failed", @@ -514,7 +580,7 @@ def compile_raptor_target( steps.append( StepRecord( - name="Compile Raptor PIM", + name="Compile Raptor Pim", duration_sec=time.perf_counter() - start, command=command, ) @@ -548,7 +614,8 @@ def run_functional_validation( pim_dir, output_bin, dump_ranges, - input_bins, + input_bins[0].parent, + batch_size, args.pimsim_mode, batch_output_dir, ) @@ -559,7 +626,7 @@ def run_functional_validation( cwd=args.pim_simulator_dir, timeout_sec=args.timeout_seconds, steps=steps, - stage="Run Functional Simulation", + stage="Run functional simulation", ) max_diffs: dict[str, float] = {} failed_iterations = [] @@ -610,9 +677,9 @@ def compile_pimcomp( (pimcomp_output_dir / name).unlink(missing_ok=True) model_name = args.pimcomp_model_name or f"compare_{model_path.stem}" frontend_json = frontend_json_dir / f"{model_name}.json" - print_step("Compile PIMCOMP", stage="Compile PIM") - # The original PIMCOMP frontend rewrites its input ONNX while loading it. - # Isolate that mutation without changing the model sent through PIMCOMP. + print_step("Compile Pimcomp", stage="Compile Pim") + # The original Pimcomp frontend rewrites its input ONNX while loading it. + # Isolate that mutation without changing the model sent through Pimcomp. with TemporaryDirectory(prefix="pimcomp-model-") as temp_dir: frontend_model = Path(temp_dir) / model_path.name shutil.copy2(model_path, frontend_model) @@ -625,12 +692,12 @@ def compile_pimcomp( str(frontend_json), ] run_logged( - "Compile PIMCOMP Frontend", + "Compile Pimcomp frontend", frontend_cmd, cwd=args.pimcomp_dir / "frontend", timeout_sec=args.timeout_seconds, steps=steps, - stage="Compile PIM", + stage="Compile Pim", print_header=False, ) backend_cmd = [ @@ -643,12 +710,12 @@ def compile_pimcomp( ] try: run_logged( - "Compile PIMCOMP Backend", + "Compile Pimcomp backend", backend_cmd, cwd=frontend_json_dir.parent, timeout_sec=args.timeout_seconds, steps=steps, - stage="Compile PIM", + stage="Compile Pim", print_header=False, ) finally: @@ -666,7 +733,7 @@ def export_pimcomp_for_pimsim_nn(simulation_info: Path, output_dir: Path) -> Pat sim_config = sim_info["config"] core_count = int(sim_config["core_cnt"]) if core_count <= 0: - raise ValueError("PIMCOMP SimulationInfo.gz must configure at least one core") + raise ValueError("Pimcomp SimulationInfo.gz must configure at least one core") core_indices = range(core_count) config = { @@ -709,7 +776,7 @@ def export_pimcomp_for_rust( output_dir: Path, ) -> Path: if len(runtime_inputs) != 1: - raise ValueError("PIMCOMP export currently requires exactly one runtime input tensor") + raise ValueError("Pimcomp export currently requires exactly one runtime input tensor") if output_dir.exists(): shutil.rmtree(output_dir) exporter = load_pimcomp_exporter() @@ -901,7 +968,7 @@ def export_pimcomp_for_rust( instructions.append(translated) out_offset += width else: - raise RuntimeError(f"Unsupported PIMCOMP op {op}") + raise RuntimeError(f"Unsupported Pimcomp op {op}") with open(output_dir / f"core_{core_idx}.json", "w", encoding="utf-8") as f: json.dump(instructions, f, separators=(",", ":")) @@ -935,7 +1002,7 @@ def run_pimsim_nn( cwd=args.pimsim_nn_build_dir, timeout_sec=args.timeout_seconds * 10.0, steps=steps, - stage="Run Non-functional Simulation", + stage="Run non-functional simulation", ) return parse_pimsim_nn_metrics(output) @@ -1091,12 +1158,9 @@ def restore_side_records( def default_common_dir(out_dir: Path) -> Path: - if out_dir.name == "latency": - return out_dir.parents[1] / "common" - if out_dir.parent.name == "throughput": - return out_dir.parents[2] / "common" - if out_dir.name.startswith("arch-"): - return out_dir.parent / "common" + for parent in (out_dir, *out_dir.parents): + if parent.name == "artifacts": + return parent / "common" return out_dir / "common" @@ -1155,19 +1219,19 @@ def write_report( ): report_path.parent.mkdir(parents=True, exist_ok=True) lines = [ - "# Raptor vs PIMCOMP Comparison Report", + "# Raptor vs Pimcomp comparison report", "", f"- Model: `{model_path}`", f"- Hardware: `{hardware.get('core_count', 'n/a')} cores`, `{hardware.get('crossbar_count', 'n/a')} xbars/core`, `{hardware.get('crossbar_size', 'n/a')}x{hardware.get('crossbar_size', 'n/a')}` crossbars, mesh `{hardware.get('mesh_rows', 'n/a')}x{hardware.get('mesh_cols', 'n/a')}`", - f"- PIMCOMP pipeline: `{pimcomp_pipeline}`", - f"- PIMCOMP replication: `{pimcomp_replication}`", + f"- Pimcomp pipeline: `{pimcomp_pipeline}`", + f"- Pimcomp replication: `{pimcomp_replication}`", "", ] if failures or any(step.status != "passed" for step in steps): lines.extend( [ - "## Failures / Skipped Work", + "## Failures / skipped work", "", "The script did not abort. The failed stage was recorded and any dependent stage was skipped when its inputs were not available.", "", @@ -1182,21 +1246,21 @@ def write_report( lines.extend( [ - "## Functional Validation", + "## Functional validation", "", f"- Raptor via `pim-simulator`: `{validation_status(raptor_validation)}`", - f"- PIMCOMP via exported `pim-simulator`: `{validation_status(pimcomp_validation)}`", + f"- Pimcomp via exported `pim-simulator`: `{validation_status(pimcomp_validation)}`", ] ) if raptor_validation.error: lines.append(f"- Raptor validation note: `{raptor_validation.error.splitlines()[0]}`") if pimcomp_validation.error: - lines.append(f"- PIMCOMP validation note: `{pimcomp_validation.error.splitlines()[0]}`") + lines.append(f"- Pimcomp validation note: `{pimcomp_validation.error.splitlines()[0]}`") - lines.extend(["", "### Max Output Differences", ""]) + lines.extend(["", "### Max output differences", ""]) diff_names = sorted(set(raptor_validation.max_diffs) | set(pimcomp_validation.max_diffs)) if diff_names: - lines.extend(["| Output | Raptor max diff | PIMCOMP max diff |", "|---|---:|---:|"]) + lines.extend(["| Output | Raptor max diff | Pimcomp max diff |", "|---|---:|---:|"]) for name in diff_names: lines.append( f"| `{name}` | {raptor_validation.max_diffs.get(name, float('nan')):.6e} | " @@ -1208,7 +1272,7 @@ def write_report( lines.extend( [ "", - "## pimsim-nn Performance", + "## Pimsim performance", "", f"- Mode: `{pimsim_mode}`", "", @@ -1221,7 +1285,7 @@ def write_report( "|---|---|---:|---:|---:|---:|---:|", f"| Raptor | {perf_status(raptor_perf)} | {perf_value(raptor_perf, 'average_latency_ms', 'ms')} | {perf_value(raptor_perf, 'throughput', 'samples/s')} | " f"{perf_value(raptor_perf, 'average_power_mw', 'mW')} | {perf_value(raptor_perf, 'average_energy_pj', 'pJ/it')} | {perf_value(raptor_perf, 'output_count')} |", - f"| PIMCOMP | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'average_latency_ms', 'ms')} | {perf_value(pimcomp_perf, 'throughput', 'samples/s')} | " + f"| Pimcomp | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'average_latency_ms', 'ms')} | {perf_value(pimcomp_perf, 'throughput', 'samples/s')} | " f"{perf_value(pimcomp_perf, 'average_power_mw', 'mW')} | {perf_value(pimcomp_perf, 'average_energy_pj', 'pJ/it')} | {perf_value(pimcomp_perf, 'output_count')} |", "", ] @@ -1233,40 +1297,40 @@ def write_report( "|---|---|---:|---:|---:|", f"| Raptor | {perf_status(raptor_perf)} | {perf_value(raptor_perf, 'latency_ms', 'ms')} | " f"{perf_value(raptor_perf, 'average_power_mw', 'mW')} | {perf_value(raptor_perf, 'average_energy_pj', 'pJ')} |", - f"| PIMCOMP | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'latency_ms', 'ms')} | " + f"| Pimcomp | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'latency_ms', 'ms')} | " f"{perf_value(pimcomp_perf, 'average_power_mw', 'mW')} | {perf_value(pimcomp_perf, 'average_energy_pj', 'pJ')} |", "", ] ) if raptor_perf.get("reason") or raptor_perf.get("error"): - lines.append(f"- Raptor pimsim-nn note: `{(raptor_perf.get('reason') or raptor_perf.get('error')).splitlines()[0]}`") + lines.append(f"- Raptor Pimsim note: `{(raptor_perf.get('reason') or raptor_perf.get('error')).splitlines()[0]}`") if pimcomp_perf.get("reason") or pimcomp_perf.get("error"): - lines.append(f"- PIMCOMP pimsim-nn note: `{(pimcomp_perf.get('reason') or pimcomp_perf.get('error')).splitlines()[0]}`") + lines.append(f"- Pimcomp Pimsim note: `{(pimcomp_perf.get('reason') or pimcomp_perf.get('error')).splitlines()[0]}`") if lines[-1] != "": lines.append("") lines.extend( [ - "## Instruction Summary", + "## Instruction summary", "", "| Compiler | Status | Active cores | Total instructions | Sends | Receives | MVMUL |", "|---|---|---:|---:|---:|---:|---:|", f"| Raptor | {'FAILED' if raptor_instr.get('error') else 'SKIPPED' if raptor_instr.get('skipped') else 'DONE'} | {raptor_instr.get('active_cores', 0)} | {raptor_instr.get('total_instructions', 0)} | {raptor_instr.get('op_counts', {}).get('send', 0)} | {raptor_instr.get('op_counts', {}).get('recv', 0)} | {raptor_instr.get('op_counts', {}).get('mvmul', 0)} |", - f"| PIMCOMP | {'FAILED' if pimcomp_instr.get('error') else 'SKIPPED' if pimcomp_instr.get('skipped') else 'DONE'} | {pimcomp_instr.get('active_cores', 0)} | {pimcomp_instr.get('total_instructions', 0)} | {pimcomp_instr.get('op_counts', {}).get('send', 0)} | {pimcomp_instr.get('op_counts', {}).get('recv', 0)} | {pimcomp_instr.get('op_counts', {}).get('mvmul', 0)} |", + f"| Pimcomp | {'FAILED' if pimcomp_instr.get('error') else 'SKIPPED' if pimcomp_instr.get('skipped') else 'DONE'} | {pimcomp_instr.get('active_cores', 0)} | {pimcomp_instr.get('total_instructions', 0)} | {pimcomp_instr.get('op_counts', {}).get('send', 0)} | {pimcomp_instr.get('op_counts', {}).get('recv', 0)} | {pimcomp_instr.get('op_counts', {}).get('mvmul', 0)} |", "", - "### Raptor Op Distribution", + "### Raptor op distribution", "", "| Op | Count | Share |", "|---|---:|---:|", *format_op_table(raptor_instr.get("op_counts", {}), raptor_instr.get("total_instructions", 0)), "", - "### PIMCOMP Op Distribution", + "### Pimcomp op distribution", "", "| Op | Count | Share |", "|---|---:|---:|", *format_op_table(pimcomp_instr.get("op_counts", {}), pimcomp_instr.get("total_instructions", 0)), "", - "## Step Timings", + "## Step timings", "", "| Step | Status | Duration (s) | Return code |", "|---|---|---:|---:|", @@ -1279,7 +1343,7 @@ def write_report( ) failed_steps = [step for step in steps if step.status != "passed"] if failed_steps: - lines.extend(["", "### Failed Step Details", ""]) + lines.extend(["", "### Failed step details", ""]) for step in failed_steps: lines.extend( [ @@ -1294,7 +1358,7 @@ def write_report( lines.append("") if raptor_pass_timings: - lines.extend(["", "## Raptor Pass Timings", "", "| Pass | Duration (s) |", "|---|---:|"]) + lines.extend(["", "## Raptor pass timings", "", "| Pass | Duration (s) |", "|---|---:|"]) for name, duration in raptor_pass_timings.items(): lines.append(f"| {name} | {duration:.4f} |") report_path.write_text("\n".join(lines) + "\n", encoding="utf-8") @@ -1325,7 +1389,7 @@ def main(): parser.add_argument( "--pimcomp-config", type=Path, - help="PIMCOMP hardware config (default: /config.json).", + help="Pimcomp hardware config (default: /config.json).", ) parser.add_argument("--pim-simulator-dir", default=REPO / "backend-simulators/pim/pim-simulator", type=Path) parser.add_argument("--pimsim-nn-build-dir", default=REPO / "backend-simulators/pim/pimsim-nn/build", type=Path) @@ -1347,7 +1411,7 @@ def main(): parser.add_argument("--pimsim-mode", choices=["latency", "throughput"], default="latency") parser.add_argument("--batch-size", type=int, default=128) parser.add_argument("--pimcomp-pipeline", choices=["element", "batch"]) - parser.add_argument("--pimcomp-model-name", help="Use a PIMCOMP built-in model name such as vgg16.") + parser.add_argument("--pimcomp-model-name", help="Use a Pimcomp built-in model name such as vgg16.") parser.add_argument( "--pimcomp-replication", choices=["balance", "W0H0", "uniform", "GA"], @@ -1361,27 +1425,41 @@ def main(): parser.add_argument( "--reuse-pimcomp-dir", type=Path, - help="Reuse a directory containing PIMCOMP SimulationInfo.gz, VerificationInfo.json, and MappingResult.txt.", + help="Reuse a directory containing Pimcomp SimulationInfo.gz, VerificationInfo.json, and MappingResult.txt.", ) parser.add_argument( "--reuse-pimcomp-report", type=Path, - help="Preserve the PIMCOMP side of an existing comparison report without rerunning it.", + help="Preserve the Pimcomp side of an existing comparison report without rerunning it.", ) parser.add_argument("--skip-pimsim-nn", action="store_true") parser.add_argument( "--no-fast", action="store_true", - help="Disable fast pimsim-nn throughput convergence for authoritative experiments.", + help="Disable fast Pimsim throughput convergence for authoritative experiments.", ) parser.add_argument("--verbose-raptor-compile", action="store_true") parser.add_argument("--raptor-extra-arg", action="append", default=[]) + parser.add_argument( + "--raptor-only", + action="store_true", + help="Run Raptor compilation, validation, and simulation without running Pimcomp.", + ) parser.add_argument( "--fail-on-error", action="store_true", help="Return a non-zero status if a stage or semantic validation fails.", ) args = parser.parse_args() + if args.raptor_only and any( + option is not None + for option in ( + args.reuse_raptor_report, + args.reuse_pimcomp_dir, + args.reuse_pimcomp_report, + ) + ): + parser.error("--raptor-only cannot be combined with report reuse options") if args.reuse_pimcomp_dir is not None and args.reuse_pimcomp_report is not None: parser.error("--reuse-pimcomp-dir and --reuse-pimcomp-report are mutually exclusive") if args.pimsim_time_ms <= 0: @@ -1439,6 +1517,7 @@ def main(): runner_path: Path | None = None reference_dir: Path | None = None reference_dirs: list[Path] = [] + reference_stage_started = False raptor_pim_dir: Path | None = None raptor_pimsim_dir: Path | None = None raptor_pass_timings: dict[str, float] = {} @@ -1451,11 +1530,19 @@ def main(): reuse_raptor = args.reuse_raptor_report is not None raptor_validation = skipped_validation("Raptor validation did not run") - pimcomp_validation = failed_validation("PIMCOMP validation did not run") - raptor_perf: dict[str, Any] = skipped_perf("pimsim-nn Raptor did not run") - pimcomp_perf: dict[str, Any] = skipped_perf("pimsim-nn PIMCOMP did not run") + pimcomp_validation = ( + skipped_validation("Skipped by --raptor-only") + if args.raptor_only + else failed_validation("Pimcomp validation did not run") + ) + raptor_perf: dict[str, Any] = skipped_perf("Pimsim Raptor did not run") + pimcomp_perf: dict[str, Any] = skipped_perf( + "Skipped by --raptor-only" if args.raptor_only else "Pimsim Pimcomp did not run" + ) raptor_instr: dict[str, Any] = empty_instruction_summary("Raptor instruction parsing did not run") - pimcomp_instr: dict[str, Any] = empty_instruction_summary("PIMCOMP instruction parsing did not run") + pimcomp_instr: dict[str, Any] = empty_instruction_summary( + "Skipped by --raptor-only" if args.raptor_only else "Pimcomp instruction parsing did not run" + ) loaded_hardware = try_stage(failures, "Load hardware configuration", load_effective_hardware, args) if loaded_hardware is not None: @@ -1476,22 +1563,21 @@ def main(): if args.reuse_pimcomp_dir is not None: reused_pimcomp_report_path = args.reuse_pimcomp_dir.resolve().parent / "comparison_report.json" reuse_pimcomp = reused_pimcomp_report_path is not None + run_pimcomp = not args.raptor_only and not reuse_pimcomp if reuse_pimcomp: reused_pimcomp_report_path = reused_pimcomp_report_path.resolve() if not reused_pimcomp_report_path.exists(): - raise ValueError(f"Missing PIMCOMP report: {reused_pimcomp_report_path}") + raise ValueError(f"Missing Pimcomp report: {reused_pimcomp_report_path}") with open(reused_pimcomp_report_path, "r", encoding="utf-8") as f: reused_pimcomp = json.load(f) if reused_pimcomp.get("pimcomp_model_source") != "original_onnx": - raise ValueError("Reused PIMCOMP artifacts were not generated from the original ONNX model") + raise ValueError("Reused Pimcomp artifacts were not generated from the original ONNX model") pimcomp_validation = CompareResult(**reused_pimcomp["pimcomp_validation"]) pimcomp_perf = reused_pimcomp["pimcomp_performance"] pimcomp_instr = reused_pimcomp["pimcomp_instruction_summary"] simulation_info = path_from_report(reused_pimcomp, "pimcomp_simulation_info") pimcomp_export_dir = path_from_report(reused_pimcomp, "pimcomp_exported_pim") pimcomp_pimsim_dir = path_from_report(reused_pimcomp, "pimcomp_pimsim_nn") - if pimcomp_pimsim_dir is None and (reused_pimcomp_report_path.parent / "pimsim_nn").is_dir(): - pimcomp_pimsim_dir = reused_pimcomp_report_path.parent / "pimsim_nn" pimsim_config = path_from_report(reused_pimcomp, "pimsim_config") restore_side_records(reused_pimcomp, "pimcomp", failures, steps) @@ -1504,6 +1590,9 @@ def main(): raise ValueError(f"Reused Raptor hardware differs: {reused_hardware} != {hardware}") reference_dir = common_dir / "outputs" runner_path = path_from_report(reused, "reference_runner") + expected_library = common_dir / "runner" / f"{functional_model_path.stem}.so" + if runner_path is None or not runner_uses_library(runner_path, expected_library): + runner_path = ensure_reference_runner(args, functional_model_path, common_dir, steps) raptor_pim_dir = path_from_report(reused, "raptor_pim") raptor_pimsim_dir = path_from_report(reused, "raptor_pimsim_nn") arrays_in_order = load_saved_inputs( @@ -1519,7 +1608,6 @@ def main(): print_step("Reuse Raptor") print(f" Report: {reuse_report_path}") - expected_runner_path = common_dir / "runner/build/runner" common_inputs_dir = common_dir / "inputs" common_reference_dir = common_dir / "outputs" @@ -1539,20 +1627,15 @@ def main(): else: inputs_ready = False - if expected_runner_path.exists(): - runner_path = expected_runner_path - else: - reference_compile = try_stage( - failures, - "Compile reference", - compile_reference, - args, - functional_model_path, - common_dir, - steps, - ) - if reference_compile is not None: - runner_path = reference_compile + runner_path = try_stage( + failures, + "Prepare reference runner", + ensure_reference_runner, + args, + functional_model_path, + common_dir, + steps, + ) if runner_path is not None and runner_path.exists() and model_io is not None: if inputs_ready and reference_outputs_exist(outputs_desc, common_reference_dir): @@ -1570,9 +1653,11 @@ def main(): steps, args, common_dir, + print_header=not reference_stage_started, ) if generated_reference is not None: reference_dir = generated_reference + reference_stage_started = True else: record_failure( failures, @@ -1589,28 +1674,32 @@ def main(): write_input_batch_csv(out_dir / "inputs.csv", input_batch) raptor_input_bins = write_input_batch_binaries(input_batch, out_dir / "simulation/raptor_inputs") if args.pimsim_mode == "throughput": + batch_reference_dir = common_dir / f"reference_batch_{args.batch_size}_seed_{args.seed}" throughput_references = try_stage( failures, - "Run throughput references", - generate_reference_batch_outputs, + "Run reference", + prepare_reference_batch_outputs, runner_path, runner_path.parent, functional_model_path, input_batch, + outputs_desc, steps, args, - out_dir / "reference", + batch_reference_dir, + print_header=not reference_stage_started, ) if runner_path is not None and runner_path.exists() else None if throughput_references is not None: reference_dirs = throughput_references - reference_dir = out_dir / "reference" + reference_dir = batch_reference_dir + reference_stage_started = True elif reference_dir is not None: reference_dirs = [reference_dir] if not reuse_raptor and model_path.exists() and hardware["core_count"] > 0: compiled_raptor = try_stage( failures, - "Compile Raptor PIM", + "Compile Raptor Pim", compile_raptor_target, model_path, out_dir / "raptor", @@ -1623,8 +1712,8 @@ def main(): elif not reuse_raptor: record_failure( failures, - "Skip Raptor PIM compile", - "Raptor PIM compile was skipped because the ONNX model or hardware configuration is not available.", + "Skip Raptor Pim compile", + "Raptor Pim compile was skipped because the ONNX model or hardware configuration is not available.", ) raptor_functional_pim_dir = raptor_pim_dir @@ -1635,7 +1724,7 @@ def main(): ): compiled_functional = try_stage( failures, - "Compile Raptor functional PIM", + "Compile Raptor functional Pim", compile_raptor_target, functional_model_path, out_dir / "raptor_functional", @@ -1659,9 +1748,9 @@ def main(): if wrote_inputs and reference_dirs and outputs_desc: validation = try_stage( failures, - "Functional Validation Raptor", + "Functional validation Raptor", run_functional_validation, - "Functional Validation Raptor", + "Functional validation Raptor", raptor_functional_pim_dir, raptor_functional_pim_dir / "config.json", out_dir / "simulation/out.bin", @@ -1679,7 +1768,7 @@ def main(): else: raptor_validation = skipped_validation("Raptor input materialization failed") elif not reuse_raptor: - raptor_validation = skipped_validation("Raptor PIM compilation did not produce a PIM directory") + raptor_validation = skipped_validation("Raptor Pim compilation did not produce a Pim directory") pimcomp_model_path = model_path @@ -1687,7 +1776,7 @@ def main(): reused_pimcomp_dir = args.reuse_pimcomp_dir.resolve() copied_pimcomp = try_stage_success( failures, - "Reuse PIMCOMP outputs", + "Reuse Pimcomp outputs", copy_pimcomp_outputs, reused_pimcomp_dir, out_dir / "pimcomp/output", @@ -1695,12 +1784,12 @@ def main(): if copied_pimcomp: verification_info = out_dir / "pimcomp/output/VerificationInfo.json" simulation_info = out_dir / "pimcomp/output/SimulationInfo.gz" - print_step("Reuse PIMCOMP") + print_step("Reuse Pimcomp") print(f" Directory: {reused_pimcomp_dir}") - elif not reuse_pimcomp: + elif run_pimcomp: compiled_pimcomp = try_stage( failures, - "Compile PIMCOMP", + "Compile Pimcomp", compile_pimcomp, args, pimcomp_model_path, @@ -1710,12 +1799,12 @@ def main(): if compiled_pimcomp is not None: verification_info, simulation_info = compiled_pimcomp - if reuse_pimcomp: + if reuse_pimcomp or args.raptor_only: pass elif verification_info is not None and simulation_info is not None and model_io is not None: exported = try_stage( failures, - "Export PIMCOMP for Functional Validation", + "Export Pimcomp for functional validation", export_pimcomp_for_rust, pimcomp_model_path, verification_info, @@ -1728,14 +1817,14 @@ def main(): elif verification_info is None or simulation_info is None: record_failure( failures, - "Export PIMCOMP for Functional Validation", - "PIMCOMP functional export failed because PIMCOMP did not produce VerificationInfo.json and SimulationInfo.gz.", + "Export Pimcomp for functional validation", + "Pimcomp functional export failed because Pimcomp did not produce VerificationInfo.json and SimulationInfo.gz.", ) else: record_failure( failures, - "Export PIMCOMP for Functional Validation", - "PIMCOMP functional export failed because model inputs are not available.", + "Export Pimcomp for functional validation", + "Pimcomp functional export failed because model inputs are not available.", ) if input_batch is not None and pimcomp_export_dir is not None: @@ -1745,12 +1834,12 @@ def main(): transform=flatten_pimcomp_input, ) - if not reuse_pimcomp and pimcomp_export_dir is not None and reference_dirs and outputs_desc: + if run_pimcomp and pimcomp_export_dir is not None and reference_dirs and outputs_desc: validation = try_stage( failures, - "Functional Validation PIMCOMP", + "Functional validation Pimcomp", run_functional_validation, - "Functional Validation PIMCOMP", + "Functional validation Pimcomp", pimcomp_export_dir, pimcomp_export_dir / "config.json", out_dir / "simulation/pimcomp.out.bin", @@ -1761,11 +1850,11 @@ def main(): args, channel_last=True, ) - pimcomp_validation = validation if validation is not None else failed_validation("PIMCOMP validation failed") - elif reuse_pimcomp: + pimcomp_validation = validation if validation is not None else failed_validation("Pimcomp validation failed") + elif reuse_pimcomp or args.raptor_only: pass elif pimcomp_export_dir is None: - pimcomp_validation = failed_validation("PIMCOMP functional export is not available") + pimcomp_validation = failed_validation("Pimcomp functional export is not available") elif not reference_dirs: pimcomp_validation = failed_validation("Reference outputs are not available") else: @@ -1774,7 +1863,7 @@ def main(): if not args.skip_pimsim_nn and hardware["core_count"] > 0: written_config = try_stage( failures, - "Prepare pimsim-nn config", + "Prepare Pimsim config", prepare_pimsim_config, args, hardware, @@ -1784,25 +1873,25 @@ def main(): elif not args.skip_pimsim_nn: record_failure( failures, - "Skip pimsim-nn config", - "pimsim-nn config was skipped because the hardware configuration is not available.", + "Skip Pimsim config", + "Pimsim config was skipped because the hardware configuration is not available.", ) if args.skip_pimsim_nn: if not reuse_raptor: raptor_perf = skipped_perf("Skipped by --skip-pimsim-nn") - if not reuse_pimcomp: + if run_pimcomp: pimcomp_perf = skipped_perf("Skipped by --skip-pimsim-nn") elif pimsim_config is None: if not reuse_raptor: - raptor_perf = skipped_perf("pimsim-nn config is not available") - if not reuse_pimcomp: - pimcomp_perf = skipped_perf("pimsim-nn config is not available") + raptor_perf = skipped_perf("Pimsim config is not available") + if run_pimcomp: + pimcomp_perf = skipped_perf("Pimsim config is not available") else: if not reuse_raptor and raptor_pim_dir is not None: raptor_pimsim_dir = try_stage( failures, - "Export Raptor for pimsim-nn", + "Export Raptor for Pimsim", export_raptor_pimsim_artifact, raptor_pim_dir, out_dir / "raptor/pimsim_nn", @@ -1810,25 +1899,25 @@ def main(): if raptor_pimsim_dir is not None: perf = try_stage( failures, - "Non-Functional Simulation Raptor", + "Non-functional simulation Raptor", run_pimsim_nn, - "Non-Functional Simulation Raptor", + "Non-functional simulation Raptor", raptor_pimsim_dir, pimsim_config, steps, args, ) - raptor_perf = perf if perf is not None else failed_perf("pimsim-nn Raptor failed") + raptor_perf = perf if perf is not None else failed_perf("Pimsim Raptor failed") else: - raptor_perf = failed_perf("Raptor pimsim-nn export failed") + raptor_perf = failed_perf("Raptor Pimsim export failed") elif not reuse_raptor: - raptor_perf = skipped_perf("Raptor PIM directory is not available") + raptor_perf = skipped_perf("Raptor Pim directory is not available") - if not reuse_pimcomp: + if run_pimcomp: if simulation_info is not None: pimcomp_pimsim_dir = try_stage( failures, - "Export PIMCOMP for pimsim-nn", + "Export Pimcomp for Pimsim", export_pimcomp_for_pimsim_nn, simulation_info, out_dir / "pimcomp/pimsim_nn", @@ -1836,32 +1925,32 @@ def main(): if pimcomp_pimsim_dir is not None: perf = try_stage( failures, - "Non-Functional Simulation PIMCOMP", + "Non-functional simulation Pimcomp", run_pimsim_nn, - "Non-Functional Simulation PIMCOMP", + "Non-functional simulation Pimcomp", pimcomp_pimsim_dir, pimsim_config, steps, args, ) - pimcomp_perf = perf if perf is not None else failed_perf("pimsim-nn PIMCOMP failed") + pimcomp_perf = perf if perf is not None else failed_perf("Pimsim Pimcomp failed") else: - pimcomp_perf = failed_perf("PIMCOMP pimsim-nn export failed") + pimcomp_perf = failed_perf("Pimcomp Pimsim export failed") else: - pimcomp_perf = skipped_perf("PIMCOMP SimulationInfo.gz is not available") + pimcomp_perf = skipped_perf("Pimcomp SimulationInfo.gz is not available") if not reuse_raptor and raptor_pim_dir is not None and raptor_pim_dir.exists(): parsed = try_stage(failures, "Parse Raptor instructions", parse_raptor_instructions, raptor_pim_dir) raptor_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse Raptor instructions") elif not reuse_raptor: - raptor_instr = empty_instruction_summary("Raptor PIM directory is not available") + raptor_instr = empty_instruction_summary("Raptor Pim directory is not available") - if not reuse_pimcomp: + if run_pimcomp: if simulation_info is not None and simulation_info.exists(): - parsed = try_stage(failures, "Parse PIMCOMP instructions", parse_pimcomp_instructions, simulation_info) - pimcomp_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse PIMCOMP instructions") + parsed = try_stage(failures, "Parse Pimcomp instructions", parse_pimcomp_instructions, simulation_info) + pimcomp_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse Pimcomp instructions") else: - pimcomp_instr = empty_instruction_summary("PIMCOMP SimulationInfo.gz is not available") + pimcomp_instr = empty_instruction_summary("Pimcomp SimulationInfo.gz is not available") report_path = out_dir / "pimcomp/comparison_report.md" write_report( @@ -1890,7 +1979,7 @@ def main(): "pimsim_time_ms": args.pimsim_time_ms, "pimcomp_pipeline": args.pimcomp_pipeline, "pimcomp_replication": args.pimcomp_replication, - "pimcomp_model_source": "original_onnx", + "pimcomp_model_source": "not_run" if args.raptor_only else "original_onnx", "pimcomp_config": str(args.pimcomp_config), "raptor_extra_args": args.raptor_extra_arg, "reused_raptor_report": optional_path(args.reuse_raptor_report.resolve()) if reuse_raptor else None, @@ -1926,7 +2015,11 @@ def main(): json.dump(json_report, f, indent=2) f.write("\n") - partial_side = "pimcomp" if reuse_raptor else "raptor" if args.reuse_pimcomp_report else None + partial_side = ( + "pimcomp" if reuse_raptor + else "raptor" if args.raptor_only or args.reuse_pimcomp_report + else None + ) other_side = "RAPTOR" if partial_side == "pimcomp" else "PIMCOMP" relevant_failures = ( failures if partial_side is None @@ -1947,7 +2040,7 @@ def main(): for result in relevant_validations ) failed = bool(relevant_failures or failed_steps or functional_failure) - print("\n" + Style.BRIGHT + Fore.GREEN + "[Completed]" + Style.RESET_ALL) + print(Style.BRIGHT + Fore.GREEN + "[Completed]" + Style.RESET_ALL) print(f" Report: {report_path}") print(f" JSON: {json_path}") if failures or failed_steps: diff --git a/validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py b/validation/tools/pim/pimcomp/compare/run_pimcomp_models.py similarity index 74% rename from validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py rename to validation/tools/pim/pimcomp/compare/run_pimcomp_models.py index b053e45..f9c2d40 100755 --- a/validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py +++ b/validation/tools/pim/pimcomp/compare/run_pimcomp_models.py @@ -19,33 +19,30 @@ from colorama import Fore, Style REPO = Path(__file__).resolve().parents[5] -SUITE = REPO / "validation/networks/pimcomp_models" sys.path.insert(0, str(REPO / "validation")) -from raptor_validation.pimsim_nn import parse_pimsim_nn_metrics # noqa: E402 -from raptor_validation.validate_one import STAGE_COLORS # noqa: E402 +from raptor_validation.pimcomp_models import ( + FUNCTIONAL_MODELS, + MODELS, + SUITE, + add_models_argument, +) +from raptor_validation.artifacts import artifacts_dir, remove_lock_files +from raptor_validation.pimsim_nn import parse_pimsim_nn_metrics +from raptor_validation.validate_one import STAGE_COLORS PIMCOMP_SOURCE = REPO / "third_party/PIMCOMP-NN" PIMCOMP_CONFIGS = REPO / "validation/pimsim_configs/pimcomp" -COMPARE = Path(__file__).resolve().with_name("compare_raptor_pimcomp.py") +COMPARE = Path(__file__).resolve().with_name("compare_raptor_pimcomp_model.py") ARCHES = tuple(sorted(path.name for path in PIMCOMP_CONFIGS.iterdir() if path.is_dir())) -MODELS = { - "vgg8": SUITE / "vgg8/vgg8-mnist-reconstructed.onnx", - "resnet18": SUITE / "resnet18/resnet18-v1-7.onnx", - "resnet34": SUITE / "resnet34/resnet34-v1-7.onnx", - "googlenet": SUITE / "googlenet/googlenet-12-pimsim-nn.onnx", - "yolo11n": SUITE / "yolo11n/yolo11n-pimsim-nn.onnx", -} -FUNCTIONAL_MODELS = { - **MODELS, - "yolo11n": REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx", -} +DEFAULT_ARCHES = ("arch-a", "arch-b") COMPARISONS = ( ("latency", 1, "element"), ("throughput", 2, "batch"), ("throughput", 4, "batch"), ("throughput", 8, "batch"), ) +RESULTS_FILENAME = "results_comparison.csv" @dataclass(frozen=True) @@ -67,39 +64,60 @@ def model_dir(root: Path | None, name: str) -> Path: return root / name if root is not None else MODELS[name].parent -def result_dir(root: Path | None, name: str, arch: str, mode: str, pipeline: int) -> Path: - base = model_dir(root, name) +def result_dir( + root: Path | None, + name: str, + arch: str, + mode: str, + pipeline: int, + ablation_variant: str | None = None, +) -> Path: + base = artifacts_dir(model_dir(root, name)) suffix = "latency" if mode == "latency" else f"throughput/pipeline{pipeline}" - return base / arch / suffix + result = base / arch / suffix + if ablation_variant is not None: + result /= Path("ablation") / ablation_variant + return result -def common_dir(root: Path | None, name: str) -> Path: +def common_dir(root: Path | None, name: str, common_root: Path | None = None) -> Path: suffix = "common" if FUNCTIONAL_MODELS[name] == MODELS[name] else "common-functional" - return model_dir(root, name) / suffix + base = common_root / name if common_root is not None else model_dir(root, name) + return artifacts_dir(base) / suffix -def clean_artifacts(root: Path | None, models: list[str], arches: list[str]) -> int: +def clean_artifacts( + root: Path | None, + models: list[str], + common_root: Path | None = None, +) -> int: removed = 0 for name in models: base = model_dir(root, name) - arch_dirs = {base / arch for arch in arches} - arch_dirs.update(path for path in base.glob("arch-*") if path.is_dir() and not path.is_symlink()) - for path in arch_dirs: + for path in (artifacts_dir(base),): if path.is_dir() and not path.is_symlink(): shutil.rmtree(path) removed += 1 - for common in (base / "common", base / "common-functional"): - if common.is_dir() and not common.is_symlink(): - shutil.rmtree(common) - removed += 1 for path in ( - (root or SUITE) / "results.csv", - (root or SUITE) / "results_latency.csv", - (root or SUITE) / "results_throughput.csv", + (root or SUITE) / RESULTS_FILENAME, + (root or SUITE) / "results_ablation.csv", ): if path.is_file() or path.is_symlink(): path.unlink() removed += 1 + summary_root = root or SUITE + if (summary_root / "ablation").is_dir(): + for path in (summary_root / "ablation").glob("*/results_comparison.csv"): + path.unlink(missing_ok=True) + removed += 1 + removed += remove_lock_files(summary_root) + if common_root is not None: + for name in models: + path = artifacts_dir(common_root / name) + if path.is_dir() and not path.is_symlink(): + shutil.rmtree(path) + removed += 1 + removed += remove_lock_files(common_root) return removed @@ -108,8 +126,11 @@ def write_results_csv( arch: str, models: list[str], comparisons: tuple[tuple[str, int, str], ...] = COMPARISONS, + ablation_variant: str | None = None, ) -> Path: - output = (root or SUITE) / "results.csv" + output = (root or SUITE) / RESULTS_FILENAME + if ablation_variant is not None: + output = output.parent / "ablation" / ablation_variant / RESULTS_FILENAME fields = ( "arch", "model", @@ -152,7 +173,9 @@ def write_results_csv( rows.append({field: row.get(field, "NA") for field in fields}) for name in models: for comparison_mode, pipeline, pimcomp_pipeline in comparisons: - report_path = result_dir(root, name, arch, comparison_mode, pipeline) / "pimcomp/comparison_report.json" + report_path = result_dir( + root, name, arch, comparison_mode, pipeline, ablation_variant + ) / "pimcomp/comparison_report.json" row = { "model": name, "arch": arch, @@ -215,6 +238,7 @@ def write_results_csv( int(row["raptor_pipeline"]), ) ) + output.parent.mkdir(parents=True, exist_ok=True) with open(output, "w", newline="", encoding="utf-8") as f: writer = csv.DictWriter(f, fieldnames=fields, lineterminator="\n") writer.writeheader() @@ -275,11 +299,11 @@ def format_value(value: float | None) -> str: def print_stage(title: str, color: str) -> None: - print("\n" + Style.BRIGHT + color + f"[{title}]" + Style.RESET_ALL, flush=True) + print(Style.BRIGHT + color + f"[{title}]" + Style.RESET_ALL, flush=True) def print_completed(label: str, output: str = "") -> None: - print("\n" + Style.BRIGHT + Fore.CYAN + f"[Completed {label}]" + Style.RESET_ALL) + print(Style.BRIGHT + Fore.CYAN + f"[Completed {label}]" + Style.RESET_ALL) if output: print(output, end="" if output.endswith("\n") else "\n") print("=" * 72, flush=True) @@ -321,7 +345,7 @@ def validate_pimcomp_source() -> None: source = header.read_text(encoding="utf-8") for setting in ("int population_num = 200;", "int max_iteration = 1000;"): if setting not in source: - raise RuntimeError(f"PIMCOMP paper setting is missing: {setting}") + raise RuntimeError(f"Pimcomp paper setting is missing: {setting}") def comparison_command( @@ -337,6 +361,8 @@ def comparison_command( batch_size: int, timeout: float, fast: bool, + raptor_extra_args: list[str] | tuple[str, ...] = (), + raptor_only: bool = False, reuse_raptor_report: Path | None = None, reuse_pimcomp_dir: Path | None = None, reuse_pimcomp_report: Path | None = None, @@ -373,7 +399,9 @@ def comparison_command( pimcomp_pipeline, "--pimcomp-replication", "GA", + *(["--raptor-only"] if raptor_only else []), f"--raptor-extra-arg=--pipeline={pipeline}", + *[f"--raptor-extra-arg={arg}" for arg in raptor_extra_args], "--timeout-seconds", str(timeout), "--fail-on-error", @@ -418,13 +446,19 @@ def comparison_command_for( args.batch_size, args.timeout_seconds, not args.no_fast, + args.raptor_extra_args, + args.raptor_only, reuse_raptor_report=( report if args.only == "pimcomp" else None ), - reuse_pimcomp_dir=spec.shared_pimcomp_dir if reuse_shared_pimcomp and args.only != "raptor" else None, - reuse_pimcomp_report=report if args.only == "raptor" else None, + reuse_pimcomp_dir=( + spec.shared_pimcomp_dir + if reuse_shared_pimcomp and args.only != "raptor" and not args.raptor_only + else None + ), + reuse_pimcomp_report=report if args.only == "raptor" and not args.raptor_only else None, ) @@ -497,20 +531,29 @@ def config_path(arch: str, mode: str, sim_time_ms: int, *, write: bool) -> Path: def main() -> int: parser = argparse.ArgumentParser( - description="Compare supported PIMCOMP models with Raptor latency and throughput schedules." + description="Compare supported Pimcomp models with Raptor latency and throughput schedules." ) parser.add_argument( "--out-dir", type=Path, - help="Result root (default: artifacts beside each model under validation/).", + help="Suite root; generated artifacts go below each model's artifacts/ directory (default: validation/networks/pimcomp_models).", ) - parser.add_argument("--models", nargs="+", choices=MODELS, default=list(MODELS)) parser.add_argument( - "--arch", + "--common-dir", + type=Path, + help="Shared root for per-model reference artifacts. Default: inside --out-dir.", + ) + parser.add_argument( + "--ablation-variant", + help="Place Raptor artifacts below /ablation/ and write the comparison summary below ablation/.", + ) + add_models_argument(parser) + parser.add_argument( + "--archs", nargs="+", choices=ARCHES, - default=list(ARCHES), - help="PIM architectures to run (default: all architectures).", + default=list(DEFAULT_ARCHES), + help=f"Pim architectures to run (default: {', '.join(DEFAULT_ARCHES)}).", ) parser.add_argument( "--mode", @@ -524,6 +567,11 @@ def main() -> int: choices=("raptor", "pimcomp"), help="Re-run only this compiler's compile, validation, and simulation stages; preserve the other side from its report.", ) + parser.add_argument( + "--raptor-only", + action="store_true", + help="Run only Raptor; do not compile, validate, or simulate Pimcomp. Default: off.", + ) parser.add_argument( "--pipeline", type=int, @@ -534,7 +582,7 @@ def main() -> int: "--pimsim-time-ms", type=int, default=1000, - help="throughput pimsim-nn convergence deadline in ms (default: 1000).", + help="Throughput Pimsim convergence deadline in ms (default: 1000).", ) parser.add_argument( "--batch-size", @@ -564,15 +612,32 @@ def main() -> int: parser.add_argument( "--no-fast", action="store_true", - help="Disable fast pimsim-nn throughput convergence for authoritative experiments.", + help="Disable fast Pimsim throughput convergence for authoritative experiments.", + ) + parser.add_argument( + "--raptor-extra-arg", + action="append", + default=[], + dest="raptor_extra_args", + help="Additional argument to pass to Raptor; repeat as needed.", ) args = parser.parse_args() + if args.ablation_variant and Path(args.ablation_variant).name != args.ablation_variant: + parser.error("--ablation-variant must be a single directory name") + + if args.only is not None and args.raptor_only: + parser.error("--only cannot be combined with --raptor-only") + out_dir = args.out_dir.resolve() if args.out_dir is not None else None - args.arch = list(dict.fromkeys(args.arch)) + common_root = args.common_dir.resolve() if args.common_dir is not None else None + args.archs = list(dict.fromkeys(args.archs)) args.mode = list(dict.fromkeys(args.mode)) if args.clean: - print(f"Removed {clean_artifacts(out_dir, args.models, args.arch)} comparison artifact path(s).") + print( + f"Removed {clean_artifacts(out_dir, args.models, common_root)} " + "comparison artifact path(s)." + ) return 0 if args.jobs < 1: @@ -585,7 +650,7 @@ def main() -> int: parser.error("--timeout-seconds must be non-negative") comparisons_by_arch: dict[str, tuple[tuple[str, int, str], ...]] = {} configs_by_arch: dict[str, dict[str, Path]] = {} - for arch in args.arch: + for arch in args.archs: comparisons = tuple( comparison for comparison in COMPARISONS if comparison[0] in args.mode @@ -611,10 +676,12 @@ def main() -> int: if args.only is not None: missing_reuse = [] - for arch in args.arch: + for arch in args.archs: for name in args.models: for mode, pipeline, _ in comparisons_by_arch[arch]: - comparison_dir = result_dir(out_dir, name, arch, mode, pipeline) + comparison_dir = result_dir( + out_dir, name, arch, mode, pipeline, args.ablation_variant + ) required = comparison_dir / "pimcomp/comparison_report.json" if not required.exists(): missing_reuse.append(str(required)) @@ -629,14 +696,15 @@ def main() -> int: + ", ".join(missing_reuse) ) - validate_pimcomp_source() + if not args.raptor_only: + validate_pimcomp_source() if out_dir is not None and not args.dry_run: out_dir.mkdir(parents=True, exist_ok=True) - print(Style.BRIGHT + f"Found {len(args.models)} PIMCOMP model(s) to compare." + Style.RESET_ALL) - print(f"Architectures: {', '.join(args.arch)}") + print(Style.BRIGHT + f"Found {len(args.models)} Pimcomp model(s) to compare." + Style.RESET_ALL) + print(f"Architectures: {', '.join(args.archs)}") print(f"Modes: {', '.join(args.mode)}") - print(f"Throughput pimsim time: {args.pimsim_time_ms} ms") + print(f"Throughput Pimsim time: {args.pimsim_time_ms} ms") print(f"Max parallel jobs: {args.jobs}") print( f"Comparison jobs: " @@ -645,29 +713,36 @@ def main() -> int: print(f"Results root: {out_dir or SUITE}") print("=" * 72) - print_stage("Prepare shared artifacts", STAGE_COLORS["Build Runner"]) - for name in args.models: - try: - run( - prepare_common_command( - FUNCTIONAL_MODELS[name], - common_dir(out_dir, name), - args.timeout_seconds, - ), - dry_run=args.dry_run, - ) - finally: - print_completed(f"shared artifacts: {name}") + print_stage("Prepare shared artifacts", STAGE_COLORS["Build runner"]) + try: + for name in args.models: + try: + run( + prepare_common_command( + FUNCTIONAL_MODELS[name], + common_dir(out_dir, name, common_root), + args.timeout_seconds, + ), + dry_run=args.dry_run, + ) + finally: + print_completed(f"shared artifacts: {name}") + except KeyboardInterrupt: + remove_lock_files(out_dir or SUITE) + print("Interrupted; cleaned validation lock files.", file=sys.stderr) + return 130 failed = [] comparison_specs: list[ComparisonSpec] = [] shared_pimcomp_by_group: dict[tuple[str, str, str], Path] = {} - for arch in args.arch: + for arch in args.archs: comparisons = comparisons_by_arch[arch] configs = configs_by_arch[arch] for name in args.models: for mode, pipeline, pimcomp_pipeline in comparisons: - model_result_dir = result_dir(out_dir, name, arch, mode, pipeline) + model_result_dir = result_dir( + out_dir, name, arch, mode, pipeline, args.ablation_variant + ) label = f"{arch}/{name}/{mode}/pipeline{pipeline}" group = (arch, name, mode) shared_pimcomp_dir = shared_pimcomp_by_group.get(group) @@ -681,7 +756,7 @@ def main() -> int: model=MODELS[name], functional_model=FUNCTIONAL_MODELS[name], output_dir=model_result_dir, - common_dir=common_dir(out_dir, name), + common_dir=common_dir(out_dir, name, common_root), config=configs[mode], mode=mode, pipeline=pipeline, @@ -701,7 +776,8 @@ def main() -> int: if run(command, dry_run=True, check=False): failed.append(spec.label) elif comparison_specs: - anchor_specs = comparison_specs if args.only == "raptor" else [ + raptor_only_run = args.only == "raptor" or args.raptor_only + anchor_specs = comparison_specs if raptor_only_run else [ spec for spec in comparison_specs if spec.anchor ] anchor_jobs = [ @@ -712,64 +788,75 @@ def main() -> int: ) for spec in anchor_specs ] - dependent_specs = [] if args.only == "raptor" else [ + dependent_specs = [] if raptor_only_run else [ spec for spec in comparison_specs if not spec.anchor ] print_directly = min(args.jobs, len(comparison_specs)) == 1 - with (nullcontext(None) if print_directly else TemporaryDirectory(prefix="raptor-pimcomp-")) as log_dir: - anchor_failed, log_offset = run_comparison_jobs( - anchor_jobs, - args.jobs, - Path(log_dir) if log_dir else None, - 0, - ) - failed.extend(anchor_failed) - dependent_jobs = [ - ( - spec.label, - comparison_command_for( - spec, - args, - reuse_shared_pimcomp=pimcomp_artifact_ready(spec.shared_pimcomp_dir), - ), - None, + try: + with (nullcontext(None) if print_directly else TemporaryDirectory(prefix="raptor-pimcomp-")) as log_dir: + anchor_failed, log_offset = run_comparison_jobs( + anchor_jobs, + args.jobs, + Path(log_dir) if log_dir else None, + 0, ) - for spec in dependent_specs - ] - dependent_failed, _ = run_comparison_jobs( - dependent_jobs, - args.jobs, - Path(log_dir) if log_dir else None, - log_offset, - ) - failed.extend(dependent_failed) + failed.extend(anchor_failed) + dependent_jobs = [ + ( + spec.label, + comparison_command_for( + spec, + args, + reuse_shared_pimcomp=pimcomp_artifact_ready(spec.shared_pimcomp_dir), + ), + None, + ) + for spec in dependent_specs + ] + dependent_failed, _ = run_comparison_jobs( + dependent_jobs, + args.jobs, + Path(log_dir) if log_dir else None, + log_offset, + ) + failed.extend(dependent_failed) + except KeyboardInterrupt: + remove_lock_files(out_dir or SUITE) + print("Interrupted; cleaned validation lock files.", file=sys.stderr) + return 130 if args.dry_run: return 1 if failed else 0 + remove_lock_files(out_dir or SUITE) + results_path = None - for arch in args.arch: + for arch in args.archs: results_path = write_results_csv( out_dir, arch, args.models, comparisons_by_arch[arch], + args.ablation_variant, ) assert results_path is not None - print_stage(results_path.name, STAGE_COLORS["Compare Outputs"]) + print_stage(results_path.name, STAGE_COLORS["Compare outputs"]) print(results_path.read_text(encoding="utf-8"), end="") - for arch in args.arch: + for arch in args.archs: for name in args.models: for mode, pipeline, _ in comparisons_by_arch[arch]: label = f"{arch}/{name}/{mode}/pipeline{pipeline}" - report_path = result_dir(out_dir, name, arch, mode, pipeline) / "pimcomp/comparison_report.json" + report_path = result_dir( + out_dir, name, arch, mode, pipeline, args.ablation_variant + ) / "pimcomp/comparison_report.json" + compiler = "raptor" if args.raptor_only else args.only if not report_path.exists() or not comparison_passed( json.loads(report_path.read_text(encoding="utf-8")), - args.only, + compiler, ): if label not in failed: failed.append(label) - print("\n" + Style.BRIGHT + Fore.CYAN + "Summary" + Style.RESET_ALL) + print(Style.BRIGHT + Fore.CYAN + "Summary" + Style.RESET_ALL) total_jobs = sum(len(args.models) * len(comparisons) for comparisons in comparisons_by_arch.values()) print(Style.BRIGHT + f"Passed: {total_jobs - len(failed)}" + Style.RESET_ALL) print(Style.BRIGHT + f"Failed: {len(failed)}" + Style.RESET_ALL) diff --git a/validation/tools/pim/pimcomp/correctness/README.md b/validation/tools/pim/pimcomp/correctness/README.md index bbdf1e5..27d2702 100644 --- a/validation/tools/pim/pimcomp/correctness/README.md +++ b/validation/tools/pim/pimcomp/correctness/README.md @@ -1,6 +1,6 @@ -# PIMCOMP batch correctness reproduction +# Pimcomp batch correctness reproduction -PIMCOMP's batch scheduler currently emits an incomplete standalone program for +Pimcomp's batch scheduler currently emits an incomplete standalone program for models containing post operations. The generated `VerificationInfo.json` uses a negative `source_address` to identify the preceding node, but the batch verifier resolves that address by copying the provider tensor directly from @@ -17,8 +17,8 @@ the missing computation visible in Rust functional validation. For the checked-in GoogLeNet throughput/pipeline2 artifact, 39 provider tensors are referenced by batch loads. Nineteen have generated stores; twenty are never written. Preloading the provider tensors with the same ONNX Runtime -intermediates used by PIMCOMP's verifier makes the exported program pass. This -reproduces the verifier's input contract; it does not repair PIMCOMP's batch +intermediates used by Pimcomp's verifier makes the exported program pass. This +reproduces the verifier's input contract; it does not repair Pimcomp's batch schedule. Run the default reproduction from the repository root: @@ -27,23 +27,59 @@ Run the default reproduction from the repository root: .venv/bin/python validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py ``` -The launcher accepts an alternate comparison directory, model, and work -work directory, and shared reference-artifact directory: +The launcher accepts an alternate comparison directory, model, work directory, +and shared reference-artifact directory: ```bash .venv/bin/python validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py \ - validation/networks/pimcomp_models/googlenet/arch-a/throughput/pipeline2 \ + validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2 \ validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx \ - /tmp/pimcomp-prefill-googlenet \ - validation/networks/pimcomp_models/googlenet/common + validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2/correctness/prefill \ + validation/networks/pimcomp_models/googlenet/artifacts/common ``` +Without the optional work-directory argument, the experiment uses the same +`correctness/prefill/` directory below the comparison artifacts. + It runs the exported artifact once with its original memory image and once with [`prefill_batch_memory.py`](prefill_batch_memory.py), then compares both outputs with the recorded native reference. The expected GoogLeNet result is a baseline maximum difference near `6.70705` and a prefilled maximum difference near `4.05e-6`. -The issue is in PIMCOMP batch scheduling/validation semantics, not in the Rust +The issue is in Pimcomp batch scheduling/validation semantics, not in the Rust simulator's vector-length interpretation. Vector lengths remain element counts as specified by the reference ISA. + +## ResNet BatchNorm correctness gap + +Pimcomp has a separate correctness limitation in its ResNet element pipeline. +`BatchNormalization` is an ONNX operation. The frontend's +[`fuse_operators()` pass](../../../../../third_party/PIMCOMP-NN/frontend/frontend.py#L535) +marks it as fused and removes the node from the +scheduled graph, but the released path does not fold the BatchNorm affine +transform into the convolution weights and bias. The corresponding verifier +workaround in [`verification.py`](../../../../../third_party/PIMCOMP-NN/verification/verification.py#L99) +replaces BatchNorm parameters with identity values (scale and variance equal +to one, bias and mean equal to zero) before running ONNX Runtime. + +Therefore Pimcomp's native verifier and exported element program agree with +each other, but they do not implement the original ResNet ONNX model. On the +current ResNet-18 Arch-A latency artifact, using the same input: + +| Comparison | Maximum absolute difference | +|---|---:| +| Pimcomp native verifier vs Rust export | `6.7e-8` | +| Rust export vs original ONNX reference | `4.8294563` | + +The Pimcomp output ranges from approximately `-0.187` to `0.220`, while the +original ONNX output ranges from `-3.572` to `4.834`; 462 of 1000 final +elements differ by more than one. This is not a Rust simulator or Python +exporter regression. It is a Pimcomp model-semantics mismatch caused by +dropping BatchNorm numerics. The same issue affects the ResNet-34 latency +artifact. VGG and GoogLeNet do not show this particular mismatch because they +do not contain the same ResNet BatchNorm path. + +The latency comparison intentionally uses the original ONNX reference, so +these Pimcomp rows must remain `FAIL` until Pimcomp folds BatchNorm correctly +or the comparison explicitly uses a BatchNorm-neutralized reference. diff --git a/validation/tools/pim/pimcomp/correctness/prefill_batch_memory.py b/validation/tools/pim/pimcomp/correctness/prefill_batch_memory.py index 6c9bd51..1f69950 100644 --- a/validation/tools/pim/pimcomp/correctness/prefill_batch_memory.py +++ b/validation/tools/pim/pimcomp/correctness/prefill_batch_memory.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Populate the host-side intermediate buffers expected by PIMCOMP batch mode.""" +"""Populate the host-side intermediate buffers expected by Pimcomp batch mode.""" from __future__ import annotations @@ -23,7 +23,7 @@ def flatten_reference(value: np.ndarray) -> np.ndarray: elif value.ndim == 2: value = value.transpose() else: - raise ValueError(f"PIMCOMP batch verification only flattens 2D/4D tensors, got {value.shape}") + raise ValueError(f"Pimcomp batch verification only flattens 2D/4D tensors, got {value.shape}") return value.astype(np.float32, copy=False).reshape(-1) @@ -94,7 +94,7 @@ def prefill_batch_memory( session = ort.InferenceSession(runtime_model.SerializeToString(), providers=["CPUExecutionProvider"]) session_inputs = session.get_inputs() if len(session_inputs) != 1: - raise ValueError("PIMCOMP export currently requires exactly one runtime input tensor") + raise ValueError("Pimcomp export currently requires exactly one runtime input tensor") input_meta = session_inputs[0] input_tensor = np.loadtxt(input_path, delimiter=",", dtype=np.float32).reshape(input_meta.shape) provider_names = [node_list[index]["name"] for index in providers] diff --git a/validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py b/validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py index 4a88933..04da777 100755 --- a/validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py +++ b/validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Reproduce the PIMCOMP batch prefill correctness experiment.""" +"""Reproduce the Pimcomp batch prefill correctness experiment.""" from __future__ import annotations @@ -7,7 +7,6 @@ import argparse import json import subprocess import sys -import tempfile from pathlib import Path import numpy as np @@ -17,9 +16,9 @@ from prefill_batch_memory import prefill_batch_memory SCRIPT_DIR = Path(__file__).resolve().parent REPO_ROOT = SCRIPT_DIR.parents[4] -DEFAULT_COMPARISON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/arch-a/throughput/pipeline2" +DEFAULT_COMPARISON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2" DEFAULT_MODEL = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx" -DEFAULT_COMMON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/common" +DEFAULT_COMMON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/common" SIMULATOR_MANIFEST = REPO_ROOT / "backend-simulators/pim/pim-simulator/Cargo.toml" @@ -28,6 +27,7 @@ def run_simulator( memory: Path, output: Path, dump: str, + input_dir: Path, ) -> None: subprocess.run( [ @@ -44,6 +44,8 @@ def run_simulator( str(comparison_dir / "pimcomp/exported"), "--memory", str(memory), + "--input-dir", + str(input_dir), "-o", str(output), "-d", @@ -86,14 +88,18 @@ def main() -> int: model = args.model.resolve() common_dir = args.common_dir.resolve() if args.work_dir is None: - work_dir = Path(tempfile.mkdtemp(prefix="pimcomp-prefill.")) + work_dir = comparison_dir / "correctness/prefill" + work_dir.mkdir(parents=True, exist_ok=True) else: work_dir = args.work_dir.resolve() work_dir.mkdir(parents=True, exist_ok=True) input_path = common_dir / "inputs/in0.csv" - if not input_path.is_file(): - input_path = comparison_dir / "inputs/in0.csv" + simulator_input_dir = work_dir / "inputs" + simulator_input_dir.mkdir(parents=True, exist_ok=True) + np.loadtxt(input_path, delimiter=",", dtype=np.float32).tofile( + simulator_input_dir / "input_0.bin" + ) prefilled_memory = work_dir / "prefilled_memory.bin" metadata_path = work_dir / "metadata.json" @@ -117,8 +123,15 @@ def main() -> int: comparison_dir / "pimcomp/exported/memory.bin", baseline_output, dump, + simulator_input_dir, + ) + run_simulator( + comparison_dir, + prefilled_memory, + prefilled_output, + dump, + simulator_input_dir, ) - run_simulator(comparison_dir, prefilled_memory, prefilled_output, dump) compare_outputs(baseline_output, prefilled_output, reference, work_dir) return 0 diff --git a/validation/tools/pim/visuals/pim_sequence_diagram.py b/validation/tools/pim/visuals/pim_sequence_diagram.py index 033cd78..e4f0fbd 100644 --- a/validation/tools/pim/visuals/pim_sequence_diagram.py +++ b/validation/tools/pim/visuals/pim_sequence_diagram.py @@ -98,9 +98,9 @@ def read_binary(path: Path) -> Program: magic, version, count = HEADER.unpack_from(data) expected_size = HEADER.size + count * RECORD.size if magic != b"PIMB": - raise ValueError(f"{path}: invalid PIM binary magic") + raise ValueError(f"{path}: invalid Pim binary magic") if version != 1: - raise ValueError(f"{path}: unsupported PIM binary version {version}") + raise ValueError(f"{path}: unsupported Pim binary version {version}") if len(data) != expected_size: raise ValueError(f"{path}: expected {expected_size} bytes, found {len(data)}") @@ -399,7 +399,7 @@ def render_text(cores: list[int], transfers: list[Transfer], programs: dict[int, def main() -> int: parser = argparse.ArgumentParser( - description="Generate SequenceDiagram.org text for PIM communication and intervening work." + description="Generate SequenceDiagram.org text for Pim communication and intervening work." ) parser.add_argument("pim_dir", type=Path, help="Directory containing core_.json or core_.pim files") selection = parser.add_mutually_exclusive_group(required=True) diff --git a/validation/tools/yolo_local_image_validation.py b/validation/tools/yolo_local_image_validation.py index f2d242d..289a7c6 100644 --- a/validation/tools/yolo_local_image_validation.py +++ b/validation/tools/yolo_local_image_validation.py @@ -22,6 +22,7 @@ if sys.version_info < (3, 10): ) from raptor_validation.onnx_utils import _ONNX_TO_NP, onnx_io, write_inputs_binary, write_inputs_to_memory_bin +from raptor_validation.artifacts import artifacts_dir from raptor_validation.validate_one import ( MODE_COMPILE_ONLY, build_dump_ranges, @@ -64,16 +65,17 @@ def find_network_onnx(network_dir: Path) -> Path: def local_case_paths(network_dir: Path, case_name: str): + artifact_root = artifacts_dir(network_dir) return { - "root": network_dir, - "runner": network_dir / "runner" / "build" / "runner", - "runner_build": network_dir / "runner" / "build", - "raptor_pim": network_dir / "raptor" / "pim", - "real_root": network_dir / "real_image_validation", - "input_csv": network_dir / "real_image_validation" / "inputs" / f"{case_name}.csv", - "ref_dir": network_dir / "real_image_validation" / "reference" / case_name, - "sim_dir": network_dir / "real_image_validation" / "simulation" / case_name, - "sim_bin": network_dir / "real_image_validation" / "simulation" / case_name / "out.bin", + "root": artifact_root, + "runner": artifact_root / "runner" / "build" / "runner", + "runner_build": artifact_root / "runner" / "build", + "raptor_pim": artifact_root / "raptor" / "pim", + "real_root": artifact_root / "real_image_validation", + "input_csv": artifact_root / "real_image_validation" / "inputs" / f"{case_name}.csv", + "ref_dir": artifact_root / "real_image_validation" / "reference" / case_name, + "sim_dir": artifact_root / "real_image_validation" / "simulation" / case_name, + "sim_bin": artifact_root / "real_image_validation" / "simulation" / case_name / "out.bin", } @@ -102,10 +104,11 @@ def ensure_local_artifacts(args, network_onnx_path: Path): def ensure_existing_artifacts(network_dir: Path): + artifact_root = artifacts_dir(network_dir) required_paths = [ - network_dir / "runner" / "build" / "runner", - network_dir / "raptor" / "pim" / "config.json", - network_dir / "raptor" / "pim" / "memory.bin", + artifact_root / "runner" / "build" / "runner", + artifact_root / "raptor" / "pim" / "config.json", + artifact_root / "raptor" / "pim" / "memory.bin", ] missing = [str(path) for path in required_paths if not path.exists()] if missing: @@ -137,7 +140,7 @@ def run_local_reference_and_simulator(args, network_dir: Path, network_onnx_path tensor = np.loadtxt(paths["input_csv"], delimiter=",", dtype=np.float32).reshape(1, 3, 640, 640) write_inputs_to_memory_bin(paths["raptor_pim"] / "memory.bin", paths["raptor_pim"] / "config.json", [tensor]) - input_bin = paths["sim_dir"] / "input.bin" + input_bin = paths["sim_dir"] / "input_0.bin" write_inputs_binary(input_bin, [tensor]) dump_ranges = build_dump_ranges(paths["raptor_pim"] / "config.json", output_descriptors) @@ -147,7 +150,8 @@ def run_local_reference_and_simulator(args, network_dir: Path, network_onnx_path paths["sim_bin"], dump_ranges, timeout_sec=args.command_timeout_seconds, - input_paths=[input_bin], + input_dir=input_bin.parent, + batch_size=1, ) return paths, output_descriptors[0] @@ -225,7 +229,7 @@ def main(): parser.add_argument( "--annotated-dir", type=Path, - default=defaults["network_dir"] / "real_image_validation" / "annotated", + default=defaults["network_dir"] / "artifacts" / "real_image_validation" / "annotated", ) args = parser.parse_args() diff --git a/validation/tools/yolo_real_image_validation.py b/validation/tools/yolo_real_image_validation.py index 391a311..dd5be55 100644 --- a/validation/tools/yolo_real_image_validation.py +++ b/validation/tools/yolo_real_image_validation.py @@ -262,7 +262,7 @@ def ensure_remote_artifacts(args): def remote_case_paths(args, case_name: str): network_dir = Path(args.network_dir) - root = Path(args.remote_project) / network_dir + root = Path(args.remote_project) / network_dir / "artifacts" return { "root": root, "runner": root / "runner" / "build" / "runner", @@ -272,7 +272,7 @@ def remote_case_paths(args, case_name: str): "input_csv": root / "real_image_validation" / "inputs" / f"{case_name}.csv", "ref_dir": root / "real_image_validation" / "reference" / case_name, "sim_dir": root / "real_image_validation" / "simulation" / case_name, - "sim_input": root / "real_image_validation" / "simulation" / case_name / "input.bin", + "sim_input": root / "real_image_validation" / "simulation" / case_name / "input_0.bin", "sim_bin": root / "real_image_validation" / "simulation" / case_name / "out.bin", } @@ -331,7 +331,7 @@ PY f"cd {quoted_project}/backend-simulators/pim/pim-simulator && " f"cargo run --no-default-features --release --package pim-simulator --bin pim-simulator -- " f"-f {quoted_pim} -o {quoted_sim_bin} -d {dump_range} " - f"--mode latency --input {shlex.quote(str(paths['sim_input']))}" + f"--mode latency --batch-size 1 --input-dir {shlex.quote(str(paths['sim_input'].parent))}" ) remote_bash(args.ssh_key, args.remote_host, sim_command) return paths @@ -408,7 +408,7 @@ def main(): parser.add_argument("--core-count", type=int, default=144) parser.add_argument("--command-timeout-seconds", type=int, default=7200) parser.add_argument("--skip-compile", action="store_true") - parser.add_argument("--annotated-dir", default="validation/networks/yolo11n/depth_51/real_image_validation/annotated") + parser.add_argument("--annotated-dir", default="validation/networks/yolo11n/depth_51/artifacts/real_image_validation/annotated") args = parser.parse_args() args.ssh_key = str(Path(args.ssh_key).expanduser()) diff --git a/validation/validate.py b/validation/validate.py index 7143615..48576d5 100755 --- a/validation/validate.py +++ b/validation/validate.py @@ -133,7 +133,7 @@ def print_average_pim_pass_timings(pass_timing_sums, pass_timing_counts, total_t if timed_benchmark_count == 0: return - print("\n" + Style.BRIGHT + Fore.CYAN + "Average PIM Pass Timings" + Style.RESET_ALL) + print("\n" + Style.BRIGHT + Fore.CYAN + "Average Pim pass timings" + Style.RESET_ALL) for _, label in PIM_PASS_LABELS: count = pass_timing_counts[label] if count == 0: @@ -210,7 +210,7 @@ def main(): ap.add_argument("--onnx-include-dir", help="Path to OnnxMlirRuntime include directory.") ap.add_argument("--operations-dir", default=None, help="Root of the operations tree (default: operations).") ap.add_argument("--simulator-dir", default=None, - help="Path to the functional pim-simulator crate root " + help="Path to the functional Pim simulator crate root " "(default: auto-detected relative to script).") ap.add_argument("--non-functional-simulator-build-dir", metavar="PATH", default=None, help="Path to the non-functional simulator build directory " @@ -221,7 +221,7 @@ def main(): ap.add_argument("--skip-non-functional-simulation", action="store_true", help="Skip non-functional simulation.") ap.add_argument("--no-fast", action="store_true", - help="Disable fast pimsim-nn throughput convergence for authoritative experiments.") + help="Disable fast Pimsim throughput convergence for authoritative experiments.") ap.add_argument("--threshold", type=float, default=1e-3, help="Absolute tolerance for per-element output comparison.") ap.add_argument("--relative-threshold", type=float, default=1e-5, @@ -243,7 +243,7 @@ def main(): help="Remove generated validation artifacts under each model workspace and exit.") mode_group = ap.add_mutually_exclusive_group() mode_group.add_argument("--compile-only", action="store_true", - help="Compile reference and PIM artifacts only; do not run reference execution, " + help="Compile reference and Pim artifacts only; do not run reference execution, " "simulations, or comparison.") mode_group.add_argument("--run-only", action="store_true", help="Reuse existing compiled artifacts and only run inputs, reference execution, " @@ -495,7 +495,7 @@ def main(): pimsim_unsupported = pimsim_statuses.count(PIMSIM_UNSUPPORTED) print( Style.BRIGHT - + f"pimsim-nn: {pimsim_statuses.count(PIMSIM_DONE)} measured, " + + f"Pimsim: {pimsim_statuses.count(PIMSIM_DONE)} measured, " f"{pimsim_failed} failed, {pimsim_unsupported} unsupported, " f"{pimsim_skipped} skipped" + Style.RESET_ALL