nnicolosi/Raptor
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

add ablation study
Validate Operations / validate-operations (push) Has been cancelled
Details

Browse Source 
normalize names and artifact paths
This commit is contained in:
NiccoloN
2026-08-20 17:58:02 +02:00
parent add20e56eb
commit b009e1ff08
67 changed files with 1573 additions and 993 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+92 -34
View File
@@ -5,7 +5,7 @@ targeting in-memory computing / processing-in-memory (PIM) architectures. It
extends ONNX-MLIR with a PIM accelerator and progressively lowers ONNX-MLIR
through custom MLIR dialects to simulator artifacts.
The current target is the PIM simulator stack under `backend-simulators/pim`.
The current target is the Pim simulator stack under `backend-simulators/pim`.
Raptor emits binary per-core `.pim` instruction files by default, plus
`memory.bin`, `config.json`, and weight binaries. It can also emit per-core JSON
instruction files with `--pim-emit-json`.
@@ -29,9 +29,9 @@ lowering, scheduling, memory layout, and code-generation optimizations.
- `backend-simulators/pim/pim-simulator` is the in-tree Rust functional
simulator used by validation. It reads Raptor's `pim/` artifact directory and
compares simulator output against native ONNX-MLIR execution.
- `backend-simulators/pim/pimsim-nn` is the non-functional simulator submodule
used internally by validation for latency, power, and energy.
The helper scripts in `pimcomp_utils/` are for comparison with PIMCOMP-NN and
- `backend-simulators/pim/pimsim-nn` contains the non-functional Pimsim
simulator used internally by validation for latency, power, and energy.
The helper scripts in `pimcomp_utils/` are for comparison with Pimcomp and
contain local paths; treat them as local utilities, not portable workflows.
## Compilation pipeline
@@ -43,7 +43,7 @@ them to ONNX-MLIR through generated shim directories under
High-level lowering flow:
```
ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> PIM artifacts
ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> Pim artifacts
```
1. **ONNX -> Spatial** (`src/PIM/Conversion/ONNXToSpatial`).
@@ -81,20 +81,20 @@ ONNX-MLIR -> Spatial -> Pim (tensor) -> Pim (bufferized) -> PIM artifacts
addressable accesses, and `PimBufferizationVerification` checks tensor
absence, contiguity, and copy address spaces.
5. **PIM local-memory planning**
5. **Pim local-memory planning**
(`src/PIM/Dialect/Pim/Passes/Transforms/LocalMemoryPlanning`).
Computes whole-core lifetimes, reuses addresses for non-overlapping
allocations, and records the explicit plan in PIM IR. Reusable lifetime
allocations, and records the explicit plan in Pim IR. Reusable lifetime
analysis lives under `src/PIM/Dialect/Pim/Passes/Analyses`.
6. **PIM verification and code generation** (`src/PIM/Passes/PimCodegen` and
6. **Pim verification and code generation** (`src/PIM/Passes/PimCodegen` and
`src/PIM/Compiler`).
Verifies the memory plan and other PIM invariants, then emits `.pim` core
Verifies the memory plan and other Pim invariants, then emits `.pim` core
files, weights, and `memory.bin` / `config.json` without rerunning liveness.
Supporting pieces:
- `src/PIM/Common` - shared IR, filesystem, diagnostics, reports, and utility
helpers.
- `src/PIM/Compiler` - PIM compiler options, planned-address materialization, binary
- `src/PIM/Compiler` - Pim compiler options, planned-address materialization, binary
instruction format, artifact writing, weight emission, and codegen entry
points.
- `src/PIM/Conversion/SpatialToGraphviz` - optional Spatial graphviz conversion
@@ -102,40 +102,97 @@ Supporting pieces:
- `src/PIM/Passes` - pass registration and auxiliary passes.
- `src/PIM/PimAccelerator.{cpp,hpp}` - ONNX-MLIR accelerator entry point.
## PIM compiler options
## Pim compiler options
Pass these to `onnx-mlir` when compiling for PIM. These are all Raptor/PIM-specific
Pass these to `onnx-mlir` when compiling for Pim. These are all Raptor/Pim-specific
options; `onnx-mlir --help` lists the inherited ONNX-MLIR options.
- `--maccel=PIM` - select the PIM accelerator.
- `--maccel=PIM` - select the Pim accelerator. Default: no Pim accelerator.
- `--EmitSpatial`, `--EmitPim`, `--EmitPimBufferized`,
`--EmitPimCodegen` - stop the PIM pipeline at the requested stage. The PIM
default is `--EmitPimCodegen`.
- `--core-count=<N>` - required positive core count for PIM compilation.
- `--crossbar-size=<N>` - crossbar width/height. Default in code is `128`.
- `--crossbar-count=<N>` - crossbars per core. Default in code is `64`.
- `--pim-target-config=<PATH>` - optional PIM target configuration used by the
`--EmitPimCodegen` - stop the Pim pipeline at the requested stage. Default:
`--EmitPimCodegen` for Pim compilation.
- `--core-count=<N>` - required positive core count for Pim compilation.
Default: none; this option is required.
- `--crossbar-size=<N>` - required positive crossbar width/height for Pim
compilation. Default: none; this option is required.
- `--crossbar-count=<N>` - required positive crossbar count per core for Pim
compilation. Default: none; this option is required.
- `--pipeline=<N>` - number of throughput pipeline stages; `1` preserves
latency scheduling. Default: `1`.
- `--pim-target-config=<PATH>` - optional Pim target configuration used by the
target adapter to construct the target-neutral Spatial scheduling cost and
topology model. Resource values must match the explicit core/crossbar flags.
Default: empty; use the built-in target model.
- `--pim-memory-report=<summary|none>` - emit the concise combined memory report
under `reports/memory_report.txt`, or disable it. Default is `summary`.
- `--pim-only-codegen` - assume input is already bufferized PIM IR and only run
the codegen tail.
under `reports/memory_report.txt`, or disable it. Default: `summary`.
- `--pim-only-codegen` - assume input is already bufferized Pim IR and only run
the codegen tail. Default: off.
- `--pim-disable-synchronization` - omit generated `wait` and `sync`
instructions for performance ablation. Default: off.
- `--pim-disable-spatial-planning` - select the first, trivial DenseNCHW layout
alternative for every Spatial plan operation, disabling cost-based layout
planning while leaving ONNX rewrites and graph-compute merging enabled.
Default: off.
### Spatial layout plan variants
Spatial plan operations advertise alternatives as an exact combination of
operand physical layouts and one result physical layout. Every plan operation
has the default `DenseNCHW -> DenseNCHW` alternative. The planner can select
the following additional variants when the operation, tensor shapes, and
target resources make them legal:
| Physical layout or plan | Meaning and current use |
|---|---|
| `DenseNCHW` | Ordinary dense NCHW storage. This is the first alternative and the one selected by `--pim-disable-spatial-planning`. |
| `NHWCRowStrip` | Row-strip storage for NCHW logical tensors: spatial rows are processed as channel vectors. This enables row-strip lowering through compatible chains. |
| `Fragmented` | Fragmented physical input accepted by `Flatten`, which reassembles it to dense NCHW. It is not currently selected as a plan result. |
| `NCHWRowStrip` | A Spatial IR layout enum value reserved for NCHW-oriented row strips; current layout-capability implementations do not advertise it as a plan alternative. |
The operation-specific non-trivial alternatives are:
| Plan operation | Additional alternatives beyond dense NCHW |
|---|---|
| `Conv2D` | Dense input to row-strip output, or row-strip input to row-strip output when the target-dependent Conv lowering supports it. |
| `Flatten` | Fragmented input to dense output, or row-strip input to dense output when legal. |
| `Relu` | Row-strip input to row-strip output. |
| `SiLU` | Row-strip input to row-strip output, with a stronger intrinsic cost preference than the generic row-strip variant. |
| `ResizeNearest` | Row-strip input to row-strip output when its lowering is legal. |
| `MaxPool2D` | Dense input to row-strip output, or row-strip input to row-strip output. |
| `GlobalAveragePool` | Dense input to row-strip output, or row-strip input to row-strip output. |
| `BiasAdd` | Row-strip data input plus a dense bias input to row-strip output when the bias shape is supported. |
| `Add` | All data inputs row-strip to row-strip output. |
| `Concat` | All inputs row-strip to row-strip output. |
Cost-based planning scores intrinsic alternative cost, operand layout
mismatches, and downstream incompatibility, then iterates in alternating
forward and reverse operation order until the bounded analysis converges.
Function results are required to remain `DenseNCHW`; explicit materialization
operations reconcile layout mismatches at boundaries. With
`--pim-disable-spatial-planning`, the pass still runs and records a valid plan,
but chooses the first dense alternative for every plan operation. Later graph
compute merging is unchanged, so elementwise operations such as `Relu` remain
separate from neighboring parallel operations and can create fan-out/fan-in
diamonds.
- `--pim-emit-json` - also emit `core_*.json` instruction files alongside
`core_*.pim`.
`core_*.pim`. Default: off.
- `--pim-export-spatial-dataflow=<none|spatial1|spatial2|spatial3|spatial4|all>` -
control Spatial dataflow CSV reports for the graph, trivially merged graph,
scheduled, and realized snapshots under `reports/`. Default is `none`.
scheduled, and realized snapshots under `reports/`. Default: `none`.
- `--pim-conv-lowering=<auto|legacy|depthwise|packed-im2col|streamed-patch|streamed-packed|output-channel-tiled|input-k-tiled|tiled-2d>` -
select the convolution lowering strategy. Default is `auto`.
select the convolution lowering strategy. Default: `auto`.
- `--pim-conv-im2col-max-elements=<N>` - maximum globally materialized im2col
elements per convolution before streaming. Default is `1048576`.
elements per convolution before streaming. Default: `1048576`.
- `--pim-conv-stream-chunk-positions=<N>` - maximum output positions per
streamed convolution chunk. Default is `1024`.
streamed convolution chunk. Default: `1024`.
- `--pim-report-conv-lowering=<true|false>` - emit a bounded convolution
lowering report. Default: `true`.
- `--pim-detect-communication-deadlock` - statically simulate expanded
send/receive ordering and reject blocking deadlocks. Default is off.
send/receive ordering and reject blocking deadlocks. Default: off.
- `--pim-verify-bufferization-copy-freedom` - run the expensive official Pim
tensor-copy freedom proof before bufferization. Default: off.
## Standard PIM hardware profile
## Standard Pim hardware profile
Raptor's standard development and YOLO validation profile is:
@@ -149,7 +206,8 @@ Canonical compiler flags:
`--crossbar-count=64 --crossbar-size=128 --core-count=144`
`--core-count` remains mandatory and must be passed explicitly to the compiler.
`--crossbar-size`, `--crossbar-count`, and `--core-count` remain mandatory and
must be passed explicitly to the compiler.
Example:
@@ -159,11 +217,11 @@ Example:
--crossbar-count=64 --crossbar-size=128 --core-count=144
```
This writes PIM artifacts under `/tmp/raptor/pim/`.
This writes Pim artifacts under `/tmp/raptor/pim/`.
## Validation
Functional validation compiles ONNX models, compares native ONNX-MLIR and PIM
Functional validation compiles ONNX models, compares native ONNX-MLIR and Pim
simulator outputs, and optionally reports latency, power, and energy. See
[`validation/README.md`](validation/README.md) for prerequisites, usage,
options, artifacts, and results.
@@ -276,7 +334,7 @@ cd backend-simulators/pim/pim-simulator
cargo test
```
## Repository Layout
## Repository layout
- `src/PIM/` - PIM accelerator implementation.
- `test/PIM/` - PIM C++ unit tests.
@@ -284,6 +342,6 @@ cargo test
slices, and pimsim config generation.
- `backend-simulators/pim/pim-simulator/` - in-tree Rust functional simulator.
- `backend-simulators/pim/pimsim-nn/` - non-functional simulator submodule.
- `pimcomp_utils/` - local comparison helpers for PIMCOMP-NN.
- `pimcomp_utils/` - local comparison helpers for Pimcomp.
- `.github/actions/` and `.github/workflows/validate_operations.yml` - CI setup
for MLIR/Protobuf caching, building Raptor, and validation.
backend-simulators/pim/pim-simulator/src/bin/pim-simulator/main.rs
+6 -13
View File
@@ -53,9 +53,9 @@ struct Args {
#[arg(long)]
batch_size: Option<u32>,
/// Input binary for one iteration; repeat once per batch entry
#[arg(long = "input")]
inputs: Vec<PathBuf>,
/// Directory containing input_*.bin files, one per batch entry
#[arg(long = "input-dir")]
input_dir: PathBuf,
/// Optional directory for per-iteration output dumps
#[arg(long)]
@@ -142,16 +142,9 @@ fn input_regions(config: &Value) -> Result<Vec<(usize, usize)>> {
}
fn retrieve_inputs(args: &Args, batch_size: u32) -> Result<Vec<Vec<u8>>> {
if args.inputs.len() != batch_size as usize {
bail!(
"batch size {batch_size} requires {} inputs, got {}",
batch_size,
args.inputs.len()
);
}
args.inputs
.iter()
.map(|path| fs::read(path).with_context(|| format!("Failed to read input file: {path:?}")))
(0..batch_size)
.map(|index| args.input_dir.join(format!("input_{index}.bin")))
.map(|path| fs::read(&path).with_context(|| format!("Failed to read input file: {path:?}")))
.collect()
}
backend-simulators/pim/pim-simulator/src/lib/binary_to_instruction/mod.rs
+4 -4
View File
@@ -80,19 +80,19 @@ fn read_i32_le(bytes: &[u8], offset: usize) -> i32 {
fn parse_binary_records(bytes: &[u8]) -> Result<Vec<InstructionRecord>> {
ensure!(bytes.len() >= HEADER_SIZE, "binary core file too small");
ensure!(&bytes[0..4] == MAGIC, "invalid PIM binary magic");
ensure!(&bytes[0..4] == MAGIC, "invalid Pim binary magic");
let version = read_u32_le(bytes, 4);
ensure!(
version == VERSION,
"unsupported PIM binary version {version}"
"unsupported Pim binary version {version}"
);
let instruction_count = read_u32_le(bytes, 8) as usize;
let expected_len = HEADER_SIZE + instruction_count * RECORD_SIZE;
ensure!(
bytes.len() == expected_len,
"PIM binary size mismatch: expected {expected_len} bytes, got {}",
"Pim binary size mismatch: expected {expected_len} bytes, got {}",
bytes.len()
);
@@ -335,7 +335,7 @@ fn append_record(
.set_offset_select_value(generic1, 0);
inst_builder.make_inst(sync, inst_data_builder.build());
}
_ => bail!("unsupported PIM binary opcode {opcode}"),
_ => bail!("unsupported Pim binary opcode {opcode}"),
}
Ok(())
}
backend-simulators/pim/pim-simulator/src/lib/pimcore.rs
+1 -1
View File
@@ -285,7 +285,7 @@ impl<'a> Executable<'a> {
pub fn dump(&self) {
let core_instructions = &self.core_instructions;
for (i, core_instruction) in core_instructions.iter().enumerate() {
eprintln!("INST OF CORE {}:", i);
eprintln!("Instructions for core {}:", i);
for inst in &core_instruction.instructions {
inst.dump();
}
src/PIM/Common/IR/AddressAnalysis.hpp
+1 -1
View File
@@ -19,7 +19,7 @@ struct ResolvedContiguousAddress {
};
/// Records compile-time facts used when interpreting address arithmetic and
/// loop-carried aliases inside PIM regions.
/// loop-carried aliases inside Pim regions.
struct StaticValueKnowledge {
llvm::DenseMap<mlir::Value, int64_t> indexValues;
llvm::DenseMap<mlir::Value, mlir::Value> aliases;
src/PIM/Common/IR/CoreBlockUtils.cpp
+4 -4
View File
@@ -85,12 +85,12 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block,
auto step = resolveIndexValue(forOp.getStep(), knowledge);
if (failed(lower) || failed(upper) || failed(step)
|| (mode == CoreWalkMode::ExecuteCommunication && *step <= 0)) {
forOp.emitOpError() << "requires statically evaluable scf.for bounds for PIM " << purpose;
forOp.emitOpError() << "requires statically evaluable scf.for bounds for Pim " << purpose;
hasFailure = true;
continue;
}
if (*step <= 0) {
forOp.emitOpError("requires positive scf.for step for PIM verification");
forOp.emitOpError("requires positive scf.for step for Pim verification");
hasFailure = true;
continue;
}
@@ -126,7 +126,7 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block,
if (auto ifOp = mlir::dyn_cast<mlir::scf::IfOp>(op)) {
auto condition = resolveIndexValue(ifOp.getCondition(), knowledge);
if (failed(condition)) {
ifOp.emitOpError() << "requires statically evaluable scf.if condition for PIM " << purpose;
ifOp.emitOpError() << "requires statically evaluable scf.if condition for Pim " << purpose;
hasFailure = true;
continue;
}
@@ -147,7 +147,7 @@ static mlir::LogicalResult walkPimCoreBlockImpl(mlir::Block& block,
if (auto switchOp = mlir::dyn_cast<mlir::scf::IndexSwitchOp>(op)) {
auto selector = resolveIndexValue(switchOp.getArg(), knowledge);
if (failed(selector)) {
switchOp.emitOpError() << "requires a statically evaluable scf.index_switch selector for PIM " << purpose;
switchOp.emitOpError() << "requires a statically evaluable scf.index_switch selector for Pim " << purpose;
hasFailure = true;
continue;
}
src/PIM/Common/IR/CoreBlockUtils.hpp
+1 -1
View File
@@ -14,7 +14,7 @@ namespace onnx_mlir {
using PimCoreCommunicationPlan = llvm::DenseMap<mlir::Block*, llvm::SmallVector<mlir::Operation*, 8>>;
/// Returns true for ops in a `pim.core` body that only participate in static
/// address or index computation and therefore do not emit PIM instructions.
/// address or index computation and therefore do not emit Pim instructions.
bool isCoreStaticAddressOp(mlir::Operation* op);
/// Walks a `pim.core` body's communication stream, statically unrolling
src/PIM/Common/IR/EntryPointUtils.cpp
+2 -2
View File
@@ -9,7 +9,7 @@ llvm::FailureOr<mlir::func::FuncOp> getPimEntryFunc(mlir::ModuleOp moduleOp) {
llvm::SmallVector<mlir::ONNXEntryPointOp> entryPoints(moduleOp.getOps<mlir::ONNXEntryPointOp>());
if (entryPoints.size() > 1) {
moduleOp.emitError("PIM pipeline requires a single ONNX entry point, but found ") << entryPoints.size();
moduleOp.emitError("Pim pipeline requires a single ONNX entry point, but found ") << entryPoints.size();
return mlir::failure();
}
if (!entryPoints.empty()) {
@@ -38,7 +38,7 @@ llvm::FailureOr<mlir::func::FuncOp> getPimEntryFunc(mlir::ModuleOp moduleOp) {
if (nonExternalFuncs.size() == 1)
return nonExternalFuncs.front();
moduleOp.emitError("could not resolve a unique PIM entry function");
moduleOp.emitError("could not resolve a unique Pim entry function");
return mlir::failure();
}
src/PIM/Common/IR/EntryPointUtils.hpp
+1 -1
View File
@@ -5,7 +5,7 @@
namespace onnx_mlir {
/// Resolves the function the PIM pipeline should treat as its entry point.
/// Resolves the function the Pim pipeline should treat as its entry point.
/// Prefers ONNX entry-point metadata, then `main_graph`, then the only
/// non-external function if the module is otherwise unambiguous.
llvm::FailureOr<mlir::func::FuncOp> getPimEntryFunc(mlir::ModuleOp moduleOp);
src/PIM/Common/IR/WeightUtils.hpp
+1 -1
View File
@@ -32,7 +32,7 @@ struct ResolvedWeightView {
bool hasWeightAlways(mlir::Operation* op);
/// Tags an op as producing a value that should stay materialized as a reusable
/// weight across later PIM lowering/codegen stages.
/// weight across later Pim lowering/codegen stages.
void markWeightAlways(mlir::Operation* op);
bool isSpatialMvmVmmWeightUse(mlir::OpOperand& use);
src/PIM/Common/Support/CheckedArithmetic.cpp
+8 -8
View File
@@ -11,7 +11,7 @@ namespace onnx_mlir::pim {
namespace {
static void emitCrashMessage(llvm::StringRef fieldName, llvm::StringRef message) {
llvm::errs() << "PIM " << fieldName << " " << message << "\n";
llvm::errs() << "Pim " << fieldName << " " << message << "\n";
}
template <typename To, typename From>
@@ -65,7 +65,7 @@ InFlightDiagnostic emitCheckedArithmeticError(Operation* anchor, llvm::StringRef
}
InFlightDiagnostic emitCheckedArithmeticError(Location loc, llvm::StringRef fieldName, llvm::StringRef message) {
return emitError(loc) << "PIM " << fieldName << " " << message;
return emitError(loc) << "Pim " << fieldName << " " << message;
}
FailureOr<int32_t> checkedI32(int64_t value, Operation* anchor, llvm::StringRef fieldName) {
@@ -174,7 +174,7 @@ FailureOr<uint64_t> getCheckedShapedTypeSizeInBytes(ShapedType type, Location lo
int32_t checkedI32OrCrash(int64_t value, llvm::StringRef fieldName) {
if (value < std::numeric_limits<int32_t>::min() || value > std::numeric_limits<int32_t>::max()) {
emitCrashMessage(fieldName, "is outside representable range");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return static_cast<int32_t>(value);
}
@@ -182,7 +182,7 @@ int32_t checkedI32OrCrash(int64_t value, llvm::StringRef fieldName) {
int32_t checkedI32OrCrash(uint64_t value, llvm::StringRef fieldName) {
if (value > static_cast<uint64_t>(std::numeric_limits<int32_t>::max())) {
emitCrashMessage(fieldName, "is outside representable range");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return static_cast<int32_t>(value);
}
@@ -190,7 +190,7 @@ int32_t checkedI32OrCrash(uint64_t value, llvm::StringRef fieldName) {
uint8_t checkedU8OrCrash(uint64_t value, llvm::StringRef fieldName) {
if (value > static_cast<uint64_t>(std::numeric_limits<uint8_t>::max())) {
emitCrashMessage(fieldName, "is outside representable range");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return static_cast<uint8_t>(value);
}
@@ -198,7 +198,7 @@ uint8_t checkedU8OrCrash(uint64_t value, llvm::StringRef fieldName) {
size_t checkedSizeOrCrash(int64_t value, llvm::StringRef fieldName) {
if (value < 0) {
emitCrashMessage(fieldName, "is outside representable range");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return static_cast<size_t>(value);
}
@@ -206,7 +206,7 @@ size_t checkedSizeOrCrash(int64_t value, llvm::StringRef fieldName) {
size_t checkedAddOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) {
if (rhs > std::numeric_limits<size_t>::max() - lhs) {
emitCrashMessage(fieldName, "addition overflow");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return lhs + rhs;
}
@@ -214,7 +214,7 @@ size_t checkedAddOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) {
size_t checkedMulOrCrash(size_t lhs, size_t rhs, llvm::StringRef fieldName) {
if (lhs != 0 && rhs > std::numeric_limits<size_t>::max() / lhs) {
emitCrashMessage(fieldName, "multiplication overflow");
llvm_unreachable("PIM checked arithmetic failure");
llvm_unreachable("Pim checked arithmetic failure");
}
return lhs * rhs;
}
src/PIM/Common/Support/FileSystemUtils.hpp
+1 -1
View File
@@ -4,7 +4,7 @@
namespace onnx_mlir {
/// Returns the directory that should hold PIM artifacts/debug dumps for the
/// Returns the directory that should hold Pim artifacts/debug dumps for the
/// current compiler invocation.
std::string getOutputDir();
src/PIM/Compiler/PimBinaryFormat.hpp
+3 -3
View File
@@ -171,19 +171,19 @@ inline Opcode opcodeFromString(llvm::StringRef opName) {
for (auto [index, name] : llvm::enumerate(kOpcodeNames))
if (opName == name)
return static_cast<Opcode>(index);
llvm_unreachable("Unsupported PIM binary opcode");
llvm_unreachable("Unsupported Pim binary opcode");
}
inline llvm::StringRef opcodeToString(Opcode opcode) {
size_t index = static_cast<size_t>(opcode);
assert(index < kOpcodeNames.size() && "Unsupported PIM binary opcode");
assert(index < kOpcodeNames.size() && "Unsupported Pim binary opcode");
return kOpcodeNames[index];
}
inline InstructionRecord makeInstructionRecord(const llvm::json::Object& instruction) {
InstructionRecord record;
std::optional<llvm::StringRef> opName = instruction.getString("op");
assert(opName && "Missing op field in PIM instruction");
assert(opName && "Missing op field in Pim instruction");
record.opcode = opcodeFromString(*opName);
const auto& format = kInstructionJsonFormats[static_cast<size_t>(record.opcode)];
if (format.rd)
src/PIM/Compiler/PimCodeGen.cpp
+19 -15
View File
@@ -125,7 +125,7 @@ static bool isZeroSplatGlobal(mlir::Value value) {
return false;
}
// PIM instruction immediates are serialized as signed int32_t fields today
// Pim instruction immediates are serialized as signed int32_t fields today
// (`sldi` goes through checkedI32OrCrash), so local addresses must stay within
// the non-negative int32_t range.
static FailureOr<size_t> checkedAlignTo(size_t value, size_t alignment, Operation* anchor, StringRef fieldName) {
@@ -141,7 +141,7 @@ static void printMemoryOverflowDiagnostic(const MemoryValueKey& key,
size_t requestedSize,
size_t currentFirstAvailableAddress,
size_t alignedEndAddress) {
llvm::errs() << "PIM local memory allocation overflow\n";
llvm::errs() << "Pim local memory allocation overflow\n";
llvm::errs() << "Requested allocation size: " << requestedSize << " bytes\n";
llvm::errs() << "Current firstAvailableAddress: " << currentFirstAvailableAddress << "\n";
llvm::errs() << "Aligned end address: " << alignedEndAddress << "\n";
@@ -187,7 +187,7 @@ size_t PimMemory::allocateAddress(size_t size, const MemoryValueKey& key) {
size,
firstAvailableAddress,
succeeded(checkedAlignedEnd) ? *checkedAlignedEnd : kPimLocalMemoryAddressLimit);
llvm_unreachable("PIM local memory allocation overflow");
llvm_unreachable("Pim local memory allocation overflow");
}
firstAvailableAddress = *checkedAlignedEnd;
return address;
@@ -276,7 +276,7 @@ void PimMemory::allocateCore(const CompiledCoreMemoryPlan& plan, std::optional<u
}
else if (*localArenaSize != plan.arenaSize || reportRow.logicalLocalAllocationCount != plan.logicalAllocationCount
|| reportRow.logicalLocalBytes != plan.logicalBytes)
llvm_unreachable("inconsistent PIM local-memory plan across core-batch lanes");
llvm_unreachable("inconsistent Pim local-memory plan across core-batch lanes");
for (const CompiledLocalMemoryEntry& entry : plan.entries) {
MemoryValueKey key = getMemoryValueKey(entry.value, lane);
ownedMemEntriesMap[key] = entry.memory;
@@ -352,8 +352,8 @@ size_t PimAcceleratorMemory::getValueAddress(mlir::Value value,
llvm_unreachable("Missing mem entry");
}
size_t byteOffset = pim::checkedSizeOrCrash(resolvedAddress->byteOffset, "resolved PIM byte offset");
return pim::checkedAddOrCrash(iter->second.address, byteOffset, "resolved PIM address");
size_t byteOffset = pim::checkedSizeOrCrash(resolvedAddress->byteOffset, "resolved Pim byte offset");
return pim::checkedAddOrCrash(iter->second.address, byteOffset, "resolved Pim address");
}
llvm::FailureOr<int64_t> PimAcceleratorMemory::getIndexValue(mlir::Value value,
@@ -706,6 +706,8 @@ void PimCodeGen::codeGenSendOp(pim::PimSendOp sendOp, const StaticValueKnowledge
void PimCodeGen::codeGenWaitOp(
pim::PimWaitOp waitOp, const StaticValueKnowledge& knowledge) const {
if (pimDisableSynchronization)
return;
auto eventRegister = indexOf(waitOp.getEventRegister(), knowledge);
auto waitValue = indexOf(waitOp.getWaitValue(), knowledge);
assert(succeeded(eventRegister) && succeeded(waitValue)
@@ -722,6 +724,8 @@ void PimCodeGen::codeGenWaitOp(
void PimCodeGen::codeGenSyncOp(
pim::PimSyncOp syncOp, const StaticValueKnowledge& knowledge) const {
if (pimDisableSynchronization)
return;
auto targetCoreId = indexOf(syncOp.getTargetCoreId(), knowledge);
auto eventRegister = indexOf(syncOp.getEventRegister(), knowledge);
assert(succeeded(targetCoreId) && succeeded(eventRegister)
@@ -958,7 +962,7 @@ static LogicalResult executeCompiledCorePlan(
auto step = node.step.evaluate(knowledge);
auto forOp = cast<mlir::scf::ForOp>(node.op);
if (failed(lowerBound) || failed(upperBound) || failed(step) || *step <= 0) {
forOp.emitOpError("requires statically evaluable scf.for bounds for PIM codegen");
forOp.emitOpError("requires statically evaluable scf.for bounds for Pim codegen");
return failure();
}
@@ -984,7 +988,7 @@ static LogicalResult executeCompiledCorePlan(
auto condition = node.condition.evaluate(knowledge);
auto ifOp = cast<mlir::scf::IfOp>(node.op);
if (failed(condition)) {
ifOp.emitOpError("requires statically evaluable scf.if condition for PIM codegen");
ifOp.emitOpError("requires statically evaluable scf.if condition for Pim codegen");
return failure();
}
@@ -998,7 +1002,7 @@ static LogicalResult executeCompiledCorePlan(
auto selector = node.condition.evaluate(knowledge);
auto switchOp = cast<mlir::scf::IndexSwitchOp>(node.op);
if (failed(selector)) {
switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for PIM codegen");
switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for Pim codegen");
return failure();
}
const llvm::SmallVectorImpl<CompiledCoreNode>* selectedBody = node.defaultBody.get();
@@ -1184,12 +1188,12 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std::
}
auto getCompiledProgram = [&](Operation* op) {
auto it = compiledPrograms.find(op);
assert(it != compiledPrograms.end() && "missing compiled PIM core program");
assert(it != compiledPrograms.end() && "missing compiled Pim core program");
return it->second.get();
};
auto getMemoryPlan = [&](Operation* op) {
auto it = memoryPlans.find(op);
assert(it != memoryPlans.end() && "missing PIM core memory plan");
assert(it != memoryPlans.end() && "missing Pim core memory plan");
return it->second.get();
};
@@ -1267,7 +1271,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std::
if (failed(weightView)) {
std::string message;
llvm::raw_string_ostream os(message);
os << "requires a statically resolvable dense global weight view during PIM codegen; weight="
os << "requires a statically resolvable dense global weight view during Pim codegen; weight="
<< vmmOp.getWeight() << " type=" << vmmOp.getWeight().getType();
result.recordDiagnostic(vmmOp, os.str());
return failure();
@@ -1275,7 +1279,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std::
if (weightView->shape.size() != 2) {
std::string message;
llvm::raw_string_ostream os(message);
os << "requires a rank-2 matrix weight view during PIM codegen; resolved shape=[";
os << "requires a rank-2 matrix weight view during Pim codegen; resolved shape=[";
llvm::interleaveComma(weightView->shape, os);
os << "] weight=" << vmmOp.getWeight() << " type=" << vmmOp.getWeight().getType();
result.recordDiagnostic(vmmOp, os.str());
@@ -1387,7 +1391,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std::
}
if (diagnostics.hasFailure())
diagnostics.emitSuppressedSummary(summaryAnchor ? summaryAnchor : moduleOp.getOperation(),
"PIM codegen diagnostic(s)");
"Pim codegen diagnostic(s)");
for (size_t jobIndex = 0; jobIndex < jobs.size(); ++jobIndex)
if (jobResults[jobIndex].status != CompilerSuccess)
@@ -1453,7 +1457,7 @@ OnnxMlirCompilerErrorCodes onnx_mlir::compileToPimCode(ModuleOp& moduleOp, std::
if (!batchPerCoreRow)
batchPerCoreRow = result.reportRow;
else if (!(*batchPerCoreRow == result.reportRow))
llvm_unreachable("one PIM core batch produced inconsistent per-core memory reports");
llvm_unreachable("one Pim core batch produced inconsistent per-core memory reports");
}
uint64_t batchReportId = jobs[group.front()].batchReportId.value_or(0);
src/PIM/Compiler/PimCompilerOptions.cpp
+51 -32
View File
@@ -9,25 +9,25 @@
namespace onnx_mlir {
llvm::cl::opt<PimEmissionTargetType> pimEmissionTarget(
llvm::cl::desc("[Optional] Choose PIM-related target to emit (once selected it will cancel the other targets):"),
llvm::cl::values(clEnumVal(EmitSpatial, "Lower model to spatial IR")),
llvm::cl::values(clEnumVal(EmitPim, "Lower model to PIM IR")),
llvm::cl::values(clEnumVal(EmitPimBufferized, "Lower model to PIM IR and bufferize it")),
llvm::cl::values(clEnumVal(EmitPimCodegen, "Lower model to PIM IR and generate code for PIM")),
llvm::cl::desc("[Optional] Choose Pim-related target to emit (once selected it will cancel the other targets):"),
llvm::cl::values(clEnumVal(EmitSpatial, "Lower model to Spatial IR")),
llvm::cl::values(clEnumVal(EmitPim, "Lower model to Pim IR")),
llvm::cl::values(clEnumVal(EmitPimBufferized, "Lower model to Pim IR and bufferize it")),
llvm::cl::values(clEnumVal(EmitPimCodegen, "Lower model to Pim IR and generate code for Pim")),
llvm::cl::init(EmitPimCodegen),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<PimMemoryReportLevel> pimMemoryReport(
"pim-memory-report",
llvm::cl::desc("Emit a human-readable PIM memory planning report"),
llvm::cl::values(clEnumValN(PimMemoryReportNone, "none", "Do not emit any PIM memory planning report")),
llvm::cl::values(clEnumValN(PimMemoryReportSummary, "summary", "Emit a concise PIM memory summary")),
llvm::cl::desc("Emit a human-readable Pim memory planning report"),
llvm::cl::values(clEnumValN(PimMemoryReportNone, "none", "Do not emit any Pim memory planning report")),
llvm::cl::values(clEnumValN(PimMemoryReportSummary, "summary", "Emit a concise Pim memory summary")),
llvm::cl::init(PimMemoryReportSummary),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<PimConvLoweringType> pimConvLowering(
"pim-conv-lowering",
llvm::cl::desc("Convolution lowering strategy for PIM"),
llvm::cl::desc("Convolution lowering strategy for Pim"),
llvm::cl::values(clEnumValN(PimConvLoweringAuto, "auto", "Select the Conv lowering strategy automatically")),
llvm::cl::values(clEnumValN(PimConvLoweringLegacy, "legacy", "Use the legacy explicit-im2col Conv lowering")),
llvm::cl::values(clEnumValN(PimConvLoweringDepthwise, "depthwise", "Force the depthwise-specialized Conv lowering")),
@@ -55,20 +55,20 @@ llvm::cl::opt<PimSpatialDataflowExportType> pimExportSpatialDataflow(
llvm::cl::desc("Emit Gephi-importable CSV dataflow reports for Spatial pipeline snapshots"),
llvm::cl::values(clEnumValN(SpatialDataflowExportNone, "none", "Do not emit Spatial dataflow CSV reports")),
llvm::cl::values(
clEnumValN(SpatialDataflowExportSpatial1, "spatial1", "Emit spatial1 graph dataflow CSV reports")),
clEnumValN(SpatialDataflowExportSpatial1, "spatial1", "Emit Spatial1 graph dataflow CSV reports")),
llvm::cl::values(
clEnumValN(SpatialDataflowExportSpatial2, "spatial2", "Emit spatial2 trivially merged graph dataflow CSV reports")),
clEnumValN(SpatialDataflowExportSpatial2, "spatial2", "Emit Spatial2 trivially merged graph dataflow CSV reports")),
llvm::cl::values(
clEnumValN(SpatialDataflowExportSpatial3, "spatial3", "Emit spatial3 scheduled dataflow CSV reports")),
clEnumValN(SpatialDataflowExportSpatial3, "spatial3", "Emit Spatial3 scheduled dataflow CSV reports")),
llvm::cl::values(
clEnumValN(SpatialDataflowExportSpatial4, "spatial4", "Emit spatial4 realized dataflow CSV reports")),
clEnumValN(SpatialDataflowExportSpatial4, "spatial4", "Emit Spatial4 realized dataflow CSV reports")),
llvm::cl::values(clEnumValN(SpatialDataflowExportAll, "all", "Emit all Spatial dataflow CSV reports")),
llvm::cl::init(SpatialDataflowExportNone),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<bool>
pimOnlyCodegen("pim-only-codegen",
llvm::cl::desc("Only generate code for PIM (assume input is already in bufferized PIM IR)"),
llvm::cl::desc("Only generate code for Pim (assume input is already in bufferized Pim IR)"),
llvm::cl::init(false),
llvm::cl::cat(OnnxMlirOptions));
@@ -96,21 +96,37 @@ llvm::cl::opt<bool> pimEmitJson("pim-emit-json",
llvm::cl::opt<bool> pimDetectCommunicationDeadlock(
"pim-detect-communication-deadlock",
llvm::cl::desc("Expensively simulate the statically expanded PIM send/receive order at verification time and fail if a blocking communication deadlock is found"),
llvm::cl::desc("Expensively simulate the statically expanded Pim send/receive order at verification time and fail if a blocking communication deadlock is found"),
llvm::cl::init(false),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<bool> pimVerifyBufferizationCopyFreedom(
"pim-verify-bufferization-copy-freedom",
llvm::cl::desc("Run the expensive official PIM tensor-copy freedom proof before bufferization"),
llvm::cl::desc("Run the expensive official Pim tensor-copy freedom proof before bufferization"),
llvm::cl::init(false),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<bool> pimDisableSynchronization(
"pim-disable-synchronization",
llvm::cl::desc("Omit Pim wait/sync instructions from generated code for performance ablation"),
llvm::cl::init(false),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<bool> pimDisableSpatialPlanning(
"pim-disable-spatial-planning",
llvm::cl::desc("Select the trivial Spatial layout plan for performance ablation"),
llvm::cl::init(false),
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<size_t>
crossbarSize("crossbar-size", llvm::cl::desc("Width and height of a single crossbar"), llvm::cl::init(128));
crossbarSize("crossbar-size",
llvm::cl::desc("Width and height of a single crossbar (required for Pim compilation)"),
llvm::cl::init(0));
llvm::cl::opt<size_t>
crossbarCountInCore("crossbar-count", llvm::cl::desc("Number of crossbars in each core"), llvm::cl::init(64));
crossbarCountInCore("crossbar-count",
llvm::cl::desc("Number of crossbars in each core (required for Pim compilation)"),
llvm::cl::init(0));
llvm::cl::opt<size_t> pipelineStages(
"pipeline",
@@ -119,32 +135,35 @@ llvm::cl::opt<size_t> pipelineStages(
llvm::cl::cat(OnnxMlirOptions));
llvm::cl::opt<long> coresCount("core-count",
llvm::cl::desc("Number of cores in the chip. Required for PIM compilation."),
llvm::cl::desc("Number of cores in the chip. Required for Pim compilation."),
llvm::cl::init(-1));
llvm::cl::opt<std::string> pimTargetConfig(
"pim-target-config",
llvm::cl::desc("PIM target configuration used to construct the Spatial scheduling cost model"),
llvm::cl::desc("Pim target configuration used to construct the Spatial scheduling cost model"),
llvm::cl::init(""),
llvm::cl::cat(OnnxMlirOptions));
bool hasExplicitPimCoreCount() { return coresCount.getNumOccurrences() != 0; }
void verifyExplicitPimCoreCount() {
if (!hasExplicitPimCoreCount())
llvm::report_fatal_error("PIM compilation requires an explicit --core-count=<positive integer>");
void verifyPimCompilerOptions() {
if (coresCount.getNumOccurrences() == 0)
llvm::report_fatal_error("Pim compilation requires an explicit --core-count=<positive integer>");
if (coresCount.getValue() <= 0)
llvm::report_fatal_error("PIM compilation requires --core-count to be a positive integer");
}
void verifyPimPipelineStages() {
llvm::report_fatal_error("Pim compilation requires --core-count to be a positive integer");
if (crossbarSize.getNumOccurrences() == 0)
llvm::report_fatal_error("Pim compilation requires an explicit --crossbar-size=<positive integer>");
if (crossbarSize.getValue() == 0)
llvm::report_fatal_error("Pim compilation requires --crossbar-size to be a positive integer");
if (crossbarCountInCore.getNumOccurrences() == 0)
llvm::report_fatal_error("Pim compilation requires an explicit --crossbar-count=<positive integer>");
if (crossbarCountInCore.getValue() == 0)
llvm::report_fatal_error("Pim compilation requires --crossbar-count to be a positive integer");
if (pipelineStages.getValue() == 0)
llvm::report_fatal_error("PIM compilation requires --pipeline to be positive");
llvm::report_fatal_error("Pim compilation requires --pipeline to be positive");
if (static_cast<size_t>(coresCount.getValue()) < pipelineStages.getValue())
llvm::report_fatal_error("PIM compilation requires --pipeline not to exceed --core-count");
llvm::report_fatal_error("Pim compilation requires --pipeline not to exceed --core-count");
if (crossbarCountInCore.getValue()
> std::numeric_limits<size_t>::max() / pipelineStages.getValue())
llvm::report_fatal_error("PIM compilation --crossbar-count * --pipeline overflows");
llvm::report_fatal_error("Pim compilation --crossbar-count * --pipeline overflows");
}
} // namespace onnx_mlir
src/PIM/Compiler/PimCompilerOptions.hpp
+3 -3
View File
@@ -59,6 +59,8 @@ extern llvm::cl::opt<bool> pimEmitJson;
extern llvm::cl::opt<bool> pimReportConvLowering;
extern llvm::cl::opt<bool> pimDetectCommunicationDeadlock;
extern llvm::cl::opt<bool> pimVerifyBufferizationCopyFreedom;
extern llvm::cl::opt<bool> pimDisableSynchronization;
extern llvm::cl::opt<bool> pimDisableSpatialPlanning;
extern llvm::cl::opt<size_t> crossbarSize;
extern llvm::cl::opt<size_t> crossbarCountInCore;
@@ -68,8 +70,6 @@ extern llvm::cl::opt<std::string> pimTargetConfig;
extern llvm::cl::opt<uint64_t> pimConvIm2colMaxElements;
extern llvm::cl::opt<uint64_t> pimConvStreamChunkPositions;
bool hasExplicitPimCoreCount();
void verifyExplicitPimCoreCount();
void verifyPimPipelineStages();
void verifyPimCompilerOptions();
} // namespace onnx_mlir
src/PIM/Compiler/PimCompilerUtils.cpp
+21 -21
View File
@@ -97,7 +97,7 @@ spatial::ConvLoweringStrategy getSpatialConvLoweringStrategy(PimConvLoweringType
case PimConvLoweringInputKTiled: return spatial::ConvLoweringStrategy::InputKTiled;
case PimConvLoweringTiled2D: return spatial::ConvLoweringStrategy::Tiled2D;
}
llvm_unreachable("unknown PIM Conv lowering strategy");
llvm_unreachable("unknown Pim Conv lowering strategy");
}
spatial::SpatialDataflowExportStage getPimSpatialDataflowExportStage(
@@ -110,7 +110,7 @@ spatial::SpatialDataflowExportStage getPimSpatialDataflowExportStage(
case SpatialDataflowExportSpatial4: return spatial::SpatialDataflowExportStage::Spatial4;
case SpatialDataflowExportAll: return spatial::SpatialDataflowExportStage::All;
}
llvm_unreachable("unknown PIM Spatial dataflow export stage");
llvm_unreachable("unknown Pim Spatial dataflow export stage");
}
spatial::SpatialTargetResources getPimSpatialTargetResources(const spatial::SchedulingTarget& target) {
@@ -120,7 +120,7 @@ spatial::SpatialTargetResources getPimSpatialTargetResources(const spatial::Sche
resources.processorCount = target.processorCount;
resources.vectorWidth = target.vectorWidth;
if (failed(resources.verify()))
llvm::report_fatal_error("PIM target resources are incomplete");
llvm::report_fatal_error("Pim target resources are incomplete");
return resources;
}
@@ -138,7 +138,7 @@ const llvm::json::Object& requireObject(const llvm::json::Object& object,
llvm::StringRef path) {
const llvm::json::Object* nested = object.getObject(key);
if (!nested)
llvm::report_fatal_error("PIM target config is missing object '" + path + "." + key + "'");
llvm::report_fatal_error("Pim target config is missing object '" + path + "." + key + "'");
return *nested;
}
@@ -151,7 +151,7 @@ Cost getConfigCost(const llvm::json::Object& object,
return fallback;
if (!std::isfinite(*number) || *number < 0.0 || (!allowZero && *number == 0.0)
|| *number > static_cast<double>(std::numeric_limits<Cost>::max()))
llvm::report_fatal_error("PIM target config field '" + key + "' must be a valid positive number");
llvm::report_fatal_error("Pim target config field '" + key + "' must be a valid positive number");
return static_cast<Cost>(std::ceil(*number));
}
@@ -159,11 +159,11 @@ std::pair<size_t, size_t> getConfigPair(const llvm::json::Object& object,
llvm::StringRef key) {
const llvm::json::Array* values = object.getArray(key);
if (!values || values->size() != 2)
llvm::report_fatal_error("PIM target config field '" + key + "' must contain two integers");
llvm::report_fatal_error("Pim target config field '" + key + "' must contain two integers");
std::optional<int64_t> first = (*values)[0].getAsInteger();
std::optional<int64_t> second = (*values)[1].getAsInteger();
if (!first || !second || *first <= 0 || *second <= 0)
llvm::report_fatal_error("PIM target config field '" + key + "' must contain two positive integers");
llvm::report_fatal_error("Pim target config field '" + key + "' must contain two positive integers");
return {static_cast<size_t>(*first), static_cast<size_t>(*second)};
}
@@ -174,7 +174,7 @@ void loadPimInterProcessorLatencies(
network.getString("net_config_file_path");
if (!filename)
llvm::report_fatal_error(
"PIM target config is missing network latency file path");
"Pim target config is missing network latency file path");
llvm::SmallString<256> networkPath(*filename);
if (!llvm::sys::path::is_absolute(networkPath)) {
@@ -187,19 +187,19 @@ void loadPimInterProcessorLatencies(
auto buffer = llvm::MemoryBuffer::getFile(networkPath);
if (!buffer)
llvm::report_fatal_error(
llvm::Twine("failed to read PIM network config '")
llvm::Twine("failed to read Pim network config '")
+ networkPath + "': " + buffer.getError().message());
auto parsed = llvm::json::parse(buffer.get()->getBuffer());
if (!parsed)
llvm::report_fatal_error(
llvm::Twine("failed to parse PIM network config '")
llvm::Twine("failed to parse Pim network config '")
+ networkPath + "': " + llvm::toString(parsed.takeError()));
const llvm::json::Object* root = parsed->getAsObject();
const llvm::json::Object* latencies =
root ? root->getObject("latency") : nullptr;
if (!latencies)
llvm::report_fatal_error(
"PIM network config is missing its latency matrix");
"Pim network config is missing its latency matrix");
target.interProcessorLatencyNs.assign(
target.processorCount * target.processorCount, 0);
@@ -210,7 +210,7 @@ void loadPimInterProcessorLatencies(
const llvm::json::Object* row = latencies->getObject(sourceKey);
if (!row)
llvm::report_fatal_error(
llvm::Twine("PIM network config is missing latency row ")
llvm::Twine("Pim network config is missing latency row ")
+ sourceKey);
for (size_t destination = 0;
destination < target.processorCount; ++destination) {
@@ -220,7 +220,7 @@ void loadPimInterProcessorLatencies(
std::optional<double> latency = row->getNumber(destinationKey);
if (!latency || !std::isfinite(*latency) || *latency <= 0.0)
llvm::report_fatal_error(
llvm::Twine("PIM network config is missing latency ")
llvm::Twine("Pim network config is missing latency ")
+ sourceKey + " -> " + destinationKey);
Cost roundedLatency = static_cast<Cost>(std::ceil(*latency));
target.interProcessorLatencyNs[
@@ -244,17 +244,17 @@ spatial::SchedulingTarget getPimSchedulingTarget() {
auto buffer = llvm::MemoryBuffer::getFile(pimTargetConfig);
if (!buffer)
llvm::report_fatal_error(
llvm::Twine("failed to read PIM target config '")
llvm::Twine("failed to read Pim target config '")
+ pimTargetConfig.getValue() + "': " + buffer.getError().message());
auto parsed = llvm::json::parse(buffer.get()->getBuffer());
if (!parsed)
llvm::report_fatal_error(
llvm::Twine("failed to parse PIM target config '")
llvm::Twine("failed to parse Pim target config '")
+ pimTargetConfig.getValue() + "': "
+ llvm::toString(parsed.takeError()));
const llvm::json::Object* root = parsed->getAsObject();
if (!root)
llvm::report_fatal_error("PIM target config must contain a JSON object");
llvm::report_fatal_error("Pim target config must contain a JSON object");
const llvm::json::Object& chip = requireObject(*root, "chip_config", "root");
const llvm::json::Object& core = requireObject(chip, "core_config", "chip_config");
@@ -267,7 +267,7 @@ spatial::SchedulingTarget getPimSchedulingTarget() {
std::optional<int64_t> coreCount = chip.getInteger("core_cnt");
if (!coreCount || *coreCount <= 0)
llvm::report_fatal_error("PIM target config field 'core_cnt' must be a positive integer");
llvm::report_fatal_error("Pim target config field 'core_cnt' must be a positive integer");
target.processorCount = static_cast<size_t>(*coreCount);
target.residentWeightCapacity =
getConfigCost(matrix, "xbar_array_count", target.residentWeightCapacity);
@@ -278,7 +278,7 @@ spatial::SchedulingTarget getPimSchedulingTarget() {
|| target.residentWeightCapacity != crossbarCountInCore.getValue()
|| target.matrixRows != crossbarSize.getValue()
|| target.matrixColumns != crossbarSize.getValue())
llvm::report_fatal_error("PIM target config resources do not match --core-count, "
llvm::report_fatal_error("Pim target config resources do not match --core-count, "
"--crossbar-count, and --crossbar-size");
loadPimInterProcessorLatencies(target, network);
@@ -331,8 +331,7 @@ void addPassesPim(OwningOpRef<ModuleOp>& module,
PassManager& pm,
EmissionTargetType& emissionTarget,
std::string outputNameNoExt) {
verifyExplicitPimCoreCount();
verifyPimPipelineStages();
verifyPimCompilerOptions();
spatial::SchedulingTarget schedulingTarget = getPimSchedulingTarget();
spatial::SpatialTargetResources targetResources = getPimSpatialTargetResources(schedulingTarget);
@@ -352,7 +351,8 @@ void addPassesPim(OwningOpRef<ModuleOp>& module,
spatial::SpatialDataflowExportStage exportStage =
getPimSpatialDataflowExportStage(pimExportSpatialDataflow.getValue());
pm.addPass(createONNXToSpatialPass(targetResources, planningOptions));
pm.addPass(createSpatialLayoutPlanningPass(targetResources));
pm.addPass(createSpatialLayoutPlanningPass(
targetResources, pimDisableSpatialPlanning.getValue()));
pm.addPass(createLowerSpatialPlansPass(targetResources, planningOptions, exportStage));
pm.addPass(createTrivialGraphComputeMergePass(
schedulingTarget.residentWeightCapacity, exportStage));
src/PIM/Compiler/PimCoreProgram.cpp
+3 -3
View File
@@ -46,7 +46,7 @@ static LogicalResult compileCoreEmissionPlan(Block& block, SmallVectorImpl<Compi
auto upper = compileIndexExpr(forOp.getUpperBound());
auto step = compileIndexExpr(forOp.getStep());
if (failed(lower) || failed(upper) || failed(step)) {
forOp.emitOpError("requires statically evaluable scf.for bounds for PIM codegen");
forOp.emitOpError("requires statically evaluable scf.for bounds for Pim codegen");
return failure();
}
CompiledCoreNode node;
@@ -63,7 +63,7 @@ static LogicalResult compileCoreEmissionPlan(Block& block, SmallVectorImpl<Compi
if (auto ifOp = dyn_cast<scf::IfOp>(op)) {
auto condition = compileIndexExpr(ifOp.getCondition());
if (failed(condition)) {
ifOp.emitOpError("requires statically evaluable scf.if condition for PIM codegen");
ifOp.emitOpError("requires statically evaluable scf.if condition for Pim codegen");
return failure();
}
CompiledCoreNode node;
@@ -82,7 +82,7 @@ static LogicalResult compileCoreEmissionPlan(Block& block, SmallVectorImpl<Compi
if (auto switchOp = dyn_cast<scf::IndexSwitchOp>(op)) {
auto selector = compileIndexExpr(switchOp.getArg());
if (failed(selector)) {
switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for PIM codegen");
switchOp.emitOpError("requires a statically evaluable scf.index_switch selector for Pim codegen");
return failure();
}
CompiledCoreNode node;
src/PIM/Conversion/ONNXToSpatial/Common/ComputeRegionBuilder.hpp
+1 -1
View File
@@ -249,7 +249,7 @@ auto createEmptySpatGraphComputeBatch(RewriterT& rewriter,
if (laneCount <= 0 || laneCount > std::numeric_limits<int32_t>::max())
return mlir::FailureOr<spatial::SpatGraphComputeBatch>(mlir::failure());
auto laneCountAttr = pim::getCheckedI32Attr(rewriter, loc, laneCount, "spatial compute_batch lane count");
auto laneCountAttr = pim::getCheckedI32Attr(rewriter, loc, laneCount, "Spatial compute_batch lane count");
if (mlir::failed(laneCountAttr))
return mlir::FailureOr<spatial::SpatGraphComputeBatch>(mlir::failure());
src/PIM/Conversion/ONNXToSpatial/Common/ShapeTilingUtils.hpp
+1 -1
View File
@@ -25,7 +25,7 @@ llvm::SmallVector<mlir::Value> sliceVector(const mlir::Value& vectorToSlice,
mlir::Location loc);
/// Partitions one logical vector into per-core crossbar-sized slices using the
/// current PIM target geometry.
/// current Pim target geometry.
llvm::DenseMap<CoreId, llvm::SmallVector<mlir::Value>> sliceVectorPerCrossbarPerCore(
const mlir::Value& vectorToSlice,
mlir::PatternRewriter& rewriter,
src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/LowerSpatialPlansPass.cpp
+1 -1
View File
@@ -46,7 +46,7 @@ struct LowerSpatialPlansPass final
}
auto entryFunc = getPimEntryFunc(moduleOp);
if (failed(entryFunc)) {
moduleOp.emitError("failed to locate the PIM entry function during LowerSpatialPlans");
moduleOp.emitError("failed to locate the Pim entry function during LowerSpatialPlans");
signalPassFailure();
return;
}
src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/ONNXToSpatialPass.cpp
+2 -2
View File
@@ -158,7 +158,7 @@ void ONNXToSpatialPass::runOnOperation() {
auto entryFunc = getPimEntryFunc(moduleOp);
if (failed(entryFunc)) {
moduleOp.emitError("failed to locate the PIM entry function during ONNX-to-Spatial lowering");
moduleOp.emitError("failed to locate the Pim entry function during ONNX-to-Spatial lowering");
signalPassFailure();
return;
}
@@ -245,7 +245,7 @@ void ONNXToSpatialPass::runOnOperation() {
RewritePatternSet postPatterns(ctx);
populatePostPatterns(postPatterns, ctx);
if (failed(applyPartialConversion(*entryFunc, postTarget, std::move(postPatterns)))) {
moduleOp.emitError("failed to normalize weight-like Spatial compute operands before Spatial-to-PIM lowering");
moduleOp.emitError("failed to normalize weight-like Spatial compute operands before Spatial-to-Pim lowering");
signalPassFailure();
return;
}
src/PIM/Conversion/ONNXToSpatial/Passes/Transforms/SpatialLayoutPlanningPass.cpp
+15 -8
View File
@@ -42,8 +42,9 @@ static SmallVector<spatial::PhysicalLayout> getOperandLayouts(
class SpatialLayoutAnalysis {
public:
SpatialLayoutAnalysis(func::FuncOp funcOp,
const spatial::SpatialTargetResources& target)
: funcOp(funcOp), target(target) {}
const spatial::SpatialTargetResources& target,
bool selectTrivialPlan)
: funcOp(funcOp), target(target), selectTrivialPlan(selectTrivialPlan) {}
FailureOr<SpatialLayoutSelection> run() {
SpatialLayoutSelection selection;
@@ -56,6 +57,9 @@ public:
selection.selectedAlternative[&op] = 0;
}
if (selectTrivialPlan)
return selection;
const size_t maxRounds = 2 * planOps.size() + 1;
for (size_t round = 0; round < maxRounds; ++round) {
bool changed = false;
@@ -168,6 +172,7 @@ private:
func::FuncOp funcOp;
const spatial::SpatialTargetResources& target;
bool selectTrivialPlan;
};
static LogicalResult materializeMismatchedUses(
@@ -251,8 +256,9 @@ struct SpatialLayoutPlanningPass final
}
SpatialLayoutPlanningPass() = default;
explicit SpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target)
: target(target), hasTarget(true) {}
SpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target,
bool selectTrivialPlan)
: target(target), selectTrivialPlan(selectTrivialPlan), hasTarget(true) {}
void runOnOperation() override {
ModuleOp moduleOp = getOperation();
@@ -263,13 +269,13 @@ struct SpatialLayoutPlanningPass final
}
auto entryFunc = getPimEntryFunc(moduleOp);
if (failed(entryFunc)) {
moduleOp.emitError("failed to locate the PIM entry function during Spatial layout planning");
moduleOp.emitError("failed to locate the Pim entry function during Spatial layout planning");
signalPassFailure();
return;
}
func::FuncOp funcOp = *entryFunc;
SpatialLayoutAnalysis analysis(funcOp, target);
SpatialLayoutAnalysis analysis(funcOp, target, selectTrivialPlan);
FailureOr<SpatialLayoutSelection> selection = analysis.run();
if (failed(selection)) {
signalPassFailure();
@@ -301,6 +307,7 @@ struct SpatialLayoutPlanningPass final
}
spatial::SpatialTargetResources target;
bool selectTrivialPlan = false;
bool hasTarget = false;
};
@@ -311,8 +318,8 @@ std::unique_ptr<Pass> createSpatialLayoutPlanningPass() {
}
std::unique_ptr<Pass> createSpatialLayoutPlanningPass(
const spatial::SpatialTargetResources& target) {
return std::make_unique<SpatialLayoutPlanningPass>(target);
const spatial::SpatialTargetResources& target, bool selectTrivialPlan) {
return std::make_unique<SpatialLayoutPlanningPass>(target, selectTrivialPlan);
}
} // namespace onnx_mlir
src/PIM/Conversion/ONNXToSpatial/Patterns/Math/Conv.cpp
+1 -1
View File
@@ -199,7 +199,7 @@ static bool writeConvLoweringReport(const ConvLoweringReportEntry& entry,
return false;
}
reportFile << "# PIM Conv Lowering Report (bounded to 512 rows)\n\n";
reportFile << "# Pim conv lowering report (bounded to 512 rows)\n\n";
reportFile << "## Plan selection\n";
writeConvReportTableHeader(reportFile, "Selector");
bool realizationSectionStarted = false;
src/PIM/Conversion/SpatialToPim/BatchCoreLoweringPatterns.cpp
+2 -2
View File
@@ -307,7 +307,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeBatchOp(spatial::SpatSchedul
"resultful compute_batch lowering currently requires a spat.in_parallel terminator");
}
auto coreIds = getRequiredScheduledBatchCoreIds(computeBatchOp, "spatial compute_batch core id");
auto coreIds = getRequiredScheduledBatchCoreIds(computeBatchOp, "Spatial compute_batch core id");
if (failed(coreIds))
return failure();
SmallVector<Value> batchWeights(computeBatchOp.getWeights().begin(), computeBatchOp.getWeights().end());
@@ -317,7 +317,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeBatchOp(spatial::SpatSchedul
rewriter.setInsertionPointAfter(computeBatchOp);
auto laneCountAttr = pim::getCheckedI32Attr(
rewriter, computeBatchOp, static_cast<uint64_t>(computeBatchOp.getLaneCount()), "pim core_batch lane count");
rewriter, computeBatchOp, static_cast<uint64_t>(computeBatchOp.getLaneCount()), "Pim core_batch lane count");
if (failed(laneCountAttr))
return failure();
auto coreBatchOp =
src/PIM/Conversion/SpatialToPim/CoreLoweringPatterns.cpp
+2 -2
View File
@@ -410,7 +410,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeOp(spatial::SpatScheduledCom
continue;
}
return computeOp.emitOpError("has an unsupported remaining result use during Spatial-to-PIM lowering");
return computeOp.emitOpError("has an unsupported remaining result use during Spatial-to-Pim lowering");
}
rewriter.setInsertionPoint(yieldOp);
@@ -420,7 +420,7 @@ LogicalResult raptor::SpatialToPimPass::lowerComputeOp(spatial::SpatScheduledCom
if (!computeOp.getWeights().empty())
computeWeights.append(computeOp.getWeights().begin(), computeOp.getWeights().end());
rewriter.setInsertionPointAfter(computeOp);
auto checkedCoreId = getRequiredScheduledCoreId(computeOp, "spatial compute core id");
auto checkedCoreId = getRequiredScheduledCoreId(computeOp, "Spatial compute core id");
if (failed(checkedCoreId))
return failure();
auto coreIdAttr = pim::getCheckedI32Attr(rewriter, computeOp, static_cast<int64_t>(*checkedCoreId), "pim core id");
src/PIM/Conversion/SpatialToPim/ReturnPathNormalization.cpp
+2 -2
View File
@@ -734,7 +734,7 @@ raptor::SpatialToPimPass::ReturnPathLoweringResult raptor::SpatialToPimPass::low
auto storedType = dyn_cast<RankedTensorType>(storedValue.getType());
if (!storedType) {
producerOp->emitOpError(
"has an unsupported non-ranked concat-return helper yield during Spatial-to-PIM lowering");
"has an unsupported non-ranked concat-return helper yield during Spatial-to-Pim lowering");
return ReturnPathLoweringResult::Failure;
}
rewriter.setInsertionPointAfterValue(storedValue);
@@ -748,7 +748,7 @@ raptor::SpatialToPimPass::ReturnPathLoweringResult raptor::SpatialToPimPass::low
SmallVector<int64_t> destinationIndices;
if (failed(mapIndicesThroughHelperChain(
sourceIndices, concatReturnUse->concatShape, concatReturnUse->helperChain, destinationIndices))) {
producerOp->emitOpError("has an unsupported concat-return helper chain during Spatial-to-PIM lowering");
producerOp->emitOpError("has an unsupported concat-return helper chain during Spatial-to-Pim lowering");
return ReturnPathLoweringResult::Failure;
}
src/PIM/Conversion/SpatialToPim/SpatialToPimPass.cpp
+5 -5
View File
@@ -88,7 +88,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() {
operationsToRemove.clear();
ModuleOp moduleOp = getOperation();
if (!hasTarget || failed(targetResources.verify())) {
moduleOp.emitError("Spatial-to-PIM lowering requires valid injected target resources");
moduleOp.emitError("Spatial-to-Pim lowering requires valid injected target resources");
signalPassFailure();
return;
}
@@ -96,7 +96,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() {
auto entryFunc = getPimEntryFunc(moduleOp);
if (failed(entryFunc)) {
moduleOp.emitError("failed to locate the PIM entry function during Spatial-to-PIM lowering");
moduleOp.emitError("failed to locate the Pim entry function during Spatial-to-Pim lowering");
signalPassFailure();
return;
}
@@ -135,7 +135,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() {
RewritePatternSet initialPatterns(ctx);
populateInitialPatterns(initialPatterns);
if (failed(applyPartialConversion(moduleOp, target, std::move(initialPatterns)))) {
moduleOp.emitError("failed to lower required Spatial ops to the initial PIM form");
moduleOp.emitError("failed to lower required Spatial ops to the initial Pim form");
signalPassFailure();
return;
}
@@ -153,7 +153,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() {
auto returnOp = cast<func::ReturnOp>(funcOp.front().getTerminator());
addReturnOutputBuffers(returnOp, rewriter);
if (failed(allocateAndInitializeCoreLocalVariables(funcOp, rewriter))) {
funcOp.emitOpError("failed to allocate or initialize core-local tensors during Spatial-to-PIM lowering");
funcOp.emitOpError("failed to allocate or initialize core-local tensors during Spatial-to-Pim lowering");
signalPassFailure();
return;
}
@@ -285,7 +285,7 @@ void onnx_mlir::raptor::SpatialToPimPass::runOnOperation() {
RewritePatternSet communicationPatterns(ctx);
populateChannelLoweringPatterns(communicationPatterns);
if (failed(applyFullConversion(funcOp, communicationTarget, std::move(communicationPatterns)))) {
funcOp.emitOpError("failed to lower Spatial communication ops to PIM communication ops");
funcOp.emitOpError("failed to lower Spatial communication ops to Pim communication ops");
signalPassFailure();
return;
}
src/PIM/Conversion/SpatialToPim/SpatialToPimPass.hpp
+1 -1
View File
@@ -26,7 +26,7 @@ namespace raptor {
struct SpatialToPimPass : mlir::PassWrapper<SpatialToPimPass, mlir::OperationPass<mlir::ModuleOp>> {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(SpatialToPimPass)
llvm::StringRef getArgument() const override { return "convert-spatial-to-pim"; }
llvm::StringRef getDescription() const override { return "Lower Spatial ops to PIM-ready format"; }
llvm::StringRef getDescription() const override { return "Lower Spatial ops to Pim-ready format"; }
SpatialToPimPass() = default;
explicit SpatialToPimPass(const spatial::SpatialTargetResources& target)
src/PIM/Dialect/Pim/Passes/Transforms/Bufferization/PimBufferizationPass.cpp
+19 -19
View File
@@ -402,7 +402,7 @@ static LogicalResult verifyPimCoresNeedNoTensorCopies(
bufferization::BufferizationState state;
if (failed(bufferization::insertTensorCopies(*clone, options, state))) {
moduleOp.emitError("official one-shot analysis failed while verifying PIM core copy freedom");
moduleOp.emitError("official one-shot analysis failed while verifying Pim core copy freedom");
return failure();
}
@@ -415,10 +415,10 @@ static LogicalResult verifyPimCoresNeedNoTensorCopies(
Operation* requiredBy = alloc->getUsers().empty()
? alloc.getOperation() : *alloc->getUsers().begin();
diagnostics.report(requiredBy, [](Operation* op) {
op->emitOpError("official one-shot bufferization requires a tensor copy inside a PIM core");
op->emitOpError("official one-shot bufferization requires a tensor copy inside a Pim core");
});
});
diagnostics.emitSuppressedSummary(moduleOp, "required PIM core tensor copies");
diagnostics.emitSuppressedSummary(moduleOp, "required Pim core tensor copies");
return success(!diagnostics.hasFailure());
}
@@ -440,7 +440,7 @@ static LogicalResult runOneShotPimBufferization(
bufferization::BufferizationState state;
if (failed(bufferization::insertTensorCopies(moduleOp, hostOptions, state))
|| failed(bufferization::bufferizeModuleOp(moduleOp, options, state))) {
moduleOp.emitError("Failed to bufferize PIM and Spatial ops");
moduleOp.emitError("Failed to bufferize Pim and Spatial ops");
return failure();
}
return success();
@@ -478,7 +478,7 @@ static LogicalResult verifyContiguousRuntimeOperands(ModuleOp moduleOp) {
if (succeeded(resolveContiguousAddress(operand, knowledge)) || succeeded(compileContiguousAddressExpr(operand)))
return;
op.emitOpError() << "operand #" << operandIndex
<< " is not backed by contiguous addressable storage after PIM bufferization";
<< " is not backed by contiguous addressable storage after Pim bufferization";
hasFailure = true;
};
@@ -552,7 +552,7 @@ static LogicalResult verifyContiguousRuntimeOperands(ModuleOp moduleOp) {
});
if (hasFailure) {
moduleOp.emitError("PIM bufferization must fully normalize executable runtime operand contiguity before codegen");
moduleOp.emitError("Pim bufferization must fully normalize executable runtime operand contiguity before codegen");
return failure();
}
return success();
@@ -589,7 +589,7 @@ static LogicalResult verifyPimCopyAddressSpaces(ModuleOp moduleOp) {
});
if (failureCount != 0)
moduleOp.emitError() << "found " << failureCount
<< " PIM copy address-space violation(s); the first is reported above";
<< " Pim copy address-space violation(s); the first is reported above";
return success(failureCount == 0);
}
@@ -673,7 +673,7 @@ static LogicalResult normalizePimMemory(ModuleOp moduleOp, func::FuncOp funcOp)
GreedyRewriteConfig contiguityConfig;
contiguityConfig.enableFolding(false);
if (failed(applyPatternsGreedily(moduleOp, std::move(contiguityPatterns), contiguityConfig))) {
moduleOp.emitError("failed to normalize PIM copy contiguity during bufferization");
moduleOp.emitError("failed to normalize Pim copy contiguity during bufferization");
return failure();
}
annotateWeightsMemrefs(moduleOp, funcOp);
@@ -684,7 +684,7 @@ static LogicalResult normalizePimMemory(ModuleOp moduleOp, func::FuncOp funcOp)
static FailureOr<func::FuncOp> requirePimEntryFunc(ModuleOp moduleOp, StringRef phase) {
auto entryFunc = getPimEntryFunc(moduleOp);
if (failed(entryFunc)) {
moduleOp.emitError("failed to locate the PIM entry function during ") << phase;
moduleOp.emitError("failed to locate the Pim entry function during ") << phase;
return failure();
}
return *entryFunc;
@@ -701,12 +701,12 @@ struct PimBufferizationPreparationPass
StringRef getArgument() const override { return "pim-bufferization-preparation"; }
StringRef getDescription() const override {
return "Prepare writable tensor destinations for PIM one-shot bufferization.";
return "Prepare writable tensor destinations for Pim one-shot bufferization.";
}
void runOnOperation() final {
ModuleOp moduleOp = getOperation();
auto funcOp = requirePimEntryFunc(moduleOp, "PIM bufferization preparation");
auto funcOp = requirePimEntryFunc(moduleOp, "Pim bufferization preparation");
if (failed(funcOp)) {
signalPassFailure();
return;
@@ -725,7 +725,7 @@ struct PimOneShotBufferizationPass
StringRef getArgument() const override { return "pim-one-shot-bufferization"; }
StringRef getDescription() const override {
return "Run one-shot bufferization for PIM and Spatial tensors.";
return "Run one-shot bufferization for Pim and Spatial tensors.";
}
void runOnOperation() final {
@@ -740,12 +740,12 @@ struct PimMemoryNormalizationPass
StringRef getArgument() const override { return "pim-memory-normalization"; }
StringRef getDescription() const override {
return "Normalize PIM memory copies and verify addressable operands.";
return "Normalize Pim memory copies and verify addressable operands.";
}
void runOnOperation() final {
ModuleOp moduleOp = getOperation();
auto funcOp = requirePimEntryFunc(moduleOp, "PIM memory normalization");
auto funcOp = requirePimEntryFunc(moduleOp, "Pim memory normalization");
if (failed(funcOp)) {
signalPassFailure();
return;
@@ -761,20 +761,20 @@ static LogicalResult verifyNoTensorValues(ModuleOp moduleOp) {
if (failureCount >= 8)
return;
if (op->getDialect()->getNamespace() == "tensor") {
op->emitOpError("tensor operation remains after PIM bufferization");
op->emitOpError("tensor operation remains after Pim bufferization");
++failureCount;
return;
}
for (Value value : op->getOperands()) {
if (isa<TensorType>(value.getType())) {
op->emitOpError("tensor operand remains after PIM bufferization");
op->emitOpError("tensor operand remains after Pim bufferization");
++failureCount;
return;
}
}
for (Value value : op->getResults()) {
if (isa<TensorType>(value.getType())) {
op->emitOpError("tensor result remains after PIM bufferization");
op->emitOpError("tensor result remains after Pim bufferization");
++failureCount;
return;
}
@@ -782,7 +782,7 @@ static LogicalResult verifyNoTensorValues(ModuleOp moduleOp) {
});
if (failureCount != 0)
moduleOp.emitError() << "found " << failureCount
<< " tensor value(s) after PIM bufferization"
<< " tensor value(s) after Pim bufferization"
<< (failureCount == 8 ? " (first 8 reported)" : "");
return success(failureCount == 0);
}
@@ -793,7 +793,7 @@ struct PimBufferizationVerificationPass
StringRef getArgument() const override { return "pim-bufferization-verification"; }
StringRef getDescription() const override {
return "Verify tensor elimination, contiguity, and PIM copy address spaces.";
return "Verify tensor elimination, contiguity, and Pim copy address spaces.";
}
void runOnOperation() final {
src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/HostConstantFoldingPass.cpp
+2 -2
View File
@@ -16,7 +16,7 @@ struct HostConstantFoldingPass : PassWrapper<HostConstantFoldingPass, OperationP
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(HostConstantFoldingPass)
StringRef getArgument() const override { return "pim-host-constant-folding-pass"; }
StringRef getDescription() const override { return "Fold host-side constant expressions before PIM verification"; }
StringRef getDescription() const override { return "Fold host-side constant expressions before Pim verification"; }
LogicalResult initialize(MLIRContext* context) override {
RewritePatternSet owningPatterns(context);
@@ -38,7 +38,7 @@ struct HostConstantFoldingPass : PassWrapper<HostConstantFoldingPass, OperationP
GreedyRewriteConfig config;
config.enableFolding();
if (failed(applyPatternsGreedily(moduleOp, *patterns, config))) {
moduleOp.emitError("PIM host constant folding failed in the greedy rewrite driver");
moduleOp.emitError("Pim host constant folding failed in the greedy rewrite driver");
signalPassFailure();
return;
}
src/PIM/Dialect/Pim/Passes/Transforms/HostConstantFolding/Patterns/Constant.cpp
+1 -1
View File
@@ -472,7 +472,7 @@ struct FoldConstantHostCopyPattern final : OpRewritePattern<memref::CopyOp> {
}
};
// Converts PIM copies from dense globals into direct folded globals before codegen.
// Converts Pim copies from dense globals into direct folded globals before codegen.
struct FoldConstantMemCpPattern final : OpRewritePattern<pim::PimMemCopyOp> {
using OpRewritePattern::OpRewritePattern;
src/PIM/Dialect/Pim/Passes/Transforms/InstructionSelection/InstructionSelectionPass.cpp
+2 -2
View File
@@ -40,7 +40,7 @@ struct LowerTransposePattern final : OpRewritePattern<pim::PimTransposeOp> {
auto sourceType = dyn_cast<MemRefType>(op.getInput().getType());
auto targetType = dyn_cast<MemRefType>(op.getOutputBuffer().getType());
if (!sourceType || !targetType || !sourceType.hasStaticShape() || !targetType.hasStaticShape())
return op.emitOpError("requires static memref operands before PIM instruction selection");
return op.emitOpError("requires static memref operands before Pim instruction selection");
ArrayRef<int64_t> sourceShape = sourceType.getShape();
size_t rank = sourceShape.size();
@@ -147,7 +147,7 @@ struct InstructionSelectionPass : PassWrapper<InstructionSelectionPass, Operatio
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(InstructionSelectionPass)
StringRef getArgument() const override { return "pim-instruction-selection"; }
StringRef getDescription() const override { return "Select explicit PIM ISA operations"; }
StringRef getDescription() const override { return "Select explicit Pim ISA operations"; }
void runOnOperation() override {
RewritePatternSet patterns(&getContext());
src/PIM/Dialect/Pim/Passes/Transforms/LocalMemoryPlanning/LocalMemoryPlanning.cpp
+3 -3
View File
@@ -36,7 +36,7 @@ struct PimLocalMemoryPlanningPass : PassWrapper<PimLocalMemoryPlanningPass, Oper
StringRef getArgument() const override { return "pim-local-memory-planning"; }
StringRef getDescription() const override {
return "Plan liveness-based addresses for PIM core-local memory";
return "Plan liveness-based addresses for Pim core-local memory";
}
void runOnOperation() override {
@@ -149,14 +149,14 @@ FailureOr<CoreMemoryPlan> buildCoreMemoryPlan(Operation* coreLikeOp) {
plan.intervals = std::move(*intervals);
auto placements = planLocalMemoryPlacements(plan.intervals, kPimLocalMemoryAddressLimit);
if (failed(placements)) {
coreLikeOp->emitError("PIM local-memory plan exceeds the signed int32 address range");
coreLikeOp->emitError("Pim local-memory plan exceeds the signed int32 address range");
return failure();
}
plan.placements = std::move(*placements);
for (const LocalMemoryPlacement& placement : plan.placements) {
auto end = alignedEnd(placement.address, placement.size, kPimLocalMemoryAddressLimit);
if (failed(end)) {
coreLikeOp->emitError("PIM local-memory plan has invalid address arithmetic");
coreLikeOp->emitError("Pim local-memory plan has invalid address arithmetic");
return failure();
}
plan.arenaSize = std::max(plan.arenaSize, *end);
src/PIM/Dialect/Pim/Passes/Transforms/Verification/VerificationPass.cpp
+22 -22
View File
@@ -117,7 +117,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag
for (StringRef name : kRemovedLocalMemoryPlanAttrNames)
if (coreLikeOp->hasAttr(name)) {
diagnostics.report(coreLikeOp, [name](Operation* op) {
op->emitError() << "contains removed PIM local-memory planning attribute '" << name << "'";
op->emitError() << "contains removed Pim local-memory planning attribute '" << name << "'";
});
hasFailure = true;
}
@@ -137,7 +137,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag
auto analyzed = pim::analyzeLocalMemoryLifetimes(coreLikeOp);
if (failed(analyzed)) {
diagnostics.report(coreLikeOp, [](Operation* op) {
op->emitError("cannot analyze PIM local-memory lifetimes for plan verification");
op->emitError("cannot analyze Pim local-memory lifetimes for plan verification");
});
return failure();
}
@@ -156,7 +156,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag
for (StringRef name : kRemovedLocalMemoryPlanAttrNames)
if (allocation->hasAttr(name)) {
diagnostics.report(allocation, [name](Operation* op) {
op->emitOpError() << "contains removed PIM local-memory planning attribute '" << name << "'";
op->emitOpError() << "contains removed Pim local-memory planning attribute '" << name << "'";
});
hasFailure = true;
}
@@ -171,7 +171,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag
uint64_t address = static_cast<uint64_t>(addressAttr.getInt());
if (address % 4 != 0 || address > arenaSize || interval.size > arenaSize - address) {
diagnostics.report(allocation, [&](Operation* op) {
op->emitOpError() << "has invalid PIM local-memory range [" << address << ", "
op->emitOpError() << "has invalid Pim local-memory range [" << address << ", "
<< (address <= arenaSize && interval.size <= arenaSize - address
? address + interval.size
: arenaSize)
@@ -221,7 +221,7 @@ verifyLocalMemoryPlan(Operation* coreLikeOp, pim::CappedDiagnosticReporter& diag
memref::AllocOp otherAllocation = other.allocation;
diagnostics.report(allocation, [&](Operation*) {
auto diagnostic = allocation.emitOpError()
<< "PIM local-memory plan assigns simultaneously live allocations to overlapping ranges; first range ["
<< "Pim local-memory plan assigns simultaneously live allocations to overlapping ranges; first range ["
<< conflicting->first << ", " << conflicting->first + other.size << "), second range [" << address
<< ", " << address + interval.size << "), live positions overlap at ["
<< std::max(interval.start, other.start) << ", " << std::min(interval.end, other.end) << "]";
@@ -471,7 +471,7 @@ static LogicalResult appendCoreCommunicationEvents(Block& block,
auto targetCoreId = resolveIndexValue(sendOp.getTargetCoreId(), knowledge);
if (failed(targetCoreId)) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("cannot statically resolve send target core for PIM communication deadlock check");
illegalOp->emitOpError("cannot statically resolve send target core for Pim communication deadlock check");
});
return failure();
}
@@ -490,7 +490,7 @@ static LogicalResult appendCoreCommunicationEvents(Block& block,
if (failed(sourceCoreId)) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError(
"cannot statically resolve receive source core for PIM communication deadlock check");
"cannot statically resolve receive source core for Pim communication deadlock check");
});
return failure();
}
@@ -530,7 +530,7 @@ static void printCommunicationWindow(llvm::raw_ostream& os,
static void printCommunicationDeadlockReport(const DenseMap<int64_t, CommunicationEventVector>& coreEvents,
const DenseMap<int64_t, size_t>& programCounters,
ArrayRef<int64_t> cycle) {
llvm::errs() << "\n=== PIM static communication deadlock report ===\n";
llvm::errs() << "\n=== Pim static communication deadlock report ===\n";
llvm::errs() << "wait cycle:";
for (int64_t coreId : cycle)
llvm::errs() << " " << coreId;
@@ -565,7 +565,7 @@ static void printCommunicationDeadlockReport(const DenseMap<int64_t, Communicati
continue;
printCommunicationWindow(llvm::errs(), coreEvents, coreId, pcIt->second);
}
llvm::errs() << "=== end PIM static communication deadlock report ===\n\n";
llvm::errs() << "=== end Pim static communication deadlock report ===\n\n";
}
static void emitCommunicationDeadlockCycle(ModuleOp moduleOp,
@@ -576,8 +576,8 @@ static void emitCommunicationDeadlockCycle(ModuleOp moduleOp,
auto diagnostic =
moduleOp.emitError()
<< "PIM communication deadlock check found a blocking send/receive cycle while statically simulating the "
"expanded per-core communication streams; see the PIM static communication deadlock report above";
<< "Pim communication deadlock check found a blocking send/receive cycle while statically simulating the "
"expanded per-core communication streams; see the Pim static communication deadlock report above";
for (int64_t coreId : cycle) {
auto eventsIt = coreEvents.find(coreId);
@@ -728,7 +728,7 @@ static LogicalResult verifyNoStaticCommunicationDeadlock(ModuleOp moduleOp,
auto diagnostic =
moduleOp.emitError()
<< "PIM communication deadlock check stalled without finding a closed wait cycle; this usually means a "
<< "Pim communication deadlock check stalled without finding a closed wait cycle; this usually means a "
"send/receive peer is missing or ordered after a finished core";
for (const auto& [coreId, events] : coreEvents) {
size_t pc = programCounters[coreId];
@@ -746,7 +746,7 @@ struct VerificationPass : PassWrapper<VerificationPass, OperationPass<ModuleOp>>
StringRef getArgument() const override { return "verify-pim-pass"; }
StringRef getDescription() const override {
return "Verify that bufferized PIM IR contains only explicit host/device transfers";
return "Verify that bufferized Pim IR contains only explicit host/device transfers";
}
VerificationPass() {}
@@ -763,7 +763,7 @@ struct VerificationPass : PassWrapper<VerificationPass, OperationPass<ModuleOp>>
pim::CappedDiagnosticReporter diagnostics;
if (!hasTarget || failed(targetResources.verify())) {
moduleOp.emitError("PIM codegen verification requires valid injected target resources");
moduleOp.emitError("Pim codegen verification requires valid injected target resources");
signalPassFailure();
return;
}
@@ -792,7 +792,7 @@ struct VerificationPass : PassWrapper<VerificationPass, OperationPass<ModuleOp>>
return;
diagnostics.report(op, [](Operation* illegalOp) {
illegalOp->emitError("illegal Spatial operation reached PIM codegen verification");
illegalOp->emitError("illegal Spatial operation reached Pim codegen verification");
});
});
@@ -833,7 +833,7 @@ struct VerificationPass : PassWrapper<VerificationPass, OperationPass<ModuleOp>>
if (!isAddressOnlyHostOp(&op)) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("illegal host-side runtime op remains after PIM bufferization; "
illegalOp->emitOpError("illegal host-side runtime op remains after Pim bufferization; "
"fold it to constants or lower it into pim.core");
});
continue;
@@ -849,7 +849,7 @@ struct VerificationPass : PassWrapper<VerificationPass, OperationPass<ModuleOp>>
if (diagnostics.hasFailure()) {
diagnostics.emitSuppressedSummary(moduleOp, "verification failures");
moduleOp.emitError("PIM codegen verification failed; see diagnostics above");
moduleOp.emitError("Pim codegen verification failed; see diagnostics above");
hasFailure = true;
}
@@ -928,7 +928,7 @@ private:
bool hasFailure = false;
if (!isSupportedCoreInstructionOp(&op)) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("unsupported executable op reached PIM codegen verification");
illegalOp->emitOpError("unsupported executable op reached Pim codegen verification");
});
hasFailure = true;
}
@@ -990,7 +990,7 @@ private:
if (failed(resolveIndexValue(storeOp.getHostTargetOffset(), knowledge))
|| failed(resolveIndexValue(storeOp.getDeviceSourceOffset(), knowledge))) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen");
illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen");
});
hasFailure = true;
}
@@ -1006,7 +1006,7 @@ private:
if (failed(resolveIndexValue(loadOp.getDeviceTargetOffset(), knowledge))
|| failed(resolveIndexValue(loadOp.getHostSourceOffset(), knowledge))) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen");
illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen");
});
hasFailure = true;
}
@@ -1022,7 +1022,7 @@ private:
if (failed(resolveIndexValue(copyOp.getTargetOffset(), knowledge))
|| failed(resolveIndexValue(copyOp.getSourceOffset(), knowledge))) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError("offset operands must be statically evaluable for PIM codegen");
illegalOp->emitOpError("offset operands must be statically evaluable for Pim codegen");
});
hasFailure = true;
}
@@ -1032,7 +1032,7 @@ private:
&& failed(resolveIndexValue(receiveOp.getOutputOffset(), knowledge))) {
diagnostics.report(&op, [](Operation* illegalOp) {
illegalOp->emitOpError(
"output offset must be statically evaluable for PIM codegen");
"output offset must be statically evaluable for Pim codegen");
});
hasFailure = true;
}
src/PIM/Dialect/Pim/Pim.td
+2 -2
View File
@@ -11,7 +11,7 @@ include "mlir/Dialect/Bufferization/IR/BufferViewFlowOpInterface.td"
def PimDialect : Dialect {
let name = "pim";
let summary = "A low-level dialect for the PIM coprocessors on ReRAM crossbars";
let summary = "A low-level dialect for the Pim coprocessors on ReRAM crossbars";
let cppNamespace = "::onnx_mlir::pim";
}
@@ -27,7 +27,7 @@ def PimTensor :
def PimCoreOp : PimOp<"core", [SingleBlock,
DeclareOpInterfaceMethods<OpAsmOpInterface, ["getAsmBlockArgumentNames"]>]> {
let summary = "Execute a block on a PIM core";
let summary = "Execute a block on a Pim core";
let regions = (region SizedRegion<1>:$body);
src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/ScheduledSpatialPasses.cpp
+1 -1
View File
@@ -28,7 +28,7 @@ static bool hasValidTarget(const SchedulingTarget& target) {
static FailureOr<func::FuncOp> requireEntry(ModuleOp moduleOp) {
auto entry = getPimEntryFunc(moduleOp);
if (failed(entry)) {
moduleOp.emitError("failed to locate the PIM entry function during Spatial scheduling and realization");
moduleOp.emitError("failed to locate the Pim entry function during Spatial scheduling and realization");
return failure();
}
return *entry;
src/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/SpatialDataflowCsvExporter.cpp
+2 -2
View File
@@ -193,7 +193,7 @@ FailureOr<TopLevelOpInfo> buildTopLevelOpInfo(Operation& op, bool isScheduled, s
if constexpr (std::is_same_v<ComputeOpTy, SpatScheduledCompute>) {
if (auto compute = dyn_cast<ComputeOpTy>(&op)) {
auto coreId = getOptionalScheduledCoreId(compute, "spatial dataflow export core id");
auto coreId = getOptionalScheduledCoreId(compute, "Spatial dataflow export core id");
if (failed(coreId))
return failure();
if (*coreId)
@@ -207,7 +207,7 @@ FailureOr<TopLevelOpInfo> buildTopLevelOpInfo(Operation& op, bool isScheduled, s
template <typename BatchOpTy>
FailureOr<SmallVector<int32_t, 8>> getBatchLaneCoreIds(BatchOpTy batch) {
if constexpr (std::is_same_v<BatchOpTy, SpatScheduledComputeBatch>) {
auto coreIds = getOptionalScheduledBatchCoreIds(batch, "spatial dataflow export core ids");
auto coreIds = getOptionalScheduledBatchCoreIds(batch, "Spatial dataflow export core ids");
if (failed(coreIds))
return failure();
if (!*coreIds)
src/PIM/Dialect/Spatial/Spatial.td
+1 -1
View File
@@ -13,7 +13,7 @@ include "mlir/Interfaces/SideEffectInterfaces.td"
def SpatialDialect : Dialect {
let name = "spat";
let summary = "Dialect designed for deep learning computation in a spatial architecture";
let summary = "Dialect designed for deep learning computation in a Spatial architecture";
let cppNamespace = "::onnx_mlir::spatial";
let useDefaultAttributePrinterParser = 0;
let extraClassDeclaration = [{
src/PIM/Passes/PIMPasses.h
+2 -1
View File
@@ -25,7 +25,8 @@ std::unique_ptr<mlir::Pass> createONNXToSpatialPass(
const spatial::SpatialTargetResources& target,
const ONNXToSpatialPlanningOptions& options);
std::unique_ptr<mlir::Pass> createSpatialLayoutPlanningPass();
std::unique_ptr<mlir::Pass> createSpatialLayoutPlanningPass(const spatial::SpatialTargetResources& target);
std::unique_ptr<mlir::Pass> createSpatialLayoutPlanningPass(
const spatial::SpatialTargetResources& target, bool selectTrivialPlan = false);
std::unique_ptr<mlir::Pass> createLowerSpatialPlansPass();
std::unique_ptr<mlir::Pass> createLowerSpatialPlansPass(
const spatial::SpatialTargetResources& target,
src/PIM/Passes/PimCodegen/EmitPimCodePass.cpp
+2 -2
View File
@@ -12,7 +12,7 @@ namespace {
struct EmitPimCodePass : PassWrapper<EmitPimCodePass, OperationPass<ModuleOp>> {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(EmitPimCodePass);
StringRef getArgument() const override { return "emit-pim-code-pass"; }
StringRef getDescription() const override { return "Emit PIM simulator code artifacts"; }
StringRef getDescription() const override { return "Emit Pim simulator code artifacts"; }
EmitPimCodePass() {}
EmitPimCodePass(const EmitPimCodePass& pass) {}
@@ -25,7 +25,7 @@ struct EmitPimCodePass : PassWrapper<EmitPimCodePass, OperationPass<ModuleOp>> {
int compiler_error_code = compileToPimCode(moduleOp, pimDir);
if (compiler_error_code != CompilerSuccess) {
moduleOp.emitError() << "failed to emit PIM simulator code artifacts; compiler error code "
moduleOp.emitError() << "failed to emit Pim simulator code artifacts; compiler error code "
<< compiler_error_code;
signalPassFailure();
}
tools/analyze_spatial_ir_cardinality.py
+1 -1
View File
@@ -238,7 +238,7 @@ def print_report(path: Path, counts: Counter, groups: dict[tuple[str, str], Chai
def main() -> None:
parser = argparse.ArgumentParser(description="Analyze repeated Spatial/PIM tensor IR cardinality patterns.")
parser = argparse.ArgumentParser(description="Analyze repeated Spatial/Pim tensor IR cardinality patterns.")
parser.add_argument("paths", nargs="+", help="MLIR files to analyze.")
parser.add_argument("--limit", type=int, default=12, help="Maximum number of hot chains to print per file.")
args = parser.parse_args()
validation/.gitignore
+4 -15
View File
@@ -1,23 +1,12 @@
operations/**/inputs
operations/**/outputs
operations/**/raptor
operations/**/runner
operations/**/simulation
operations/**/artifacts
operations/**/*.csv
!operations/validation_results.csv
networks/**/inputs
networks/**/outputs
networks/**/raptor
networks/**/raptor_functional
networks/**/pimcomp
networks/**/runner
networks/**/simulation
networks/**/real_image_val
networks/**/artifacts
networks/**/*.png
networks/**/*.jpg
networks/**/*.csv
!networks/validation_results.csv
!networks/full_net/validation_results.csv
!networks/pimcomp_models/validation_results.csv
!networks/pimcomp_models/results.csv
!networks/pimcomp_models/results_comparison.csv
!networks/pimcomp_models/results_ablation.csv
validation/README.md
+41 -54
View File
@@ -1,14 +1,14 @@
# Raptor Validation
# Raptor validation
`validate.py` validates every ONNX model below a selected directory. For each
model it can:
1. compile an ONNX-MLIR reference library and runner;
2. generate deterministic random inputs;
3. compile PIM artifacts with Raptor;
4. run the reference implementation and functional PIM simulator;
3. compile Pim artifacts with Raptor;
4. run the reference implementation and functional Pim simulator;
5. compare their outputs;
6. run `pimsim-nn` to report latency, throughput, power, and energy.
6. run Pimsim to report latency, throughput, power, and energy.
Run the script from the repository root with the repository Python environment.
@@ -61,29 +61,12 @@ Validate a network or network slice:
The script discovers them recursively and writes `validation_results.csv` in
that directory while retaining separate latency and throughput terminal tables.
## Raptor vs PIMCOMP comparison
## Pim validation tools
The PIMCOMP paper-model suite has a one-command multi-architecture comparison:
```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py
```
The runner verifies PIMCOMP's population-200, 1000-iteration GA settings,
expects Raptor and the existing `third_party/PIMCOMP-NN/build` tree to already
be built, and compares the configured paper models in parallel. Set `--jobs` to
control comparison workers;
the runner leaves `OMP_NUM_THREADS` at its environment default. Use
`--models vgg8` for one model or `--dry-run` to print the commands.
Generated artifacts are stored beside each model and ignored by Git. The
comparison reuses model-level `common/inputs/`, `common/outputs/`,
`common/runner/` across architectures, modes, and pipelines. Pimsim configs and
network meshes are referenced directly from
`validation/pimsim_configs/pimcomp/`. Architecture- and pipeline-specific `raptor/`, `simulation/`, and
PIMCOMP artifacts stay under each comparison directory. See
[`networks/pimcomp_models/README.md`](networks/pimcomp_models/README.md) for
profiles, model provenance, limitations, and remote execution.
- [Pimcomp model suite](networks/pimcomp_models/README.md)
- [Pimcomp model comparison tools](tools/pim/pimcomp/compare/README.md)
- [Pimcomp correctness study](tools/pim/pimcomp/correctness/README.md)
- [Raptor compiler ablation study](tools/pim/ablation/README.md)
## Validation modes
@@ -92,7 +75,7 @@ uses one input, while throughput uses `--pipeline=4` with four distinct inputs.
Both modes reuse the generated input batch, native runner, and reference
outputs, and every throughput output is compared with its own reference.
Use `--compile-only` to build the reference runner and PIM artifacts without
Use `--compile-only` to build the reference runner and Pim artifacts without
executing either implementation:
```bash
@@ -139,23 +122,23 @@ count with `-j` or `--jobs`:
| `--onnx-include-dir PATH` | ONNX-MLIR runtime include directory. Required unless `--clean` is used. |
| `--operations-dir PATH` | Directory tree containing models. Defaults to `validation/operations`. |
| `--simulator-dir PATH` | Functional `pim-simulator` crate directory. Defaults to the in-tree simulator. |
| `--non-functional-simulator-build-dir PATH` | `pimsim-nn` build directory. Defaults to the in-tree build. |
| `--non-functional-simulator-build-dir PATH` | Pimsim build directory. Defaults to the in-tree build. |
| `--pimcomp-config {arch-a,arch-b,arch-c}` | Non-functional hardware/timing profile. Defaults to `arch-a`. |
| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency, throughput, power, and energy measurement. |
| `--no-fast` | Disable fast throughput convergence for authoritative full-duration `pimsim-nn` experiments. |
| `--skip-non-functional-simulation` | Skip Pimsim latency, throughput, power, and energy measurement. |
| `--no-fast` | Disable fast throughput convergence for authoritative full-duration Pimsim experiments. |
| `--threshold FLOAT` | Absolute output-comparison tolerance. Defaults to `1e-3`. |
| `--relative-threshold FLOAT` | Relative output-comparison tolerance. Defaults to `1e-5`. |
| `--seed INT` | Seed for generated inputs. Defaults to `0`. |
| `--crossbar-size INT` | Crossbar dimensions passed to Raptor. Defaults to the Arch-A value, `128`. |
| `--crossbar-count INT` | Crossbars per core passed to Raptor. Defaults to the Arch-A value, `96`. |
| `--core-count INT` | PIM core count passed to Raptor. Defaults to the Arch-A value, `168`. |
| `--core-count INT` | Pim core count passed to Raptor. Defaults to the Arch-A value, `168`. |
| `--raptor-extra-arg=ARG` | Additional Raptor compiler argument. Repeat for multiple arguments. |
| `--command-timeout-seconds FLOAT` | Timeout for each compiler, runner, and simulator subprocess. Defaults to `1000000.0`. |
| `-j INT`, `--jobs INT` | Parallel validation workers. Defaults to all available CPUs and must be at least one. |
| `--clean` | Remove generated validation artifacts and exit. |
| `--compile-only` | Compile reference and PIM artifacts without execution or comparison. |
| `--compile-only` | Compile reference and Pim artifacts without execution or comparison. |
| `--run-only` | Reuse compiled artifacts and perform execution, simulation, and comparison. |
| `--verbose` | Print passing per-stage and subprocess logs, plus average PIM pass timings. |
| `--verbose` | Print passing per-stage and subprocess logs, plus average Pim pass timings. |
Arguments beginning with `--` that are passed through to Raptor should use the
equals form:
@@ -166,7 +149,7 @@ equals form:
## Hardware profiles and non-functional simulation
The selected PIMCOMP profile must match `--core-count`, `--crossbar-count`, and
The selected Pimcomp profile must match `--core-count`, `--crossbar-count`, and
`--crossbar-size`. A mismatch disables only non-functional simulation and
prints the incompatible values; functional validation still runs.
@@ -189,27 +172,30 @@ comparison. A non-functional simulation failure remains visible as `ERROR` in
the corresponding latency, throughput, power, or energy columns but does not
change a functional PASS.
`pimsim-nn` does not currently implement the `vsoftmax` instruction. When its
Pimsim does not currently implement the `vsoftmax` instruction. When its
explicit unsupported-op diagnostic is encountered, Softmax validations retain
their functional PASS and show `UNSUPPORTED` in the non-functional columns.
Other `pimsim-nn` failures remain `ERROR`.
Other Pimsim failures remain `ERROR`.
## Generated artifacts
Artifacts are written beside each model:
Generated files are grouped below an `artifacts/` directory beside each model
or operation case. This keeps checked-in ONNX files and generated trees
separate and lets `--clean` remove the complete workspace, including stale
validation lock files:
| Path | Contents |
|---|---|
| `inputs.csv` | Generated inputs, one batch entry per line. |
| `inputs/`, `outputs/`, `runner/` | Inputs, reference outputs, and the runner shared by latency and throughput validation. |
| `raptor/pim/`, `simulation/latency/` | Latency PIM artifacts and functional simulator outputs. |
| `raptor/throughput/pim/`, `simulation/throughput/` | Pipeline-4, batch-4 throughput PIM artifacts and functional simulator outputs. |
| `common/inputs/` | Shared generated input CSV files. |
| `common/outputs/` | Shared ONNX-MLIR reference output CSV files. |
| `common/runner/` | Shared reference runner source, build tree, and library. |
| `<arch>/<mode>[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and PIM artifacts. |
| `<arch>/<mode>[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. |
| `<arch>/<mode>[/pipelineN]/pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
| `artifacts/inputs.csv` | Generated inputs, one batch entry per line. |
| `artifacts/inputs/`, `artifacts/outputs/`, `artifacts/runner/` | Inputs, reference outputs, and the runner shared by latency and throughput validation. |
| `artifacts/raptor/pim/`, `artifacts/simulation/latency/` | Latency Pim artifacts and functional simulator outputs. |
| `artifacts/raptor/throughput/pim/`, `artifacts/simulation/throughput/` | Pipeline-4, batch-4 throughput Pim artifacts and functional simulator outputs. |
| `artifacts/common/inputs/` | Shared generated input CSV files. |
| `artifacts/common/outputs/` | Shared ONNX-MLIR reference output CSV files. |
| `artifacts/common/runner/` | Shared reference runner source, build tree, and library. |
| `artifacts/<arch>/<mode>[/pipelineN]/raptor/` | Architecture- and pipeline-specific Raptor MLIR and Pim artifacts. |
| `artifacts/<arch>/<mode>[/pipelineN]/simulation/` | Functional simulator outputs for that comparison. |
| `artifacts/<arch>/<mode>[/pipelineN]/pimcomp/` | Pimcomp graph, instruction, simulator, and comparison-report artifacts. |
Each comparison's `raptor/` directory may include `spatial0.mlir`,
`spatial1_graph.mlir`, `spatial2_trivial_merged.mlir`,
@@ -240,22 +226,23 @@ The generated operation inventory is documented in
## Manual functional simulator tracing
After validation has produced a `raptor/pim/` directory, rerun the functional
After validation has produced an `artifacts/raptor/pim/` directory, rerun the functional
simulator with tracing from its crate directory:
```bash
cd backend-simulators/pim/pim-simulator
cargo run --no-default-features --features tracing --release \
--package pim-simulator --bin pim-simulator -- \
-f /path/to/workspace/raptor/pim \
-o /path/to/workspace/simulation/out.bin \
-f /path/to/workspace/artifacts/raptor/pim \
-o /path/to/workspace/artifacts/simulation/out.bin \
-d <addr0>,<size0>,<addr1>,<size1>,... \
--mode latency \
--input /path/to/workspace/simulation/inputs/input_0.bin
--batch-size 1 \
--input-dir /path/to/workspace/artifacts/simulation/inputs
```
Throughput mode additionally requires `--batch-size N` and exactly `N`
`--input` arguments. Each input binary concatenates the model tensors in graph
Throughput mode additionally requires `--batch-size N` and at least `N`
`input_<index>.bin` files in `--input-dir`. Each input binary concatenates the model tensors in graph
input order. The comparison validator also writes one native reference and one
`simulation/*_iterations/output_*.bin` dump per batch entry, and checks every
entry rather than only the final output.
@@ -268,7 +255,7 @@ validator normally derives the `-d` address and byte ranges from
The final table reports latency and throughput functional pass/fail state plus
non-functional latency, throughput, power, and energy. The summary includes pass/fail totals, non-functional simulation
counts, total measured latency, and average PIM pass timings when `--verbose`
counts, total measured latency, and average Pim pass timings when `--verbose`
is enabled.
- Exit status `0`: all discovered models passed, or cleanup completed.
validation/networks/pimcomp_models/README.md
+66 -62
View File
@@ -1,38 +1,38 @@
# PIMCOMP comparison models
# Pimcomp comparison models
This directory contains the four networks evaluated in
[PIMCOMP: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159):
[Pimcomp: An End-to-End DNN Compiler for Processing-In-Memory Accelerators](https://arxiv.org/pdf/2411.09159):
VGG-8, ResNet-18, ResNet-34, and GoogLeNet. It also contains YOLO11n as an
additional compiler comparison model.
See the runner-generated [results.csv](results.csv) for the current comparison
See the runner-generated [results_comparison.csv](results_comparison.csv) for the current comparison
results. Rows are retained separately for each model, architecture, mode, and
pipeline. It records separate `PASS`/`FAIL` functional-validation fields for
the Raptor and PIMCOMP artifacts; rows without a generated report contain `NA`.
Running the runner with `--arch arch-b` or `--arch arch-c` appends those
architecture rows without replacing the existing `arch-a` entries.
the Raptor and Pimcomp artifacts; rows without a generated report contain `NA`.
Use `--archs` to select the architecture rows to generate; existing rows for
other architectures remain unchanged.
## Models and provenance
| Directory | Model | Input | Provenance |
|--------------|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `resnet18/` | ResNet-18 v1 | `1x3x224x224` | Symlink to the complete [ONNX Model Zoo `resnet18-v1-7`](https://huggingface.co/onnxmodelzoo/resnet18-v1-7) model already present at `../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as PIMCOMP's frontend does. |
| `resnet34/` | ResNet-34 v1 | `1x3x224x224` | [ONNX Model Zoo `resnet34-v1-7`](https://huggingface.co/onnxmodelzoo/resnet34-v1-7), with its symbolic batch fixed to 1 as Pimcomp's frontend does. |
| `googlenet/` | GoogLeNet | `1x3x224x224` | Unmodified [ONNX Model Zoo `googlenet-12`](https://huggingface.co/onnxmodelzoo/googlenet-12). |
| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [PIMCOMP VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. |
| `vgg8/` | VGG-8 reconstruction | `1x1x28x28` | Reconstruction of the [Pimcomp VGG-8 benchmark](https://arxiv.org/html/2411.09159#S8.SS1), with six convolution and two fully connected layers. |
| `yolo11n/` | YOLO11n detection | `1x3x640x640` | Derived from the canonical local model at `../yolo11n/depth_51/yolo11n_depth_51.onnx`, exported from [Ultralytics YOLO11n](https://github.com/ultralytics/ultralytics/blob/main/docs/en/models/yolo11.md). |
`googlenet/googlenet-12-pimsim-nn.onnx` is the explicit pimsim-nn-ready GoogLeNet model.
`googlenet/googlenet-12-pimsim-nn.onnx` is the explicit Pimsim-ready GoogLeNet model.
It removes the two LRN nodes and terminal Softmax from the original model,
so that the comparison covers only operations scheduled by PIMCOMP and supported by pimsim-nn.
so that the comparison covers only operations scheduled by Pimcomp and supported by Pimsim.
`yolo11n/yolo11n-pimsim-nn.onnx` is the explicit pimsim-nn-ready YOLO11n model.
`yolo11n/yolo11n-pimsim-nn.onnx` is the explicit Pimsim-ready YOLO11n model.
It removes the Softmax nodes from the original model,
so that the compiled artifact can be simulated in pimsim-nn.
so that the compiled artifact can be simulated in Pimsim.
## Unsupported and ignored operations
PIMCOMP's frontend accepts exactly these ONNX operations:
Pimcomp's frontend accepts exactly these ONNX operations:
```text
Add, AveragePool, BatchNormalization, Clip, Concat, Conv, Dropout, Flatten,
@@ -40,14 +40,14 @@ Gather, Gemm, GlobalAveragePool, LRN, MatMul, MaxPool, Mul, Pad, Relu, Reshape,
Shape, Sigmoid, Softmax, Squeeze, Sub, Sum, Tanh, Transpose, Unsqueeze
```
`Constant` is consumed as frontend metadata rather than emitted as a PIMCOMP
`Constant` is consumed as frontend metadata rather than emitted as a Pimcomp
node. Every other ONNX operation is unsupported: the frontend prints
`operation: <type> not considered` and stops at the first occurrence. Thus the
complete unsupported set is the complement of the allowlist above for the
model's ONNX opset. In particular, YOLO11n contains unsupported `Split` and
`Resize` nodes.
PIMCOMP's low-latency scheduler, hierarchy mapper, and genetic algorithm use
Pimcomp's low-latency scheduler, hierarchy mapper, and genetic algorithm use
this complete explicit no-consider set:
```text
@@ -63,7 +63,7 @@ specific Shape-Gather-Unsqueeze-Concat shape chain, and merge Pad into its
consumer. These transformations do not make an otherwise standalone ignored
operation timed.
`pimsim-nn` consumes PIM ISA instructions. It supports every named opcode
Pimsim consumes Pim ISA instructions. It supports every named opcode
in the shared serialized range except `vsoftmax` (opcode 21),
which is rejected explicitly in both JSON and binary input. It
silently ignores no opcode; unknown names and numbers are errors.
@@ -71,12 +71,12 @@ silently ignores no opcode; unknown names and numbers are errors.
These boundaries explain the dedicated artifacts:
- GoogLeNet's two LRN nodes and terminal Softmax perform real computation but
are ignored by PIMCOMP, so the common latency artifact removes them. Its
are ignored by Pimcomp, so the common latency artifact removes them. Its
inference Dropout and shape-only Reshape can remain without adding compute.
- YOLO11n's latency artifact bypasses exactly its two Softmax nodes so it can
run in `pimsim-nn`. Every other node, including MatMul, Transpose, and the
run in Pimsim. Every other node, including MatMul, Transpose, and the
final detection-decoding tail, remains present and timed by Raptor. No
PIMCOMP latency is reported because its frontend stops at `Split` and also
Pimcomp latency is reported because its frontend stops at `Split` and also
lacks `Resize`; compiling that prefix would not represent YOLO11n.
The authoritative lists are in
@@ -85,16 +85,16 @@ The authoritative lists are in
[`ISA.h`](../../../backend-simulators/pim/pimsim-nn/src/isa/ISA.h), and
[`Instruction.cpp`](../../../backend-simulators/pim/pimsim-nn/src/isa/Instruction.cpp).
The PIMCOMP authors did not publish the ONNX checkpoints used by the paper.
Running PIMCOMP's frontend on the three Model Zoo files above produces JSON
graphs exactly equal to PIMCOMP-NN's bundled `resnet18.json`, `resnet34.json`,
The Pimcomp authors did not publish the ONNX checkpoints used by the paper.
Running Pimcomp's frontend on the three Model Zoo files above produces JSON
graphs exactly equal to Pimcomp's bundled `resnet18.json`, `resnet34.json`,
and `googlenet.json`.
There is no VGG-8 artifact in the ONNX Model Zoo or any PIMCOMP-NN revision.
There is no VGG-8 artifact in the ONNX Model Zoo or any Pimcomp revision.
The included VGG-8 therefore has deterministic random weights and is suitable
for compiler and simulator comparison, not paper-accuracy reproduction. The
paper also says that VGG-8 and ResNet-18 were trained on MNIST, while the
published PIMCOMP graphs and ResNet Model Zoo artifacts use ImageNet shapes.
published Pimcomp graphs and ResNet Model Zoo artifacts use ImageNet shapes.
Current SHA-256 checksums:
@@ -113,10 +113,10 @@ The files in
[`../../pimsim_configs/pimcomp/`](../../pimsim_configs/pimcomp/)
encode Table V's explicit resource parameters.
Each profile subdirectory contains pre-generated latency and throughput
`pimsim-nn` configs plus its matching mesh; comparison and validation reference
Pimsim configs plus its matching mesh; comparison and validation reference
these canonical artifacts directly.
| Config | Cores | Crossbars/core | Crossbar | Cell | PIMCOMP layout |
| Config | Cores | Crossbars/core | Crossbar | Cell | Pimcomp layout |
|------------------------------|------------------:|---------------:|------------|------:|-----------------|
| `arch-a/latency_config.json` | 168 | 96 | `128x128` | 2-bit | `12x14` |
| `arch-b/latency_config.json` | 138 | 128 | `128x128` | 2-bit | `6x23` |
@@ -125,10 +125,10 @@ these canonical artifacts directly.
`adc_count` is 16, matching the paper's 16-bit fixed-point weight precision.
The paper does not give a two-dimensional core topology for Arch-A/B, so the
factorizations above preserve core count but cannot reproduce unpublished NoC
placement details. Released PIMCOMP-NN has no chip-count field; Arch-C is
placement details. Released Pimcomp has no chip-count field; Arch-C is
therefore flattened to 64 cores and does not model chip boundaries.
The remaining latency and power values come from PIMCOMP-NN's released default
The remaining latency and power values come from Pimcomp's released default
configuration. Consequently, instruction/resource comparisons are
reproducible, but absolute paper power and energy numbers are not.
@@ -147,9 +147,9 @@ cmake --build third_party/PIMCOMP-NN/build --target PIMCOMP-NN
Do not build either project with `ninja` directly.
## Compile with PIMCOMP
## Compile with Pimcomp
PIMCOMP-NN reads `third_party/PIMCOMP-NN/config.json` directly. Back it up,
Pimcomp reads `third_party/PIMCOMP-NN/config.json` directly. Back it up,
select one paper profile, and restore it when the shell exits:
```bash
@@ -162,7 +162,7 @@ trap 'cp "$CONFIG_BACKUP" "$PIMCOMP/config.json"' EXIT
cp "$PIMCOMP_CONFIGS/arch-a/latency_config.json" "$PIMCOMP/config.json"
```
The Model Zoo files map exactly to PIMCOMP's bundled model names, so compile
The Model Zoo files map exactly to Pimcomp's bundled model names, so compile
them directly:
```bash
@@ -179,7 +179,7 @@ cd "$PIMCOMP/build"
./PIMCOMP-NN -m=googlenet -r=balance -p=element -o=YES -v=YES -s=YES
```
VGG-8 first needs PIMCOMP's JSON frontend. Use a temporary ONNX copy because
VGG-8 first needs Pimcomp's JSON frontend. Use a temporary ONNX copy because
the released frontend rewrites the input batch dimension in place:
```bash
@@ -201,62 +201,66 @@ random placement code occasionally segfaults; an unchanged retry succeeded in
the observed cases.
The paper's optimizer uses a genetic algorithm with population 200 and up to
1000 iterations. The checked-out PIMCOMP submodule already has both paper
1000 iterations. The checked-out Pimcomp submodule already has both paper
settings in `backend/GeneticAlgorithm.h`; select them with `-r=GA`. Fitness
evaluation uses OpenMP and bounded bandwidth timelines. Set `OMP_NUM_THREADS`
to control its parallelism; otherwise OpenMP uses the available CPUs. The GA
uses the fixed seed `1`, so repeated serial and parallel runs are reproducible.
## Compare Raptor and PIMCOMP
## Compare Raptor and Pimcomp
The comparison driver uses one random input and one native ONNX-MLIR reference,
compiles both instruction streams, runs both through `pimsim-nn`, runs
compiles both instruction streams, runs both through Pimsim, runs
functional validation through `pim-simulator`, and writes Markdown and JSON
reports.
To reproduce all configured architectures and both latency/throughput modes,
use the model-by-model runner. It verifies the paper GA settings, expects Raptor
To reproduce the default `arch-a`/`arch-b` architectures and both
latency/throughput modes, use the model-by-model runner. Use `--archs` to
specify a different architecture set. It verifies the paper GA settings, expects Raptor
and the existing `third_party/PIMCOMP-NN/build` tree to already be built, then
runs the comparisons in parallel and regenerates `results.csv` from the JSON
runs the comparisons in parallel and regenerates `results_comparison.csv` from the JSON
reports:
```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
```
Use `--arch arch-a --mode latency` for only the Arch-A latency experiment.
Use `--archs arch-a --mode latency` for only the Arch-A latency experiment, or
`--archs arch-a arch-b arch-c` to run all three architectures.
Each model directory has a shared ignored `common/` directory containing
Each model directory has a shared ignored `artifacts/common/` directory containing
`inputs/`, `outputs/`, and the native `runner/`.
Pimsim-nn configs and network meshes remain canonical under
Pimsim configs and network meshes remain canonical under
`validation/pimsim_configs/pimcomp/` and are referenced in place.
Architecture- and pipeline-specific `raptor/`, `simulation/`, and PIMCOMP
artifacts remain under each comparison directory; PIMCOMP outputs are prepared
Architecture- and pipeline-specific `raptor/`, `simulation/`, and Pimcomp
artifacts remain under each `artifacts/<arch>/<mode>[/pipelineN]/` comparison
directory; ablation variants use
`artifacts/<arch>/<mode>[/pipelineN]/ablation/<variant>/`. Pimcomp outputs are prepared
once per model/architecture/mode and linked into the other pipeline directories;
`comparison_report.{md,json}`
live under its `pimcomp/`. The frontend regenerates
one isolated `models/JSON/` graph because PIMCOMP requires that relative
one isolated `models/JSON/` graph because Pimcomp requires that relative
layout; it is removed after a successful backend run and the shared submodule
model directory is never modified. The original ONNX model is passed to the
frontend unchanged.
PIMCOMP's source tree and build directory remain unchanged at runtime. Use
Pimcomp's source tree and build directory remain unchanged at runtime. Use
`--models vgg8` to run one model, `--mode throughput` to select one mode,
`--pipeline 4` to select one throughput pipeline, `--only raptor` or
`--only pimcomp` to reuse the other compiler's existing artifacts, `--dry-run`
to inspect every command, or `--out-dir PATH` to keep results outside
`validation/`. Use `--clean` to remove generated comparison artifacts and
summaries. Selecting a subset replaces only those comparison rows and
recomputes the aggregate `results.csv`; missing shared inputs, outputs, or the
summaries, including stale reference lock files. Selecting a subset replaces only those comparison rows and
recomputes the aggregate `results_comparison.csv`; missing shared inputs, outputs, or the
reference runner are generated even for an isolated run. Use `--jobs 4` to cap
parallel comparisons. The per-stage timeout is unlimited by default; pass a
positive `--timeout-seconds` value to impose one.
Throughput comparisons default to `pimsim-nn --fast` with a 1000 ms
convergence deadline. Add `--no-fast` for authoritative full-duration runs.
PIMCOMP receives the original ONNX model, and its frontend applies native
Pimcomp receives the original ONNX model, and its frontend applies native
BatchNormalization fusion when the graph matches its supported Conv/Gemm pattern.
The runner continues after a failed model so all reports are produced.
The known PIMCOMP batch-scheduling correctness issue and a reproducible
The known Pimcomp batch-scheduling correctness issue and a reproducible
reference-intermediate prefill experiment are documented in
[`validation/tools/pim/pimcomp/correctness/README.md`](../../tools/pim/pimcomp/correctness/README.md).
@@ -265,9 +269,9 @@ Arch-A low-latency example:
```bash
RAPTOR_ROOT=$PWD
"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py" \
"$RAPTOR_ROOT/.venv/bin/python" "$RAPTOR_ROOT/validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py" \
--model "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/resnet34-v1-7.onnx" \
--out-dir "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/arch-a/latency" \
--out-dir "$RAPTOR_ROOT/validation/networks/pimcomp_models/resnet34/artifacts/arch-a/latency" \
--pimcomp-config "$RAPTOR_ROOT/validation/pimsim_configs/pimcomp/arch-a/latency_config.json" \
--core-count 168 \
--crossbar-count 96 \
@@ -279,10 +283,10 @@ RAPTOR_ROOT=$PWD
```
Use the same command with
`yolo11n/yolo11n-pimsim-nn.onnx` to probe YOLO11n. Released PIMCOMP-NN cannot
`yolo11n/yolo11n-pimsim-nn.onnx` to probe YOLO11n. Released Pimcomp cannot
compile it: the frontend stops at `/model.2/Split`, and it also has no mapping
for YOLO11n's two nearest-neighbor `Resize` nodes. Treating the emitted prefix
as YOLO11n would produce a misleading latency, so no PIMCOMP number is
as YOLO11n would produce a misleading latency, so no Pimcomp number is
reported for this model.
For Arch-A high throughput, use `--pimsim-mode throughput
@@ -303,18 +307,18 @@ generated report.
The functional and non-functional simulators intentionally consume different
artifacts:
- Raptor and PIMCOMP are validated against the native ONNX-MLIR reference as
- Raptor and Pimcomp are validated against the native ONNX-MLIR reference as
FP32 programs in the Rust simulator. Raptor's emitted program is already
FP32. The PIMCOMP-to-Rust export expands its element-addressed storage and
FP32. The Pimcomp-to-Rust export expands its element-addressed storage and
byte-sized transfers to FP32, emits `setbw 32, 32`, and keeps vector
`imm_len` fields as element counts.
- PIMCOMP's original `SimulationInfo.gz` is copied unchanged for `pimsim-nn`.
PIMCOMP hardcodes `setbw 8, 8` and one byte per element without performing
- Pimcomp's original `SimulationInfo.gz` is copied unchanged for Pimsim.
Pimcomp hardcodes `setbw 8, 8` and one byte per element without performing
numerical quantization; this artifact is used only for latency estimation.
- Raptor's original FP32 artifact remains unchanged for functional validation.
A separate `raptor/pimsim_nn/` view uses `setbw 8, 8` and scales its
byte-addressed storage and transfer sizes from four bytes to one byte per
element. Vector `imm_len` fields remain element counts. Like PIMCOMP's
element. Vector `imm_len` fields remain element counts. Like Pimcomp's
artifact, this view is not numerically valid and is used only for a fair
non-functional comparison.
@@ -326,10 +330,10 @@ Current Raptor status:
- VGG-8, ResNet-18, fixed-batch ResNet-34, and GoogLeNet compile on Arch-A.
- Use `googlenet-12-pimsim-nn.onnx` for the paper-matched latency comparison.
It removes the two LRN nodes and terminal softmax that PIMCOMP does not
It removes the two LRN nodes and terminal softmax that Pimcomp does not
schedule.
- Raptor currently accepts one square `--crossbar-size`; Arch-C's rectangular
`512x1024` arrays can therefore be compiled by PIMCOMP but not compared
`512x1024` arrays can therefore be compiled by Pimcomp but not compared
exactly with Raptor.
Do not change the hardware profile to bypass either limitation; that would no
@@ -365,7 +369,7 @@ python3 -m venv .venv
.venv/bin/python -m pip install numpy onnx onnxruntime colorama
# Run every configured comparison in parallel.
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
```
Copy reports back without transferring large compiler artifacts:
validation/networks/pimcomp_models/results.csv
-61
View File
@@ -1,61 +0,0 @@
arch,model,mode,raptor_pipeline,pimcomp_pipeline,raptor_functional_validation,pimcomp_functional_validation,raptor_latency_ms,pimcomp_latency_ms,raptor_throughput_samples_s,pimcomp_throughput_samples_s,raptor_power_mw,pimcomp_power_mw,raptor_energy_pj,pimcomp_energy_pj,better_compiler,speedup
arch-a,vgg8,latency,1,element,PASS,PASS,1.47,7.99,NA,NA,325.67,200.11,477232782.04,1597904071.12,raptor,5.45
arch-a,vgg8,throughput,2,batch,PASS,PASS,0.94,1.04,1060.00,965.00,377.54,482.60,474325940.70,541691691.00,raptor,1.10
arch-a,vgg8,throughput,4,batch,PASS,PASS,1.19,1.04,839.00,965.00,310.32,482.60,500148858.20,541691691.00,pimcomp,1.15
arch-a,vgg8,throughput,8,batch,PASS,PASS,1.52,1.04,658.00,965.00,279.37,482.60,554348483.90,541691691.00,pimcomp,1.47
arch-a,resnet18,latency,1,element,PASS,FAIL,15.65,58.18,NA,NA,425.87,238.39,6666000553.12,13869146503.12,raptor,3.72
arch-a,resnet18,throughput,2,batch,PASS,FAIL,30.55,13.46,32.70,74.30,295.90,481.60,10218790630.00,6562535800.00,pimcomp,2.27
arch-a,resnet18,throughput,4,batch,PASS,FAIL,26.60,13.46,37.60,74.30,313.14,481.60,10993189350.00,6562535800.00,pimcomp,1.98
arch-a,resnet18,throughput,8,batch,PASS,FAIL,35.51,13.46,28.20,74.30,275.95,481.60,12844245960.00,6562535800.00,pimcomp,2.63
arch-a,resnet34,latency,1,element,PASS,FAIL,32.37,66.71,NA,NA,391.82,277.48,12684227802.68,18511540483.68,raptor,2.06
arch-a,resnet34,throughput,2,batch,FAIL,FAIL,NA,23.58,NA,42.40,NA,515.66,NA,12277699310.00,NA,NA
arch-a,resnet34,throughput,4,batch,FAIL,FAIL,NA,23.58,NA,42.40,NA,515.66,NA,12277699310.00,NA,NA
arch-a,resnet34,throughput,8,batch,PASS,FAIL,115.11,23.58,8.69,42.40,229.85,515.66,29881854960.00,12277699310.00,pimcomp,4.88
arch-a,googlenet,latency,1,element,PASS,PASS,13.03,62.92,NA,NA,465.11,231.19,6060646341.92,14547510894.24,raptor,4.83
arch-a,googlenet,throughput,2,batch,PASS,FAIL,24.01,18.27,41.70,54.70,328.17,435.11,8529823770.00,8164469406.00,pimcomp,1.31
arch-a,googlenet,throughput,4,batch,PASS,FAIL,27.01,18.27,37.00,54.70,305.68,435.11,10498372450.00,8164469406.00,pimcomp,1.48
arch-a,googlenet,throughput,8,batch,PASS,FAIL,40.03,18.27,25.00,54.70,268.61,435.11,15290418970.00,8164469406.00,pimcomp,2.19
arch-a,yolo11n,latency,1,element,PASS,FAIL,267.73,NA,NA,NA,235.73,NA,63111612279.00,NA,NA,NA
arch-a,yolo11n,throughput,2,batch,PASS,FAIL,500.00,NA,2.00,NA,227.74,NA,113867559100.00,NA,NA,NA
arch-a,yolo11n,throughput,4,batch,PASS,FAIL,500.00,NA,2.00,NA,233.16,NA,116581237400.00,NA,NA,NA
arch-a,yolo11n,throughput,8,batch,PASS,FAIL,500.00,NA,2.00,NA,253.59,NA,126797224900.00,NA,NA,NA
arch-b,vgg8,latency,1,element,PASS,PASS,1.44,7.15,NA,NA,304.72,173.77,438329594.04,1242814988.12,raptor,4.97
arch-b,vgg8,throughput,2,batch,PASS,PASS,1.56,1.07,640.00,932.00,287.50,409.47,478592086.00,439288835.80,pimcomp,1.46
arch-b,vgg8,throughput,4,batch,PASS,PASS,1.14,1.07,874.00,932.00,323.79,409.47,457617420.30,439288835.80,pimcomp,1.07
arch-b,vgg8,throughput,8,batch,PASS,PASS,1.48,1.07,676.00,932.00,282.34,409.47,437422264.40,439288835.80,pimcomp,1.38
arch-b,resnet18,latency,1,element,PASS,FAIL,17.98,64.38,NA,NA,362.63,201.66,6520518736.12,12983607699.12,raptor,3.58
arch-b,resnet18,throughput,2,batch,PASS,FAIL,33.84,13.43,29.60,74.40,253.32,484.80,9423636948.00,6569655551.00,pimcomp,2.51
arch-b,resnet18,throughput,4,batch,PASS,FAIL,26.14,13.43,38.30,74.40,286.86,484.80,9493311384.00,6569655551.00,pimcomp,1.94
arch-b,resnet18,throughput,8,batch,PASS,FAIL,28.29,13.43,35.30,74.40,272.66,484.80,11314980340.00,6569655551.00,pimcomp,2.11
arch-b,resnet34,latency,1,element,PASS,FAIL,49.51,107.20,NA,NA,284.72,206.47,14097763205.68,22134333948.68,raptor,2.17
arch-b,resnet34,throughput,2,batch,PASS,FAIL,93.13,22.80,10.70,43.90,215.38,493.39,21278771500.00,11381213070.00,pimcomp,4.10
arch-b,resnet34,throughput,4,batch,PASS,FAIL,67.55,22.80,14.80,43.90,237.97,493.39,19346594270.00,11381213070.00,pimcomp,2.97
arch-b,resnet34,throughput,8,batch,PASS,FAIL,115.01,22.80,8.70,43.90,197.27,493.39,26678372930.00,11381213070.00,pimcomp,5.05
arch-b,googlenet,latency,1,element,PASS,PASS,16.08,37.94,NA,NA,378.99,242.04,6095729659.92,9181933205.24,raptor,2.36
arch-b,googlenet,throughput,2,batch,PASS,FAIL,28.37,21.39,35.20,46.70,267.15,417.83,8684222064.00,9014655684.00,pimcomp,1.33
arch-b,googlenet,throughput,4,batch,PASS,FAIL,26.16,21.39,38.20,46.70,280.21,417.83,9300527914.00,9014655684.00,pimcomp,1.22
arch-b,googlenet,throughput,8,batch,PASS,FAIL,46.45,21.39,21.50,46.70,237.73,417.83,14078049200.00,9014655684.00,pimcomp,2.17
arch-b,yolo11n,latency,1,element,PASS,FAIL,316.70,NA,NA,NA,195.43,NA,61892225295.00,NA,NA,NA
arch-b,yolo11n,throughput,2,batch,PASS,FAIL,333.33,NA,3.00,NA,214.01,NA,71336401290.00,NA,NA,NA
arch-b,yolo11n,throughput,4,batch,PASS,FAIL,333.33,NA,3.00,NA,216.24,NA,72079194110.00,NA,NA,NA
arch-b,yolo11n,throughput,8,batch,PASS,FAIL,500.00,NA,2.00,NA,218.42,NA,109211119900.00,NA,NA,NA
arch-c,vgg8,latency,1,element,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,vgg8,throughput,2,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,vgg8,throughput,4,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,vgg8,throughput,8,batch,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet18,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet18,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet18,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet18,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet34,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet34,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet34,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,resnet34,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,googlenet,latency,1,element,FAIL,PASS,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,googlenet,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,googlenet,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,googlenet,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,yolo11n,latency,1,element,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,yolo11n,throughput,2,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,yolo11n,throughput,4,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
arch-c,yolo11n,throughput,8,batch,FAIL,FAIL,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA
1 arch model mode raptor_pipeline pimcomp_pipeline raptor_functional_validation pimcomp_functional_validation raptor_latency_ms pimcomp_latency_ms raptor_throughput_samples_s pimcomp_throughput_samples_s raptor_power_mw pimcomp_power_mw raptor_energy_pj pimcomp_energy_pj better_compiler speedup
2 arch-a vgg8 latency 1 element PASS PASS 1.47 7.99 NA NA 325.67 200.11 477232782.04 1597904071.12 raptor 5.45
3 arch-a vgg8 throughput 2 batch PASS PASS 0.94 1.04 1060.00 965.00 377.54 482.60 474325940.70 541691691.00 raptor 1.10
4 arch-a vgg8 throughput 4 batch PASS PASS 1.19 1.04 839.00 965.00 310.32 482.60 500148858.20 541691691.00 pimcomp 1.15
5 arch-a vgg8 throughput 8 batch PASS PASS 1.52 1.04 658.00 965.00 279.37 482.60 554348483.90 541691691.00 pimcomp 1.47
6 arch-a resnet18 latency 1 element PASS FAIL 15.65 58.18 NA NA 425.87 238.39 6666000553.12 13869146503.12 raptor 3.72
7 arch-a resnet18 throughput 2 batch PASS FAIL 30.55 13.46 32.70 74.30 295.90 481.60 10218790630.00 6562535800.00 pimcomp 2.27
8 arch-a resnet18 throughput 4 batch PASS FAIL 26.60 13.46 37.60 74.30 313.14 481.60 10993189350.00 6562535800.00 pimcomp 1.98
9 arch-a resnet18 throughput 8 batch PASS FAIL 35.51 13.46 28.20 74.30 275.95 481.60 12844245960.00 6562535800.00 pimcomp 2.63
10 arch-a resnet34 latency 1 element PASS FAIL 32.37 66.71 NA NA 391.82 277.48 12684227802.68 18511540483.68 raptor 2.06
11 arch-a resnet34 throughput 2 batch FAIL FAIL NA 23.58 NA 42.40 NA 515.66 NA 12277699310.00 NA NA
12 arch-a resnet34 throughput 4 batch FAIL FAIL NA 23.58 NA 42.40 NA 515.66 NA 12277699310.00 NA NA
13 arch-a resnet34 throughput 8 batch PASS FAIL 115.11 23.58 8.69 42.40 229.85 515.66 29881854960.00 12277699310.00 pimcomp 4.88
14 arch-a googlenet latency 1 element PASS PASS 13.03 62.92 NA NA 465.11 231.19 6060646341.92 14547510894.24 raptor 4.83
15 arch-a googlenet throughput 2 batch PASS FAIL 24.01 18.27 41.70 54.70 328.17 435.11 8529823770.00 8164469406.00 pimcomp 1.31
16 arch-a googlenet throughput 4 batch PASS FAIL 27.01 18.27 37.00 54.70 305.68 435.11 10498372450.00 8164469406.00 pimcomp 1.48
17 arch-a googlenet throughput 8 batch PASS FAIL 40.03 18.27 25.00 54.70 268.61 435.11 15290418970.00 8164469406.00 pimcomp 2.19
18 arch-a yolo11n latency 1 element PASS FAIL 267.73 NA NA NA 235.73 NA 63111612279.00 NA NA NA
19 arch-a yolo11n throughput 2 batch PASS FAIL 500.00 NA 2.00 NA 227.74 NA 113867559100.00 NA NA NA
20 arch-a yolo11n throughput 4 batch PASS FAIL 500.00 NA 2.00 NA 233.16 NA 116581237400.00 NA NA NA
21 arch-a yolo11n throughput 8 batch PASS FAIL 500.00 NA 2.00 NA 253.59 NA 126797224900.00 NA NA NA
22 arch-b vgg8 latency 1 element PASS PASS 1.44 7.15 NA NA 304.72 173.77 438329594.04 1242814988.12 raptor 4.97
23 arch-b vgg8 throughput 2 batch PASS PASS 1.56 1.07 640.00 932.00 287.50 409.47 478592086.00 439288835.80 pimcomp 1.46
24 arch-b vgg8 throughput 4 batch PASS PASS 1.14 1.07 874.00 932.00 323.79 409.47 457617420.30 439288835.80 pimcomp 1.07
25 arch-b vgg8 throughput 8 batch PASS PASS 1.48 1.07 676.00 932.00 282.34 409.47 437422264.40 439288835.80 pimcomp 1.38
26 arch-b resnet18 latency 1 element PASS FAIL 17.98 64.38 NA NA 362.63 201.66 6520518736.12 12983607699.12 raptor 3.58
27 arch-b resnet18 throughput 2 batch PASS FAIL 33.84 13.43 29.60 74.40 253.32 484.80 9423636948.00 6569655551.00 pimcomp 2.51
28 arch-b resnet18 throughput 4 batch PASS FAIL 26.14 13.43 38.30 74.40 286.86 484.80 9493311384.00 6569655551.00 pimcomp 1.94
29 arch-b resnet18 throughput 8 batch PASS FAIL 28.29 13.43 35.30 74.40 272.66 484.80 11314980340.00 6569655551.00 pimcomp 2.11
30 arch-b resnet34 latency 1 element PASS FAIL 49.51 107.20 NA NA 284.72 206.47 14097763205.68 22134333948.68 raptor 2.17
31 arch-b resnet34 throughput 2 batch PASS FAIL 93.13 22.80 10.70 43.90 215.38 493.39 21278771500.00 11381213070.00 pimcomp 4.10
32 arch-b resnet34 throughput 4 batch PASS FAIL 67.55 22.80 14.80 43.90 237.97 493.39 19346594270.00 11381213070.00 pimcomp 2.97
33 arch-b resnet34 throughput 8 batch PASS FAIL 115.01 22.80 8.70 43.90 197.27 493.39 26678372930.00 11381213070.00 pimcomp 5.05
34 arch-b googlenet latency 1 element PASS PASS 16.08 37.94 NA NA 378.99 242.04 6095729659.92 9181933205.24 raptor 2.36
35 arch-b googlenet throughput 2 batch PASS FAIL 28.37 21.39 35.20 46.70 267.15 417.83 8684222064.00 9014655684.00 pimcomp 1.33
36 arch-b googlenet throughput 4 batch PASS FAIL 26.16 21.39 38.20 46.70 280.21 417.83 9300527914.00 9014655684.00 pimcomp 1.22
37 arch-b googlenet throughput 8 batch PASS FAIL 46.45 21.39 21.50 46.70 237.73 417.83 14078049200.00 9014655684.00 pimcomp 2.17
38 arch-b yolo11n latency 1 element PASS FAIL 316.70 NA NA NA 195.43 NA 61892225295.00 NA NA NA
39 arch-b yolo11n throughput 2 batch PASS FAIL 333.33 NA 3.00 NA 214.01 NA 71336401290.00 NA NA NA
40 arch-b yolo11n throughput 4 batch PASS FAIL 333.33 NA 3.00 NA 216.24 NA 72079194110.00 NA NA NA
41 arch-b yolo11n throughput 8 batch PASS FAIL 500.00 NA 2.00 NA 218.42 NA 109211119900.00 NA NA NA
42 arch-c vgg8 latency 1 element FAIL PASS NA NA NA NA NA NA NA NA NA NA
43 arch-c vgg8 throughput 2 batch FAIL PASS NA NA NA NA NA NA NA NA NA NA
44 arch-c vgg8 throughput 4 batch FAIL PASS NA NA NA NA NA NA NA NA NA NA
45 arch-c vgg8 throughput 8 batch FAIL PASS NA NA NA NA NA NA NA NA NA NA
46 arch-c resnet18 latency 1 element FAIL FAIL NA NA NA NA NA NA NA NA NA NA
47 arch-c resnet18 throughput 2 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
48 arch-c resnet18 throughput 4 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
49 arch-c resnet18 throughput 8 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
50 arch-c resnet34 latency 1 element FAIL FAIL NA NA NA NA NA NA NA NA NA NA
51 arch-c resnet34 throughput 2 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
52 arch-c resnet34 throughput 4 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
53 arch-c resnet34 throughput 8 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
54 arch-c googlenet latency 1 element FAIL PASS NA NA NA NA NA NA NA NA NA NA
55 arch-c googlenet throughput 2 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
56 arch-c googlenet throughput 4 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
57 arch-c googlenet throughput 8 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
58 arch-c yolo11n latency 1 element FAIL FAIL NA NA NA NA NA NA NA NA NA NA
59 arch-c yolo11n throughput 2 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
60 arch-c yolo11n throughput 4 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
61 arch-c yolo11n throughput 8 batch FAIL FAIL NA NA NA NA NA NA NA NA NA NA
validation/networks/pimcomp_models/validation_results.csv
-6
View File
@@ -1,6 +0,0 @@
Operation,Result,Compile,Host mem,Cores mem,Cores,Xbars,Latency,Power,Energy
vgg8-mnist-reconstructed,PASS,1.009 s,1.37 MiB,3.14 MiB,141,761,1.465778 ms,325.627854 mW,477298145.040001 pJ
resnet18-v1-7,PASS,11.548 s,9.89 MiB,40.24 MiB,168,7676,28.099952 ms,312.513408 mW,8781611766.119984 pJ
resnet34-v1-7,PASS,28.495 s,9.90 MiB,48.89 MiB,168,15292,45.781486 ms,326.833870 mW,14962940227.679951 pJ
googlenet-12-pimsim-nn,PASS,6.573 s,10.74 MiB,22.41 MiB,168,7176,13.371204 ms,457.538139 mW,6117835798.919991 pJ
yolo11n-pimsim-nn,FAIL,58.572 s,82.55 MiB,185.68 MiB,168,6484,885.264931 ms,189.218985 mW,167508931321.001465 pJ
1 Operation Result Compile Host mem Cores mem Cores Xbars Latency Power Energy
2 vgg8-mnist-reconstructed PASS 1.009 s 1.37 MiB 3.14 MiB 141 761 1.465778 ms 325.627854 mW 477298145.040001 pJ
3 resnet18-v1-7 PASS 11.548 s 9.89 MiB 40.24 MiB 168 7676 28.099952 ms 312.513408 mW 8781611766.119984 pJ
4 resnet34-v1-7 PASS 28.495 s 9.90 MiB 48.89 MiB 168 15292 45.781486 ms 326.833870 mW 14962940227.679951 pJ
5 googlenet-12-pimsim-nn PASS 6.573 s 10.74 MiB 22.41 MiB 168 7176 13.371204 ms 457.538139 mW 6117835798.919991 pJ
6 yolo11n-pimsim-nn FAIL 58.572 s 82.55 MiB 185.68 MiB 168 6484 885.264931 ms 189.218985 mW 167508931321.001465 pJ
validation/operations/README.md
+2 -2
View File
@@ -1,7 +1,7 @@
# Operation Validation Suite
# Operation validation suite
This directory contains the ONNX models used by `validation/validate.py` to
validate individual operations through compilation, PIM simulation, and
validate individual operations through compilation, Pim simulation, and
comparison with the ONNX-MLIR reference runtime.
## Naming
validation/operations/validation_results.csv
-178
View File
@@ -1,178 +0,0 @@
Operation,Arch,Result (l),Result (t),Compile (l),Host mem (l),Cores mem (l),Cores (l),Xbars (l),Latency (l),Power (l),Energy (l),Compile (t),Host mem (t),Cores mem (t),Cores (t),Xbars (t),Avg latency (t),Throughput (t),Avg power (t),Avg energy (t)
add/after_gemm,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it
add/basic,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
add/broadcast_row,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
add/channel_broadcast_1024,arch-a,PASS,PASS,0.052 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.036 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it
add/leading_dimension_broadcast,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
concat/channel_axis,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.16 mW,35718.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,2200000.00 samples/s,0.00 ms,2.16 mW,934.67 pJ/it
concat/negative_axis,arch-a,PASS,PASS,0.046 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.09 mW,81450.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,961000.00 samples/s,0.00 ms,2.09 mW,2108.00 pJ/it
concat/three_inputs_channel_axis,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.15 mW,50328.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1560000.00 samples/s,0.00 ms,2.15 mW,1332.67 pJ/it
conv/batch_2,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,2,2,0.01 ms,82.62 mW,1131451.48 pJ,0.044 s,0.00 MiB,0.01 MiB,4,2,147000.00 samples/s,0.01 ms,92.90 mW,669920.48 pJ/it
conv/batch_4_pointwise,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,116.08 mW,456420.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,69.62 mW,291187.04 pJ/it
conv/depthwise_1024_channels,arch-a,PASS,PASS,0.082 s,0.19 MiB,0.38 MiB,129,128,0.22 ms,178.45 mW,39393966.72 pJ,0.117 s,0.38 MiB,0.46 MiB,45,118,4930.00 samples/s,0.20 ms,144.34 mW,33005602.67 pJ/it
conv/depthwise_grouped,arch-a,PASS,PASS,0.055 s,0.01 MiB,0.00 MiB,5,4,0.01 ms,107.78 mW,671878.96 pJ,0.042 s,0.01 MiB,0.00 MiB,7,4,418000.00 samples/s,0.00 ms,168.64 mW,479471.12 pJ/it
conv/dilated_3x3,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,118.77 mW,1034819.16 pJ,0.057 s,0.01 MiB,0.01 MiB,12,9,104000.00 samples/s,0.01 ms,155.40 mW,1574973.57 pJ/it
conv/dynamic,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,92.28 mW,169336.00 pJ,0.044 s,0.00 MiB,0.00 MiB,6,0,778000.00 samples/s,0.00 ms,86.62 mW,124092.33 pJ/it
conv/explicit_padding,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.02 MiB,17,16,0.01 ms,145.34 mW,1454397.84 pJ,0.048 s,0.01 MiB,0.02 MiB,19,16,179000.00 samples/s,0.01 ms,201.80 mW,1234440.20 pJ/it
conv/grouped_many_groups,arch-a,PASS,PASS,0.474 s,0.05 MiB,0.09 MiB,65,64,0.18 ms,142.21 mW,25867112.36 pJ,0.476 s,0.08 MiB,0.73 MiB,87,64,3660.00 samples/s,0.27 ms,97.46 mW,30277556.57 pJ/it
conv/grouped_two_groups,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,101.46 mW,543914.48 pJ,0.047 s,0.00 MiB,0.01 MiB,9,2,378000.00 samples/s,0.00 ms,105.40 mW,353700.73 pJ/it
conv/huge_pointwise_1024,arch-a,PASS,PASS,0.151 s,0.01 MiB,0.11 MiB,73,64,0.02 ms,249.58 mW,3895759.36 pJ,0.163 s,0.08 MiB,0.10 MiB,52,64,19200.00 samples/s,0.05 ms,163.09 mW,8722111.37 pJ/it
conv/huge_pointwise_1024_dynamic,arch-a,PASS,PASS,0.080 s,8.04 MiB,12.61 MiB,168,0,2.63 ms,169.52 mW,445489032.00 pJ,0.224 s,12.75 MiB,6.82 MiB,45,0,188.00 samples/s,5.31 ms,134.20 mW,746263826.70 pJ/it
conv/input_224_7x7_stride2,arch-a,PASS,PASS,0.759 s,24.14 MiB,61.87 MiB,168,169,38.41 ms,185.26 mW,7116544212.12 pJ,1.060 s,51.25 MiB,67.85 MiB,45,84,24.70 samples/s,40.56 ms,148.32 mW,6561548886.00 pJ/it
conv/kernel_2x2,arch-a,PASS,PASS,0.061 s,0.00 MiB,0.00 MiB,1,1,0.00 ms,83.83 mW,360568.24 pJ,0.046 s,0.00 MiB,0.00 MiB,3,1,356000.00 samples/s,0.00 ms,93.82 mW,298919.07 pJ/it
conv/kernel_3x3,arch-a,PASS,PASS,0.061 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,123.80 mW,889640.16 pJ,0.046 s,0.01 MiB,0.01 MiB,12,9,271000.00 samples/s,0.00 ms,195.49 mW,787478.85 pJ/it
conv/kernel_equals_input_spatial,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,89.61 mW,415689.48 pJ,0.041 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,104.42 mW,305239.85 pJ/it
conv/large_input_channels_1x1,arch-a,PASS,PASS,0.085 s,0.01 MiB,0.02 MiB,9,8,0.01 ms,117.84 mW,900569.92 pJ,0.081 s,0.02 MiB,0.02 MiB,10,8,135000.00 samples/s,0.01 ms,113.25 mW,910683.04 pJ/it
conv/large_output_channels_1x1,arch-a,PASS,PASS,0.095 s,0.01 MiB,0.02 MiB,17,8,0.01 ms,128.44 mW,1139415.92 pJ,0.078 s,0.02 MiB,0.02 MiB,18,8,93600.00 samples/s,0.01 ms,151.83 mW,1755770.60 pJ/it
conv/large_spatial,arch-a,PASS,PASS,0.061 s,0.01 MiB,0.04 MiB,37,36,0.02 ms,172.07 mW,2928344.64 pJ,0.061 s,0.01 MiB,0.04 MiB,39,36,84200.00 samples/s,0.01 ms,208.30 mW,2603711.62 pJ/it
conv/multi_channel,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,105.68 mW,685040.72 pJ,0.050 s,0.00 MiB,0.00 MiB,4,3,140000.00 samples/s,0.01 ms,54.54 mW,393755.39 pJ/it
conv/non_square_kernel_1x3,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,99.35 mW,679752.48 pJ,0.044 s,0.00 MiB,0.00 MiB,3,2,140000.00 samples/s,0.01 ms,51.01 mW,367557.81 pJ/it
conv/non_square_kernel_3x1,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,3,2,0.01 ms,95.89 mW,1292976.48 pJ,0.045 s,0.00 MiB,0.00 MiB,3,2,72200.00 samples/s,0.01 ms,47.79 mW,664683.81 pJ/it
conv/non_uniform_stride,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,104.05 mW,790874.72 pJ,0.044 s,0.00 MiB,0.00 MiB,4,3,126000.00 samples/s,0.01 ms,53.56 mW,429978.05 pJ/it
conv/output_channel_grouping_minimal,arch-a,PASS,PASS,0.080 s,0.10 MiB,0.34 MiB,131,128,0.26 ms,170.73 mW,44125916.72 pJ,0.148 s,0.28 MiB,0.96 MiB,87,84,2630.00 samples/s,0.38 ms,136.74 mW,58542917.01 pJ/it
conv/pointwise_1x1,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,1,0.01 ms,80.24 mW,987244.24 pJ,0.041 s,0.00 MiB,0.00 MiB,3,1,131000.00 samples/s,0.01 ms,89.08 mW,719413.57 pJ/it
conv/pointwise_tiled_chain,arch-a,PASS,PASS,0.642 s,0.01 MiB,0.04 MiB,20,80,0.04 ms,153.90 mW,6443957.20 pJ,0.620 s,0.06 MiB,0.08 MiB,22,80,16400.00 samples/s,0.06 ms,186.69 mW,12410041.37 pJ/it
conv/real_asymmetric_padding,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.03 MiB,29,28,0.01 ms,153.67 mW,2221606.72 pJ,0.055 s,0.01 MiB,0.03 MiB,31,28,105000.00 samples/s,0.01 ms,204.35 mW,2058223.49 pJ/it
conv/relu_conv_store,arch-a,PASS,PASS,0.085 s,0.16 MiB,0.67 MiB,168,184,0.56 ms,183.08 mW,103057723.80 pJ,0.209 s,0.37 MiB,0.58 MiB,60,58,1500.00 samples/s,0.67 ms,146.73 mW,101291048.70 pJ/it
conv/same_lower_3x3,arch-a,PASS,PASS,0.060 s,0.01 MiB,0.02 MiB,26,25,0.01 ms,166.15 mW,2215009.00 pJ,0.054 s,0.01 MiB,0.03 MiB,28,25,119000.00 samples/s,0.01 ms,204.48 mW,1837219.40 pJ/it
conv/same_padding_3x3,arch-a,PASS,PASS,0.058 s,0.01 MiB,0.02 MiB,26,25,0.01 ms,166.15 mW,2215009.00 pJ,0.056 s,0.01 MiB,0.03 MiB,28,25,119000.00 samples/s,0.01 ms,204.48 mW,1837219.40 pJ/it
conv/strategy_depthwise_16,arch-a,PASS,PASS,0.093 s,0.06 MiB,0.35 MiB,168,168,0.34 ms,197.94 mW,66331479.08 pJ,0.257 s,0.20 MiB,0.17 MiB,45,84,2650.00 samples/s,0.38 ms,153.71 mW,60322816.49 pJ/it
conv/strategy_input_k_tiled,arch-a,PASS,PASS,0.082 s,0.08 MiB,0.27 MiB,109,108,0.35 ms,170.81 mW,60422605.92 pJ,0.100 s,0.22 MiB,0.28 MiB,45,90,4250.00 samples/s,0.24 ms,143.52 mW,36750587.75 pJ/it
conv/strategy_output_channel_tiled,arch-a,PASS,PASS,0.073 s,0.03 MiB,0.16 MiB,74,72,0.09 ms,155.74 mW,14244739.28 pJ,0.110 s,0.13 MiB,0.22 MiB,81,72,9220.00 samples/s,0.11 ms,145.73 mW,18577796.14 pJ/it
conv/strategy_streamed_packed,arch-a,PASS,PASS,0.155 s,3.34 MiB,7.89 MiB,168,168,9.35 ms,179.86 mW,1682364509.56 pJ,0.357 s,5.32 MiB,7.79 MiB,43,42,113.00 samples/s,8.87 ms,61.68 mW,558465684.90 pJ/it
conv/strategy_streamed_patch,arch-a,PASS,PASS,0.115 s,0.34 MiB,1.32 MiB,168,168,1.90 ms,181.91 mW,346476645.64 pJ,0.299 s,0.82 MiB,1.21 MiB,43,42,501.00 samples/s,1.99 ms,62.44 mW,125148059.00 pJ/it
conv/strategy_tiled_2d,arch-a,PASS,PASS,0.111 s,0.11 MiB,0.44 MiB,168,168,0.42 ms,182.13 mW,75690907.84 pJ,0.219 s,0.36 MiB,0.46 MiB,47,144,4070.00 samples/s,0.25 ms,155.26 mW,43400768.51 pJ/it
conv/stride_2,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.00 MiB,5,4,0.01 ms,110.78 mW,580154.96 pJ,0.043 s,0.01 MiB,0.00 MiB,7,4,476000.00 samples/s,0.00 ms,175.93 mW,424103.12 pJ/it
conv/with_bias_3x3,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,4,3,0.01 ms,104.16 mW,776220.72 pJ,0.046 s,0.00 MiB,0.01 MiB,4,3,133000.00 samples/s,0.01 ms,53.85 mW,416288.72 pJ/it
conv/with_constant,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,1,0.01 ms,81.74 mW,541270.24 pJ,0.043 s,0.00 MiB,0.00 MiB,4,1,231000.00 samples/s,0.00 ms,133.09 mW,655848.91 pJ/it
conv/without_kernel_shape_attr,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.01 MiB,10,9,0.01 ms,123.80 mW,889640.16 pJ,0.047 s,0.01 MiB,0.01 MiB,12,9,271000.00 samples/s,0.00 ms,195.49 mW,787478.85 pJ/it
conv/yolo11n_depthwise_head,arch-a,PASS,PASS,1.709 s,8.66 MiB,34.24 MiB,168,255,42.70 ms,200.52 mW,8562452412.00 pJ,2.373 s,27.15 MiB,20.31 MiB,87,214,20.60 samples/s,48.55 ms,157.12 mW,8007922821.00 pJ/it
conv/yolo11n_heavy,arch-a,PASS,PASS,0.519 s,4.82 MiB,19.10 MiB,161,800,8.53 ms,350.87 mW,2994683017.00 pJ,1.007 s,11.06 MiB,13.87 MiB,85,420,79.10 samples/s,12.64 ms,218.17 mW,2857451109.00 pJ/it
conv/yolo11n_stem,arch-a,PASS,PASS,0.893 s,12.86 MiB,37.59 MiB,168,488,14.24 ms,301.23 mW,4289558246.00 pJ,1.374 s,25.19 MiB,21.24 MiB,85,126,44.60 samples/s,22.42 ms,176.88 mW,4028789243.00 pJ/it
div/after_gemm,arch-a,PASS,PASS,0.053 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it
div/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
div/channel_broadcast_1024,arch-a,PASS,PASS,0.049 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.039 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it
div/leading_dimension_broadcast,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
div/runtime_scalar_rhs,arch-a,PASS,PASS,0.053 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.040 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it
div/scalar_constant,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
gather/3d_input_axis1,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.08 mW,45990.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,1700000.00 samples/s,0.00 ms,2.08 mW,1174.67 pJ/it
gather/axis0_matrix_indices,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.07 mW,54414.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,1440000.00 samples/s,0.00 ms,2.07 mW,1390.67 pJ/it
gather/axis1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.06 mW,62526.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,1250000.00 samples/s,0.00 ms,2.06 mW,1598.67 pJ/it
gather/negative_axis,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.03 mW,112134.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,697000.00 samples/s,0.00 ms,2.03 mW,2870.67 pJ/it
gather/negative_indices,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.13 mW,29376.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,2670000.00 samples/s,0.00 ms,2.12 mW,748.67 pJ/it
gemm/alpha_beta,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.27 mW,784908.96 pJ,0.050 s,0.01 MiB,0.01 MiB,6,4,212000.00 samples/s,0.00 ms,107.78 mW,574589.95 pJ/it
gemm/bias_rank2_broadcast,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,105.98 mW,749484.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,229000.00 samples/s,0.00 ms,109.50 mW,540708.12 pJ/it
gemm/dynamic,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.48 mW,221475.00 pJ,0.044 s,0.00 MiB,0.00 MiB,5,0,489000.00 samples/s,0.00 ms,44.34 mW,93754.67 pJ/it
gemm/dynamic_alpha,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.42 mW,298198.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,464000.00 samples/s,0.00 ms,44.39 mW,105291.67 pJ/it
gemm/dynamic_beta,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.32 mW,397230.00 pJ,0.042 s,0.00 MiB,0.00 MiB,5,0,247000.00 samples/s,0.00 ms,20.21 mW,81901.75 pJ/it
gemm/dynamic_bias,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.45 mW,243703.00 pJ,0.046 s,0.00 MiB,0.00 MiB,5,0,422000.00 samples/s,0.00 ms,20.28 mW,48009.75 pJ/it
gemm/dynamic_bias_alpha_beta,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,5,0,0.01 ms,91.28 mW,513811.00 pJ,0.043 s,0.00 MiB,0.00 MiB,5,0,188000.00 samples/s,0.01 ms,20.20 mW,107673.75 pJ/it
gemm/dynamic_transpose_b,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.38 mW,118883.00 pJ,0.040 s,0.00 MiB,0.00 MiB,5,0,803000.00 samples/s,0.00 ms,44.55 mW,58232.00 pJ/it
gemm/huge_1024,arch-a,PASS,PASS,0.149 s,0.01 MiB,0.10 MiB,73,64,0.02 ms,215.07 mW,3767010.36 pJ,0.160 s,0.05 MiB,0.09 MiB,51,64,26400.00 samples/s,0.04 ms,135.13 mW,5670462.59 pJ/it
gemm/large,arch-a,PASS,PASS,0.058 s,0.02 MiB,0.03 MiB,17,16,0.01 ms,140.15 mW,1573768.84 pJ,0.050 s,0.03 MiB,0.03 MiB,17,16,78500.00 samples/s,0.01 ms,79.60 mW,1008067.12 pJ/it
gemm/large_k_small_n,arch-a,PASS,PASS,0.088 s,0.01 MiB,0.01 MiB,9,8,0.00 ms,133.53 mW,633217.92 pJ,0.082 s,0.01 MiB,0.01 MiB,9,8,182000.00 samples/s,0.01 ms,84.31 mW,476982.66 pJ/it
gemm/non_square,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,118.96 mW,419565.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,71.16 mW,302182.45 pJ/it
gemm/scalar_bias,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,105.98 mW,749484.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,229000.00 samples/s,0.00 ms,109.50 mW,540708.12 pJ/it
gemm/small,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,90.14 mW,398436.48 pJ,0.043 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,105.00 mW,297577.19 pJ/it
gemm/small_k_large_n,arch-a,PASS,PASS,0.090 s,0.01 MiB,0.02 MiB,17,8,0.01 ms,131.01 mW,1043061.92 pJ,0.082 s,0.02 MiB,0.02 MiB,18,8,92700.00 samples/s,0.01 ms,152.38 mW,1742739.72 pJ/it
gemm/square_weights,arch-a,PASS,PASS,0.073 s,0.03 MiB,0.08 MiB,42,40,0.02 ms,151.77 mW,3284393.60 pJ,0.075 s,0.07 MiB,0.09 MiB,44,40,25700.00 samples/s,0.04 ms,155.80 mW,6344327.95 pJ/it
gemm/transpose_a,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.14 mW,628868.96 pJ,0.047 s,0.01 MiB,0.01 MiB,6,4,290000.00 samples/s,0.00 ms,115.23 mW,456854.62 pJ/it
gemm/transpose_a_and_b,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.14 mW,628868.96 pJ,0.044 s,0.01 MiB,0.01 MiB,6,4,290000.00 samples/s,0.00 ms,115.23 mW,456854.62 pJ/it
gemm/transpose_b,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.01 MiB,5,4,0.00 ms,118.96 mW,419565.96 pJ,0.045 s,0.01 MiB,0.01 MiB,5,4,242000.00 samples/s,0.00 ms,71.16 mW,302182.45 pJ/it
gemm/transpose_b_with_bias,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,110.55 mW,557818.96 pJ,0.046 s,0.01 MiB,0.01 MiB,5,4,218000.00 samples/s,0.00 ms,67.31 mW,333372.79 pJ/it
gemm/with_bias,arch-a,PASS,PASS,0.057 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,108.77 mW,604966.96 pJ,0.043 s,0.01 MiB,0.01 MiB,5,4,203000.00 samples/s,0.00 ms,66.54 mW,341027.79 pJ/it
gemv/all_constant,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,0,0,0.00 ms,2.00 mW,0.00 pJ,0.039 s,0.00 MiB,0.00 MiB,0,0,0.00 samples/s,0.00 ms,2.00 mW,0.00 pJ/it
gemv/constant_weight,arch-a,PASS,PASS,0.065 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,111.15 mW,573535.96 pJ,0.051 s,0.01 MiB,0.01 MiB,8,4,263000.00 samples/s,0.00 ms,154.40 mW,641346.96 pJ/it
gemv/non_uniform_bias,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.051 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it
gemv/scalar_bias,arch-a,PASS,PASS,0.061 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.052 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it
gemv/uniform_bias,arch-a,PASS,PASS,0.066 s,0.00 MiB,0.01 MiB,6,4,0.01 ms,109.81 mW,609463.96 pJ,0.049 s,0.01 MiB,0.01 MiB,8,4,243000.00 samples/s,0.00 ms,152.55 mW,697203.20 pJ/it
matmul/basic,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,2,2,0.00 ms,90.14 mW,398436.48 pJ,0.040 s,0.00 MiB,0.00 MiB,4,2,376000.00 samples/s,0.00 ms,105.00 mW,297577.19 pJ/it
matmul/batched_3d,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.59 mW,646972.96 pJ,0.048 s,0.01 MiB,0.01 MiB,6,4,275000.00 samples/s,0.00 ms,114.15 mW,469999.95 pJ/it
matmul/batched_3d_dynamic,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,92.19 mW,167975.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,758000.00 samples/s,0.00 ms,18.49 mW,24624.50 pJ/it
matmul/batched_left_constant,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.02 MiB,9,8,0.01 ms,114.39 mW,1009105.92 pJ,0.059 s,0.01 MiB,0.02 MiB,11,8,149000.00 samples/s,0.01 ms,159.00 mW,1186296.28 pJ/it
matmul/batched_lhs_broadcast,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,109.39 mW,621440.96 pJ,0.042 s,0.01 MiB,0.01 MiB,6,4,276000.00 samples/s,0.00 ms,114.66 mW,463736.29 pJ/it
matmul/batched_rhs_broadcast,arch-a,PASS,PASS,0.058 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.59 mW,646972.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,275000.00 samples/s,0.00 ms,114.15 mW,469999.95 pJ/it
matmul/dynamic,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.42 mW,148195.00 pJ,0.041 s,0.00 MiB,0.00 MiB,5,0,660000.00 samples/s,0.00 ms,44.46 mW,70471.33 pJ/it
matmul/huge_1024,arch-a,PASS,PASS,0.146 s,0.01 MiB,0.10 MiB,73,64,0.02 ms,215.07 mW,3767010.36 pJ,0.162 s,0.05 MiB,0.09 MiB,51,64,26400.00 samples/s,0.04 ms,135.13 mW,5670462.59 pJ/it
matmul/left_constant,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.01 MiB,5,4,0.01 ms,108.86 mW,637168.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,297000.00 samples/s,0.00 ms,115.66 mW,451626.62 pJ/it
matmul/matrix_vector,arch-a,PASS,PASS,0.098 s,0.52 MiB,0.78 MiB,168,173,0.38 ms,202.13 mW,77751476.88 pJ,0.289 s,1.04 MiB,0.84 MiB,44,171,2150.00 samples/s,0.46 ms,119.16 mW,65096859.53 pJ/it
matmul/vector_matrix,arch-a,PASS,PASS,0.089 s,0.01 MiB,0.01 MiB,9,8,0.01 ms,118.70 mW,878749.92 pJ,0.083 s,0.02 MiB,0.02 MiB,10,8,140000.00 samples/s,0.01 ms,115.31 mW,910373.16 pJ/it
matmul/yolo_attention,arch-a,PASS,PASS,0.474 s,1.02 MiB,43.44 MiB,168,0,8.15 ms,170.00 mW,1385775865.00 pJ,0.645 s,5.57 MiB,42.54 MiB,75,0,145.00 samples/s,6.89 ms,119.04 mW,932561637.00 pJ/it
mul/after_conv,arch-a,PASS,PASS,0.059 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.64 mW,586955.72 pJ,0.050 s,0.00 MiB,0.00 MiB,4,3,194000.00 samples/s,0.01 ms,58.31 mW,304072.05 pJ/it
mul/after_conv_scalar_constant,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.64 mW,586955.72 pJ,0.046 s,0.00 MiB,0.00 MiB,4,3,194000.00 samples/s,0.01 ms,58.31 mW,304072.05 pJ/it
mul/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
mul/channel_broadcast_1024,arch-a,PASS,PASS,0.050 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.038 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it
mul/leading_dimension_broadcast,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
mul/scalar_constant,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
pool/avg_basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,929400.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,84200.00 samples/s,0.01 ms,2.02 mW,24013.00 pJ/it
pool/avg_ceil_mode,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.03 mW,339210.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,230000.00 samples/s,0.00 ms,2.03 mW,8786.67 pJ/it
pool/avg_explicit_padding,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.03 mW,685860.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,114000.00 samples/s,0.01 ms,2.03 mW,17745.00 pJ/it
pool/avg_include_pad,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,661116.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,118000.00 samples/s,0.01 ms,2.02 mW,17017.00 pJ/it
pool/avg_large_channels,arch-a,PASS,PASS,0.059 s,0.04 MiB,0.02 MiB,1,0,0.24 ms,78.00 mW,18397284.00 pJ,0.046 s,0.04 MiB,0.02 MiB,1,0,4250.00 samples/s,0.24 ms,2.00 mW,471380.00 pJ/it
pool/avg_non_uniform_stride,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,1129134.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,69300.00 samples/s,0.01 ms,2.02 mW,29111.00 pJ/it
pool/avg_real_asymmetric_padding,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.03 ms,78.02 mW,1959204.00 pJ,0.041 s,0.00 MiB,0.00 MiB,1,0,39900.00 samples/s,0.03 ms,2.02 mW,50769.00 pJ/it
pool/max_after_conv,arch-a,PASS,PASS,0.063 s,0.00 MiB,0.00 MiB,5,4,0.01 ms,99.12 mW,1209961.96 pJ,0.051 s,0.00 MiB,0.00 MiB,5,4,140000.00 samples/s,0.01 ms,58.47 mW,468396.45 pJ/it
pool/max_basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.06 mW,324744.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,241000.00 samples/s,0.00 ms,2.06 mW,8532.67 pJ/it
pool/max_ceil_mode,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.07 mW,151464.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,516000.00 samples/s,0.00 ms,2.07 mW,3972.67 pJ/it
pool/max_global_style_kernel_equals_input,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.01 mW,657534.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,119000.00 samples/s,0.01 ms,2.01 mW,16843.00 pJ/it
pool/max_non_square_kernel,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.02 mW,1060572.00 pJ,0.047 s,0.00 MiB,0.00 MiB,1,0,73700.00 samples/s,0.01 ms,2.02 mW,27353.00 pJ/it
pool/max_real_asymmetric_padding,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.03 mW,814992.00 pJ,0.040 s,0.00 MiB,0.00 MiB,1,0,96100.00 samples/s,0.01 ms,2.03 mW,21173.00 pJ/it
pool/max_same_upper,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.04 mW,625068.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,125000.00 samples/s,0.01 ms,2.04 mW,16233.00 pJ/it
pool/max_stride2_multichannel,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.02 ms,78.02 mW,1245870.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,62800.00 samples/s,0.02 ms,2.02 mW,32117.00 pJ/it
reduce_mean/4d_spatial,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.02 mW,326814.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,239000.00 samples/s,0.00 ms,2.02 mW,8390.67 pJ/it
reduce_mean/4d_spatial_keepdims_0,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,94.35 mW,61801.00 pJ,0.041 s,0.00 MiB,0.00 MiB,4,0,1470000.00 samples/s,0.00 ms,44.66 mW,31425.33 pJ/it
reduce_mean/after_conv,arch-a,PASS,PASS,0.062 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,100.79 mW,1119699.72 pJ,0.057 s,0.00 MiB,0.00 MiB,4,3,125000.00 samples/s,0.01 ms,54.14 mW,474782.17 pJ/it
reduce_mean/all_axes_keepdims_0,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.24 mW,30982.00 pJ,0.038 s,0.00 MiB,0.00 MiB,2,0,2720000.00 samples/s,0.00 ms,44.37 mW,16707.00 pJ/it
reduce_mean/all_axes_keepdims_1,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it
reduce_mean/basic,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.51 mW,34881.00 pJ,0.042 s,0.00 MiB,0.00 MiB,4,0,2600000.00 samples/s,0.00 ms,5.85 mW,2235.67 pJ/it
reduce_mean/channel_axis_nchw,arch-a,PASS,PASS,0.054 s,0.03 MiB,0.02 MiB,4,0,0.16 ms,93.60 mW,15436518.00 pJ,0.040 s,0.03 MiB,0.08 MiB,4,0,12900.00 samples/s,0.08 ms,5.00 mW,388853.50 pJ/it
reduce_mean/keepdims_0,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,5,0,0.00 ms,91.40 mW,68368.00 pJ,0.043 s,0.00 MiB,0.00 MiB,5,0,1300000.00 samples/s,0.00 ms,44.76 mW,36032.00 pJ/it
reduce_mean/large_dimension_1024,arch-a,PASS,PASS,0.048 s,0.01 MiB,0.00 MiB,1,0,0.00 ms,78.02 mW,217278.00 pJ,0.040 s,0.01 MiB,0.00 MiB,1,0,359000.00 samples/s,0.00 ms,2.02 mW,5274.00 pJ/it
reduce_mean/legacy_axes_1_2_keepdims_1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.35 mW,21505.00 pJ,0.042 s,0.00 MiB,0.00 MiB,2,0,3620000.00 samples/s,0.00 ms,3.45 mW,898.00 pJ/it
reduce_mean/legacy_axis1_keepdims_0,arch-a,PASS,PASS,0.056 s,0.00 MiB,0.00 MiB,9,0,0.00 ms,92.50 mW,183708.00 pJ,0.042 s,0.00 MiB,0.00 MiB,9,0,654000.00 samples/s,0.00 ms,45.89 mW,72573.50 pJ/it
reduce_mean/legacy_axis1_keepdims_1,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,8,0,0.00 ms,94.56 mW,129830.00 pJ,0.049 s,0.00 MiB,0.00 MiB,8,0,1340000.00 samples/s,0.00 ms,10.15 mW,7594.50 pJ/it
reduce_mean/legacy_empty_axes_noop,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it
reduce_mean/legacy_nchw_spatial,arch-a,PASS,PASS,0.055 s,0.00 MiB,0.00 MiB,1,0,0.01 ms,78.01 mW,498648.00 pJ,0.044 s,0.00 MiB,0.00 MiB,1,0,156000.00 samples/s,0.01 ms,2.01 mW,12796.67 pJ/it
reduce_mean/legacy_negative_axis,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.52 mW,51717.00 pJ,0.041 s,0.00 MiB,0.00 MiB,6,0,1760000.00 samples/s,0.00 ms,8.07 mW,4588.50 pJ/it
reduce_mean/legacy_reduce_all_keepdims_1,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it
reduce_mean/negative_axis,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.52 mW,51717.00 pJ,0.041 s,0.00 MiB,0.00 MiB,6,0,1760000.00 samples/s,0.00 ms,8.07 mW,4588.50 pJ/it
relu/4d,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.18 mW,40734.00 pJ,0.046 s,0.00 MiB,0.00 MiB,1,0,1930000.00 samples/s,0.00 ms,2.18 mW,1014.00 pJ/it
relu/after_conv,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.00 MiB,4,3,0.01 ms,107.89 mW,577437.72 pJ,0.048 s,0.00 MiB,0.00 MiB,4,3,191000.00 samples/s,0.01 ms,58.26 mW,304800.72 pJ/it
relu/after_gemm,arch-a,PASS,PASS,0.054 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.16 mW,790056.96 pJ,0.042 s,0.01 MiB,0.01 MiB,6,4,230000.00 samples/s,0.00 ms,109.21 mW,545663.29 pJ/it
relu/basic,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it
reshape/4d_to_2d_flatten,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.28 mW,20196.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,3910000.00 samples/s,0.00 ms,2.28 mW,488.00 pJ/it
reshape/infer_dim_minus_one,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it
reshape/same_rank,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it
reshape/zero_copies_input_dim,arch-a,PASS,PASS,0.054 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,12684.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,6250000.00 samples/s,0.00 ms,2.30 mW,308.00 pJ/it
resize/height_only,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.55 mW,64833.00 pJ,0.038 s,0.00 MiB,0.00 MiB,4,0,1880000.00 samples/s,0.00 ms,5.60 mW,2986.00 pJ/it
resize/nearest_2x,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,4,0,0.00 ms,93.57 mW,109761.00 pJ,0.038 s,0.00 MiB,0.00 MiB,4,0,1450000.00 samples/s,0.00 ms,5.46 mW,3776.00 pJ/it
resize/nearest_downsample,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.45 mW,33925.00 pJ,0.039 s,0.00 MiB,0.00 MiB,2,0,2330000.00 samples/s,0.00 ms,3.28 mW,1360.50 pJ/it
resize/non_uniform_scales,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,6,0,0.00 ms,93.58 mW,164037.00 pJ,0.042 s,0.00 MiB,0.00 MiB,6,0,1250000.00 samples/s,0.00 ms,7.76 mW,6207.25 pJ/it
resize/width_only,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,2,0,0.00 ms,79.50 mW,53029.00 pJ,0.037 s,0.00 MiB,0.00 MiB,2,0,1700000.00 samples/s,0.00 ms,3.20 mW,1833.50 pJ/it
resize/with_sizes,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,3,0,0.00 ms,92.54 mW,73756.00 pJ,0.039 s,0.00 MiB,0.00 MiB,3,0,1700000.00 samples/s,0.00 ms,4.39 mW,2586.75 pJ/it
sigmoid/4d,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.18 mW,40734.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1930000.00 samples/s,0.00 ms,2.18 mW,1014.00 pJ/it
sigmoid/after_gemm,arch-a,PASS,PASS,0.054 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,105.16 mW,790056.96 pJ,0.043 s,0.01 MiB,0.01 MiB,6,4,230000.00 samples/s,0.00 ms,109.21 mW,545663.29 pJ/it
sigmoid/basic,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,17286.00 pJ,0.036 s,0.00 MiB,0.00 MiB,1,0,4570000.00 samples/s,0.00 ms,2.22 mW,437.33 pJ/it
slice/2d_basic,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it
slice/after_conv,arch-a,PASS,PASS,0.057 s,0.00 MiB,0.01 MiB,7,6,0.01 ms,118.19 mW,1335082.88 pJ,0.053 s,0.00 MiB,0.01 MiB,7,6,106000.00 samples/s,0.01 ms,73.96 mW,732218.55 pJ/it
slice/default_axes,arch-a,PASS,PASS,0.047 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it
slice/large_channel_1024,arch-a,PASS,PASS,0.049 s,0.01 MiB,0.00 MiB,1,0,0.00 ms,78.14 mW,221304.00 pJ,0.036 s,0.01 MiB,0.00 MiB,1,0,353000.00 samples/s,0.00 ms,2.14 mW,5058.00 pJ/it
slice/nchw_spatial_crop,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.24 mW,101868.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,769000.00 samples/s,0.00 ms,2.24 mW,2851.67 pJ/it
slice/negative_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,44004.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,1790000.00 samples/s,0.00 ms,2.30 mW,1227.67 pJ/it
slice/negative_indices,arch-a,PASS,PASS,0.052 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,25212.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.30 mW,675.67 pJ/it
slice/step2,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.29 mW,159876.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,490000.00 samples/s,0.00 ms,2.29 mW,4619.67 pJ/it
softmax/3d_last_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.035 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED
softmax/basic,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED
softmax/channel_axis,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.040 s,0.00 MiB,0.00 MiB,3,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED
softmax/large_dimension_1024,arch-a,PASS,PASS,0.049 s,0.01 MiB,0.01 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.01 MiB,0.01 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED
softmax/negative_axis,arch-a,PASS,PASS,0.048 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,0.036 s,0.00 MiB,0.00 MiB,1,0,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED,UNSUPPORTED
split/basic,arch-a,PASS,PASS,0.051 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,31554.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,2490000.00 samples/s,0.00 ms,2.30 mW,861.67 pJ/it
split/equal_three_way,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,44160.00 pJ,0.035 s,0.00 MiB,0.00 MiB,1,0,1780000.00 samples/s,0.00 ms,2.30 mW,1231.67 pJ/it
split/negative_axis,arch-a,PASS,PASS,0.053 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.29 mW,84786.00 pJ,0.039 s,0.00 MiB,0.00 MiB,1,0,925000.00 samples/s,0.00 ms,2.29 mW,2413.67 pJ/it
split/uneven_channel_axis_4d,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.30 mW,18948.00 pJ,0.046 s,0.00 MiB,0.00 MiB,1,0,4170000.00 samples/s,0.00 ms,2.30 mW,491.67 pJ/it
sub/after_gemm,arch-a,PASS,PASS,0.056 s,0.01 MiB,0.01 MiB,5,4,0.01 ms,104.70 mW,815012.96 pJ,0.045 s,0.01 MiB,0.01 MiB,6,4,218000.00 samples/s,0.00 ms,107.94 mW,570766.12 pJ/it
sub/basic,arch-a,PASS,PASS,0.046 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.037 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
sub/broadcast_row,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.042 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
sub/channel_broadcast_1024,arch-a,PASS,PASS,0.052 s,0.02 MiB,0.01 MiB,1,0,0.01 ms,78.12 mW,540030.00 pJ,0.037 s,0.02 MiB,0.01 MiB,1,0,145000.00 samples/s,0.01 ms,2.11 mW,13388.67 pJ/it
sub/constant_lhs_broadcast,arch-a,PASS,PASS,0.049 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25188.00 pJ,0.034 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,656.67 pJ/it
sub/leading_dimension_broadcast,arch-a,PASS,PASS,0.050 s,0.00 MiB,0.00 MiB,1,0,0.00 ms,78.22 mW,25266.00 pJ,0.038 s,0.00 MiB,0.00 MiB,1,0,3120000.00 samples/s,0.00 ms,2.23 mW,658.67 pJ/it
1 Operation Arch Result (l) Result (t) Compile (l) Host mem (l) Cores mem (l) Cores (l) Xbars (l) Latency (l) Power (l) Energy (l) Compile (t) Host mem (t) Cores mem (t) Cores (t) Xbars (t) Avg latency (t) Throughput (t) Avg power (t) Avg energy (t)
2 add/after_gemm arch-a PASS PASS 0.056 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 104.70 mW 815012.96 pJ 0.043 s 0.01 MiB 0.01 MiB 6 4 218000.00 samples/s 0.00 ms 107.94 mW 570766.12 pJ/it
3 add/basic arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.041 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
4 add/broadcast_row arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
5 add/channel_broadcast_1024 arch-a PASS PASS 0.052 s 0.02 MiB 0.01 MiB 1 0 0.01 ms 78.12 mW 540030.00 pJ 0.036 s 0.02 MiB 0.01 MiB 1 0 145000.00 samples/s 0.01 ms 2.11 mW 13388.67 pJ/it
6 add/leading_dimension_broadcast arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
7 concat/channel_axis arch-a PASS PASS 0.047 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.16 mW 35718.00 pJ 0.034 s 0.00 MiB 0.00 MiB 1 0 2200000.00 samples/s 0.00 ms 2.16 mW 934.67 pJ/it
8 concat/negative_axis arch-a PASS PASS 0.046 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.09 mW 81450.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 961000.00 samples/s 0.00 ms 2.09 mW 2108.00 pJ/it
9 concat/three_inputs_channel_axis arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.15 mW 50328.00 pJ 0.035 s 0.00 MiB 0.00 MiB 1 0 1560000.00 samples/s 0.00 ms 2.15 mW 1332.67 pJ/it
10 conv/batch_2 arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 2 2 0.01 ms 82.62 mW 1131451.48 pJ 0.044 s 0.00 MiB 0.01 MiB 4 2 147000.00 samples/s 0.01 ms 92.90 mW 669920.48 pJ/it
11 conv/batch_4_pointwise arch-a PASS PASS 0.055 s 0.00 MiB 0.01 MiB 5 4 0.00 ms 116.08 mW 456420.96 pJ 0.043 s 0.01 MiB 0.01 MiB 5 4 242000.00 samples/s 0.00 ms 69.62 mW 291187.04 pJ/it
12 conv/depthwise_1024_channels arch-a PASS PASS 0.082 s 0.19 MiB 0.38 MiB 129 128 0.22 ms 178.45 mW 39393966.72 pJ 0.117 s 0.38 MiB 0.46 MiB 45 118 4930.00 samples/s 0.20 ms 144.34 mW 33005602.67 pJ/it
13 conv/depthwise_grouped arch-a PASS PASS 0.055 s 0.01 MiB 0.00 MiB 5 4 0.01 ms 107.78 mW 671878.96 pJ 0.042 s 0.01 MiB 0.00 MiB 7 4 418000.00 samples/s 0.00 ms 168.64 mW 479471.12 pJ/it
14 conv/dilated_3x3 arch-a PASS PASS 0.057 s 0.01 MiB 0.01 MiB 10 9 0.01 ms 118.77 mW 1034819.16 pJ 0.057 s 0.01 MiB 0.01 MiB 12 9 104000.00 samples/s 0.01 ms 155.40 mW 1574973.57 pJ/it
15 conv/dynamic arch-a PASS PASS 0.062 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 92.28 mW 169336.00 pJ 0.044 s 0.00 MiB 0.00 MiB 6 0 778000.00 samples/s 0.00 ms 86.62 mW 124092.33 pJ/it
16 conv/explicit_padding arch-a PASS PASS 0.053 s 0.01 MiB 0.02 MiB 17 16 0.01 ms 145.34 mW 1454397.84 pJ 0.048 s 0.01 MiB 0.02 MiB 19 16 179000.00 samples/s 0.01 ms 201.80 mW 1234440.20 pJ/it
17 conv/grouped_many_groups arch-a PASS PASS 0.474 s 0.05 MiB 0.09 MiB 65 64 0.18 ms 142.21 mW 25867112.36 pJ 0.476 s 0.08 MiB 0.73 MiB 87 64 3660.00 samples/s 0.27 ms 97.46 mW 30277556.57 pJ/it
18 conv/grouped_two_groups arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 3 2 0.01 ms 101.46 mW 543914.48 pJ 0.047 s 0.00 MiB 0.01 MiB 9 2 378000.00 samples/s 0.00 ms 105.40 mW 353700.73 pJ/it
19 conv/huge_pointwise_1024 arch-a PASS PASS 0.151 s 0.01 MiB 0.11 MiB 73 64 0.02 ms 249.58 mW 3895759.36 pJ 0.163 s 0.08 MiB 0.10 MiB 52 64 19200.00 samples/s 0.05 ms 163.09 mW 8722111.37 pJ/it
20 conv/huge_pointwise_1024_dynamic arch-a PASS PASS 0.080 s 8.04 MiB 12.61 MiB 168 0 2.63 ms 169.52 mW 445489032.00 pJ 0.224 s 12.75 MiB 6.82 MiB 45 0 188.00 samples/s 5.31 ms 134.20 mW 746263826.70 pJ/it
21 conv/input_224_7x7_stride2 arch-a PASS PASS 0.759 s 24.14 MiB 61.87 MiB 168 169 38.41 ms 185.26 mW 7116544212.12 pJ 1.060 s 51.25 MiB 67.85 MiB 45 84 24.70 samples/s 40.56 ms 148.32 mW 6561548886.00 pJ/it
22 conv/kernel_2x2 arch-a PASS PASS 0.061 s 0.00 MiB 0.00 MiB 1 1 0.00 ms 83.83 mW 360568.24 pJ 0.046 s 0.00 MiB 0.00 MiB 3 1 356000.00 samples/s 0.00 ms 93.82 mW 298919.07 pJ/it
23 conv/kernel_3x3 arch-a PASS PASS 0.061 s 0.01 MiB 0.01 MiB 10 9 0.01 ms 123.80 mW 889640.16 pJ 0.046 s 0.01 MiB 0.01 MiB 12 9 271000.00 samples/s 0.00 ms 195.49 mW 787478.85 pJ/it
24 conv/kernel_equals_input_spatial arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 2 2 0.00 ms 89.61 mW 415689.48 pJ 0.041 s 0.00 MiB 0.00 MiB 4 2 376000.00 samples/s 0.00 ms 104.42 mW 305239.85 pJ/it
25 conv/large_input_channels_1x1 arch-a PASS PASS 0.085 s 0.01 MiB 0.02 MiB 9 8 0.01 ms 117.84 mW 900569.92 pJ 0.081 s 0.02 MiB 0.02 MiB 10 8 135000.00 samples/s 0.01 ms 113.25 mW 910683.04 pJ/it
26 conv/large_output_channels_1x1 arch-a PASS PASS 0.095 s 0.01 MiB 0.02 MiB 17 8 0.01 ms 128.44 mW 1139415.92 pJ 0.078 s 0.02 MiB 0.02 MiB 18 8 93600.00 samples/s 0.01 ms 151.83 mW 1755770.60 pJ/it
27 conv/large_spatial arch-a PASS PASS 0.061 s 0.01 MiB 0.04 MiB 37 36 0.02 ms 172.07 mW 2928344.64 pJ 0.061 s 0.01 MiB 0.04 MiB 39 36 84200.00 samples/s 0.01 ms 208.30 mW 2603711.62 pJ/it
28 conv/multi_channel arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 105.68 mW 685040.72 pJ 0.050 s 0.00 MiB 0.00 MiB 4 3 140000.00 samples/s 0.01 ms 54.54 mW 393755.39 pJ/it
29 conv/non_square_kernel_1x3 arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 3 2 0.01 ms 99.35 mW 679752.48 pJ 0.044 s 0.00 MiB 0.00 MiB 3 2 140000.00 samples/s 0.01 ms 51.01 mW 367557.81 pJ/it
30 conv/non_square_kernel_3x1 arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 3 2 0.01 ms 95.89 mW 1292976.48 pJ 0.045 s 0.00 MiB 0.00 MiB 3 2 72200.00 samples/s 0.01 ms 47.79 mW 664683.81 pJ/it
31 conv/non_uniform_stride arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 104.05 mW 790874.72 pJ 0.044 s 0.00 MiB 0.00 MiB 4 3 126000.00 samples/s 0.01 ms 53.56 mW 429978.05 pJ/it
32 conv/output_channel_grouping_minimal arch-a PASS PASS 0.080 s 0.10 MiB 0.34 MiB 131 128 0.26 ms 170.73 mW 44125916.72 pJ 0.148 s 0.28 MiB 0.96 MiB 87 84 2630.00 samples/s 0.38 ms 136.74 mW 58542917.01 pJ/it
33 conv/pointwise_1x1 arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 1 0.01 ms 80.24 mW 987244.24 pJ 0.041 s 0.00 MiB 0.00 MiB 3 1 131000.00 samples/s 0.01 ms 89.08 mW 719413.57 pJ/it
34 conv/pointwise_tiled_chain arch-a PASS PASS 0.642 s 0.01 MiB 0.04 MiB 20 80 0.04 ms 153.90 mW 6443957.20 pJ 0.620 s 0.06 MiB 0.08 MiB 22 80 16400.00 samples/s 0.06 ms 186.69 mW 12410041.37 pJ/it
35 conv/real_asymmetric_padding arch-a PASS PASS 0.057 s 0.01 MiB 0.03 MiB 29 28 0.01 ms 153.67 mW 2221606.72 pJ 0.055 s 0.01 MiB 0.03 MiB 31 28 105000.00 samples/s 0.01 ms 204.35 mW 2058223.49 pJ/it
36 conv/relu_conv_store arch-a PASS PASS 0.085 s 0.16 MiB 0.67 MiB 168 184 0.56 ms 183.08 mW 103057723.80 pJ 0.209 s 0.37 MiB 0.58 MiB 60 58 1500.00 samples/s 0.67 ms 146.73 mW 101291048.70 pJ/it
37 conv/same_lower_3x3 arch-a PASS PASS 0.060 s 0.01 MiB 0.02 MiB 26 25 0.01 ms 166.15 mW 2215009.00 pJ 0.054 s 0.01 MiB 0.03 MiB 28 25 119000.00 samples/s 0.01 ms 204.48 mW 1837219.40 pJ/it
38 conv/same_padding_3x3 arch-a PASS PASS 0.058 s 0.01 MiB 0.02 MiB 26 25 0.01 ms 166.15 mW 2215009.00 pJ 0.056 s 0.01 MiB 0.03 MiB 28 25 119000.00 samples/s 0.01 ms 204.48 mW 1837219.40 pJ/it
39 conv/strategy_depthwise_16 arch-a PASS PASS 0.093 s 0.06 MiB 0.35 MiB 168 168 0.34 ms 197.94 mW 66331479.08 pJ 0.257 s 0.20 MiB 0.17 MiB 45 84 2650.00 samples/s 0.38 ms 153.71 mW 60322816.49 pJ/it
40 conv/strategy_input_k_tiled arch-a PASS PASS 0.082 s 0.08 MiB 0.27 MiB 109 108 0.35 ms 170.81 mW 60422605.92 pJ 0.100 s 0.22 MiB 0.28 MiB 45 90 4250.00 samples/s 0.24 ms 143.52 mW 36750587.75 pJ/it
41 conv/strategy_output_channel_tiled arch-a PASS PASS 0.073 s 0.03 MiB 0.16 MiB 74 72 0.09 ms 155.74 mW 14244739.28 pJ 0.110 s 0.13 MiB 0.22 MiB 81 72 9220.00 samples/s 0.11 ms 145.73 mW 18577796.14 pJ/it
42 conv/strategy_streamed_packed arch-a PASS PASS 0.155 s 3.34 MiB 7.89 MiB 168 168 9.35 ms 179.86 mW 1682364509.56 pJ 0.357 s 5.32 MiB 7.79 MiB 43 42 113.00 samples/s 8.87 ms 61.68 mW 558465684.90 pJ/it
43 conv/strategy_streamed_patch arch-a PASS PASS 0.115 s 0.34 MiB 1.32 MiB 168 168 1.90 ms 181.91 mW 346476645.64 pJ 0.299 s 0.82 MiB 1.21 MiB 43 42 501.00 samples/s 1.99 ms 62.44 mW 125148059.00 pJ/it
44 conv/strategy_tiled_2d arch-a PASS PASS 0.111 s 0.11 MiB 0.44 MiB 168 168 0.42 ms 182.13 mW 75690907.84 pJ 0.219 s 0.36 MiB 0.46 MiB 47 144 4070.00 samples/s 0.25 ms 155.26 mW 43400768.51 pJ/it
45 conv/stride_2 arch-a PASS PASS 0.056 s 0.01 MiB 0.00 MiB 5 4 0.01 ms 110.78 mW 580154.96 pJ 0.043 s 0.01 MiB 0.00 MiB 7 4 476000.00 samples/s 0.00 ms 175.93 mW 424103.12 pJ/it
46 conv/with_bias_3x3 arch-a PASS PASS 0.059 s 0.00 MiB 0.01 MiB 4 3 0.01 ms 104.16 mW 776220.72 pJ 0.046 s 0.00 MiB 0.01 MiB 4 3 133000.00 samples/s 0.01 ms 53.85 mW 416288.72 pJ/it
47 conv/with_constant arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 1 0.01 ms 81.74 mW 541270.24 pJ 0.043 s 0.00 MiB 0.00 MiB 4 1 231000.00 samples/s 0.00 ms 133.09 mW 655848.91 pJ/it
48 conv/without_kernel_shape_attr arch-a PASS PASS 0.053 s 0.01 MiB 0.01 MiB 10 9 0.01 ms 123.80 mW 889640.16 pJ 0.047 s 0.01 MiB 0.01 MiB 12 9 271000.00 samples/s 0.00 ms 195.49 mW 787478.85 pJ/it
49 conv/yolo11n_depthwise_head arch-a PASS PASS 1.709 s 8.66 MiB 34.24 MiB 168 255 42.70 ms 200.52 mW 8562452412.00 pJ 2.373 s 27.15 MiB 20.31 MiB 87 214 20.60 samples/s 48.55 ms 157.12 mW 8007922821.00 pJ/it
50 conv/yolo11n_heavy arch-a PASS PASS 0.519 s 4.82 MiB 19.10 MiB 161 800 8.53 ms 350.87 mW 2994683017.00 pJ 1.007 s 11.06 MiB 13.87 MiB 85 420 79.10 samples/s 12.64 ms 218.17 mW 2857451109.00 pJ/it
51 conv/yolo11n_stem arch-a PASS PASS 0.893 s 12.86 MiB 37.59 MiB 168 488 14.24 ms 301.23 mW 4289558246.00 pJ 1.374 s 25.19 MiB 21.24 MiB 85 126 44.60 samples/s 22.42 ms 176.88 mW 4028789243.00 pJ/it
52 div/after_gemm arch-a PASS PASS 0.053 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 104.70 mW 815012.96 pJ 0.045 s 0.01 MiB 0.01 MiB 6 4 218000.00 samples/s 0.00 ms 107.94 mW 570766.12 pJ/it
53 div/basic arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
54 div/channel_broadcast_1024 arch-a PASS PASS 0.049 s 0.02 MiB 0.01 MiB 1 0 0.01 ms 78.12 mW 540030.00 pJ 0.039 s 0.02 MiB 0.01 MiB 1 0 145000.00 samples/s 0.01 ms 2.11 mW 13388.67 pJ/it
55 div/leading_dimension_broadcast arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.040 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
56 div/runtime_scalar_rhs arch-a PASS PASS 0.053 s 0.02 MiB 0.01 MiB 1 0 0.01 ms 78.12 mW 540030.00 pJ 0.040 s 0.02 MiB 0.01 MiB 1 0 145000.00 samples/s 0.01 ms 2.11 mW 13388.67 pJ/it
57 div/scalar_constant arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
58 gather/3d_input_axis1 arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.08 mW 45990.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 1700000.00 samples/s 0.00 ms 2.08 mW 1174.67 pJ/it
59 gather/axis0_matrix_indices arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.07 mW 54414.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 1440000.00 samples/s 0.00 ms 2.07 mW 1390.67 pJ/it
60 gather/axis1 arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.06 mW 62526.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 1250000.00 samples/s 0.00 ms 2.06 mW 1598.67 pJ/it
61 gather/negative_axis arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.03 mW 112134.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 697000.00 samples/s 0.00 ms 2.03 mW 2870.67 pJ/it
62 gather/negative_indices arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.13 mW 29376.00 pJ 0.041 s 0.00 MiB 0.00 MiB 1 0 2670000.00 samples/s 0.00 ms 2.12 mW 748.67 pJ/it
63 gemm/alpha_beta arch-a PASS PASS 0.056 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 105.27 mW 784908.96 pJ 0.050 s 0.01 MiB 0.01 MiB 6 4 212000.00 samples/s 0.00 ms 107.78 mW 574589.95 pJ/it
64 gemm/bias_rank2_broadcast arch-a PASS PASS 0.056 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 105.98 mW 749484.96 pJ 0.043 s 0.01 MiB 0.01 MiB 6 4 229000.00 samples/s 0.00 ms 109.50 mW 540708.12 pJ/it
65 gemm/dynamic arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.48 mW 221475.00 pJ 0.044 s 0.00 MiB 0.00 MiB 5 0 489000.00 samples/s 0.00 ms 44.34 mW 93754.67 pJ/it
66 gemm/dynamic_alpha arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.42 mW 298198.00 pJ 0.041 s 0.00 MiB 0.00 MiB 5 0 464000.00 samples/s 0.00 ms 44.39 mW 105291.67 pJ/it
67 gemm/dynamic_beta arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.32 mW 397230.00 pJ 0.042 s 0.00 MiB 0.00 MiB 5 0 247000.00 samples/s 0.00 ms 20.21 mW 81901.75 pJ/it
68 gemm/dynamic_bias arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.45 mW 243703.00 pJ 0.046 s 0.00 MiB 0.00 MiB 5 0 422000.00 samples/s 0.00 ms 20.28 mW 48009.75 pJ/it
69 gemm/dynamic_bias_alpha_beta arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 5 0 0.01 ms 91.28 mW 513811.00 pJ 0.043 s 0.00 MiB 0.00 MiB 5 0 188000.00 samples/s 0.01 ms 20.20 mW 107673.75 pJ/it
70 gemm/dynamic_transpose_b arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.38 mW 118883.00 pJ 0.040 s 0.00 MiB 0.00 MiB 5 0 803000.00 samples/s 0.00 ms 44.55 mW 58232.00 pJ/it
71 gemm/huge_1024 arch-a PASS PASS 0.149 s 0.01 MiB 0.10 MiB 73 64 0.02 ms 215.07 mW 3767010.36 pJ 0.160 s 0.05 MiB 0.09 MiB 51 64 26400.00 samples/s 0.04 ms 135.13 mW 5670462.59 pJ/it
72 gemm/large arch-a PASS PASS 0.058 s 0.02 MiB 0.03 MiB 17 16 0.01 ms 140.15 mW 1573768.84 pJ 0.050 s 0.03 MiB 0.03 MiB 17 16 78500.00 samples/s 0.01 ms 79.60 mW 1008067.12 pJ/it
73 gemm/large_k_small_n arch-a PASS PASS 0.088 s 0.01 MiB 0.01 MiB 9 8 0.00 ms 133.53 mW 633217.92 pJ 0.082 s 0.01 MiB 0.01 MiB 9 8 182000.00 samples/s 0.01 ms 84.31 mW 476982.66 pJ/it
74 gemm/non_square arch-a PASS PASS 0.059 s 0.00 MiB 0.01 MiB 5 4 0.00 ms 118.96 mW 419565.96 pJ 0.043 s 0.01 MiB 0.01 MiB 5 4 242000.00 samples/s 0.00 ms 71.16 mW 302182.45 pJ/it
75 gemm/scalar_bias arch-a PASS PASS 0.055 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 105.98 mW 749484.96 pJ 0.045 s 0.01 MiB 0.01 MiB 6 4 229000.00 samples/s 0.00 ms 109.50 mW 540708.12 pJ/it
76 gemm/small arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 2 2 0.00 ms 90.14 mW 398436.48 pJ 0.043 s 0.00 MiB 0.00 MiB 4 2 376000.00 samples/s 0.00 ms 105.00 mW 297577.19 pJ/it
77 gemm/small_k_large_n arch-a PASS PASS 0.090 s 0.01 MiB 0.02 MiB 17 8 0.01 ms 131.01 mW 1043061.92 pJ 0.082 s 0.02 MiB 0.02 MiB 18 8 92700.00 samples/s 0.01 ms 152.38 mW 1742739.72 pJ/it
78 gemm/square_weights arch-a PASS PASS 0.073 s 0.03 MiB 0.08 MiB 42 40 0.02 ms 151.77 mW 3284393.60 pJ 0.075 s 0.07 MiB 0.09 MiB 44 40 25700.00 samples/s 0.04 ms 155.80 mW 6344327.95 pJ/it
79 gemm/transpose_a arch-a PASS PASS 0.056 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 109.14 mW 628868.96 pJ 0.047 s 0.01 MiB 0.01 MiB 6 4 290000.00 samples/s 0.00 ms 115.23 mW 456854.62 pJ/it
80 gemm/transpose_a_and_b arch-a PASS PASS 0.054 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 109.14 mW 628868.96 pJ 0.044 s 0.01 MiB 0.01 MiB 6 4 290000.00 samples/s 0.00 ms 115.23 mW 456854.62 pJ/it
81 gemm/transpose_b arch-a PASS PASS 0.056 s 0.00 MiB 0.01 MiB 5 4 0.00 ms 118.96 mW 419565.96 pJ 0.045 s 0.01 MiB 0.01 MiB 5 4 242000.00 samples/s 0.00 ms 71.16 mW 302182.45 pJ/it
82 gemm/transpose_b_with_bias arch-a PASS PASS 0.057 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 110.55 mW 557818.96 pJ 0.046 s 0.01 MiB 0.01 MiB 5 4 218000.00 samples/s 0.00 ms 67.31 mW 333372.79 pJ/it
83 gemm/with_bias arch-a PASS PASS 0.057 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 108.77 mW 604966.96 pJ 0.043 s 0.01 MiB 0.01 MiB 5 4 203000.00 samples/s 0.00 ms 66.54 mW 341027.79 pJ/it
84 gemv/all_constant arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 0 0 0.00 ms 2.00 mW 0.00 pJ 0.039 s 0.00 MiB 0.00 MiB 0 0 0.00 samples/s 0.00 ms 2.00 mW 0.00 pJ/it
85 gemv/constant_weight arch-a PASS PASS 0.065 s 0.00 MiB 0.01 MiB 6 4 0.01 ms 111.15 mW 573535.96 pJ 0.051 s 0.01 MiB 0.01 MiB 8 4 263000.00 samples/s 0.00 ms 154.40 mW 641346.96 pJ/it
86 gemv/non_uniform_bias arch-a PASS PASS 0.062 s 0.00 MiB 0.01 MiB 6 4 0.01 ms 109.81 mW 609463.96 pJ 0.051 s 0.01 MiB 0.01 MiB 8 4 243000.00 samples/s 0.00 ms 152.55 mW 697203.20 pJ/it
87 gemv/scalar_bias arch-a PASS PASS 0.061 s 0.00 MiB 0.01 MiB 6 4 0.01 ms 109.81 mW 609463.96 pJ 0.052 s 0.01 MiB 0.01 MiB 8 4 243000.00 samples/s 0.00 ms 152.55 mW 697203.20 pJ/it
88 gemv/uniform_bias arch-a PASS PASS 0.066 s 0.00 MiB 0.01 MiB 6 4 0.01 ms 109.81 mW 609463.96 pJ 0.049 s 0.01 MiB 0.01 MiB 8 4 243000.00 samples/s 0.00 ms 152.55 mW 697203.20 pJ/it
89 matmul/basic arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 2 2 0.00 ms 90.14 mW 398436.48 pJ 0.040 s 0.00 MiB 0.00 MiB 4 2 376000.00 samples/s 0.00 ms 105.00 mW 297577.19 pJ/it
90 matmul/batched_3d arch-a PASS PASS 0.059 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 108.59 mW 646972.96 pJ 0.048 s 0.01 MiB 0.01 MiB 6 4 275000.00 samples/s 0.00 ms 114.15 mW 469999.95 pJ/it
91 matmul/batched_3d_dynamic arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 4 0 0.00 ms 92.19 mW 167975.00 pJ 0.041 s 0.00 MiB 0.00 MiB 5 0 758000.00 samples/s 0.00 ms 18.49 mW 24624.50 pJ/it
92 matmul/batched_left_constant arch-a PASS PASS 0.057 s 0.00 MiB 0.02 MiB 9 8 0.01 ms 114.39 mW 1009105.92 pJ 0.059 s 0.01 MiB 0.02 MiB 11 8 149000.00 samples/s 0.01 ms 159.00 mW 1186296.28 pJ/it
93 matmul/batched_lhs_broadcast arch-a PASS PASS 0.057 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 109.39 mW 621440.96 pJ 0.042 s 0.01 MiB 0.01 MiB 6 4 276000.00 samples/s 0.00 ms 114.66 mW 463736.29 pJ/it
94 matmul/batched_rhs_broadcast arch-a PASS PASS 0.058 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 108.59 mW 646972.96 pJ 0.043 s 0.01 MiB 0.01 MiB 6 4 275000.00 samples/s 0.00 ms 114.15 mW 469999.95 pJ/it
95 matmul/dynamic arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.42 mW 148195.00 pJ 0.041 s 0.00 MiB 0.00 MiB 5 0 660000.00 samples/s 0.00 ms 44.46 mW 70471.33 pJ/it
96 matmul/huge_1024 arch-a PASS PASS 0.146 s 0.01 MiB 0.10 MiB 73 64 0.02 ms 215.07 mW 3767010.36 pJ 0.162 s 0.05 MiB 0.09 MiB 51 64 26400.00 samples/s 0.04 ms 135.13 mW 5670462.59 pJ/it
97 matmul/left_constant arch-a PASS PASS 0.055 s 0.00 MiB 0.01 MiB 5 4 0.01 ms 108.86 mW 637168.96 pJ 0.045 s 0.01 MiB 0.01 MiB 6 4 297000.00 samples/s 0.00 ms 115.66 mW 451626.62 pJ/it
98 matmul/matrix_vector arch-a PASS PASS 0.098 s 0.52 MiB 0.78 MiB 168 173 0.38 ms 202.13 mW 77751476.88 pJ 0.289 s 1.04 MiB 0.84 MiB 44 171 2150.00 samples/s 0.46 ms 119.16 mW 65096859.53 pJ/it
99 matmul/vector_matrix arch-a PASS PASS 0.089 s 0.01 MiB 0.01 MiB 9 8 0.01 ms 118.70 mW 878749.92 pJ 0.083 s 0.02 MiB 0.02 MiB 10 8 140000.00 samples/s 0.01 ms 115.31 mW 910373.16 pJ/it
100 matmul/yolo_attention arch-a PASS PASS 0.474 s 1.02 MiB 43.44 MiB 168 0 8.15 ms 170.00 mW 1385775865.00 pJ 0.645 s 5.57 MiB 42.54 MiB 75 0 145.00 samples/s 6.89 ms 119.04 mW 932561637.00 pJ/it
101 mul/after_conv arch-a PASS PASS 0.059 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 107.64 mW 586955.72 pJ 0.050 s 0.00 MiB 0.00 MiB 4 3 194000.00 samples/s 0.01 ms 58.31 mW 304072.05 pJ/it
102 mul/after_conv_scalar_constant arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 107.64 mW 586955.72 pJ 0.046 s 0.00 MiB 0.00 MiB 4 3 194000.00 samples/s 0.01 ms 58.31 mW 304072.05 pJ/it
103 mul/basic arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.040 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
104 mul/channel_broadcast_1024 arch-a PASS PASS 0.050 s 0.02 MiB 0.01 MiB 1 0 0.01 ms 78.12 mW 540030.00 pJ 0.038 s 0.02 MiB 0.01 MiB 1 0 145000.00 samples/s 0.01 ms 2.11 mW 13388.67 pJ/it
105 mul/leading_dimension_broadcast arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
106 mul/scalar_constant arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.034 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
107 pool/avg_basic arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.02 mW 929400.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 84200.00 samples/s 0.01 ms 2.02 mW 24013.00 pJ/it
108 pool/avg_ceil_mode arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.03 mW 339210.00 pJ 0.044 s 0.00 MiB 0.00 MiB 1 0 230000.00 samples/s 0.00 ms 2.03 mW 8786.67 pJ/it
109 pool/avg_explicit_padding arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.03 mW 685860.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 114000.00 samples/s 0.01 ms 2.03 mW 17745.00 pJ/it
110 pool/avg_include_pad arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.02 mW 661116.00 pJ 0.041 s 0.00 MiB 0.00 MiB 1 0 118000.00 samples/s 0.01 ms 2.02 mW 17017.00 pJ/it
111 pool/avg_large_channels arch-a PASS PASS 0.059 s 0.04 MiB 0.02 MiB 1 0 0.24 ms 78.00 mW 18397284.00 pJ 0.046 s 0.04 MiB 0.02 MiB 1 0 4250.00 samples/s 0.24 ms 2.00 mW 471380.00 pJ/it
112 pool/avg_non_uniform_stride arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.02 mW 1129134.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 69300.00 samples/s 0.01 ms 2.02 mW 29111.00 pJ/it
113 pool/avg_real_asymmetric_padding arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 0 0.03 ms 78.02 mW 1959204.00 pJ 0.041 s 0.00 MiB 0.00 MiB 1 0 39900.00 samples/s 0.03 ms 2.02 mW 50769.00 pJ/it
114 pool/max_after_conv arch-a PASS PASS 0.063 s 0.00 MiB 0.00 MiB 5 4 0.01 ms 99.12 mW 1209961.96 pJ 0.051 s 0.00 MiB 0.00 MiB 5 4 140000.00 samples/s 0.01 ms 58.47 mW 468396.45 pJ/it
115 pool/max_basic arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.06 mW 324744.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 241000.00 samples/s 0.00 ms 2.06 mW 8532.67 pJ/it
116 pool/max_ceil_mode arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.07 mW 151464.00 pJ 0.035 s 0.00 MiB 0.00 MiB 1 0 516000.00 samples/s 0.00 ms 2.07 mW 3972.67 pJ/it
117 pool/max_global_style_kernel_equals_input arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.01 mW 657534.00 pJ 0.042 s 0.00 MiB 0.00 MiB 1 0 119000.00 samples/s 0.01 ms 2.01 mW 16843.00 pJ/it
118 pool/max_non_square_kernel arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.02 mW 1060572.00 pJ 0.047 s 0.00 MiB 0.00 MiB 1 0 73700.00 samples/s 0.01 ms 2.02 mW 27353.00 pJ/it
119 pool/max_real_asymmetric_padding arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.03 mW 814992.00 pJ 0.040 s 0.00 MiB 0.00 MiB 1 0 96100.00 samples/s 0.01 ms 2.03 mW 21173.00 pJ/it
120 pool/max_same_upper arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.04 mW 625068.00 pJ 0.042 s 0.00 MiB 0.00 MiB 1 0 125000.00 samples/s 0.01 ms 2.04 mW 16233.00 pJ/it
121 pool/max_stride2_multichannel arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.02 ms 78.02 mW 1245870.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 62800.00 samples/s 0.02 ms 2.02 mW 32117.00 pJ/it
122 reduce_mean/4d_spatial arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.02 mW 326814.00 pJ 0.044 s 0.00 MiB 0.00 MiB 1 0 239000.00 samples/s 0.00 ms 2.02 mW 8390.67 pJ/it
123 reduce_mean/4d_spatial_keepdims_0 arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 4 0 0.00 ms 94.35 mW 61801.00 pJ 0.041 s 0.00 MiB 0.00 MiB 4 0 1470000.00 samples/s 0.00 ms 44.66 mW 31425.33 pJ/it
124 reduce_mean/after_conv arch-a PASS PASS 0.062 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 100.79 mW 1119699.72 pJ 0.057 s 0.00 MiB 0.00 MiB 4 3 125000.00 samples/s 0.01 ms 54.14 mW 474782.17 pJ/it
125 reduce_mean/all_axes_keepdims_0 arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 2 0 0.00 ms 79.24 mW 30982.00 pJ 0.038 s 0.00 MiB 0.00 MiB 2 0 2720000.00 samples/s 0.00 ms 44.37 mW 16707.00 pJ/it
126 reduce_mean/all_axes_keepdims_1 arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 17286.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 4570000.00 samples/s 0.00 ms 2.22 mW 437.33 pJ/it
127 reduce_mean/basic arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 4 0 0.00 ms 93.51 mW 34881.00 pJ 0.042 s 0.00 MiB 0.00 MiB 4 0 2600000.00 samples/s 0.00 ms 5.85 mW 2235.67 pJ/it
128 reduce_mean/channel_axis_nchw arch-a PASS PASS 0.054 s 0.03 MiB 0.02 MiB 4 0 0.16 ms 93.60 mW 15436518.00 pJ 0.040 s 0.03 MiB 0.08 MiB 4 0 12900.00 samples/s 0.08 ms 5.00 mW 388853.50 pJ/it
129 reduce_mean/keepdims_0 arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 5 0 0.00 ms 91.40 mW 68368.00 pJ 0.043 s 0.00 MiB 0.00 MiB 5 0 1300000.00 samples/s 0.00 ms 44.76 mW 36032.00 pJ/it
130 reduce_mean/large_dimension_1024 arch-a PASS PASS 0.048 s 0.01 MiB 0.00 MiB 1 0 0.00 ms 78.02 mW 217278.00 pJ 0.040 s 0.01 MiB 0.00 MiB 1 0 359000.00 samples/s 0.00 ms 2.02 mW 5274.00 pJ/it
131 reduce_mean/legacy_axes_1_2_keepdims_1 arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 2 0 0.00 ms 79.35 mW 21505.00 pJ 0.042 s 0.00 MiB 0.00 MiB 2 0 3620000.00 samples/s 0.00 ms 3.45 mW 898.00 pJ/it
132 reduce_mean/legacy_axis1_keepdims_0 arch-a PASS PASS 0.056 s 0.00 MiB 0.00 MiB 9 0 0.00 ms 92.50 mW 183708.00 pJ 0.042 s 0.00 MiB 0.00 MiB 9 0 654000.00 samples/s 0.00 ms 45.89 mW 72573.50 pJ/it
133 reduce_mean/legacy_axis1_keepdims_1 arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 8 0 0.00 ms 94.56 mW 129830.00 pJ 0.049 s 0.00 MiB 0.00 MiB 8 0 1340000.00 samples/s 0.00 ms 10.15 mW 7594.50 pJ/it
134 reduce_mean/legacy_empty_axes_noop arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 17286.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 4570000.00 samples/s 0.00 ms 2.22 mW 437.33 pJ/it
135 reduce_mean/legacy_nchw_spatial arch-a PASS PASS 0.055 s 0.00 MiB 0.00 MiB 1 0 0.01 ms 78.01 mW 498648.00 pJ 0.044 s 0.00 MiB 0.00 MiB 1 0 156000.00 samples/s 0.01 ms 2.01 mW 12796.67 pJ/it
136 reduce_mean/legacy_negative_axis arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 6 0 0.00 ms 93.52 mW 51717.00 pJ 0.041 s 0.00 MiB 0.00 MiB 6 0 1760000.00 samples/s 0.00 ms 8.07 mW 4588.50 pJ/it
137 reduce_mean/legacy_reduce_all_keepdims_1 arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 17286.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 4570000.00 samples/s 0.00 ms 2.22 mW 437.33 pJ/it
138 reduce_mean/negative_axis arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 6 0 0.00 ms 93.52 mW 51717.00 pJ 0.041 s 0.00 MiB 0.00 MiB 6 0 1760000.00 samples/s 0.00 ms 8.07 mW 4588.50 pJ/it
139 relu/4d arch-a PASS PASS 0.047 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.18 mW 40734.00 pJ 0.046 s 0.00 MiB 0.00 MiB 1 0 1930000.00 samples/s 0.00 ms 2.18 mW 1014.00 pJ/it
140 relu/after_conv arch-a PASS PASS 0.057 s 0.00 MiB 0.00 MiB 4 3 0.01 ms 107.89 mW 577437.72 pJ 0.048 s 0.00 MiB 0.00 MiB 4 3 191000.00 samples/s 0.01 ms 58.26 mW 304800.72 pJ/it
141 relu/after_gemm arch-a PASS PASS 0.054 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 105.16 mW 790056.96 pJ 0.042 s 0.01 MiB 0.01 MiB 6 4 230000.00 samples/s 0.00 ms 109.21 mW 545663.29 pJ/it
142 relu/basic arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 17286.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 4570000.00 samples/s 0.00 ms 2.22 mW 437.33 pJ/it
143 reshape/4d_to_2d_flatten arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.28 mW 20196.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 3910000.00 samples/s 0.00 ms 2.28 mW 488.00 pJ/it
144 reshape/infer_dim_minus_one arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 12684.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 6250000.00 samples/s 0.00 ms 2.30 mW 308.00 pJ/it
145 reshape/same_rank arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 12684.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 6250000.00 samples/s 0.00 ms 2.30 mW 308.00 pJ/it
146 reshape/zero_copies_input_dim arch-a PASS PASS 0.054 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 12684.00 pJ 0.034 s 0.00 MiB 0.00 MiB 1 0 6250000.00 samples/s 0.00 ms 2.30 mW 308.00 pJ/it
147 resize/height_only arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 4 0 0.00 ms 93.55 mW 64833.00 pJ 0.038 s 0.00 MiB 0.00 MiB 4 0 1880000.00 samples/s 0.00 ms 5.60 mW 2986.00 pJ/it
148 resize/nearest_2x arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 4 0 0.00 ms 93.57 mW 109761.00 pJ 0.038 s 0.00 MiB 0.00 MiB 4 0 1450000.00 samples/s 0.00 ms 5.46 mW 3776.00 pJ/it
149 resize/nearest_downsample arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 2 0 0.00 ms 79.45 mW 33925.00 pJ 0.039 s 0.00 MiB 0.00 MiB 2 0 2330000.00 samples/s 0.00 ms 3.28 mW 1360.50 pJ/it
150 resize/non_uniform_scales arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 6 0 0.00 ms 93.58 mW 164037.00 pJ 0.042 s 0.00 MiB 0.00 MiB 6 0 1250000.00 samples/s 0.00 ms 7.76 mW 6207.25 pJ/it
151 resize/width_only arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 2 0 0.00 ms 79.50 mW 53029.00 pJ 0.037 s 0.00 MiB 0.00 MiB 2 0 1700000.00 samples/s 0.00 ms 3.20 mW 1833.50 pJ/it
152 resize/with_sizes arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 3 0 0.00 ms 92.54 mW 73756.00 pJ 0.039 s 0.00 MiB 0.00 MiB 3 0 1700000.00 samples/s 0.00 ms 4.39 mW 2586.75 pJ/it
153 sigmoid/4d arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.18 mW 40734.00 pJ 0.035 s 0.00 MiB 0.00 MiB 1 0 1930000.00 samples/s 0.00 ms 2.18 mW 1014.00 pJ/it
154 sigmoid/after_gemm arch-a PASS PASS 0.054 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 105.16 mW 790056.96 pJ 0.043 s 0.01 MiB 0.01 MiB 6 4 230000.00 samples/s 0.00 ms 109.21 mW 545663.29 pJ/it
155 sigmoid/basic arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 17286.00 pJ 0.036 s 0.00 MiB 0.00 MiB 1 0 4570000.00 samples/s 0.00 ms 2.22 mW 437.33 pJ/it
156 slice/2d_basic arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 18948.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 4170000.00 samples/s 0.00 ms 2.30 mW 491.67 pJ/it
157 slice/after_conv arch-a PASS PASS 0.057 s 0.00 MiB 0.01 MiB 7 6 0.01 ms 118.19 mW 1335082.88 pJ 0.053 s 0.00 MiB 0.01 MiB 7 6 106000.00 samples/s 0.01 ms 73.96 mW 732218.55 pJ/it
158 slice/default_axes arch-a PASS PASS 0.047 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 18948.00 pJ 0.035 s 0.00 MiB 0.00 MiB 1 0 4170000.00 samples/s 0.00 ms 2.30 mW 491.67 pJ/it
159 slice/large_channel_1024 arch-a PASS PASS 0.049 s 0.01 MiB 0.00 MiB 1 0 0.00 ms 78.14 mW 221304.00 pJ 0.036 s 0.01 MiB 0.00 MiB 1 0 353000.00 samples/s 0.00 ms 2.14 mW 5058.00 pJ/it
160 slice/nchw_spatial_crop arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.24 mW 101868.00 pJ 0.042 s 0.00 MiB 0.00 MiB 1 0 769000.00 samples/s 0.00 ms 2.24 mW 2851.67 pJ/it
161 slice/negative_axis arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 44004.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 1790000.00 samples/s 0.00 ms 2.30 mW 1227.67 pJ/it
162 slice/negative_indices arch-a PASS PASS 0.052 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 25212.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.30 mW 675.67 pJ/it
163 slice/step2 arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.29 mW 159876.00 pJ 0.042 s 0.00 MiB 0.00 MiB 1 0 490000.00 samples/s 0.00 ms 2.29 mW 4619.67 pJ/it
164 softmax/3d_last_axis arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED 0.035 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED UNSUPPORTED
165 softmax/basic arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED 0.036 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED UNSUPPORTED
166 softmax/channel_axis arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED 0.040 s 0.00 MiB 0.00 MiB 3 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED UNSUPPORTED
167 softmax/large_dimension_1024 arch-a PASS PASS 0.049 s 0.01 MiB 0.01 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED 0.036 s 0.01 MiB 0.01 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED UNSUPPORTED
168 softmax/negative_axis arch-a PASS PASS 0.048 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED 0.036 s 0.00 MiB 0.00 MiB 1 0 UNSUPPORTED UNSUPPORTED UNSUPPORTED UNSUPPORTED
169 split/basic arch-a PASS PASS 0.051 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 31554.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 2490000.00 samples/s 0.00 ms 2.30 mW 861.67 pJ/it
170 split/equal_three_way arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 44160.00 pJ 0.035 s 0.00 MiB 0.00 MiB 1 0 1780000.00 samples/s 0.00 ms 2.30 mW 1231.67 pJ/it
171 split/negative_axis arch-a PASS PASS 0.053 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.29 mW 84786.00 pJ 0.039 s 0.00 MiB 0.00 MiB 1 0 925000.00 samples/s 0.00 ms 2.29 mW 2413.67 pJ/it
172 split/uneven_channel_axis_4d arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.30 mW 18948.00 pJ 0.046 s 0.00 MiB 0.00 MiB 1 0 4170000.00 samples/s 0.00 ms 2.30 mW 491.67 pJ/it
173 sub/after_gemm arch-a PASS PASS 0.056 s 0.01 MiB 0.01 MiB 5 4 0.01 ms 104.70 mW 815012.96 pJ 0.045 s 0.01 MiB 0.01 MiB 6 4 218000.00 samples/s 0.00 ms 107.94 mW 570766.12 pJ/it
174 sub/basic arch-a PASS PASS 0.046 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.037 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
175 sub/broadcast_row arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.042 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
176 sub/channel_broadcast_1024 arch-a PASS PASS 0.052 s 0.02 MiB 0.01 MiB 1 0 0.01 ms 78.12 mW 540030.00 pJ 0.037 s 0.02 MiB 0.01 MiB 1 0 145000.00 samples/s 0.01 ms 2.11 mW 13388.67 pJ/it
177 sub/constant_lhs_broadcast arch-a PASS PASS 0.049 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25188.00 pJ 0.034 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 656.67 pJ/it
178 sub/leading_dimension_broadcast arch-a PASS PASS 0.050 s 0.00 MiB 0.00 MiB 1 0 0.00 ms 78.22 mW 25266.00 pJ 0.038 s 0.00 MiB 0.00 MiB 1 0 3120000.00 samples/s 0.00 ms 2.23 mW 658.67 pJ/it
validation/raptor_validation/artifacts.py
+33
View File
@@ -0,0 +1,33 @@
from pathlib import Path
ARTIFACTS_DIRNAME = "artifacts"
def artifacts_dir(workspace_dir: str | Path) -> Path:
return Path(workspace_dir) / ARTIFACTS_DIRNAME
def runner_uses_library(runner_path: str | Path, library_path: str | Path) -> bool:
runner_path = Path(runner_path)
library_path = Path(library_path)
try:
return (
runner_path.is_file()
and library_path.is_file()
and str(library_path.resolve()).encode() in runner_path.read_bytes()
)
except OSError:
return False
def remove_lock_files(root: str | Path) -> int:
root = Path(root)
if not root.exists():
return 0
removed = 0
for path in root.rglob("*.lock"):
if path.is_file() or path.is_symlink():
path.unlink(missing_ok=True)
removed += 1
return removed
validation/raptor_validation/pimcomp_models.py
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
import argparse
from pathlib import Path
REPO = Path(__file__).resolve().parents[2]
SUITE = REPO / "validation/networks/pimcomp_models"
MODELS = {
"vgg8": SUITE / "vgg8/vgg8-mnist-reconstructed.onnx",
"resnet18": SUITE / "resnet18/resnet18-v1-7.onnx",
"resnet34": SUITE / "resnet34/resnet34-v1-7.onnx",
"googlenet": SUITE / "googlenet/googlenet-12-pimsim-nn.onnx",
"yolo11n": SUITE / "yolo11n/yolo11n-pimsim-nn.onnx",
}
FUNCTIONAL_MODELS = {
**MODELS,
"yolo11n": REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx",
}
MODEL_NAMES = tuple(MODELS)
DEFAULT_MODELS = MODEL_NAMES
ABLATION_DEFAULT_MODELS = ("vgg8", "resnet18", "resnet34", "googlenet")
def add_models_argument(
parser: argparse.ArgumentParser,
default: tuple[str, ...] = DEFAULT_MODELS,
) -> None:
help_text = "Models to run (default: " + ", ".join(default) + ")."
if "yolo11n" not in default:
help_text += " Select yolo11n explicitly when needed."
parser.add_argument(
"--models",
nargs="+",
choices=MODEL_NAMES,
default=list(default),
metavar="MODEL",
help=help_text,
)
validation/raptor_validation/raptor.py
+7 -7
View File
@@ -8,13 +8,13 @@ from .subprocess_utils import run_command_with_reporter
PIM_PASS_LABELS = (
("ONNXToSpatialPass", "ONNX to Spatial"),
("MergeComputeNodesPass", "Merge Compute Nodes"),
("SpatialToPimPass", "Spatial to PIM"),
("PimBufferizationPass", "Bufferize PIM"),
("HostConstantFoldingPass", "Fold Host Constants"),
("PimLocalMemoryPlanningPass", "Plan Local Memory"),
("VerificationPass", "Verify PIM"),
("EmitPimCodePass", "Emit PIM Code"),
("MergeComputeNodesPass", "Merge compute nodes"),
("SpatialToPimPass", "Spatial to Pim"),
("PimBufferizationPass", "Bufferize Pim"),
("HostConstantFoldingPass", "Fold host constants"),
("PimLocalMemoryPlanningPass", "Plan local memory"),
("VerificationPass", "Verify Pim"),
("EmitPimCodePass", "Emit Pim code"),
)
PIM_PASS_LABEL_BY_SUFFIX = dict(PIM_PASS_LABELS)
TIMING_LINE_RE = re.compile(r"^\s*([0-9]+\.[0-9]+)\s+\(\s*[0-9.]+%\)\s+(.+?)\s*$")
validation/raptor_validation/validate_one.py
+49 -39
View File
@@ -9,6 +9,7 @@ import numpy as np
from dataclasses import dataclass, field
from pathlib import Path
from colorama import Style, Fore
from .artifacts import artifacts_dir, runner_uses_library
from .gen_network_runner import gen_network_runner
from .onnx_utils import (
_ONNX_TO_NP,
@@ -26,13 +27,13 @@ from .subprocess_utils import run_command_with_reporter
STAGE_TITLES = (
"Compile ONNX",
"Build Runner",
"Generate Inputs",
"Run Reference",
"Compile PIM",
"Run Functional Simulation",
"Compare Outputs",
"Run Non-functional Simulation",
"Build runner",
"Generate inputs",
"Run reference",
"Compile Pim",
"Run functional simulation",
"Compare outputs",
"Run non-functional simulation",
)
STAGE_COLORS = {
STAGE_TITLES[0]: Fore.BLUE,
@@ -46,8 +47,7 @@ STAGE_COLORS = {
}
STAGE_COUNT = len(STAGE_TITLES)
GENERATED_DIR_NAMES = (
"inputs", "outputs", "pimcomp", "raptor", "runner", "simulation",
"throughput_validation",
"artifacts",
)
MODE_FULL = "full"
@@ -58,15 +58,15 @@ MODE_STAGE_TITLES = {
MODE_FULL: STAGE_TITLES,
MODE_COMPILE_ONLY: (
"Compile ONNX",
"Build Runner",
"Compile PIM",
"Build runner",
"Compile Pim",
),
MODE_RUN_ONLY: (
"Generate Inputs",
"Run Reference",
"Run Functional Simulation",
"Compare Outputs",
"Run Non-functional Simulation",
"Generate inputs",
"Run reference",
"Run functional simulation",
"Compare outputs",
"Run non-functional simulation",
),
}
@@ -75,7 +75,7 @@ PIMSIM_FAILED = "ERROR"
PIMSIM_UNSUPPORTED = "UNSUPPORTED"
PIMSIM_SKIPPED = "SKIP"
PIMSIM_NOT_RUN = "-"
PIMSIM_UNSUPPORTED_VSOFTMAX = "pimsim-nn does not support opcode vsoftmax"
PIMSIM_UNSUPPORTED_VSOFTMAX = "Pimsim does not support opcode vsoftmax"
class PimSimUnsupportedError(RuntimeError):
@@ -338,7 +338,7 @@ def run_pimsim_nn(
else ("throughput", "average_latency_ms", "average_power_mw", "average_energy_pj")
)
if any(name not in metrics for name in required):
raise RuntimeError(f"pimsim-nn output did not contain required {execution_mode} metrics")
raise RuntimeError(f"Pimsim output did not contain required {execution_mode} metrics")
return metrics
@@ -361,6 +361,11 @@ def clean_workspace_artifacts(workspace_dir, model_stem):
for suffix in (".onnx.mlir", ".so", ".tmp"):
remove_path(workspace_dir / f"{model_stem}{suffix}")
for path in workspace_dir.rglob("*.lock"):
if path.is_file() or path.is_symlink():
path.unlink(missing_ok=True)
removed_paths.append(path)
return removed_paths
@@ -407,32 +412,34 @@ def build_dump_ranges(config_path, outputs_descriptor):
def build_pim_simulator_command(
pim_dir, output_bin_path, dump_ranges, input_paths, mode="latency",
pim_dir, output_bin_path, dump_ranges, input_dir, batch_size, mode="latency",
batch_output_dir=None):
if mode not in ("latency", "throughput"):
raise ValueError(f"unknown simulator mode: {mode}")
if not input_paths:
if batch_size < 1:
raise ValueError("simulator requires at least one input")
if input_dir is None:
raise ValueError("simulator requires an input directory")
command = [
"cargo", "run", "--no-default-features", "--release", "--package", "pim-simulator", "--bin", "pim-simulator",
"--", "-f", str(pim_dir), "-o", str(output_bin_path), "-d", dump_ranges,
"--mode", mode, "--batch-size", str(len(input_paths)),
"--mode", mode, "--batch-size", str(batch_size),
"--input-dir", str(input_dir),
]
if batch_output_dir is not None:
command += ["--batch-output-dir", str(batch_output_dir)]
for path in input_paths:
command += ["--input", str(path)]
return command
def run_pim_simulator(
simulator_dir, pim_dir, output_bin_path, dump_ranges, reporter=None,
timeout_sec=None, input_paths=(), mode="latency", batch_output_dir=None):
timeout_sec=None, input_dir=None, batch_size=1, mode="latency", batch_output_dir=None):
command = build_pim_simulator_command(
pim_dir,
output_bin_path,
dump_ranges,
input_paths,
input_dir,
batch_size,
mode=mode,
batch_output_dir=batch_output_dir,
)
@@ -516,7 +523,7 @@ def validate_execution(
try:
print_stage(
reporter, model_index, model_total, model_name,
"Run Functional Simulation", name,
"Run functional simulation", name,
)
write_inputs_to_memory_bin(
pim_dir / "memory.bin", pim_dir / "config.json", input_batch[0])
@@ -526,13 +533,13 @@ def validate_execution(
run_pim_simulator(
simulator_dir, pim_dir, simulation_dir / "out.bin", dump_ranges,
reporter=reporter, timeout_sec=command_timeout_seconds,
input_paths=input_paths[:batch_size], mode=name,
input_dir=input_paths[0].parent, batch_size=batch_size, mode=name,
batch_output_dir=output_dir)
reporter.advance()
print_stage(
reporter, model_index, model_total, model_name,
"Compare Outputs", name,
"Compare outputs", name,
)
reporter.suspend()
try:
@@ -553,7 +560,7 @@ def validate_execution(
print_stage(
reporter, model_index, model_total, model_name,
"Run Non-functional Simulation", name,
"Run non-functional simulation", name,
)
config_path = execution["pimsim_config"]
if state["compiled"] and pimsim_nn_build_dir is not None and config_path is not None:
@@ -580,7 +587,7 @@ def validate_execution(
elif not state["compiled"]:
state["pimsim_status"] = PIMSIM_NOT_RUN
else:
print_info(reporter, "pimsim-nn non-functional simulation skipped")
print_info(reporter, "Pimsim non-functional simulation skipped")
reporter.advance()
@@ -609,12 +616,12 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
owns_reporter = reporter is None
reporter = reporter or ProgressReporter(model_total, stages_per_model=len(MODE_STAGE_TITLES[mode]), verbose=verbose)
workspace_dir = network_onnx_path.parent
workspace_dir = artifacts_dir(network_onnx_path.parent)
raptor_dir = workspace_dir / "raptor"
runner_dir = workspace_dir / "runner"
runner_build_dir = runner_dir / "build"
if mode != MODE_RUN_ONLY:
clean_workspace_artifacts(workspace_dir, network_onnx_path.stem)
clean_workspace_artifacts(network_onnx_path.parent, network_onnx_path.stem)
Path.mkdir(raptor_dir, parents=True, exist_ok=True)
Path.mkdir(runner_build_dir, parents=True, exist_ok=True)
@@ -669,7 +676,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
print_info(reporter, f"Shared library saved to {network_so_path}")
reporter.advance()
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Build Runner")
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Build runner")
gen_network_runner(
network_onnx_path, network_so_path, onnx_include_dir,
entry="run_main_graph", out=runner_dir / "runner.c", verbose=False)
@@ -683,7 +690,10 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
report_validation_failure(reporter, "reference", "compilation", exc)
else:
required_paths = (network_so_path, network_mlir_path, runner_path)
reference_ready = all(path.exists() for path in required_paths)
reference_ready = (
all(path.exists() for path in required_paths)
and runner_uses_library(runner_path, network_so_path)
)
if not reference_ready:
report_validation_failure(reporter, "reference", "artifact lookup", FileNotFoundError(
"run-only mode requires the compiled shared library, ONNX MLIR, and runner"))
@@ -696,7 +706,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
states[name]["compiled"] = (pim_dir / "config.json").exists()
if not states[name]["compiled"]:
report_validation_failure(reporter, name, "artifact lookup", FileNotFoundError(
f"run-only mode requires compiled PIM artifacts at {pim_dir}"))
f"run-only mode requires compiled Pim artifacts at {pim_dir}"))
else:
states[name]["resource_metrics"] = collect_pim_resource_metrics(pim_dir)
if name == "latency":
@@ -705,7 +715,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
try:
print_stage(
reporter, model_index, model_total, network_onnx_path.name,
"Compile PIM", name,
"Compile Pim", name,
)
root.mkdir(parents=True, exist_ok=True)
started = time.perf_counter()
@@ -724,7 +734,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
states[name]["resource_metrics"] = collect_pim_resource_metrics(pim_dir)
if name == "latency":
resource_metrics = states[name]["resource_metrics"]
print_info(reporter, f"PIM artifacts saved to {pim_dir}")
print_info(reporter, f"Pim artifacts saved to {pim_dir}")
except Exception as exc:
report_validation_failure(reporter, name, "compilation", exc)
reporter.advance()
@@ -735,7 +745,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
else:
input_batch = input_paths = reference_dirs = outputs_descriptor = None
try:
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Generate Inputs")
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Generate inputs")
inputs_descriptor, outputs_descriptor = onnx_io(network_onnx_path)
first_inputs, _ = gen_random_inputs(inputs_descriptor, seed=seed)
input_batch = generate_input_batch(
@@ -754,7 +764,7 @@ def validate_network(network_onnx_path, raptor_path, onnx_include_dir,
if not reference_ready:
raise FileNotFoundError("reference runner is unavailable")
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Run Reference")
print_stage(reporter, model_index, model_total, network_onnx_path.name, "Run reference")
reference_dirs = []
for index, flags in enumerate(input_flags):
reference_dir = workspace_dir / "outputs" / f"{index:06d}"
validation/tools/classification_local_image_validation.py
+14 -11
View File
@@ -16,6 +16,7 @@ if str(VALIDATION_DIR) not in sys.path:
sys.path.insert(0, str(VALIDATION_DIR))
from raptor_validation.onnx_utils import _ONNX_TO_NP, onnx_io, write_inputs_binary, write_inputs_to_memory_bin
from raptor_validation.artifacts import artifacts_dir
from raptor_validation.validate_one import (
MODE_COMPILE_ONLY,
build_dump_ranges,
@@ -75,10 +76,11 @@ def ensure_local_artifacts(args, model_path: Path):
def ensure_existing_artifacts(model_dir: Path):
artifact_root = artifacts_dir(model_dir)
required_paths = [
model_dir / "runner" / "build" / "runner",
model_dir / "raptor" / "pim" / "config.json",
model_dir / "raptor" / "pim" / "memory.bin",
artifact_root / "runner" / "build" / "runner",
artifact_root / "raptor" / "pim" / "config.json",
artifact_root / "raptor" / "pim" / "memory.bin",
]
missing = [str(path) for path in required_paths if not path.exists()]
if missing:
@@ -185,13 +187,13 @@ def draw_classification_panel(image: Image.Image, results, output_path: Path):
def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray):
model_dir = model_path.parent
runner_build_dir = model_dir / "runner" / "build"
artifact_root = artifacts_dir(model_path.parent)
runner_build_dir = artifact_root / "runner" / "build"
runner_path = runner_build_dir / "runner"
pim_dir = model_dir / "raptor" / "pim"
simulation_dir = model_dir / "classification_demo" / "simulation"
reference_dir = model_dir / "classification_demo" / "reference"
inputs_dir = model_dir / "classification_demo" / "inputs"
pim_dir = artifact_root / "raptor" / "pim"
simulation_dir = artifact_root / "classification_demo" / "simulation"
reference_dir = artifact_root / "classification_demo" / "reference"
inputs_dir = artifact_root / "classification_demo" / "inputs"
simulation_dir.mkdir(parents=True, exist_ok=True)
reference_dir.mkdir(parents=True, exist_ok=True)
@@ -222,7 +224,7 @@ def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray):
subprocess.run(runner_cmd, cwd=runner_build_dir, check=True)
write_inputs_to_memory_bin(pim_dir / "memory.bin", pim_dir / "config.json", [tensor])
input_bin_path = simulation_dir / "input.bin"
input_bin_path = simulation_dir / "input_0.bin"
write_inputs_binary(input_bin_path, [tensor])
dump_ranges = build_dump_ranges(pim_dir / "config.json", output_descriptors)
output_bin_path = simulation_dir / "out.bin"
@@ -232,7 +234,8 @@ def run_reference_and_simulator(args, model_path: Path, tensor: np.ndarray):
output_bin_path,
dump_ranges,
timeout_sec=args.command_timeout_seconds,
input_paths=[input_bin_path],
input_dir=input_bin_path.parent,
batch_size=1,
)
output_index, output_name, output_dtype_code, output_shape = output_descriptors[0]
validation/tools/conv_lowering_tiles_drawio.py
+22 -22
View File
@@ -37,55 +37,55 @@ class Tile:
TILES = (
Tile("classic-reference", "REFERENCE", "Classic Conv + exact weight unfolding",
Tile("classic-reference", "Reference", "Classic Conv + exact weight unfolding",
"Original OIHW weights; the two implementations choose different K orders.",
"reference", "Y[p,o] = Σc,kh,kw Xpatch[p,c,kh,kw] · W[o,c,kh,kw]",
"At every output position, multiply the patch by one filter and add every product."),
Tile("pimcomp-element", "PIMCOMP", "Element pipeline",
Tile("pimcomp-element", "Pimcomp", "Element pipeline",
"One patch vector per input cycle; mapped weights stay fixed.",
"pimcomp_element", "patchPIM[1×K] · WflatPIM[K×O] → Yp[1×O]",
"Keep Wflat in the arrays; stream one patch each cycle to produce all O outputs."),
Tile("pimcomp-batch", "PIMCOMP", "Batch / replicated pipeline",
Tile("pimcomp-batch", "Pimcomp", "Batch / replicated pipeline",
"Complete Wflat copies divide patches or input samples.",
"pimcomp_batch", "for replica r: Yr = patchr[1×K] · WflatPIM[K×O]",
"Copy all weights R times and send different patches to the copies in parallel."),
Tile("raptor-legacy-im2col", "RAPTOR", "Legacy explicit im2col",
Tile("raptor-legacy-im2col", "Raptor", "Legacy explicit im2col",
"Every patch becomes one row of a global P×K matrix.",
"legacy", "Y[P×O] = im2col(X)[P×K] · WflatR[K×O]",
"Write every image patch as one matrix row, then multiply the two large matrices."),
Tile("raptor-packed-im2col", "RAPTOR", "Packed im2col",
Tile("raptor-packed-im2col", "Raptor", "Packed im2col",
"Pack q patch rows and repeat Wflat on a block diagonal.",
"packed", "packedY[1×qO] = [patch0|…|patchq−1] · diag(WflatR,…,WflatR)",
"Join q patches and use diagonal weight copies so one multiply computes q independent outputs."),
Tile("raptor-streamed-patch", "RAPTOR", "Streamed patch",
Tile("raptor-streamed-patch", "Raptor", "Streamed patch",
"Gather one patch into bounded scratch; avoid global im2col.",
"streamed_patch", "Yp[1×O] = scratchPatchp[1×K] · WflatR[K×O]",
"Gather one patch, multiply it, write its output, and reuse scratch for the next patch."),
Tile("raptor-streamed-packed", "RAPTOR", "Streamed packed",
Tile("raptor-streamed-packed", "Raptor", "Streamed packed",
"Gather q patch rows in bounded scratch, then block-diagonal pack them.",
"streamed_packed", "packedY = packedScratch[1×qK] · diag(WflatR×q)[qK×qO]",
"Gather q patches in small scratch, join them, multiply by diagonal weights, then unpack q outputs."),
Tile("raptor-depthwise", "RAPTOR", "Depthwise special case",
Tile("raptor-depthwise", "Raptor", "Depthwise special case",
"Each channel owns one row-major 3×3 kernel; channels never reduce together.",
"depthwise", "Y[p,c] = Σkh,kw Xpatch[p,c,kh,kw] · W[c,kh,kw]",
"For each channel separately, multiply its nine patch values by its nine weights and add."),
Tile("raptor-output-channel-tiled", "RAPTOR", "Output-channel tiled",
Tile("raptor-output-channel-tiled", "Raptor", "Output-channel tiled",
"Every O tile retains all channel-major K rows and selects output columns.",
"c_tiled", "Y[:,Oj] = patch[1×K] · Wflat[:,Oj][K×|Oj|]; concat j",
"Reuse the full patch for each output-filter group, then join the output groups."),
Tile("raptor-input-k-tiled", "RAPTOR", "Input-K tiled",
Tile("raptor-input-k-tiled", "Raptor", "Input-K tiled",
"Split matching K ranges; add their partial output vectors.",
"k_tiled", "Y[1×O] = Σi patch[Ki] · Wflat[Ki,:]",
"Multiply matching K slices independently, then add their partial output vectors."),
Tile("raptor-tiled-2d", "RAPTOR", "Two-dimensional tiled",
Tile("raptor-tiled-2d", "Raptor", "Two-dimensional tiled",
"Partition both K rows and output-filter columns.",
"tiled_2d", "Y[:,Oj] = Σi patch[Ki] · Wflat[Ki,Oj]; concat j",
"Split both directions: add results down K and join results across output groups."),
Tile("raptor-row-strip", "RAPTOR", "Pixel-major row-strip",
Tile("raptor-row-strip", "Raptor", "Pixel-major row-strip",
"A lane forms patches across one output row and slices K.",
"row_strip", "for x: Y[r,x,:] = Σi patch[r,x,Ki] · Wflat[Ki,:]",
"Move across one output row; at each x form a patch, multiply its K slices, and add."),
Tile("raptor-row-strip-c-tiled", "RAPTOR", "Row-strip + output tiling",
Tile("raptor-row-strip-c-tiled", "Raptor", "Row-strip + output tiling",
"Each row lane is duplicated across disjoint output-column tiles.",
"row_strip_c", "for x,j: Y[r,x,Oj] = patch[r,x,:] · Wflat[:,Oj]",
"Give each output-filter group a copy of the row lane, then join their output columns."),
@@ -473,7 +473,7 @@ def operation_scene(d: Drawio, parent: str, scene: str) -> None:
def reference_body(d: Drawio, parent: str) -> None:
add_box(d, parent, 24, 252, 712, 158, "", fill="#ffffff", stroke=PIMCOMP)
add_text(d, parent, 40, 260, 680, 22,
"PIMCOMP: spatial-major, row-wise positions; C interleaved", 11,
"Pimcomp: Spatial-major, row-wise positions; C interleaved", 11,
color=PIMCOMP, align="center", bold=True)
order_vector(d, parent, 306, "pimcomp", weight=False, label="Input patch")
order_vector(d, parent, 366, "pimcomp", weight=True, label="Matching W")
@@ -483,7 +483,7 @@ def reference_body(d: Drawio, parent: str) -> None:
add_box(d, parent, 24, 424, 712, 158, "", fill="#ffffff", stroke=RAPTOR)
add_text(d, parent, 40, 432, 680, 22,
"RAPTOR: channel-major; each 3×3 plane is row-major", 11,
"Raptor: channel-major; each 3×3 plane is row-major", 11,
color=RAPTOR, align="center", bold=True)
order_vector(d, parent, 478, "raptor", weight=False, label="Input patch")
order_vector(d, parent, 538, "raptor", weight=True, label="Matching W")
@@ -500,22 +500,22 @@ def reference_body(d: Drawio, parent: str) -> None:
def layout_reference(d: Drawio, parent: str, tile: Tile, accent: str) -> None:
if tile.scene == "depthwise":
layout = "independent row-major Kc=9 per channel"
elif tile.owner == "PIMCOMP":
layout = "PIMCOMP spatial-major K order"
elif tile.owner == "Pimcomp":
layout = "Pimcomp Spatial-major K order"
else:
layout = "RAPTOR channel-major K order"
layout = "Raptor channel-major K order"
add_box(d, parent, 24, 140, 712, 42, "", fill=PALE, stroke=accent)
add_text(d, parent, 38, 146, 684, 30,
f"LAYOUT → see REFERENCE tile: {layout}", 10,
f"Layout → see Reference tile: {layout}", 10,
color=accent, align="center", bold=True)
def algorithm_card(d: Drawio, parent: str, tile: Tile, accent: str) -> None:
add_box(d, parent, 24, 638, 712, 102, "", fill="#ffffff", stroke=accent)
add_text(d, parent, 40, 646, 90, 34, "ALGORITHM", 9,
add_text(d, parent, 40, 646, 90, 34, "Algorithm", 9,
color=accent, bold=True)
add_text(d, parent, 132, 644, 588, 38, tile.algorithm, 10)
add_text(d, parent, 40, 690, 90, 34, "MATH", 9,
add_text(d, parent, 40, 690, 90, 34, "Math", 9,
color=accent, bold=True)
add_text(d, parent, 132, 686, 588, 42, tile.formula, 10,
color=accent, bold=True)
@@ -524,7 +524,7 @@ def algorithm_card(d: Drawio, parent: str, tile: Tile, accent: str) -> None:
def render_tile(d: Drawio, tile: Tile, index: int) -> None:
col, row = index % COLS, index // COLS
parent = d.group(col * (TILE + GAP), row * (TILE + GAP), tile.slug)
accent = {"REFERENCE": REFERENCE, "PIMCOMP": PIMCOMP, "RAPTOR": RAPTOR}[tile.owner]
accent = {"Reference": REFERENCE, "Pimcomp": PIMCOMP, "Raptor": RAPTOR}[tile.owner]
add_box(d, parent, 0, 0, TILE, TILE, "", fill="#fbfcff", stroke=accent,
stroke_width=3)
add_box(d, parent, 24, 20, 106, 28, tile.owner, fill=accent, stroke=accent,
validation/tools/pim/ablation/README.md
+66
View File
@@ -0,0 +1,66 @@
# Raptor compiler ablation
`run_ablation.py` performs the complete synchronization/Spatial-planning
ablation study on the Pimcomp model suite. By default it runs `vgg8`,
`resnet18`, `resnet34`, and `googlenet` across `arch-a` and `arch-b`, latency,
and throughput pipeline 4. `arch-c` and `yolo11n` are run only when selected
explicitly.
Latency is pipeline 1; the wrapper invokes the suite runner separately for
latency and throughput pipeline 4.
```bash
.venv/bin/python validation/tools/pim/ablation/run_ablation.py \
--jobs 4
```
## Variants
| Variant | Raptor options |
|---|---|
| `baseline` | None. |
| `no-synchronization` | `--pim-disable-synchronization` |
| `no-spatial-planning` | `--pim-disable-spatial-planning` |
| `no-synchronization-no-spatial-planning` | Both options. |
Every variant runs Raptor only. Pimcomp is not compiled, validated, or
simulated. Reference inputs and outputs are generated once under the shared
common-artifact root and reused by every variant. Ctrl+C terminates the active
variant and all of its worker jobs.
The percentage baseline is `no-synchronization-no-spatial-planning`: both
ablation features are disabled, so its values are `+0.00%`. Every other
variant reports the signed percentage difference of its Raptor metrics from
that reference for the same model, architecture, mode, and pipeline. Positive
values mean the metric is higher; negative values mean it is lower.
## Options
| Option | Description and default |
|---|---|
| `-h`, `--help` | Show help and exit. |
| `--out-dir PATH` | Suite root. Default: `validation/networks/pimcomp_models`. Model artifacts are stored below `<out-dir>/<model>/artifacts`; disabled variants use `<arch>/<mode>[/pipelineN]/ablation/<variant>`. Variant summaries remain under `<out-dir>/ablation/<variant>/`. |
| `--models MODEL [...]` | Models to run. Default: `vgg8 resnet18 resnet34 googlenet`; include `yolo11n` explicitly when needed. |
| `--variant NAME` | Select a variant from the table above; repeat for multiple variants. Default: all variants. The feature-full baseline and percentage reference are added automatically when needed. |
| `--dry-run` | Print the suite-runner commands that would run without modifying files. Default: off. |
All options of
[`run_pimcomp_models.py`](../pimcomp/compare/README.md), including
`--archs`, `--mode`, `--pipeline`, `--no-fast`, and
`--raptor-extra-arg=ARG`, are forwarded to each variant. `--out-dir`,
`--variant`, and `--dry-run` belong to this wrapper; `--only` is reserved for
the suite runner's comparison mode and is rejected by this wrapper, which
always invokes `--raptor-only`. If neither `--mode` nor `--pipeline` is
forwarded, the wrapper uses its default latency and throughput/pipeline-4
case set. Supplying either option overrides that default and is passed through
as one suite-runner invocation per variant.
The feature-full `baseline` variant uses the normal `run_pimcomp_models.py`
artifact paths and is rerun with the same selected cases and forwarded options
as the disabled variants. The three disabled variants are stored below each model's
`artifacts/<arch>/<mode>[/pipelineN]/ablation/` directory. Shared reference
artifacts remain under each model's `artifacts/common` directory. Transient
per-variant comparison summaries are removed after aggregation. The combined
table is written to `<out-dir>/results_ablation.csv`; it contains only the variant, case
identifiers, and signed `latency_percent`, `throughput_percent`,
`power_percent`, and `energy_percent` columns. These are Raptor metrics;
Pimcomp metrics are omitted because the ablation invokes Raptor only.
validation/tools/pim/ablation/run_ablation.py
Executable
+317
View File
@@ -0,0 +1,317 @@
#!/usr/bin/env python3
"""Run the synchronization and Spatial-planning ablation matrix on Pimcomp models."""
from __future__ import annotations
import argparse
import csv
import math
import os
import shlex
import signal
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
REPO = Path(__file__).resolve().parents[4]
RUNNER = REPO / "validation/tools/pim/pimcomp/compare/run_pimcomp_models.py"
DEFAULT_OUT_DIR = REPO / "validation/networks/pimcomp_models"
sys.path.insert(0, str(REPO / "validation"))
from raptor_validation.pimcomp_models import (
ABLATION_DEFAULT_MODELS,
add_models_argument,
)
from raptor_validation.artifacts import remove_lock_files
@dataclass(frozen=True)
class Variant:
name: str
raptor_args: tuple[str, ...]
VARIANTS = (
Variant("baseline", ()),
Variant("no-synchronization", ("--pim-disable-synchronization",)),
Variant("no-spatial-planning", ("--pim-disable-spatial-planning",)),
Variant(
"no-synchronization-no-spatial-planning",
("--pim-disable-synchronization", "--pim-disable-spatial-planning"),
),
)
VARIANT_BY_NAME = {variant.name: variant for variant in VARIANTS}
REFERENCE_VARIANT = "no-synchronization-no-spatial-planning"
COMPARISON_RESULTS_FILENAME = "results_comparison.csv"
ABLATION_RESULTS_FILENAME = "results_ablation.csv"
CASE_FIELDS = ("arch", "model", "mode", "raptor_pipeline")
PERCENTAGE_FIELDS = (
("raptor_latency_ms", "latency_percent"),
("raptor_throughput_samples_s", "throughput_percent"),
("raptor_power_mw", "power_percent"),
("raptor_energy_pj", "energy_percent"),
)
RESULT_FIELDS = (*CASE_FIELDS, *(target for _, target in PERCENTAGE_FIELDS))
DEFAULT_CASE_ARGUMENTS = (
("latency", ("--mode", "latency")),
("throughput/pipeline4", ("--mode", "throughput", "--pipeline", "4")),
)
DEFAULT_CASE_KEYS = frozenset({("latency", "1"), ("throughput", "4")})
def parse_args(argv: list[str] | None = None) -> tuple[argparse.Namespace, list[str]]:
parser = argparse.ArgumentParser(
description="Run the complete compiler ablation matrix on the Pimcomp model suite.",
allow_abbrev=False,
)
parser.add_argument(
"--out-dir",
type=Path,
default=DEFAULT_OUT_DIR,
help="Artifact root (default: validation/networks/pimcomp_models).",
)
add_models_argument(parser, ABLATION_DEFAULT_MODELS)
parser.add_argument(
"--variant",
choices=tuple(VARIANT_BY_NAME),
action="append",
dest="variants",
help="Run only this variant; baseline is added when needed. Repeat as needed.",
)
parser.add_argument("--dry-run", action="store_true", help="Print runner commands without modifying files.")
args, forwarded = parser.parse_known_args(argv)
if any(option == "--only" or option.startswith("--only=") for option in forwarded):
parser.error("--only is not supported; the ablation wrapper always runs Raptor only")
return args, forwarded
def selected_variants(names: list[str] | None) -> list[Variant]:
requested = set(names or VARIANT_BY_NAME)
requested.add("baseline")
requested.add(REFERENCE_VARIANT)
return [variant for variant in VARIANTS if variant.name in requested]
def has_option(arguments: list[str], option: str) -> bool:
return any(argument == option or argument.startswith(f"{option}=") for argument in arguments)
def runner_argument_sets(forwarded: list[str]) -> tuple[tuple[str, list[str]], ...]:
if has_option(forwarded, "--mode") or has_option(forwarded, "--pipeline"):
return (("requested", forwarded),)
return tuple(
(label, [*forwarded, *case_arguments])
for label, case_arguments in DEFAULT_CASE_ARGUMENTS
)
def variant_output_dir(out_dir: Path, variant: Variant) -> Path:
return out_dir if variant.name == "baseline" else out_dir / "ablation" / variant.name
def remove_variant_summaries(out_dir: Path) -> None:
summary_root = out_dir / "ablation"
if not summary_root.is_dir():
return
for summary in summary_root.glob("*/results_comparison.csv"):
summary.unlink()
for variant_dir in summary_root.iterdir():
if variant_dir.is_dir() and not any(variant_dir.iterdir()):
variant_dir.rmdir()
if not any(summary_root.iterdir()):
summary_root.rmdir()
def runner_command(
out_dir: Path,
variant: Variant,
models: list[str],
forwarded: list[str],
common_root: Path,
dry_run: bool,
) -> list[str]:
command = [
sys.executable,
str(RUNNER),
"--out-dir",
str(out_dir),
"--raptor-only",
"--models",
*models,
]
if variant.name != "baseline":
command.extend(("--ablation-variant", variant.name))
if not has_option(forwarded, "--common-dir"):
command.extend(("--common-dir", str(common_root)))
command.extend(forwarded)
command.extend(f"--raptor-extra-arg={arg}" for arg in variant.raptor_args)
if dry_run:
command.append("--dry-run")
return command
def terminate_process_group(process: subprocess.Popen[bytes]) -> None:
if process.poll() is not None:
return
try:
os.killpg(process.pid, signal.SIGTERM)
except ProcessLookupError:
return
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
try:
os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError:
return
process.wait()
def percentage_difference(value: str | None, reference: str | None) -> str:
try:
current = float(value) if value is not None else math.nan
baseline = float(reference) if reference is not None else math.nan
except ValueError:
return "NA"
if not math.isfinite(current) or not math.isfinite(baseline) or baseline == 0:
return "NA"
return f"{(current - baseline) / baseline * 100:+.2f}%"
def aggregate_results(
out_dir: Path,
variants: list[Variant],
selected_cases: frozenset[tuple[str, str]] | None = None,
selected_models: frozenset[str] | None = None,
) -> tuple[Path | None, list[str]]:
fields: list[str] | None = None
rows_by_variant: dict[str, list[dict[str, str]]] = {}
failures = []
for variant in variants:
results_path = variant_output_dir(out_dir, variant) / COMPARISON_RESULTS_FILENAME
if not results_path.is_file():
failures.append(f"{variant.name}: missing {results_path}")
continue
with results_path.open(newline="", encoding="utf-8") as stream:
reader = csv.DictReader(stream)
if reader.fieldnames is None:
failures.append(f"{variant.name}: empty {results_path}")
continue
if fields is None:
fields = reader.fieldnames
elif reader.fieldnames != fields:
failures.append(f"{variant.name}: inconsistent columns in {results_path}")
continue
selected_rows = rows_by_variant.setdefault(variant.name, [])
for row in reader:
if selected_models is not None and row.get("model") not in selected_models:
continue
if selected_cases is not None and (
row.get("mode"), row.get("raptor_pipeline")
) not in selected_cases:
continue
selected_rows.append(row)
if fields is None:
return None, failures
reference_rows = {
tuple(row.get(field, "") for field in CASE_FIELDS): row
for row in rows_by_variant.get(REFERENCE_VARIANT, [])
}
if REFERENCE_VARIANT not in rows_by_variant:
failures.append(f"{REFERENCE_VARIANT}: missing percentage reference results")
output = out_dir / ABLATION_RESULTS_FILENAME
with output.open("w", newline="", encoding="utf-8") as stream:
writer = csv.DictWriter(stream, fieldnames=("variant", *RESULT_FIELDS), lineterminator="\n")
writer.writeheader()
for variant in variants:
for row in rows_by_variant.get(variant.name, []):
reference = reference_rows.get(tuple(row.get(field, "") for field in CASE_FIELDS))
writer.writerow(
{
"variant": variant.name,
**{field: row.get(field, "") for field in CASE_FIELDS},
**{
target: percentage_difference(
row.get(source), reference.get(source) if reference else None
)
for source, target in PERCENTAGE_FIELDS
},
}
)
return output, failures
def main(argv: list[str] | None = None) -> int:
args, forwarded = parse_args(argv)
if not RUNNER.is_file():
print(f"Missing Pimcomp runner: {RUNNER}", file=sys.stderr)
return 1
variants = selected_variants(args.variants)
out_dir = args.out_dir.resolve()
case_sets = runner_argument_sets(forwarded)
if args.dry_run:
for variant in variants:
for _, case_forwarded in case_sets:
print(
shlex.join(
runner_command(
out_dir,
variant,
args.models,
case_forwarded,
out_dir,
True,
)
)
)
return 0
out_dir.mkdir(parents=True, exist_ok=True)
failed = []
current_process = None
try:
for variant in variants:
for case_label, case_forwarded in case_sets:
command = runner_command(
out_dir, variant, args.models, case_forwarded, out_dir, False
)
print(f"[{variant.name}/{case_label}] {shlex.join(command)}")
current_process = subprocess.Popen(
command,
cwd=REPO,
start_new_session=True,
)
returncode = current_process.wait()
current_process = None
if returncode:
failed.append(f"{variant.name}/{case_label}: runner exited with {returncode}")
except KeyboardInterrupt:
if current_process is not None:
terminate_process_group(current_process)
remove_lock_files(out_dir)
remove_variant_summaries(out_dir)
print("Interrupted; terminated the active ablation job.", file=sys.stderr)
return 130
remove_lock_files(out_dir)
selected_cases = DEFAULT_CASE_KEYS if len(case_sets) > 1 else None
output, aggregation_failures = aggregate_results(
out_dir,
variants,
selected_cases,
frozenset(args.models),
)
remove_variant_summaries(out_dir)
failed.extend(aggregation_failures)
if output is not None:
print(f"Ablation results: {output}")
if failed:
print("Failed: " + "; ".join(failed), file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
validation/tools/pim/pimcomp/compare/README.md
+62
View File
@@ -0,0 +1,62 @@
# Pimcomp model comparison
These scripts compare Raptor with Pimcomp on the models in
`validation/networks/pimcomp_models`.
`run_pimcomp_models.py` runs the supported model suite across the configured
architectures and simulation modes. `compare_raptor_pimcomp_model.py` is the
lower-level one-model comparison used by the suite runner.
## Suite runner
Run all five models, Arch-A and Arch-B, latency, and throughput:
```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
```
### Options
| Option | Description and default |
|---|---|
| `-h`, `--help` | Show help and exit. |
| `--out-dir PATH` | Suite root; artifacts are placed below `<out-dir>/<model>/artifacts/`. Default: `validation/networks/pimcomp_models`. |
| `--common-dir PATH` | Shared root for per-model reference artifacts. Default: `<out-dir>/<model>/artifacts/common`. |
| `--models MODEL [...]` | Models to run: `vgg8`, `resnet18`, `resnet34`, `googlenet`, `yolo11n`. Default: all five; pass a subset to select specific models. |
| `--archs ARCH [...]` | Pimcomp hardware profiles to run. Default: `arch-a arch-b`. |
| `--mode {latency,throughput} [...]` | Simulation modes. Default: both. |
| `--only {raptor,pimcomp}` | Re-run only one compiler side and reuse the other side's existing report. Default: run both sides. |
| `--raptor-only` | Run Raptor without compiling, validating, or simulating Pimcomp. Default: off. |
| `--pipeline {1,2,4,8}` | Select one Raptor pipeline. Default: latency pipeline 1 and throughput pipelines 2, 4, and 8. |
| `--pimsim-time-ms INT` | Throughput convergence deadline. Default: `1000`. |
| `--batch-size INT` | Functional throughput batch size. Default: `128`. |
| `--timeout-seconds FLOAT` | Per-stage timeout; `0` means unlimited. Default: `0`. |
| `-j INT`, `--jobs INT` | Parallel comparison workers. Default: `4`. |
| `--clean` | Remove generated comparison artifacts and summaries, then exit. Default: off. |
| `--ablation-variant NAME` | Put generated artifacts below `<mode>[/pipelineN]/ablation/NAME` and write the summary below `ablation/NAME`. Default: none. |
| `--dry-run` | Print commands without modifying files. Default: off. |
| `--no-fast` | Disable fast throughput convergence. Default: off. |
| `--raptor-extra-arg=ARG` | Extra Raptor compiler argument; repeat for multiple arguments. Default: none. |
Arguments beginning with `--` must use the equals form when passed through:
```bash
.venv/bin/python validation/tools/pim/pimcomp/compare/run_pimcomp_models.py \
--models vgg8 \
--raptor-extra-arg=--pim-disable-synchronization
```
The runner writes `results_comparison.csv` under the selected result root. It reuses
shared model inputs, outputs, and reference runners, and reuses Pimcomp
artifacts between pipelines in the same model/architecture/mode group. With
`--raptor-only`, only Raptor results are generated and Pimcomp is not invoked.
All generated files are kept below each model's `artifacts/` directory;
`--clean` also removes any `.lock` files left by an interrupted reference
generation.
## One-model comparator
`compare_raptor_pimcomp_model.py` compares one ONNX model. Its required
arguments are `--model PATH` and `--out-dir PATH`; use `--help` for the full
lower-level interface. The suite runner supplies the model, hardware profile,
simulation mode, and reuse paths automatically.
validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp.py → validation/tools/pim/pimcomp/compare/compare_raptor_pimcomp_model.py
+243 -150
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import argparse
import fcntl
import gzip
import importlib.util
import json
@@ -30,8 +31,8 @@ PIMSIM_CONFIG_DIR = VALIDATION_DIR / "pimsim_configs/pimcomp"
PIMCOMP_OUTPUT_FILES = ("SimulationInfo.gz", "VerificationInfo.json", "MappingResult.txt")
sys.path.insert(0, str(VALIDATION_DIR))
from raptor_validation.gen_network_runner import gen_network_runner # noqa: E402
from raptor_validation.onnx_utils import ( # noqa: E402
from raptor_validation.gen_network_runner import gen_network_runner
from raptor_validation.onnx_utils import (
_ONNX_TO_NP,
generate_input_batch,
gen_random_inputs,
@@ -41,12 +42,13 @@ from raptor_validation.onnx_utils import ( # noqa: E402
write_input_batch_csv,
write_inputs_to_memory_bin,
)
from raptor_validation.raptor import compile_with_raptor # noqa: E402
from raptor_validation.pimsim_nn import ( # noqa: E402
from raptor_validation.raptor import compile_with_raptor
from raptor_validation.pimsim_nn import (
export_raptor_pimsim_artifact,
parse_pimsim_nn_metrics,
)
from raptor_validation.validate_one import ( # noqa: E402
from raptor_validation.artifacts import runner_uses_library
from raptor_validation.validate_one import (
STAGE_COLORS,
build_pim_simulator_command,
build_dump_ranges,
@@ -99,7 +101,7 @@ def print_step(
stage: str | None = None,
):
color = STAGE_COLORS.get(stage or name, Fore.WHITE)
print("\n" + Style.BRIGHT + color + f"[{name}]" + Style.RESET_ALL)
print(Style.BRIGHT + color + f"[{name}]" + Style.RESET_ALL)
if cmd is not None:
print(f" cwd: {cwd or REPO}")
print(f" $ {shell_join(cmd)}")
@@ -134,7 +136,7 @@ def exception_message(exc: BaseException) -> str:
def print_failure(name: str, exc: BaseException | str) -> None:
message = exc if isinstance(exc, str) else exception_message(exc)
print(
"\n" + Style.BRIGHT + Fore.RED + f"[{name} FAILED]" + Style.RESET_ALL,
Style.BRIGHT + Fore.RED + f"[{name} FAILED]" + Style.RESET_ALL,
file=sys.stderr,
)
for line in message.splitlines()[:20]:
@@ -239,6 +241,17 @@ def reference_inputs_exist(
)
def reference_batch_dirs(root: Path, batch_size: int) -> list[Path]:
return [root / f"batch_{index:06d}" for index in range(batch_size)]
def reference_batch_outputs_exist(
outputs_desc: list[tuple[int, str, int, list[int]]],
reference_dirs: list[Path],
) -> bool:
return all(reference_outputs_exist(outputs_desc, reference_dir) for reference_dir in reference_dirs)
def compare_simulator_outputs(
output_bin: Path,
outputs_desc: list[tuple[int, str, int, list[int]]],
@@ -308,7 +321,7 @@ def select_pimsim_config(args: argparse.Namespace, hardware: dict[str, int]) ->
if args.pimsim_mode == "latency" or config["sim_config"]["sim_time"] == args.pimsim_time_ms:
return path
raise ValueError(
f"No pre-generated {args.pimsim_mode} pimsim-nn config for {args.pimsim_time_ms} ms matches {hardware}"
f"No pre-generated {args.pimsim_mode} Pimsim config for {args.pimsim_time_ms} ms matches {hardware}"
)
@@ -329,13 +342,14 @@ def compile_reference(
runner_dir = work_dir / "runner"
build_dir = runner_dir / "build"
raptor_dir.mkdir(parents=True, exist_ok=True)
shutil.rmtree(build_dir, ignore_errors=True)
build_dir.mkdir(parents=True, exist_ok=True)
stem = model_path.stem
onnx_ir_base = raptor_dir / stem
runner_base = runner_dir / stem
run_logged(
"Compile Reference ONNX IR",
"Compile reference ONNX IR",
[str(args.raptor_path), str(model_path), "-o", str(onnx_ir_base), "--EmitONNXIR",
"--mlir-elide-elementsattrs-if-larger=16", "--enable-conv-opt-pass=false"],
cwd=REPO,
@@ -344,7 +358,7 @@ def compile_reference(
stage="Compile ONNX",
)
run_logged(
"Compile Reference Native",
"Compile reference native",
[str(args.raptor_path), "-O3", str(model_path), "-o", str(runner_base)],
cwd=REPO,
timeout_sec=args.timeout_seconds,
@@ -353,7 +367,7 @@ def compile_reference(
)
network_so = runner_base.with_suffix(".so")
print_step("Generate Runner Source", stage="Build Runner")
print_step("Generate runner source", stage="Build runner")
gen_network_runner(
model_path,
network_so,
@@ -364,15 +378,15 @@ def compile_reference(
)
run_logged(
"Configure Runner",
"Configure runner",
["cmake", str(runner_dir), "-DCMAKE_BUILD_TYPE=Release", "-DCMAKE_C_FLAGS_RELEASE=-O3"],
cwd=build_dir,
timeout_sec=args.timeout_seconds,
steps=steps,
stage="Build Runner",
stage="Build runner",
)
run_logged(
"Build Runner",
"Build runner",
["cmake", "--build", ".", "-j"],
cwd=build_dir,
timeout_sec=args.timeout_seconds,
@@ -382,6 +396,20 @@ def compile_reference(
return build_dir / "runner"
def ensure_reference_runner(
args: argparse.Namespace,
model_path: Path,
work_dir: Path,
steps: list[StepRecord],
) -> Path:
runner_dir = work_dir / "runner"
runner_path = runner_dir / "build/runner"
library_path = runner_dir / f"{model_path.stem}.so"
if runner_uses_library(runner_path, library_path):
return runner_path
return compile_reference(args, model_path, work_dir, steps)
def generate_reference_outputs(
runner_path: Path,
runner_build_dir: Path,
@@ -390,6 +418,8 @@ def generate_reference_outputs(
steps: list[StepRecord],
args: argparse.Namespace,
out_dir: Path,
*,
print_header: bool = True,
) -> Path:
inputs_dir = out_dir / "inputs"
reference_dir = out_dir / "outputs"
@@ -397,11 +427,12 @@ def generate_reference_outputs(
reference_dir.mkdir(parents=True, exist_ok=True)
flags, _ = save_inputs_to_files(model_path, arrays_in_order, inputs_dir)
run_logged(
"Run Reference",
"Run reference",
[str(runner_path), *flags, "--save-csv-dir", str(reference_dir)],
cwd=runner_build_dir,
timeout_sec=args.timeout_seconds,
steps=steps,
print_header=print_header,
)
return reference_dir
@@ -414,19 +445,56 @@ def generate_reference_batch_outputs(
steps: list[StepRecord],
args: argparse.Namespace,
out_dir: Path,
*,
print_header: bool = True,
) -> list[Path]:
return [
generate_reference_outputs(
if print_header:
print_step("Run reference")
references = []
for index, sample in enumerate(input_batch):
references.append(
generate_reference_outputs(
runner_path,
runner_build_dir,
model_path,
sample,
steps,
args,
out_dir / f"batch_{index:06d}",
print_header=False,
)
)
return references
def prepare_reference_batch_outputs(
runner_path: Path,
runner_build_dir: Path,
model_path: Path,
input_batch: list[list[np.ndarray]],
outputs_desc: list[tuple[int, str, int, list[int]]],
steps: list[StepRecord],
args: argparse.Namespace,
out_dir: Path,
*,
print_header: bool = True,
) -> list[Path]:
reference_dirs = reference_batch_dirs(out_dir, len(input_batch))
lock_path = out_dir.parent / ".reference.lock"
with lock_path.open("w", encoding="utf-8") as lock:
fcntl.flock(lock.fileno(), fcntl.LOCK_EX)
if reference_batch_outputs_exist(outputs_desc, reference_dirs):
return reference_dirs
return generate_reference_batch_outputs(
runner_path,
runner_build_dir,
model_path,
sample,
input_batch,
steps,
args,
out_dir / f"batch_{index:06d}",
out_dir,
print_header=print_header,
)
for index, sample in enumerate(input_batch)
]
def prepare_common_artifacts(
@@ -437,10 +505,8 @@ def prepare_common_artifacts(
inputs_desc, outputs_desc, arrays_in_order = load_model_inputs(model_path, args.seed)
inputs_dir = common_dir / "inputs"
outputs_dir = common_dir / "outputs"
runner_path = common_dir / "runner/build/runner"
steps: list[StepRecord] = []
if not runner_path.exists():
runner_path = compile_reference(args, model_path, common_dir, steps)
runner_path = ensure_reference_runner(args, model_path, common_dir, steps)
inputs_ready = reference_inputs_exist(inputs_desc, inputs_dir)
outputs_ready = reference_outputs_exist(outputs_desc, outputs_dir)
@@ -478,7 +544,7 @@ def compile_raptor_target(
"--pim-emit-json",
*args.raptor_extra_arg,
]
print_step("Compile Raptor", cmd, REPO, "Compile PIM")
print_step("Compile Raptor", cmd, REPO, "Compile Pim")
start = time.perf_counter()
command = shell_join(cmd)
raptor_extra_args = [
@@ -503,7 +569,7 @@ def compile_raptor_target(
except Exception as exc:
steps.append(
StepRecord(
name="Compile Raptor PIM",
name="Compile Raptor Pim",
duration_sec=time.perf_counter() - start,
command=command,
status="failed",
@@ -514,7 +580,7 @@ def compile_raptor_target(
steps.append(
StepRecord(
name="Compile Raptor PIM",
name="Compile Raptor Pim",
duration_sec=time.perf_counter() - start,
command=command,
)
@@ -548,7 +614,8 @@ def run_functional_validation(
pim_dir,
output_bin,
dump_ranges,
input_bins,
input_bins[0].parent,
batch_size,
args.pimsim_mode,
batch_output_dir,
)
@@ -559,7 +626,7 @@ def run_functional_validation(
cwd=args.pim_simulator_dir,
timeout_sec=args.timeout_seconds,
steps=steps,
stage="Run Functional Simulation",
stage="Run functional simulation",
)
max_diffs: dict[str, float] = {}
failed_iterations = []
@@ -610,9 +677,9 @@ def compile_pimcomp(
(pimcomp_output_dir / name).unlink(missing_ok=True)
model_name = args.pimcomp_model_name or f"compare_{model_path.stem}"
frontend_json = frontend_json_dir / f"{model_name}.json"
print_step("Compile PIMCOMP", stage="Compile PIM")
# The original PIMCOMP frontend rewrites its input ONNX while loading it.
# Isolate that mutation without changing the model sent through PIMCOMP.
print_step("Compile Pimcomp", stage="Compile Pim")
# The original Pimcomp frontend rewrites its input ONNX while loading it.
# Isolate that mutation without changing the model sent through Pimcomp.
with TemporaryDirectory(prefix="pimcomp-model-") as temp_dir:
frontend_model = Path(temp_dir) / model_path.name
shutil.copy2(model_path, frontend_model)
@@ -625,12 +692,12 @@ def compile_pimcomp(
str(frontend_json),
]
run_logged(
"Compile PIMCOMP Frontend",
"Compile Pimcomp frontend",
frontend_cmd,
cwd=args.pimcomp_dir / "frontend",
timeout_sec=args.timeout_seconds,
steps=steps,
stage="Compile PIM",
stage="Compile Pim",
print_header=False,
)
backend_cmd = [
@@ -643,12 +710,12 @@ def compile_pimcomp(
]
try:
run_logged(
"Compile PIMCOMP Backend",
"Compile Pimcomp backend",
backend_cmd,
cwd=frontend_json_dir.parent,
timeout_sec=args.timeout_seconds,
steps=steps,
stage="Compile PIM",
stage="Compile Pim",
print_header=False,
)
finally:
@@ -666,7 +733,7 @@ def export_pimcomp_for_pimsim_nn(simulation_info: Path, output_dir: Path) -> Pat
sim_config = sim_info["config"]
core_count = int(sim_config["core_cnt"])
if core_count <= 0:
raise ValueError("PIMCOMP SimulationInfo.gz must configure at least one core")
raise ValueError("Pimcomp SimulationInfo.gz must configure at least one core")
core_indices = range(core_count)
config = {
@@ -709,7 +776,7 @@ def export_pimcomp_for_rust(
output_dir: Path,
) -> Path:
if len(runtime_inputs) != 1:
raise ValueError("PIMCOMP export currently requires exactly one runtime input tensor")
raise ValueError("Pimcomp export currently requires exactly one runtime input tensor")
if output_dir.exists():
shutil.rmtree(output_dir)
exporter = load_pimcomp_exporter()
@@ -901,7 +968,7 @@ def export_pimcomp_for_rust(
instructions.append(translated)
out_offset += width
else:
raise RuntimeError(f"Unsupported PIMCOMP op {op}")
raise RuntimeError(f"Unsupported Pimcomp op {op}")
with open(output_dir / f"core_{core_idx}.json", "w", encoding="utf-8") as f:
json.dump(instructions, f, separators=(",", ":"))
@@ -935,7 +1002,7 @@ def run_pimsim_nn(
cwd=args.pimsim_nn_build_dir,
timeout_sec=args.timeout_seconds * 10.0,
steps=steps,
stage="Run Non-functional Simulation",
stage="Run non-functional simulation",
)
return parse_pimsim_nn_metrics(output)
@@ -1091,12 +1158,9 @@ def restore_side_records(
def default_common_dir(out_dir: Path) -> Path:
if out_dir.name == "latency":
return out_dir.parents[1] / "common"
if out_dir.parent.name == "throughput":
return out_dir.parents[2] / "common"
if out_dir.name.startswith("arch-"):
return out_dir.parent / "common"
for parent in (out_dir, *out_dir.parents):
if parent.name == "artifacts":
return parent / "common"
return out_dir / "common"
@@ -1155,19 +1219,19 @@ def write_report(
):
report_path.parent.mkdir(parents=True, exist_ok=True)
lines = [
"# Raptor vs PIMCOMP Comparison Report",
"# Raptor vs Pimcomp comparison report",
"",
f"- Model: `{model_path}`",
f"- Hardware: `{hardware.get('core_count', 'n/a')} cores`, `{hardware.get('crossbar_count', 'n/a')} xbars/core`, `{hardware.get('crossbar_size', 'n/a')}x{hardware.get('crossbar_size', 'n/a')}` crossbars, mesh `{hardware.get('mesh_rows', 'n/a')}x{hardware.get('mesh_cols', 'n/a')}`",
f"- PIMCOMP pipeline: `{pimcomp_pipeline}`",
f"- PIMCOMP replication: `{pimcomp_replication}`",
f"- Pimcomp pipeline: `{pimcomp_pipeline}`",
f"- Pimcomp replication: `{pimcomp_replication}`",
"",
]
if failures or any(step.status != "passed" for step in steps):
lines.extend(
[
"## Failures / Skipped Work",
"## Failures / skipped work",
"",
"The script did not abort. The failed stage was recorded and any dependent stage was skipped when its inputs were not available.",
"",
@@ -1182,21 +1246,21 @@ def write_report(
lines.extend(
[
"## Functional Validation",
"## Functional validation",
"",
f"- Raptor via `pim-simulator`: `{validation_status(raptor_validation)}`",
f"- PIMCOMP via exported `pim-simulator`: `{validation_status(pimcomp_validation)}`",
f"- Pimcomp via exported `pim-simulator`: `{validation_status(pimcomp_validation)}`",
]
)
if raptor_validation.error:
lines.append(f"- Raptor validation note: `{raptor_validation.error.splitlines()[0]}`")
if pimcomp_validation.error:
lines.append(f"- PIMCOMP validation note: `{pimcomp_validation.error.splitlines()[0]}`")
lines.append(f"- Pimcomp validation note: `{pimcomp_validation.error.splitlines()[0]}`")
lines.extend(["", "### Max Output Differences", ""])
lines.extend(["", "### Max output differences", ""])
diff_names = sorted(set(raptor_validation.max_diffs) | set(pimcomp_validation.max_diffs))
if diff_names:
lines.extend(["| Output | Raptor max diff | PIMCOMP max diff |", "|---|---:|---:|"])
lines.extend(["| Output | Raptor max diff | Pimcomp max diff |", "|---|---:|---:|"])
for name in diff_names:
lines.append(
f"| `{name}` | {raptor_validation.max_diffs.get(name, float('nan')):.6e} | "
@@ -1208,7 +1272,7 @@ def write_report(
lines.extend(
[
"",
"## pimsim-nn Performance",
"## Pimsim performance",
"",
f"- Mode: `{pimsim_mode}`",
"",
@@ -1221,7 +1285,7 @@ def write_report(
"|---|---|---:|---:|---:|---:|---:|",
f"| Raptor | {perf_status(raptor_perf)} | {perf_value(raptor_perf, 'average_latency_ms', 'ms')} | {perf_value(raptor_perf, 'throughput', 'samples/s')} | "
f"{perf_value(raptor_perf, 'average_power_mw', 'mW')} | {perf_value(raptor_perf, 'average_energy_pj', 'pJ/it')} | {perf_value(raptor_perf, 'output_count')} |",
f"| PIMCOMP | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'average_latency_ms', 'ms')} | {perf_value(pimcomp_perf, 'throughput', 'samples/s')} | "
f"| Pimcomp | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'average_latency_ms', 'ms')} | {perf_value(pimcomp_perf, 'throughput', 'samples/s')} | "
f"{perf_value(pimcomp_perf, 'average_power_mw', 'mW')} | {perf_value(pimcomp_perf, 'average_energy_pj', 'pJ/it')} | {perf_value(pimcomp_perf, 'output_count')} |",
"",
]
@@ -1233,40 +1297,40 @@ def write_report(
"|---|---|---:|---:|---:|",
f"| Raptor | {perf_status(raptor_perf)} | {perf_value(raptor_perf, 'latency_ms', 'ms')} | "
f"{perf_value(raptor_perf, 'average_power_mw', 'mW')} | {perf_value(raptor_perf, 'average_energy_pj', 'pJ')} |",
f"| PIMCOMP | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'latency_ms', 'ms')} | "
f"| Pimcomp | {perf_status(pimcomp_perf)} | {perf_value(pimcomp_perf, 'latency_ms', 'ms')} | "
f"{perf_value(pimcomp_perf, 'average_power_mw', 'mW')} | {perf_value(pimcomp_perf, 'average_energy_pj', 'pJ')} |",
"",
]
)
if raptor_perf.get("reason") or raptor_perf.get("error"):
lines.append(f"- Raptor pimsim-nn note: `{(raptor_perf.get('reason') or raptor_perf.get('error')).splitlines()[0]}`")
lines.append(f"- Raptor Pimsim note: `{(raptor_perf.get('reason') or raptor_perf.get('error')).splitlines()[0]}`")
if pimcomp_perf.get("reason") or pimcomp_perf.get("error"):
lines.append(f"- PIMCOMP pimsim-nn note: `{(pimcomp_perf.get('reason') or pimcomp_perf.get('error')).splitlines()[0]}`")
lines.append(f"- Pimcomp Pimsim note: `{(pimcomp_perf.get('reason') or pimcomp_perf.get('error')).splitlines()[0]}`")
if lines[-1] != "":
lines.append("")
lines.extend(
[
"## Instruction Summary",
"## Instruction summary",
"",
"| Compiler | Status | Active cores | Total instructions | Sends | Receives | MVMUL |",
"|---|---|---:|---:|---:|---:|---:|",
f"| Raptor | {'FAILED' if raptor_instr.get('error') else 'SKIPPED' if raptor_instr.get('skipped') else 'DONE'} | {raptor_instr.get('active_cores', 0)} | {raptor_instr.get('total_instructions', 0)} | {raptor_instr.get('op_counts', {}).get('send', 0)} | {raptor_instr.get('op_counts', {}).get('recv', 0)} | {raptor_instr.get('op_counts', {}).get('mvmul', 0)} |",
f"| PIMCOMP | {'FAILED' if pimcomp_instr.get('error') else 'SKIPPED' if pimcomp_instr.get('skipped') else 'DONE'} | {pimcomp_instr.get('active_cores', 0)} | {pimcomp_instr.get('total_instructions', 0)} | {pimcomp_instr.get('op_counts', {}).get('send', 0)} | {pimcomp_instr.get('op_counts', {}).get('recv', 0)} | {pimcomp_instr.get('op_counts', {}).get('mvmul', 0)} |",
f"| Pimcomp | {'FAILED' if pimcomp_instr.get('error') else 'SKIPPED' if pimcomp_instr.get('skipped') else 'DONE'} | {pimcomp_instr.get('active_cores', 0)} | {pimcomp_instr.get('total_instructions', 0)} | {pimcomp_instr.get('op_counts', {}).get('send', 0)} | {pimcomp_instr.get('op_counts', {}).get('recv', 0)} | {pimcomp_instr.get('op_counts', {}).get('mvmul', 0)} |",
"",
"### Raptor Op Distribution",
"### Raptor op distribution",
"",
"| Op | Count | Share |",
"|---|---:|---:|",
*format_op_table(raptor_instr.get("op_counts", {}), raptor_instr.get("total_instructions", 0)),
"",
"### PIMCOMP Op Distribution",
"### Pimcomp op distribution",
"",
"| Op | Count | Share |",
"|---|---:|---:|",
*format_op_table(pimcomp_instr.get("op_counts", {}), pimcomp_instr.get("total_instructions", 0)),
"",
"## Step Timings",
"## Step timings",
"",
"| Step | Status | Duration (s) | Return code |",
"|---|---|---:|---:|",
@@ -1279,7 +1343,7 @@ def write_report(
)
failed_steps = [step for step in steps if step.status != "passed"]
if failed_steps:
lines.extend(["", "### Failed Step Details", ""])
lines.extend(["", "### Failed step details", ""])
for step in failed_steps:
lines.extend(
[
@@ -1294,7 +1358,7 @@ def write_report(
lines.append("")
if raptor_pass_timings:
lines.extend(["", "## Raptor Pass Timings", "", "| Pass | Duration (s) |", "|---|---:|"])
lines.extend(["", "## Raptor pass timings", "", "| Pass | Duration (s) |", "|---|---:|"])
for name, duration in raptor_pass_timings.items():
lines.append(f"| {name} | {duration:.4f} |")
report_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
@@ -1325,7 +1389,7 @@ def main():
parser.add_argument(
"--pimcomp-config",
type=Path,
help="PIMCOMP hardware config (default: <pimcomp-dir>/config.json).",
help="Pimcomp hardware config (default: <pimcomp-dir>/config.json).",
)
parser.add_argument("--pim-simulator-dir", default=REPO / "backend-simulators/pim/pim-simulator", type=Path)
parser.add_argument("--pimsim-nn-build-dir", default=REPO / "backend-simulators/pim/pimsim-nn/build", type=Path)
@@ -1347,7 +1411,7 @@ def main():
parser.add_argument("--pimsim-mode", choices=["latency", "throughput"], default="latency")
parser.add_argument("--batch-size", type=int, default=128)
parser.add_argument("--pimcomp-pipeline", choices=["element", "batch"])
parser.add_argument("--pimcomp-model-name", help="Use a PIMCOMP built-in model name such as vgg16.")
parser.add_argument("--pimcomp-model-name", help="Use a Pimcomp built-in model name such as vgg16.")
parser.add_argument(
"--pimcomp-replication",
choices=["balance", "W0H0", "uniform", "GA"],
@@ -1361,27 +1425,41 @@ def main():
parser.add_argument(
"--reuse-pimcomp-dir",
type=Path,
help="Reuse a directory containing PIMCOMP SimulationInfo.gz, VerificationInfo.json, and MappingResult.txt.",
help="Reuse a directory containing Pimcomp SimulationInfo.gz, VerificationInfo.json, and MappingResult.txt.",
)
parser.add_argument(
"--reuse-pimcomp-report",
type=Path,
help="Preserve the PIMCOMP side of an existing comparison report without rerunning it.",
help="Preserve the Pimcomp side of an existing comparison report without rerunning it.",
)
parser.add_argument("--skip-pimsim-nn", action="store_true")
parser.add_argument(
"--no-fast",
action="store_true",
help="Disable fast pimsim-nn throughput convergence for authoritative experiments.",
help="Disable fast Pimsim throughput convergence for authoritative experiments.",
)
parser.add_argument("--verbose-raptor-compile", action="store_true")
parser.add_argument("--raptor-extra-arg", action="append", default=[])
parser.add_argument(
"--raptor-only",
action="store_true",
help="Run Raptor compilation, validation, and simulation without running Pimcomp.",
)
parser.add_argument(
"--fail-on-error",
action="store_true",
help="Return a non-zero status if a stage or semantic validation fails.",
)
args = parser.parse_args()
if args.raptor_only and any(
option is not None
for option in (
args.reuse_raptor_report,
args.reuse_pimcomp_dir,
args.reuse_pimcomp_report,
)
):
parser.error("--raptor-only cannot be combined with report reuse options")
if args.reuse_pimcomp_dir is not None and args.reuse_pimcomp_report is not None:
parser.error("--reuse-pimcomp-dir and --reuse-pimcomp-report are mutually exclusive")
if args.pimsim_time_ms <= 0:
@@ -1439,6 +1517,7 @@ def main():
runner_path: Path | None = None
reference_dir: Path | None = None
reference_dirs: list[Path] = []
reference_stage_started = False
raptor_pim_dir: Path | None = None
raptor_pimsim_dir: Path | None = None
raptor_pass_timings: dict[str, float] = {}
@@ -1451,11 +1530,19 @@ def main():
reuse_raptor = args.reuse_raptor_report is not None
raptor_validation = skipped_validation("Raptor validation did not run")
pimcomp_validation = failed_validation("PIMCOMP validation did not run")
raptor_perf: dict[str, Any] = skipped_perf("pimsim-nn Raptor did not run")
pimcomp_perf: dict[str, Any] = skipped_perf("pimsim-nn PIMCOMP did not run")
pimcomp_validation = (
skipped_validation("Skipped by --raptor-only")
if args.raptor_only
else failed_validation("Pimcomp validation did not run")
)
raptor_perf: dict[str, Any] = skipped_perf("Pimsim Raptor did not run")
pimcomp_perf: dict[str, Any] = skipped_perf(
"Skipped by --raptor-only" if args.raptor_only else "Pimsim Pimcomp did not run"
)
raptor_instr: dict[str, Any] = empty_instruction_summary("Raptor instruction parsing did not run")
pimcomp_instr: dict[str, Any] = empty_instruction_summary("PIMCOMP instruction parsing did not run")
pimcomp_instr: dict[str, Any] = empty_instruction_summary(
"Skipped by --raptor-only" if args.raptor_only else "Pimcomp instruction parsing did not run"
)
loaded_hardware = try_stage(failures, "Load hardware configuration", load_effective_hardware, args)
if loaded_hardware is not None:
@@ -1476,22 +1563,21 @@ def main():
if args.reuse_pimcomp_dir is not None:
reused_pimcomp_report_path = args.reuse_pimcomp_dir.resolve().parent / "comparison_report.json"
reuse_pimcomp = reused_pimcomp_report_path is not None
run_pimcomp = not args.raptor_only and not reuse_pimcomp
if reuse_pimcomp:
reused_pimcomp_report_path = reused_pimcomp_report_path.resolve()
if not reused_pimcomp_report_path.exists():
raise ValueError(f"Missing PIMCOMP report: {reused_pimcomp_report_path}")
raise ValueError(f"Missing Pimcomp report: {reused_pimcomp_report_path}")
with open(reused_pimcomp_report_path, "r", encoding="utf-8") as f:
reused_pimcomp = json.load(f)
if reused_pimcomp.get("pimcomp_model_source") != "original_onnx":
raise ValueError("Reused PIMCOMP artifacts were not generated from the original ONNX model")
raise ValueError("Reused Pimcomp artifacts were not generated from the original ONNX model")
pimcomp_validation = CompareResult(**reused_pimcomp["pimcomp_validation"])
pimcomp_perf = reused_pimcomp["pimcomp_performance"]
pimcomp_instr = reused_pimcomp["pimcomp_instruction_summary"]
simulation_info = path_from_report(reused_pimcomp, "pimcomp_simulation_info")
pimcomp_export_dir = path_from_report(reused_pimcomp, "pimcomp_exported_pim")
pimcomp_pimsim_dir = path_from_report(reused_pimcomp, "pimcomp_pimsim_nn")
if pimcomp_pimsim_dir is None and (reused_pimcomp_report_path.parent / "pimsim_nn").is_dir():
pimcomp_pimsim_dir = reused_pimcomp_report_path.parent / "pimsim_nn"
pimsim_config = path_from_report(reused_pimcomp, "pimsim_config")
restore_side_records(reused_pimcomp, "pimcomp", failures, steps)
@@ -1504,6 +1590,9 @@ def main():
raise ValueError(f"Reused Raptor hardware differs: {reused_hardware} != {hardware}")
reference_dir = common_dir / "outputs"
runner_path = path_from_report(reused, "reference_runner")
expected_library = common_dir / "runner" / f"{functional_model_path.stem}.so"
if runner_path is None or not runner_uses_library(runner_path, expected_library):
runner_path = ensure_reference_runner(args, functional_model_path, common_dir, steps)
raptor_pim_dir = path_from_report(reused, "raptor_pim")
raptor_pimsim_dir = path_from_report(reused, "raptor_pimsim_nn")
arrays_in_order = load_saved_inputs(
@@ -1519,7 +1608,6 @@ def main():
print_step("Reuse Raptor")
print(f" Report: {reuse_report_path}")
expected_runner_path = common_dir / "runner/build/runner"
common_inputs_dir = common_dir / "inputs"
common_reference_dir = common_dir / "outputs"
@@ -1539,20 +1627,15 @@ def main():
else:
inputs_ready = False
if expected_runner_path.exists():
runner_path = expected_runner_path
else:
reference_compile = try_stage(
failures,
"Compile reference",
compile_reference,
args,
functional_model_path,
common_dir,
steps,
)
if reference_compile is not None:
runner_path = reference_compile
runner_path = try_stage(
failures,
"Prepare reference runner",
ensure_reference_runner,
args,
functional_model_path,
common_dir,
steps,
)
if runner_path is not None and runner_path.exists() and model_io is not None:
if inputs_ready and reference_outputs_exist(outputs_desc, common_reference_dir):
@@ -1570,9 +1653,11 @@ def main():
steps,
args,
common_dir,
print_header=not reference_stage_started,
)
if generated_reference is not None:
reference_dir = generated_reference
reference_stage_started = True
else:
record_failure(
failures,
@@ -1589,28 +1674,32 @@ def main():
write_input_batch_csv(out_dir / "inputs.csv", input_batch)
raptor_input_bins = write_input_batch_binaries(input_batch, out_dir / "simulation/raptor_inputs")
if args.pimsim_mode == "throughput":
batch_reference_dir = common_dir / f"reference_batch_{args.batch_size}_seed_{args.seed}"
throughput_references = try_stage(
failures,
"Run throughput references",
generate_reference_batch_outputs,
"Run reference",
prepare_reference_batch_outputs,
runner_path,
runner_path.parent,
functional_model_path,
input_batch,
outputs_desc,
steps,
args,
out_dir / "reference",
batch_reference_dir,
print_header=not reference_stage_started,
) if runner_path is not None and runner_path.exists() else None
if throughput_references is not None:
reference_dirs = throughput_references
reference_dir = out_dir / "reference"
reference_dir = batch_reference_dir
reference_stage_started = True
elif reference_dir is not None:
reference_dirs = [reference_dir]
if not reuse_raptor and model_path.exists() and hardware["core_count"] > 0:
compiled_raptor = try_stage(
failures,
"Compile Raptor PIM",
"Compile Raptor Pim",
compile_raptor_target,
model_path,
out_dir / "raptor",
@@ -1623,8 +1712,8 @@ def main():
elif not reuse_raptor:
record_failure(
failures,
"Skip Raptor PIM compile",
"Raptor PIM compile was skipped because the ONNX model or hardware configuration is not available.",
"Skip Raptor Pim compile",
"Raptor Pim compile was skipped because the ONNX model or hardware configuration is not available.",
)
raptor_functional_pim_dir = raptor_pim_dir
@@ -1635,7 +1724,7 @@ def main():
):
compiled_functional = try_stage(
failures,
"Compile Raptor functional PIM",
"Compile Raptor functional Pim",
compile_raptor_target,
functional_model_path,
out_dir / "raptor_functional",
@@ -1659,9 +1748,9 @@ def main():
if wrote_inputs and reference_dirs and outputs_desc:
validation = try_stage(
failures,
"Functional Validation Raptor",
"Functional validation Raptor",
run_functional_validation,
"Functional Validation Raptor",
"Functional validation Raptor",
raptor_functional_pim_dir,
raptor_functional_pim_dir / "config.json",
out_dir / "simulation/out.bin",
@@ -1679,7 +1768,7 @@ def main():
else:
raptor_validation = skipped_validation("Raptor input materialization failed")
elif not reuse_raptor:
raptor_validation = skipped_validation("Raptor PIM compilation did not produce a PIM directory")
raptor_validation = skipped_validation("Raptor Pim compilation did not produce a Pim directory")
pimcomp_model_path = model_path
@@ -1687,7 +1776,7 @@ def main():
reused_pimcomp_dir = args.reuse_pimcomp_dir.resolve()
copied_pimcomp = try_stage_success(
failures,
"Reuse PIMCOMP outputs",
"Reuse Pimcomp outputs",
copy_pimcomp_outputs,
reused_pimcomp_dir,
out_dir / "pimcomp/output",
@@ -1695,12 +1784,12 @@ def main():
if copied_pimcomp:
verification_info = out_dir / "pimcomp/output/VerificationInfo.json"
simulation_info = out_dir / "pimcomp/output/SimulationInfo.gz"
print_step("Reuse PIMCOMP")
print_step("Reuse Pimcomp")
print(f" Directory: {reused_pimcomp_dir}")
elif not reuse_pimcomp:
elif run_pimcomp:
compiled_pimcomp = try_stage(
failures,
"Compile PIMCOMP",
"Compile Pimcomp",
compile_pimcomp,
args,
pimcomp_model_path,
@@ -1710,12 +1799,12 @@ def main():
if compiled_pimcomp is not None:
verification_info, simulation_info = compiled_pimcomp
if reuse_pimcomp:
if reuse_pimcomp or args.raptor_only:
pass
elif verification_info is not None and simulation_info is not None and model_io is not None:
exported = try_stage(
failures,
"Export PIMCOMP for Functional Validation",
"Export Pimcomp for functional validation",
export_pimcomp_for_rust,
pimcomp_model_path,
verification_info,
@@ -1728,14 +1817,14 @@ def main():
elif verification_info is None or simulation_info is None:
record_failure(
failures,
"Export PIMCOMP for Functional Validation",
"PIMCOMP functional export failed because PIMCOMP did not produce VerificationInfo.json and SimulationInfo.gz.",
"Export Pimcomp for functional validation",
"Pimcomp functional export failed because Pimcomp did not produce VerificationInfo.json and SimulationInfo.gz.",
)
else:
record_failure(
failures,
"Export PIMCOMP for Functional Validation",
"PIMCOMP functional export failed because model inputs are not available.",
"Export Pimcomp for functional validation",
"Pimcomp functional export failed because model inputs are not available.",
)
if input_batch is not None and pimcomp_export_dir is not None:
@@ -1745,12 +1834,12 @@ def main():
transform=flatten_pimcomp_input,
)
if not reuse_pimcomp and pimcomp_export_dir is not None and reference_dirs and outputs_desc:
if run_pimcomp and pimcomp_export_dir is not None and reference_dirs and outputs_desc:
validation = try_stage(
failures,
"Functional Validation PIMCOMP",
"Functional validation Pimcomp",
run_functional_validation,
"Functional Validation PIMCOMP",
"Functional validation Pimcomp",
pimcomp_export_dir,
pimcomp_export_dir / "config.json",
out_dir / "simulation/pimcomp.out.bin",
@@ -1761,11 +1850,11 @@ def main():
args,
channel_last=True,
)
pimcomp_validation = validation if validation is not None else failed_validation("PIMCOMP validation failed")
elif reuse_pimcomp:
pimcomp_validation = validation if validation is not None else failed_validation("Pimcomp validation failed")
elif reuse_pimcomp or args.raptor_only:
pass
elif pimcomp_export_dir is None:
pimcomp_validation = failed_validation("PIMCOMP functional export is not available")
pimcomp_validation = failed_validation("Pimcomp functional export is not available")
elif not reference_dirs:
pimcomp_validation = failed_validation("Reference outputs are not available")
else:
@@ -1774,7 +1863,7 @@ def main():
if not args.skip_pimsim_nn and hardware["core_count"] > 0:
written_config = try_stage(
failures,
"Prepare pimsim-nn config",
"Prepare Pimsim config",
prepare_pimsim_config,
args,
hardware,
@@ -1784,25 +1873,25 @@ def main():
elif not args.skip_pimsim_nn:
record_failure(
failures,
"Skip pimsim-nn config",
"pimsim-nn config was skipped because the hardware configuration is not available.",
"Skip Pimsim config",
"Pimsim config was skipped because the hardware configuration is not available.",
)
if args.skip_pimsim_nn:
if not reuse_raptor:
raptor_perf = skipped_perf("Skipped by --skip-pimsim-nn")
if not reuse_pimcomp:
if run_pimcomp:
pimcomp_perf = skipped_perf("Skipped by --skip-pimsim-nn")
elif pimsim_config is None:
if not reuse_raptor:
raptor_perf = skipped_perf("pimsim-nn config is not available")
if not reuse_pimcomp:
pimcomp_perf = skipped_perf("pimsim-nn config is not available")
raptor_perf = skipped_perf("Pimsim config is not available")
if run_pimcomp:
pimcomp_perf = skipped_perf("Pimsim config is not available")
else:
if not reuse_raptor and raptor_pim_dir is not None:
raptor_pimsim_dir = try_stage(
failures,
"Export Raptor for pimsim-nn",
"Export Raptor for Pimsim",
export_raptor_pimsim_artifact,
raptor_pim_dir,
out_dir / "raptor/pimsim_nn",
@@ -1810,25 +1899,25 @@ def main():
if raptor_pimsim_dir is not None:
perf = try_stage(
failures,
"Non-Functional Simulation Raptor",
"Non-functional simulation Raptor",
run_pimsim_nn,
"Non-Functional Simulation Raptor",
"Non-functional simulation Raptor",
raptor_pimsim_dir,
pimsim_config,
steps,
args,
)
raptor_perf = perf if perf is not None else failed_perf("pimsim-nn Raptor failed")
raptor_perf = perf if perf is not None else failed_perf("Pimsim Raptor failed")
else:
raptor_perf = failed_perf("Raptor pimsim-nn export failed")
raptor_perf = failed_perf("Raptor Pimsim export failed")
elif not reuse_raptor:
raptor_perf = skipped_perf("Raptor PIM directory is not available")
raptor_perf = skipped_perf("Raptor Pim directory is not available")
if not reuse_pimcomp:
if run_pimcomp:
if simulation_info is not None:
pimcomp_pimsim_dir = try_stage(
failures,
"Export PIMCOMP for pimsim-nn",
"Export Pimcomp for Pimsim",
export_pimcomp_for_pimsim_nn,
simulation_info,
out_dir / "pimcomp/pimsim_nn",
@@ -1836,32 +1925,32 @@ def main():
if pimcomp_pimsim_dir is not None:
perf = try_stage(
failures,
"Non-Functional Simulation PIMCOMP",
"Non-functional simulation Pimcomp",
run_pimsim_nn,
"Non-Functional Simulation PIMCOMP",
"Non-functional simulation Pimcomp",
pimcomp_pimsim_dir,
pimsim_config,
steps,
args,
)
pimcomp_perf = perf if perf is not None else failed_perf("pimsim-nn PIMCOMP failed")
pimcomp_perf = perf if perf is not None else failed_perf("Pimsim Pimcomp failed")
else:
pimcomp_perf = failed_perf("PIMCOMP pimsim-nn export failed")
pimcomp_perf = failed_perf("Pimcomp Pimsim export failed")
else:
pimcomp_perf = skipped_perf("PIMCOMP SimulationInfo.gz is not available")
pimcomp_perf = skipped_perf("Pimcomp SimulationInfo.gz is not available")
if not reuse_raptor and raptor_pim_dir is not None and raptor_pim_dir.exists():
parsed = try_stage(failures, "Parse Raptor instructions", parse_raptor_instructions, raptor_pim_dir)
raptor_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse Raptor instructions")
elif not reuse_raptor:
raptor_instr = empty_instruction_summary("Raptor PIM directory is not available")
raptor_instr = empty_instruction_summary("Raptor Pim directory is not available")
if not reuse_pimcomp:
if run_pimcomp:
if simulation_info is not None and simulation_info.exists():
parsed = try_stage(failures, "Parse PIMCOMP instructions", parse_pimcomp_instructions, simulation_info)
pimcomp_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse PIMCOMP instructions")
parsed = try_stage(failures, "Parse Pimcomp instructions", parse_pimcomp_instructions, simulation_info)
pimcomp_instr = parsed if parsed is not None else empty_instruction_summary(error="Failed to parse Pimcomp instructions")
else:
pimcomp_instr = empty_instruction_summary("PIMCOMP SimulationInfo.gz is not available")
pimcomp_instr = empty_instruction_summary("Pimcomp SimulationInfo.gz is not available")
report_path = out_dir / "pimcomp/comparison_report.md"
write_report(
@@ -1890,7 +1979,7 @@ def main():
"pimsim_time_ms": args.pimsim_time_ms,
"pimcomp_pipeline": args.pimcomp_pipeline,
"pimcomp_replication": args.pimcomp_replication,
"pimcomp_model_source": "original_onnx",
"pimcomp_model_source": "not_run" if args.raptor_only else "original_onnx",
"pimcomp_config": str(args.pimcomp_config),
"raptor_extra_args": args.raptor_extra_arg,
"reused_raptor_report": optional_path(args.reuse_raptor_report.resolve()) if reuse_raptor else None,
@@ -1926,7 +2015,11 @@ def main():
json.dump(json_report, f, indent=2)
f.write("\n")
partial_side = "pimcomp" if reuse_raptor else "raptor" if args.reuse_pimcomp_report else None
partial_side = (
"pimcomp" if reuse_raptor
else "raptor" if args.raptor_only or args.reuse_pimcomp_report
else None
)
other_side = "RAPTOR" if partial_side == "pimcomp" else "PIMCOMP"
relevant_failures = (
failures if partial_side is None
@@ -1947,7 +2040,7 @@ def main():
for result in relevant_validations
)
failed = bool(relevant_failures or failed_steps or functional_failure)
print("\n" + Style.BRIGHT + Fore.GREEN + "[Completed]" + Style.RESET_ALL)
print(Style.BRIGHT + Fore.GREEN + "[Completed]" + Style.RESET_ALL)
print(f" Report: {report_path}")
print(f" JSON: {json_path}")
if failures or failed_steps:
validation/tools/pim/pimcomp/compare/run_pimcomp_paper_latency.py → validation/tools/pim/pimcomp/compare/run_pimcomp_models.py
+192 -105
View File
@@ -19,33 +19,30 @@ from colorama import Fore, Style
REPO = Path(__file__).resolve().parents[5]
SUITE = REPO / "validation/networks/pimcomp_models"
sys.path.insert(0, str(REPO / "validation"))
from raptor_validation.pimsim_nn import parse_pimsim_nn_metrics # noqa: E402
from raptor_validation.validate_one import STAGE_COLORS # noqa: E402
from raptor_validation.pimcomp_models import (
FUNCTIONAL_MODELS,
MODELS,
SUITE,
add_models_argument,
)
from raptor_validation.artifacts import artifacts_dir, remove_lock_files
from raptor_validation.pimsim_nn import parse_pimsim_nn_metrics
from raptor_validation.validate_one import STAGE_COLORS
PIMCOMP_SOURCE = REPO / "third_party/PIMCOMP-NN"
PIMCOMP_CONFIGS = REPO / "validation/pimsim_configs/pimcomp"
COMPARE = Path(__file__).resolve().with_name("compare_raptor_pimcomp.py")
COMPARE = Path(__file__).resolve().with_name("compare_raptor_pimcomp_model.py")
ARCHES = tuple(sorted(path.name for path in PIMCOMP_CONFIGS.iterdir() if path.is_dir()))
MODELS = {
"vgg8": SUITE / "vgg8/vgg8-mnist-reconstructed.onnx",
"resnet18": SUITE / "resnet18/resnet18-v1-7.onnx",
"resnet34": SUITE / "resnet34/resnet34-v1-7.onnx",
"googlenet": SUITE / "googlenet/googlenet-12-pimsim-nn.onnx",
"yolo11n": SUITE / "yolo11n/yolo11n-pimsim-nn.onnx",
}
FUNCTIONAL_MODELS = {
**MODELS,
"yolo11n": REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx",
}
DEFAULT_ARCHES = ("arch-a", "arch-b")
COMPARISONS = (
("latency", 1, "element"),
("throughput", 2, "batch"),
("throughput", 4, "batch"),
("throughput", 8, "batch"),
)
RESULTS_FILENAME = "results_comparison.csv"
@dataclass(frozen=True)
@@ -67,39 +64,60 @@ def model_dir(root: Path | None, name: str) -> Path:
return root / name if root is not None else MODELS[name].parent
def result_dir(root: Path | None, name: str, arch: str, mode: str, pipeline: int) -> Path:
base = model_dir(root, name)
def result_dir(
root: Path | None,
name: str,
arch: str,
mode: str,
pipeline: int,
ablation_variant: str | None = None,
) -> Path:
base = artifacts_dir(model_dir(root, name))
suffix = "latency" if mode == "latency" else f"throughput/pipeline{pipeline}"
return base / arch / suffix
result = base / arch / suffix
if ablation_variant is not None:
result /= Path("ablation") / ablation_variant
return result
def common_dir(root: Path | None, name: str) -> Path:
def common_dir(root: Path | None, name: str, common_root: Path | None = None) -> Path:
suffix = "common" if FUNCTIONAL_MODELS[name] == MODELS[name] else "common-functional"
return model_dir(root, name) / suffix
base = common_root / name if common_root is not None else model_dir(root, name)
return artifacts_dir(base) / suffix
def clean_artifacts(root: Path | None, models: list[str], arches: list[str]) -> int:
def clean_artifacts(
root: Path | None,
models: list[str],
common_root: Path | None = None,
) -> int:
removed = 0
for name in models:
base = model_dir(root, name)
arch_dirs = {base / arch for arch in arches}
arch_dirs.update(path for path in base.glob("arch-*") if path.is_dir() and not path.is_symlink())
for path in arch_dirs:
for path in (artifacts_dir(base),):
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
removed += 1
for common in (base / "common", base / "common-functional"):
if common.is_dir() and not common.is_symlink():
shutil.rmtree(common)
removed += 1
for path in (
(root or SUITE) / "results.csv",
(root or SUITE) / "results_latency.csv",
(root or SUITE) / "results_throughput.csv",
(root or SUITE) / RESULTS_FILENAME,
(root or SUITE) / "results_ablation.csv",
):
if path.is_file() or path.is_symlink():
path.unlink()
removed += 1
summary_root = root or SUITE
if (summary_root / "ablation").is_dir():
for path in (summary_root / "ablation").glob("*/results_comparison.csv"):
path.unlink(missing_ok=True)
removed += 1
removed += remove_lock_files(summary_root)
if common_root is not None:
for name in models:
path = artifacts_dir(common_root / name)
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
removed += 1
removed += remove_lock_files(common_root)
return removed
@@ -108,8 +126,11 @@ def write_results_csv(
arch: str,
models: list[str],
comparisons: tuple[tuple[str, int, str], ...] = COMPARISONS,
ablation_variant: str | None = None,
) -> Path:
output = (root or SUITE) / "results.csv"
output = (root or SUITE) / RESULTS_FILENAME
if ablation_variant is not None:
output = output.parent / "ablation" / ablation_variant / RESULTS_FILENAME
fields = (
"arch",
"model",
@@ -152,7 +173,9 @@ def write_results_csv(
rows.append({field: row.get(field, "NA") for field in fields})
for name in models:
for comparison_mode, pipeline, pimcomp_pipeline in comparisons:
report_path = result_dir(root, name, arch, comparison_mode, pipeline) / "pimcomp/comparison_report.json"
report_path = result_dir(
root, name, arch, comparison_mode, pipeline, ablation_variant
) / "pimcomp/comparison_report.json"
row = {
"model": name,
"arch": arch,
@@ -215,6 +238,7 @@ def write_results_csv(
int(row["raptor_pipeline"]),
)
)
output.parent.mkdir(parents=True, exist_ok=True)
with open(output, "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fields, lineterminator="\n")
writer.writeheader()
@@ -275,11 +299,11 @@ def format_value(value: float | None) -> str:
def print_stage(title: str, color: str) -> None:
print("\n" + Style.BRIGHT + color + f"[{title}]" + Style.RESET_ALL, flush=True)
print(Style.BRIGHT + color + f"[{title}]" + Style.RESET_ALL, flush=True)
def print_completed(label: str, output: str = "") -> None:
print("\n" + Style.BRIGHT + Fore.CYAN + f"[Completed {label}]" + Style.RESET_ALL)
print(Style.BRIGHT + Fore.CYAN + f"[Completed {label}]" + Style.RESET_ALL)
if output:
print(output, end="" if output.endswith("\n") else "\n")
print("=" * 72, flush=True)
@@ -321,7 +345,7 @@ def validate_pimcomp_source() -> None:
source = header.read_text(encoding="utf-8")
for setting in ("int population_num = 200;", "int max_iteration = 1000;"):
if setting not in source:
raise RuntimeError(f"PIMCOMP paper setting is missing: {setting}")
raise RuntimeError(f"Pimcomp paper setting is missing: {setting}")
def comparison_command(
@@ -337,6 +361,8 @@ def comparison_command(
batch_size: int,
timeout: float,
fast: bool,
raptor_extra_args: list[str] | tuple[str, ...] = (),
raptor_only: bool = False,
reuse_raptor_report: Path | None = None,
reuse_pimcomp_dir: Path | None = None,
reuse_pimcomp_report: Path | None = None,
@@ -373,7 +399,9 @@ def comparison_command(
pimcomp_pipeline,
"--pimcomp-replication",
"GA",
*(["--raptor-only"] if raptor_only else []),
f"--raptor-extra-arg=--pipeline={pipeline}",
*[f"--raptor-extra-arg={arg}" for arg in raptor_extra_args],
"--timeout-seconds",
str(timeout),
"--fail-on-error",
@@ -418,13 +446,19 @@ def comparison_command_for(
args.batch_size,
args.timeout_seconds,
not args.no_fast,
args.raptor_extra_args,
args.raptor_only,
reuse_raptor_report=(
report
if args.only == "pimcomp"
else None
),
reuse_pimcomp_dir=spec.shared_pimcomp_dir if reuse_shared_pimcomp and args.only != "raptor" else None,
reuse_pimcomp_report=report if args.only == "raptor" else None,
reuse_pimcomp_dir=(
spec.shared_pimcomp_dir
if reuse_shared_pimcomp and args.only != "raptor" and not args.raptor_only
else None
),
reuse_pimcomp_report=report if args.only == "raptor" and not args.raptor_only else None,
)
@@ -497,20 +531,29 @@ def config_path(arch: str, mode: str, sim_time_ms: int, *, write: bool) -> Path:
def main() -> int:
parser = argparse.ArgumentParser(
description="Compare supported PIMCOMP models with Raptor latency and throughput schedules."
description="Compare supported Pimcomp models with Raptor latency and throughput schedules."
)
parser.add_argument(
"--out-dir",
type=Path,
help="Result root (default: artifacts beside each model under validation/).",
help="Suite root; generated artifacts go below each model's artifacts/ directory (default: validation/networks/pimcomp_models).",
)
parser.add_argument("--models", nargs="+", choices=MODELS, default=list(MODELS))
parser.add_argument(
"--arch",
"--common-dir",
type=Path,
help="Shared root for per-model reference artifacts. Default: inside --out-dir.",
)
parser.add_argument(
"--ablation-variant",
help="Place Raptor artifacts below <mode>/ablation/<variant> and write the comparison summary below ablation/<variant>.",
)
add_models_argument(parser)
parser.add_argument(
"--archs",
nargs="+",
choices=ARCHES,
default=list(ARCHES),
help="PIM architectures to run (default: all architectures).",
default=list(DEFAULT_ARCHES),
help=f"Pim architectures to run (default: {', '.join(DEFAULT_ARCHES)}).",
)
parser.add_argument(
"--mode",
@@ -524,6 +567,11 @@ def main() -> int:
choices=("raptor", "pimcomp"),
help="Re-run only this compiler's compile, validation, and simulation stages; preserve the other side from its report.",
)
parser.add_argument(
"--raptor-only",
action="store_true",
help="Run only Raptor; do not compile, validate, or simulate Pimcomp. Default: off.",
)
parser.add_argument(
"--pipeline",
type=int,
@@ -534,7 +582,7 @@ def main() -> int:
"--pimsim-time-ms",
type=int,
default=1000,
help="throughput pimsim-nn convergence deadline in ms (default: 1000).",
help="Throughput Pimsim convergence deadline in ms (default: 1000).",
)
parser.add_argument(
"--batch-size",
@@ -564,15 +612,32 @@ def main() -> int:
parser.add_argument(
"--no-fast",
action="store_true",
help="Disable fast pimsim-nn throughput convergence for authoritative experiments.",
help="Disable fast Pimsim throughput convergence for authoritative experiments.",
)
parser.add_argument(
"--raptor-extra-arg",
action="append",
default=[],
dest="raptor_extra_args",
help="Additional argument to pass to Raptor; repeat as needed.",
)
args = parser.parse_args()
if args.ablation_variant and Path(args.ablation_variant).name != args.ablation_variant:
parser.error("--ablation-variant must be a single directory name")
if args.only is not None and args.raptor_only:
parser.error("--only cannot be combined with --raptor-only")
out_dir = args.out_dir.resolve() if args.out_dir is not None else None
args.arch = list(dict.fromkeys(args.arch))
common_root = args.common_dir.resolve() if args.common_dir is not None else None
args.archs = list(dict.fromkeys(args.archs))
args.mode = list(dict.fromkeys(args.mode))
if args.clean:
print(f"Removed {clean_artifacts(out_dir, args.models, args.arch)} comparison artifact path(s).")
print(
f"Removed {clean_artifacts(out_dir, args.models, common_root)} "
"comparison artifact path(s)."
)
return 0
if args.jobs < 1:
@@ -585,7 +650,7 @@ def main() -> int:
parser.error("--timeout-seconds must be non-negative")
comparisons_by_arch: dict[str, tuple[tuple[str, int, str], ...]] = {}
configs_by_arch: dict[str, dict[str, Path]] = {}
for arch in args.arch:
for arch in args.archs:
comparisons = tuple(
comparison for comparison in COMPARISONS
if comparison[0] in args.mode
@@ -611,10 +676,12 @@ def main() -> int:
if args.only is not None:
missing_reuse = []
for arch in args.arch:
for arch in args.archs:
for name in args.models:
for mode, pipeline, _ in comparisons_by_arch[arch]:
comparison_dir = result_dir(out_dir, name, arch, mode, pipeline)
comparison_dir = result_dir(
out_dir, name, arch, mode, pipeline, args.ablation_variant
)
required = comparison_dir / "pimcomp/comparison_report.json"
if not required.exists():
missing_reuse.append(str(required))
@@ -629,14 +696,15 @@ def main() -> int:
+ ", ".join(missing_reuse)
)
validate_pimcomp_source()
if not args.raptor_only:
validate_pimcomp_source()
if out_dir is not None and not args.dry_run:
out_dir.mkdir(parents=True, exist_ok=True)
print(Style.BRIGHT + f"Found {len(args.models)} PIMCOMP model(s) to compare." + Style.RESET_ALL)
print(f"Architectures: {', '.join(args.arch)}")
print(Style.BRIGHT + f"Found {len(args.models)} Pimcomp model(s) to compare." + Style.RESET_ALL)
print(f"Architectures: {', '.join(args.archs)}")
print(f"Modes: {', '.join(args.mode)}")
print(f"Throughput pimsim time: {args.pimsim_time_ms} ms")
print(f"Throughput Pimsim time: {args.pimsim_time_ms} ms")
print(f"Max parallel jobs: {args.jobs}")
print(
f"Comparison jobs: "
@@ -645,29 +713,36 @@ def main() -> int:
print(f"Results root: {out_dir or SUITE}")
print("=" * 72)
print_stage("Prepare shared artifacts", STAGE_COLORS["Build Runner"])
for name in args.models:
try:
run(
prepare_common_command(
FUNCTIONAL_MODELS[name],
common_dir(out_dir, name),
args.timeout_seconds,
),
dry_run=args.dry_run,
)
finally:
print_completed(f"shared artifacts: {name}")
print_stage("Prepare shared artifacts", STAGE_COLORS["Build runner"])
try:
for name in args.models:
try:
run(
prepare_common_command(
FUNCTIONAL_MODELS[name],
common_dir(out_dir, name, common_root),
args.timeout_seconds,
),
dry_run=args.dry_run,
)
finally:
print_completed(f"shared artifacts: {name}")
except KeyboardInterrupt:
remove_lock_files(out_dir or SUITE)
print("Interrupted; cleaned validation lock files.", file=sys.stderr)
return 130
failed = []
comparison_specs: list[ComparisonSpec] = []
shared_pimcomp_by_group: dict[tuple[str, str, str], Path] = {}
for arch in args.arch:
for arch in args.archs:
comparisons = comparisons_by_arch[arch]
configs = configs_by_arch[arch]
for name in args.models:
for mode, pipeline, pimcomp_pipeline in comparisons:
model_result_dir = result_dir(out_dir, name, arch, mode, pipeline)
model_result_dir = result_dir(
out_dir, name, arch, mode, pipeline, args.ablation_variant
)
label = f"{arch}/{name}/{mode}/pipeline{pipeline}"
group = (arch, name, mode)
shared_pimcomp_dir = shared_pimcomp_by_group.get(group)
@@ -681,7 +756,7 @@ def main() -> int:
model=MODELS[name],
functional_model=FUNCTIONAL_MODELS[name],
output_dir=model_result_dir,
common_dir=common_dir(out_dir, name),
common_dir=common_dir(out_dir, name, common_root),
config=configs[mode],
mode=mode,
pipeline=pipeline,
@@ -701,7 +776,8 @@ def main() -> int:
if run(command, dry_run=True, check=False):
failed.append(spec.label)
elif comparison_specs:
anchor_specs = comparison_specs if args.only == "raptor" else [
raptor_only_run = args.only == "raptor" or args.raptor_only
anchor_specs = comparison_specs if raptor_only_run else [
spec for spec in comparison_specs if spec.anchor
]
anchor_jobs = [
@@ -712,64 +788,75 @@ def main() -> int:
)
for spec in anchor_specs
]
dependent_specs = [] if args.only == "raptor" else [
dependent_specs = [] if raptor_only_run else [
spec for spec in comparison_specs if not spec.anchor
]
print_directly = min(args.jobs, len(comparison_specs)) == 1
with (nullcontext(None) if print_directly else TemporaryDirectory(prefix="raptor-pimcomp-")) as log_dir:
anchor_failed, log_offset = run_comparison_jobs(
anchor_jobs,
args.jobs,
Path(log_dir) if log_dir else None,
0,
)
failed.extend(anchor_failed)
dependent_jobs = [
(
spec.label,
comparison_command_for(
spec,
args,
reuse_shared_pimcomp=pimcomp_artifact_ready(spec.shared_pimcomp_dir),
),
None,
try:
with (nullcontext(None) if print_directly else TemporaryDirectory(prefix="raptor-pimcomp-")) as log_dir:
anchor_failed, log_offset = run_comparison_jobs(
anchor_jobs,
args.jobs,
Path(log_dir) if log_dir else None,
0,
)
for spec in dependent_specs
]
dependent_failed, _ = run_comparison_jobs(
dependent_jobs,
args.jobs,
Path(log_dir) if log_dir else None,
log_offset,
)
failed.extend(dependent_failed)
failed.extend(anchor_failed)
dependent_jobs = [
(
spec.label,
comparison_command_for(
spec,
args,
reuse_shared_pimcomp=pimcomp_artifact_ready(spec.shared_pimcomp_dir),
),
None,
)
for spec in dependent_specs
]
dependent_failed, _ = run_comparison_jobs(
dependent_jobs,
args.jobs,
Path(log_dir) if log_dir else None,
log_offset,
)
failed.extend(dependent_failed)
except KeyboardInterrupt:
remove_lock_files(out_dir or SUITE)
print("Interrupted; cleaned validation lock files.", file=sys.stderr)
return 130
if args.dry_run:
return 1 if failed else 0
remove_lock_files(out_dir or SUITE)
results_path = None
for arch in args.arch:
for arch in args.archs:
results_path = write_results_csv(
out_dir,
arch,
args.models,
comparisons_by_arch[arch],
args.ablation_variant,
)
assert results_path is not None
print_stage(results_path.name, STAGE_COLORS["Compare Outputs"])
print_stage(results_path.name, STAGE_COLORS["Compare outputs"])
print(results_path.read_text(encoding="utf-8"), end="")
for arch in args.arch:
for arch in args.archs:
for name in args.models:
for mode, pipeline, _ in comparisons_by_arch[arch]:
label = f"{arch}/{name}/{mode}/pipeline{pipeline}"
report_path = result_dir(out_dir, name, arch, mode, pipeline) / "pimcomp/comparison_report.json"
report_path = result_dir(
out_dir, name, arch, mode, pipeline, args.ablation_variant
) / "pimcomp/comparison_report.json"
compiler = "raptor" if args.raptor_only else args.only
if not report_path.exists() or not comparison_passed(
json.loads(report_path.read_text(encoding="utf-8")),
args.only,
compiler,
):
if label not in failed:
failed.append(label)
print("\n" + Style.BRIGHT + Fore.CYAN + "Summary" + Style.RESET_ALL)
print(Style.BRIGHT + Fore.CYAN + "Summary" + Style.RESET_ALL)
total_jobs = sum(len(args.models) * len(comparisons) for comparisons in comparisons_by_arch.values())
print(Style.BRIGHT + f"Passed: {total_jobs - len(failed)}" + Style.RESET_ALL)
print(Style.BRIGHT + f"Failed: {len(failed)}" + Style.RESET_ALL)
validation/tools/pim/pimcomp/correctness/README.md
+46 -10
View File
@@ -1,6 +1,6 @@
# PIMCOMP batch correctness reproduction
# Pimcomp batch correctness reproduction
PIMCOMP's batch scheduler currently emits an incomplete standalone program for
Pimcomp's batch scheduler currently emits an incomplete standalone program for
models containing post operations. The generated `VerificationInfo.json` uses a
negative `source_address` to identify the preceding node, but the batch
verifier resolves that address by copying the provider tensor directly from
@@ -17,8 +17,8 @@ the missing computation visible in Rust functional validation.
For the checked-in GoogLeNet throughput/pipeline2 artifact, 39 provider tensors
are referenced by batch loads. Nineteen have generated stores; twenty are
never written. Preloading the provider tensors with the same ONNX Runtime
intermediates used by PIMCOMP's verifier makes the exported program pass. This
reproduces the verifier's input contract; it does not repair PIMCOMP's batch
intermediates used by Pimcomp's verifier makes the exported program pass. This
reproduces the verifier's input contract; it does not repair Pimcomp's batch
schedule.
Run the default reproduction from the repository root:
@@ -27,23 +27,59 @@ Run the default reproduction from the repository root:
.venv/bin/python validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py
```
The launcher accepts an alternate comparison directory, model, and work
work directory, and shared reference-artifact directory:
The launcher accepts an alternate comparison directory, model, work directory,
and shared reference-artifact directory:
```bash
.venv/bin/python validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py \
validation/networks/pimcomp_models/googlenet/arch-a/throughput/pipeline2 \
validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2 \
validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx \
/tmp/pimcomp-prefill-googlenet \
validation/networks/pimcomp_models/googlenet/common
validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2/correctness/prefill \
validation/networks/pimcomp_models/googlenet/artifacts/common
```
Without the optional work-directory argument, the experiment uses the same
`correctness/prefill/` directory below the comparison artifacts.
It runs the exported artifact once with its original memory image and once
with [`prefill_batch_memory.py`](prefill_batch_memory.py), then compares both
outputs with the recorded native reference. The expected GoogLeNet result is a
baseline maximum difference near `6.70705` and a prefilled maximum difference
near `4.05e-6`.
The issue is in PIMCOMP batch scheduling/validation semantics, not in the Rust
The issue is in Pimcomp batch scheduling/validation semantics, not in the Rust
simulator's vector-length interpretation. Vector lengths remain element counts
as specified by the reference ISA.
## ResNet BatchNorm correctness gap
Pimcomp has a separate correctness limitation in its ResNet element pipeline.
`BatchNormalization` is an ONNX operation. The frontend's
[`fuse_operators()` pass](../../../../../third_party/PIMCOMP-NN/frontend/frontend.py#L535)
marks it as fused and removes the node from the
scheduled graph, but the released path does not fold the BatchNorm affine
transform into the convolution weights and bias. The corresponding verifier
workaround in [`verification.py`](../../../../../third_party/PIMCOMP-NN/verification/verification.py#L99)
replaces BatchNorm parameters with identity values (scale and variance equal
to one, bias and mean equal to zero) before running ONNX Runtime.
Therefore Pimcomp's native verifier and exported element program agree with
each other, but they do not implement the original ResNet ONNX model. On the
current ResNet-18 Arch-A latency artifact, using the same input:
| Comparison | Maximum absolute difference |
|---|---:|
| Pimcomp native verifier vs Rust export | `6.7e-8` |
| Rust export vs original ONNX reference | `4.8294563` |
The Pimcomp output ranges from approximately `-0.187` to `0.220`, while the
original ONNX output ranges from `-3.572` to `4.834`; 462 of 1000 final
elements differ by more than one. This is not a Rust simulator or Python
exporter regression. It is a Pimcomp model-semantics mismatch caused by
dropping BatchNorm numerics. The same issue affects the ResNet-34 latency
artifact. VGG and GoogLeNet do not show this particular mismatch because they
do not contain the same ResNet BatchNorm path.
The latency comparison intentionally uses the original ONNX reference, so
these Pimcomp rows must remain `FAIL` until Pimcomp folds BatchNorm correctly
or the comparison explicitly uses a BatchNorm-neutralized reference.
validation/tools/pim/pimcomp/correctness/prefill_batch_memory.py
+3 -3
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Populate the host-side intermediate buffers expected by PIMCOMP batch mode."""
"""Populate the host-side intermediate buffers expected by Pimcomp batch mode."""
from __future__ import annotations
@@ -23,7 +23,7 @@ def flatten_reference(value: np.ndarray) -> np.ndarray:
elif value.ndim == 2:
value = value.transpose()
else:
raise ValueError(f"PIMCOMP batch verification only flattens 2D/4D tensors, got {value.shape}")
raise ValueError(f"Pimcomp batch verification only flattens 2D/4D tensors, got {value.shape}")
return value.astype(np.float32, copy=False).reshape(-1)
@@ -94,7 +94,7 @@ def prefill_batch_memory(
session = ort.InferenceSession(runtime_model.SerializeToString(), providers=["CPUExecutionProvider"])
session_inputs = session.get_inputs()
if len(session_inputs) != 1:
raise ValueError("PIMCOMP export currently requires exactly one runtime input tensor")
raise ValueError("Pimcomp export currently requires exactly one runtime input tensor")
input_meta = session_inputs[0]
input_tensor = np.loadtxt(input_path, delimiter=",", dtype=np.float32).reshape(input_meta.shape)
provider_names = [node_list[index]["name"] for index in providers]
validation/tools/pim/pimcomp/correctness/run_prefill_experiment.py
+21 -8
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Reproduce the PIMCOMP batch prefill correctness experiment."""
"""Reproduce the Pimcomp batch prefill correctness experiment."""
from __future__ import annotations
@@ -7,7 +7,6 @@ import argparse
import json
import subprocess
import sys
import tempfile
from pathlib import Path
import numpy as np
@@ -17,9 +16,9 @@ from prefill_batch_memory import prefill_batch_memory
SCRIPT_DIR = Path(__file__).resolve().parent
REPO_ROOT = SCRIPT_DIR.parents[4]
DEFAULT_COMPARISON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/arch-a/throughput/pipeline2"
DEFAULT_COMPARISON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2"
DEFAULT_MODEL = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx"
DEFAULT_COMMON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/common"
DEFAULT_COMMON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/common"
SIMULATOR_MANIFEST = REPO_ROOT / "backend-simulators/pim/pim-simulator/Cargo.toml"
@@ -28,6 +27,7 @@ def run_simulator(
memory: Path,
output: Path,
dump: str,
input_dir: Path,
) -> None:
subprocess.run(
[
@@ -44,6 +44,8 @@ def run_simulator(
str(comparison_dir / "pimcomp/exported"),
"--memory",
str(memory),
"--input-dir",
str(input_dir),
"-o",
str(output),
"-d",
@@ -86,14 +88,18 @@ def main() -> int:
model = args.model.resolve()
common_dir = args.common_dir.resolve()
if args.work_dir is None:
work_dir = Path(tempfile.mkdtemp(prefix="pimcomp-prefill."))
work_dir = comparison_dir / "correctness/prefill"
work_dir.mkdir(parents=True, exist_ok=True)
else:
work_dir = args.work_dir.resolve()
work_dir.mkdir(parents=True, exist_ok=True)
input_path = common_dir / "inputs/in0.csv"
if not input_path.is_file():
input_path = comparison_dir / "inputs/in0.csv"
simulator_input_dir = work_dir / "inputs"
simulator_input_dir.mkdir(parents=True, exist_ok=True)
np.loadtxt(input_path, delimiter=",", dtype=np.float32).tofile(
simulator_input_dir / "input_0.bin"
)
prefilled_memory = work_dir / "prefilled_memory.bin"
metadata_path = work_dir / "metadata.json"
@@ -117,8 +123,15 @@ def main() -> int:
comparison_dir / "pimcomp/exported/memory.bin",
baseline_output,
dump,
simulator_input_dir,
)
run_simulator(
comparison_dir,
prefilled_memory,
prefilled_output,
dump,
simulator_input_dir,
)
run_simulator(comparison_dir, prefilled_memory, prefilled_output, dump)
compare_outputs(baseline_output, prefilled_output, reference, work_dir)
return 0
validation/tools/pim/visuals/pim_sequence_diagram.py
+3 -3
View File
@@ -98,9 +98,9 @@ def read_binary(path: Path) -> Program:
magic, version, count = HEADER.unpack_from(data)
expected_size = HEADER.size + count * RECORD.size
if magic != b"PIMB":
raise ValueError(f"{path}: invalid PIM binary magic")
raise ValueError(f"{path}: invalid Pim binary magic")
if version != 1:
raise ValueError(f"{path}: unsupported PIM binary version {version}")
raise ValueError(f"{path}: unsupported Pim binary version {version}")
if len(data) != expected_size:
raise ValueError(f"{path}: expected {expected_size} bytes, found {len(data)}")
@@ -399,7 +399,7 @@ def render_text(cores: list[int], transfers: list[Transfer], programs: dict[int,
def main() -> int:
parser = argparse.ArgumentParser(
description="Generate SequenceDiagram.org text for PIM communication and intervening work."
description="Generate SequenceDiagram.org text for Pim communication and intervening work."
)
parser.add_argument("pim_dir", type=Path, help="Directory containing core_<id>.json or core_<id>.pim files")
selection = parser.add_mutually_exclusive_group(required=True)
validation/tools/yolo_local_image_validation.py
+19 -15
View File
@@ -22,6 +22,7 @@ if sys.version_info < (3, 10):
)
from raptor_validation.onnx_utils import _ONNX_TO_NP, onnx_io, write_inputs_binary, write_inputs_to_memory_bin
from raptor_validation.artifacts import artifacts_dir
from raptor_validation.validate_one import (
MODE_COMPILE_ONLY,
build_dump_ranges,
@@ -64,16 +65,17 @@ def find_network_onnx(network_dir: Path) -> Path:
def local_case_paths(network_dir: Path, case_name: str):
artifact_root = artifacts_dir(network_dir)
return {
"root": network_dir,
"runner": network_dir / "runner" / "build" / "runner",
"runner_build": network_dir / "runner" / "build",
"raptor_pim": network_dir / "raptor" / "pim",
"real_root": network_dir / "real_image_validation",
"input_csv": network_dir / "real_image_validation" / "inputs" / f"{case_name}.csv",
"ref_dir": network_dir / "real_image_validation" / "reference" / case_name,
"sim_dir": network_dir / "real_image_validation" / "simulation" / case_name,
"sim_bin": network_dir / "real_image_validation" / "simulation" / case_name / "out.bin",
"root": artifact_root,
"runner": artifact_root / "runner" / "build" / "runner",
"runner_build": artifact_root / "runner" / "build",
"raptor_pim": artifact_root / "raptor" / "pim",
"real_root": artifact_root / "real_image_validation",
"input_csv": artifact_root / "real_image_validation" / "inputs" / f"{case_name}.csv",
"ref_dir": artifact_root / "real_image_validation" / "reference" / case_name,
"sim_dir": artifact_root / "real_image_validation" / "simulation" / case_name,
"sim_bin": artifact_root / "real_image_validation" / "simulation" / case_name / "out.bin",
}
@@ -102,10 +104,11 @@ def ensure_local_artifacts(args, network_onnx_path: Path):
def ensure_existing_artifacts(network_dir: Path):
artifact_root = artifacts_dir(network_dir)
required_paths = [
network_dir / "runner" / "build" / "runner",
network_dir / "raptor" / "pim" / "config.json",
network_dir / "raptor" / "pim" / "memory.bin",
artifact_root / "runner" / "build" / "runner",
artifact_root / "raptor" / "pim" / "config.json",
artifact_root / "raptor" / "pim" / "memory.bin",
]
missing = [str(path) for path in required_paths if not path.exists()]
if missing:
@@ -137,7 +140,7 @@ def run_local_reference_and_simulator(args, network_dir: Path, network_onnx_path
tensor = np.loadtxt(paths["input_csv"], delimiter=",", dtype=np.float32).reshape(1, 3, 640, 640)
write_inputs_to_memory_bin(paths["raptor_pim"] / "memory.bin", paths["raptor_pim"] / "config.json", [tensor])
input_bin = paths["sim_dir"] / "input.bin"
input_bin = paths["sim_dir"] / "input_0.bin"
write_inputs_binary(input_bin, [tensor])
dump_ranges = build_dump_ranges(paths["raptor_pim"] / "config.json", output_descriptors)
@@ -147,7 +150,8 @@ def run_local_reference_and_simulator(args, network_dir: Path, network_onnx_path
paths["sim_bin"],
dump_ranges,
timeout_sec=args.command_timeout_seconds,
input_paths=[input_bin],
input_dir=input_bin.parent,
batch_size=1,
)
return paths, output_descriptors[0]
@@ -225,7 +229,7 @@ def main():
parser.add_argument(
"--annotated-dir",
type=Path,
default=defaults["network_dir"] / "real_image_validation" / "annotated",
default=defaults["network_dir"] / "artifacts" / "real_image_validation" / "annotated",
)
args = parser.parse_args()
validation/tools/yolo_real_image_validation.py
+4 -4
View File
@@ -262,7 +262,7 @@ def ensure_remote_artifacts(args):
def remote_case_paths(args, case_name: str):
network_dir = Path(args.network_dir)
root = Path(args.remote_project) / network_dir
root = Path(args.remote_project) / network_dir / "artifacts"
return {
"root": root,
"runner": root / "runner" / "build" / "runner",
@@ -272,7 +272,7 @@ def remote_case_paths(args, case_name: str):
"input_csv": root / "real_image_validation" / "inputs" / f"{case_name}.csv",
"ref_dir": root / "real_image_validation" / "reference" / case_name,
"sim_dir": root / "real_image_validation" / "simulation" / case_name,
"sim_input": root / "real_image_validation" / "simulation" / case_name / "input.bin",
"sim_input": root / "real_image_validation" / "simulation" / case_name / "input_0.bin",
"sim_bin": root / "real_image_validation" / "simulation" / case_name / "out.bin",
}
@@ -331,7 +331,7 @@ PY
f"cd {quoted_project}/backend-simulators/pim/pim-simulator && "
f"cargo run --no-default-features --release --package pim-simulator --bin pim-simulator -- "
f"-f {quoted_pim} -o {quoted_sim_bin} -d {dump_range} "
f"--mode latency --input {shlex.quote(str(paths['sim_input']))}"
f"--mode latency --batch-size 1 --input-dir {shlex.quote(str(paths['sim_input'].parent))}"
)
remote_bash(args.ssh_key, args.remote_host, sim_command)
return paths
@@ -408,7 +408,7 @@ def main():
parser.add_argument("--core-count", type=int, default=144)
parser.add_argument("--command-timeout-seconds", type=int, default=7200)
parser.add_argument("--skip-compile", action="store_true")
parser.add_argument("--annotated-dir", default="validation/networks/yolo11n/depth_51/real_image_validation/annotated")
parser.add_argument("--annotated-dir", default="validation/networks/yolo11n/depth_51/artifacts/real_image_validation/annotated")
args = parser.parse_args()
args.ssh_key = str(Path(args.ssh_key).expanduser())
validation/validate.py
+5 -5
View File
@@ -133,7 +133,7 @@ def print_average_pim_pass_timings(pass_timing_sums, pass_timing_counts, total_t
if timed_benchmark_count == 0:
return
print("\n" + Style.BRIGHT + Fore.CYAN + "Average PIM Pass Timings" + Style.RESET_ALL)
print("\n" + Style.BRIGHT + Fore.CYAN + "Average Pim pass timings" + Style.RESET_ALL)
for _, label in PIM_PASS_LABELS:
count = pass_timing_counts[label]
if count == 0:
@@ -210,7 +210,7 @@ def main():
ap.add_argument("--onnx-include-dir", help="Path to OnnxMlirRuntime include directory.")
ap.add_argument("--operations-dir", default=None, help="Root of the operations tree (default: operations).")
ap.add_argument("--simulator-dir", default=None,
help="Path to the functional pim-simulator crate root "
help="Path to the functional Pim simulator crate root "
"(default: auto-detected relative to script).")
ap.add_argument("--non-functional-simulator-build-dir", metavar="PATH", default=None,
help="Path to the non-functional simulator build directory "
@@ -221,7 +221,7 @@ def main():
ap.add_argument("--skip-non-functional-simulation", action="store_true",
help="Skip non-functional simulation.")
ap.add_argument("--no-fast", action="store_true",
help="Disable fast pimsim-nn throughput convergence for authoritative experiments.")
help="Disable fast Pimsim throughput convergence for authoritative experiments.")
ap.add_argument("--threshold", type=float, default=1e-3,
help="Absolute tolerance for per-element output comparison.")
ap.add_argument("--relative-threshold", type=float, default=1e-5,
@@ -243,7 +243,7 @@ def main():
help="Remove generated validation artifacts under each model workspace and exit.")
mode_group = ap.add_mutually_exclusive_group()
mode_group.add_argument("--compile-only", action="store_true",
help="Compile reference and PIM artifacts only; do not run reference execution, "
help="Compile reference and Pim artifacts only; do not run reference execution, "
"simulations, or comparison.")
mode_group.add_argument("--run-only", action="store_true",
help="Reuse existing compiled artifacts and only run inputs, reference execution, "
@@ -495,7 +495,7 @@ def main():
pimsim_unsupported = pimsim_statuses.count(PIMSIM_UNSUPPORTED)
print(
Style.BRIGHT
+ f"pimsim-nn: {pimsim_statuses.count(PIMSIM_DONE)} measured, "
+ f"Pimsim: {pimsim_statuses.count(PIMSIM_DONE)} measured, "
f"{pimsim_failed} failed, {pimsim_unsupported} unsupported, "
f"{pimsim_skipped} skipped"
+ Style.RESET_ALL