better compile times for throughput scheduling
Validate Operations / validate-operations (push) Has been cancelled

fix failing no-plan throughput ablation runs
update results
This commit is contained in:
NiccoloN
2026-10-28 16:34:48 +01:00
parent 3c7679594b
commit 5f2db946f0
28 changed files with 909 additions and 197 deletions
+24 -4
View File
@@ -63,8 +63,11 @@ lanes of an operation and computes one ordered partition with a prefix DP.
The DP jointly assigns operation prefixes and at most sixteen units of spare
core capacity. Each configured stage reserves one physical core. Quantization
limits initialization only; refinement uses geometric core transfers, including one core.
Quantization can miss a feasible allocation. Initialization failure reports a
bounded-search limitation, not proved hardware infeasibility.
If the quality grid has no feasible state, a minimum-resource prefix DP checks
every physical core count for each segment. This preserves feasibility over
the segment assignments found by the constructive packers, independently of
resource quantization. Packing failure remains a bounded-search limitation,
not proved hardware infeasibility.
The seed minimizes maximum stage packing/dependency priority over the legal
active-stage counts within the configured depth. These priorities provide
a starting point, not a timing guarantee. Refinement can remove stage boundaries.
@@ -76,6 +79,13 @@ constructive ordered-packing curves and uses reuse-first resident-weight packing
when order alone cannot realize a capacity bound. A feasible packing remains
feasible with idle extra cores. All placements pass the existing topology-aware
physical packing, operation-group, adjacency and capacity checks.
Ordered packing returns an unavailable cost when no ordered assignment fits.
Every finite feasibility cost has an ordered or reuse-first constructive assignment.
Initialization retains that assignment before adding idle spare cores. If topology-aware
greedy placement and reconstruction in physical task order fail, it uses the
retained assignment.
Task reordering cannot discard the initial partition's constructive capacity evidence.
This evidence does not prove final backend legality.
Refinement uses a beam of two schedules ranked by the larger queue estimate.
Its ordered neighborhood moves operation groups across adjacent stage boundaries,
@@ -163,7 +173,9 @@ Unsupported or incomplete target timing produces an `unavailable` report.
The analysis brackets the maximum cycle ratio within one nanosecond. It tests
edge delays `w - II*d`, sweeping the distance-zero DAG between recurrence
relaxations. At most `R+1` sweeps detect a positive cycle, where `R` counts
relaxations. A positive predecessor cycle can terminate an infeasible probe early.
The final witness pass retains the full relaxation procedure and deterministic
witness selection. At most `R+1` sweeps detect a positive cycle, where `R` counts
positive-distance edges. Bisection uses at most 64 tests. Timing outside the
supported numeric range produces an explicit unavailable result. A predecessor
cycle supplies a witness, including delay, iteration distance, cores, and links.
@@ -337,6 +349,11 @@ Functional simulation still executes Softmax. Estimated timing and energy exclud
The shared queue engine models controller arrival arbitration and cross-generation dependencies.
The compiler does not execute candidate programs or invoke a simulator.
Spatial extraction permits at most one million operation visits per physical core.
The total visit count is diagnostic; adding cores does not reduce another core's analysis allowance.
Abstract queue construction permits two million non-work event visits per candidate.
The complete-state recurrence limits remain unchanged.
Both arbitration policies must establish complete-state recurrence within bounded analysis budgets.
Their estimates are sensitivity checks, not formal bounds.
The candidate with the lowest larger estimate must separate from the initialized schedule.
@@ -362,6 +379,8 @@ If every candidate fails, one diagnostic lists all attempted candidates and caus
`reports/pipeline_selection.json` records Spatial costs, eligibility, selection, and backend attempts.
It also records search parents, compact task-stage runs, core budgets, recurrence contributions, termination, attempts, queue budget, and phase times.
Phase times separate neighborhood scoring, constructive packing, dependency-witness
analysis, and queue execution with its complete-state proof.
`--enable-timing` adds per-pass lowering times and IR operation/value counts.
Final instruction generation uses the normal compiler timing report.
Explicit Spatial exports and IR dumps reconstruct pre-lowering artifacts for the selected schedule.
@@ -424,7 +443,8 @@ fragment/consumer demands, and resource quantization `Q <= 16`.
| Weight indexing | Up to `O(R W)` equality comparisons |
| Weight lower bounds | `O(R + W)` / `O(W)` bits |
| Constructive best fit | `O(T log T + P R + P W)` / `O(P W)` bits plus `O(T + P)`; bounded retries only on failure |
| Joint initialization DP | `O(G² Q Kpack + S G² Q²)` time / `O(G² Q + S G Q)` storage; `Kpack` is one ordered-pack cost query |
| Joint initialization DP | `O(G² Q Kpack + S G² Q²)` time / `O(G² Q + S G Q)` storage; `Kpack` is one constructive pack-cost query |
| Feasibility completion, only if the quality grid fails | `O(G² P Kpack + S G²)` time / `O(G² + S G)` additional storage |
| Neighborhood memory service | Expected `O(T + F)` per partition / `O(T + F)` scratch |
| Topology clustering / cached PEFT distances | `O(P²)` time |
| Concat cost extraction | `O(A D)` for `A` inputs of rank `D`; constant auxiliary space |
+8 -1
View File
@@ -369,7 +369,7 @@ public:
std::vector<std::pair<size_t, Cost>> memoryServices;
std::array<Cost, pim_binary::kOpcodeCount> opcodeService{}, opcodeCounts{};
std::string error;
size_t abstractBudget = 1000000;
size_t abstractBudget = 2000000;
bool readCore(int64_t core, llvm::StringRef data, llvm::ArrayRef<int64_t> groups);
bool readExecution(int64_t core, const std::vector<PimExecutionEvent> &program);
@@ -845,13 +845,17 @@ bool Analysis::connect() {
EvaluatedAnalysis Analysis::evaluate() {
EvaluatedAnalysis result;
double dependencySeconds = 0;
if (cores.empty()) result.error = "artifact has no active repeating instruction stream";
else if (!connect()) result.error = error;
if (!result.error.empty()) { result.cost.reason = result.error; return result; }
if (options.detailed || options.searchGuidance) {
auto start = std::chrono::steady_clock::now();
auto recurrence = analyzePeriodicDependencies(events.size(), edges, result.error);
if (mlir::failed(recurrence)) { result.cost.reason = result.error; return result; }
result.recurrence = std::move(*recurrence);
dependencySeconds = std::chrono::duration<double>(
std::chrono::steady_clock::now() - start).count();
}
if (!options.detailed || memoryQueue) {
auto start = std::chrono::steady_clock::now();
@@ -885,6 +889,7 @@ EvaluatedAnalysis Analysis::evaluate() {
+= static_cast<double>(edge.delayNs) / distance;
}
}
result.cost.timing.dependencySeconds = dependencySeconds;
return result;
}
@@ -894,6 +899,8 @@ llvm::json::Object Analysis::report(const EvaluatedAnalysis &evaluation) const {
result["status"] = "unavailable"; result["reason"] = evaluation.error; return result;
}
result["global_memory_service_ns"] = static_cast<int64_t>(memory.serviceNs);
if (options.measureTime)
result["dependency_analysis_ms"] = evaluation.cost.timing.dependencySeconds * 1000;
result["global_read_bytes"] = static_cast<int64_t>(memory.readBytes);
result["global_write_bytes"] = static_cast<int64_t>(memory.writeBytes);
result["global_read_requests"] = static_cast<int64_t>(memory.readRequests);
+5 -1
View File
@@ -113,7 +113,7 @@ struct PimPipelineSelectionPass final : PassWrapper<PimPipelineSelectionPass, Op
PreparedSpatialSchedules &prepared, std::vector<WholeScheduleCost> &costs,
llvm::json::Object &report) {
llvm::json::Array summaries;
double realization = 0, extraction = 0, expansion = 0, queue = 0, evaluation = 0;
double realization = 0, extraction = 0, expansion = 0, queue = 0, evaluation = 0, dependency = 0;
auto evaluate = [&](const MergeScheduleResult &schedule) {
size_t index = summaries.size();
llvm::json::Object summary{{"index", static_cast<int64_t>(index)}};
@@ -131,6 +131,7 @@ struct PimPipelineSelectionPass final : PassWrapper<PimPipelineSelectionPass, Op
extraction += cost.timing.extractionSeconds;
expansion += cost.timing.expansionSeconds;
queue += cost.timing.evaluationSeconds;
dependency += cost.timing.dependencySeconds;
auto convergence = cost.convergenceProved(), support = cost.loweringSupported();
summary["cost_available"] = cost.available();
summary["convergence_proved"] = convergence
@@ -183,6 +184,7 @@ struct PimPipelineSelectionPass final : PassWrapper<PimPipelineSelectionPass, Op
report["event_extraction_seconds"] = extraction;
report["event_expansion_seconds"] = expansion;
report["queue_seconds"] = queue;
report["dependency_analysis_seconds"] = dependency;
report["candidates"] = std::move(summaries);
if (failed(search)) {
report["status"] = "incomplete-costs";
@@ -204,6 +206,8 @@ struct PimPipelineSelectionPass final : PassWrapper<PimPipelineSelectionPass, Op
report["single_candidate_analysis_skipped"] = single;
report["search_rounds"] = static_cast<int64_t>(search->rounds);
report["packing_attempts"] = static_cast<int64_t>(search->packingAttempts);
report["neighborhood_seconds"] = search->neighborhoodSeconds;
report["constructive_packing_seconds"] = search->packingSeconds;
report["considered_assignments"] = static_cast<int64_t>(search->consideredAssignments);
report["search_termination"] = search->termination;
report["queue_evaluation_budget"] = static_cast<int64_t>(kPipelineQueueEvaluationBudget);
+11 -1
View File
@@ -58,6 +58,10 @@ public:
program(target.processorCount) {}
bool build(ModuleOp module);
private:
// Bound interpretation per physical core, independently of batch lane count.
static constexpr size_t maximumCoreVisits = 1000000;
size_t coreVisits = 0;
int64_t currentCore = -1;
llvm::SmallDenseSet<int64_t, 16> assignedCores;
DenseMap<Value, Residence> residence;
DenseMap<Value, CopyAccessPlan> publicationAccess;
@@ -378,7 +382,12 @@ bool Extraction::operation(Operation &op, Sequence &sequence) {
bool Extraction::block(Block &body, Sequence &sequence) {
for (Operation &op : body) {
if (++visited > 1000000) { error = "abstract extraction budget exceeded"; return false; }
++visited;
if (++coreVisits > maximumCoreVisits) {
error = "abstract extraction budget exceeded on physical core "
+ std::to_string(currentCore);
return false;
}
if (!operation(op, sequence)) return false;
for (Value result : op.getResults())
if (auto outputs = directOutputs.find(result); outputs != directOutputs.end())
@@ -400,6 +409,7 @@ bool Extraction::core(Operation *op, int64_t physicalCore, unsigned lane) {
loweringSupport = StaticLoweringSupport::Unsupported;
error = "duplicate physical core assignment: " + std::to_string(physicalCore); return false;
}
coreVisits = 0; currentCore = physicalCore;
residence.clear(); publicationAccess.clear(); directOutputs.clear(); compactStorage.clear();
environment.bindings.clear();
ValueRange inputs, weights;
@@ -57,7 +57,29 @@ mlir::FailureOr<PeriodicDependencyEstimate> analyzePeriodicDependencies(
const size_t absent = std::numeric_limits<size_t>::max();
std::vector<double> potential(nodeCount);
std::vector<size_t> predecessor(nodeCount, absent);
auto infeasible = [&](double interval) {
std::vector<size_t> cycleSeen(nodeCount);
auto hasPositivePredecessorCycle = [&](double interval) {
std::fill(cycleSeen.begin(), cycleSeen.end(), absent);
for (size_t start = 0; start < nodeCount; ++start) {
size_t node = start;
while (cycleSeen[node] == absent && predecessor[node] != absent) {
cycleSeen[node] = start;
node = edges[predecessor[node]].source;
}
if (cycleSeen[node] != start || predecessor[node] == absent) continue;
const size_t first = node;
Cost delay = 0, distance = 0;
do {
const auto &edge = edges[predecessor[node]];
delay = checkedAdd(delay, edge.delayNs);
distance = checkedAdd(distance, static_cast<Cost>(edge.distance));
node = edge.source;
} while (node != first);
if (static_cast<double>(delay) - interval * distance > 1e-7) return true;
}
return false;
};
auto infeasible = [&](double interval, bool earlyCycle = true) {
std::fill(potential.begin(), potential.end(), 0);
std::fill(predecessor.begin(), predecessor.end(), absent);
auto relax = [&](size_t index) {
@@ -85,6 +107,9 @@ mlir::FailureOr<PeriodicDependencyEstimate> analyzePeriodicDependencies(
changed |= relax(index);
if (!changed)
return false;
// A positive predecessor cycle already proves infeasibility. Keep the
// final witness pass unchanged so search guidance remains deterministic.
if (earlyCycle && hasPositivePredecessorCycle(interval)) return true;
}
return true;
};
@@ -104,7 +129,7 @@ mlir::FailureOr<PeriodicDependencyEstimate> analyzePeriodicDependencies(
error = "periodic timing exceeds one-nanosecond floating-point precision";
return mlir::failure();
}
if (!infeasible(result.recurrenceLowerNs))
if (!infeasible(result.recurrenceLowerNs, false))
return result;
// Recover witness cycles from the last longest-path predecessor forest.
@@ -5,6 +5,7 @@
#include <algorithm>
#include <cmath>
#include <chrono>
#include <limits>
#include <numeric>
#include <optional>
@@ -71,6 +72,7 @@ struct PipelineGroup {
struct PipelineStageAssignment {
std::vector<size_t> taskStages;
std::vector<size_t> stageSizes;
std::vector<size_t> constructivePacks;
};
static bool consumesPipelineInput(const ComputeGraphNode &node) {
@@ -229,7 +231,7 @@ static FailureOr<PipelineStageAssignment> initialPipelinePartition(
std::string &error) {
size_t count = constraints.groups.size(), stages = layout.getStageCount();
if (!count) return PipelineStageAssignment{{},
{layout.getStageSizes().begin(), layout.getStageSizes().end()}};
{layout.getStageSizes().begin(), layout.getStageSizes().end()}, {}};
size_t extra = layout.getProcessorCount() - stages;
// Resource quantization bounds seed construction only. Refinement uses
// individual physical cores and final allocation uses every available core.
@@ -259,9 +261,9 @@ static FailureOr<PipelineStageAssignment> initialPipelinePartition(
for (size_t amount = 0; amount <= units; ++amount) {
size_t cores = coreCount(amount);
if (weights.count() > checkedMultiply(cores, residentWeightCapacity)) continue;
costs[start][end][amount] = checkedAdd(span - largest,
estimateOrderedPackCost(model.schedulingCosts, model.weights.taskWeightIds,
model.weights.weightCount, tasks, residentWeightCapacity, cores));
if (auto packing = model.weights.estimatePackCost(tasks,
model.schedulingCosts, residentWeightCapacity, cores))
costs[start][end][amount] = checkedAdd(span - largest, *packing);
}
}
}
@@ -290,8 +292,53 @@ static FailureOr<PipelineStageAssignment> initialPipelinePartition(
&& (!selected || best[stage][count][units].priority
< best[selected][count][units].priority))
selected = stage;
// Quantization controls quality, not feasibility. Complete the same prefix
// DP with minimum constructive core counts if no quantized state survives.
std::vector<std::vector<Cell>> feasible;
if (!selected) {
feasible.resize(stages + 1, std::vector<Cell>(count + 1));
feasible[0][0].valid = true;
std::vector<std::vector<size_t>> minimum(count + 1,
std::vector<size_t>(count + 1, std::numeric_limits<size_t>::max()));
for (size_t start = 0; start < count; ++start) {
TaskList tasks;
for (size_t end = start + 1; end <= count; ++end) {
llvm::append_range(tasks, constraints.groups[constraints.groupOrder[end - 1]].tasks);
const size_t lower = std::max<size_t>(1,
model.weights.getPackCountLowerBound(tasks, residentWeightCapacity));
const size_t maximum = layout.getProcessorCount() - stages + 1;
for (size_t cores = lower; cores <= maximum; ++cores)
if (model.weights.estimatePackCost(tasks, model.schedulingCosts,
residentWeightCapacity, cores)) {
minimum[start][end] = cores;
break;
}
}
}
for (size_t stage = 0; stage < stages; ++stage)
for (size_t start = 0; start < count; ++start) {
if (!feasible[stage][start].valid) continue;
for (size_t end = start + 1; end <= count; ++end) {
if (stage == 0 && end < constraints.minimumStageZeroEnd) continue;
if (stage && constraints.hasCrossingEdge
&& constraints.furthestSuccessorBefore[start] >= end) continue;
const size_t cores = minimum[start][end];
if (cores > layout.getProcessorCount()) continue;
Cost used = checkedAdd(feasible[stage][start].priority, static_cast<Cost>(cores));
if (used > layout.getProcessorCount() - (stages - stage - 1)) continue;
auto &current = feasible[stage + 1][end];
if (!current.valid || used < current.priority)
current = {used, start, cores, true};
}
}
for (size_t stage = 1; stage <= stages; ++stage)
if (feasible[stage][count].valid
&& (!selected || feasible[stage][count].priority < feasible[selected][count].priority))
selected = stage;
}
if (!selected && count) {
error = "pipeline initialization found no dependency-adjacent operation partition within its bounded core-budget search";
error = "pipeline initialization found no constructively packable dependency-adjacent partition; "
"bounded packing failure does not prove physical infeasibility";
return failure();
}
PipelineStageAssignment result;
@@ -299,13 +346,31 @@ static FailureOr<PipelineStageAssignment> initialPipelinePartition(
result.taskStages.resize(constraints.taskToGroup.size());
size_t end = count, remaining = units;
for (size_t stage = selected; stage; --stage) {
const auto &cell = best[stage][end][remaining];
result.stageSizes[stage - 1] = coreCount(cell.units);
const auto &cell = feasible.empty() ? best[stage][end][remaining] : feasible[stage][end];
result.stageSizes[stage - 1] = feasible.empty() ? coreCount(cell.units) : cell.units;
for (size_t position = cell.start; position < end; ++position)
for (size_t task : constraints.groups[constraints.groupOrder[position]].tasks)
result.taskStages[task] = stage - 1;
end = cell.start;
remaining -= cell.units;
if (feasible.empty()) remaining -= cell.units;
}
// Preserve the constructive witness before adding spare cores. Reordering
// tasks for topology-aware placement must not lose DP feasibility evidence.
result.constructivePacks.resize(model.schedulingCosts.size());
std::vector<TaskList> stageTasks(selected);
for (size_t group : constraints.groupOrder)
for (size_t task : constraints.groups[group].tasks)
stageTasks[result.taskStages[task]].push_back(task);
for (size_t stage = 0; stage < selected; ++stage) {
const auto &tasks = stageTasks[stage];
TaskList packs;
if (!model.weights.estimatePackCost(tasks, model.schedulingCosts,
residentWeightCapacity, result.stageSizes[stage], &packs)) {
error = "pipeline initializer could not reconstruct its constructive packing witness";
return failure();
}
for (auto [index, task] : llvm::enumerate(tasks))
result.constructivePacks[task] = packs[index];
}
size_t assigned = std::accumulate(result.stageSizes.begin(), result.stageSizes.end(), size_t{0});
for (; assigned < layout.getProcessorCount(); ++assigned) {
@@ -431,7 +496,7 @@ static LogicalResult packPipelineStages(
std::vector<TaskList> &tasksByCpu, const PipelineCoreLayout &layout,
size_t pipelineStages, size_t residentWeightCapacity,
const SchedulingTarget &target, size_t &failedStage,
std::string &error) {
std::string &error, ArrayRef<size_t> constructivePacks) {
std::vector<size_t> indegree(graph.nodes.size());
std::vector<TaskList> successors(graph.nodes.size());
for (size_t task = 0; task < graph.nodes.size(); ++task)
@@ -493,13 +558,11 @@ static LogicalResult packPipelineStages(
llvm::sort(tasks, [&](size_t lhs, size_t rhs) {
return topologicalPosition[lhs] < topologicalPosition[rhs];
});
estimateOrderedPackCost(model.schedulingCosts, model.weights.taskWeightIds,
model.weights.weightCount, tasks, residentWeightCapacity, range.size, &taskPacks);
if (taskPacks.size() != tasks.size()) {
auto assignment = model.weights.pack(
tasks, model.schedulingCosts, residentWeightCapacity, range.size);
if (assignment)
taskPacks = std::move(*assignment);
model.weights.estimatePackCost(tasks, model.schedulingCosts,
residentWeightCapacity, range.size, &taskPacks);
if (taskPacks.size() != tasks.size() && !constructivePacks.empty()) {
taskPacks.clear();
for (size_t task : tasks) taskPacks.push_back(constructivePacks[task]);
}
packed = taskPacks.size() == tasks.size() && packPipelineStage(
graph, model.schedulingCosts, tasksByCpu,
@@ -508,9 +571,9 @@ static LogicalResult packPipelineStages(
}
if (!packed) {
failedStage = stage;
error = "pipeline resident-weight heuristic search could not pack stage "
error = "pipeline constructive physical-placement search could not pack stage "
+ std::to_string(stage)
+ " within its physical core budget";
+ " within its physical core budget; bounded search failure does not prove physical infeasibility";
return failure();
}
}
@@ -633,7 +696,7 @@ static LogicalResult packPipelineAssignment(const ComputeGraph &graph,
if (succeeded(packPipelineStages(
graph, taskModel, tasksByPhysicalCpu, candidateLayout,
pipelineStages, physicalTarget.residentWeightCapacity,
physicalTarget, failedStage, packingError))) {
physicalTarget, failedStage, packingError, assignment.constructivePacks))) {
packed = true;
break;
}
@@ -651,6 +714,7 @@ static LogicalResult packPipelineAssignment(const ComputeGraph &graph,
donor = stage;
if (!donor)
break;
assignment.constructivePacks.clear();
--assignment.stageSizes[*donor];
++assignment.stageSizes[failedStage];
}
@@ -748,24 +812,10 @@ static std::optional<PipelineStageAssignment> allocatePipelineStageCores(
largest = std::max(largest, model.schedulingCosts[task]);
}
for (size_t cores = minimum; cores <= target.processorCount; ++cores) {
TaskList packs;
Cost packedCost = std::max(largest, work[stage] / cores + (work[stage] % cores != 0));
if (constructPacks)
packedCost = estimateOrderedPackCost(model.schedulingCosts,
model.weights.taskWeightIds, model.weights.weightCount, tasks[stage],
target.residentWeightCapacity, cores, &packs);
if (constructPacks && packs.size() != tasks[stage].size()) {
auto constructive = model.weights.pack(tasks[stage], model.schedulingCosts,
target.residentWeightCapacity, cores);
packedCost = std::numeric_limits<Cost>::max();
if (constructive) {
TaskCosts loads(cores);
for (size_t i = 0; i < tasks[stage].size(); ++i)
loads[(*constructive)[i]] = checkedAdd(loads[(*constructive)[i]],
model.schedulingCosts[tasks[stage][i]]);
packedCost = *llvm::max_element(loads);
}
}
packedCost = model.weights.estimatePackCost(tasks[stage], model.schedulingCosts,
target.residentWeightCapacity, cores).value_or(std::numeric_limits<Cost>::max());
// A feasible assignment remains feasible when extra cores stay idle.
if (cores > minimum) packedCost = std::min(packedCost, costs[stage][cores - 1]);
costs[stage][cores] = packedCost;
@@ -1060,9 +1110,8 @@ static std::vector<PipelineProposal> pipelineNeighborhood(const ComputeGraph &gr
auto packingCost = [&](const TaskList &tasks, size_t cores, size_t stage) {
auto *cached = tasks == originalView.tasks[stage] ? &packCosts[stage][cores] : nullptr;
if (cached && *cached) return **cached;
Cost value = estimateOrderedPackCost(model.schedulingCosts,
model.weights.taskWeightIds, model.weights.weightCount, tasks,
target.residentWeightCapacity, cores);
Cost value = model.weights.estimatePackCost(tasks, model.schedulingCosts,
target.residentWeightCapacity, cores).value_or(std::numeric_limits<Cost>::max());
if (cached) *cached = value;
return value;
};
@@ -1093,6 +1142,7 @@ static std::vector<PipelineProposal> pipelineNeighborhood(const ComputeGraph &gr
if (model.weights.getPackCountLowerBound(view.tasks[stage],
target.residentWeightCapacity) > assignment.stageSizes[stage]) return;
Cost work = packingCost(view.tasks[stage], assignment.stageSizes[stage], stage);
if (work == std::numeric_limits<Cost>::max()) return;
maximum = std::max(maximum, work);
// A within-generation span is latency, not an initiation-interval bound.
total = checkedAdd(total, view.spans[stage]);
@@ -1174,7 +1224,10 @@ mlir::FailureOr<PipelineSearchResult> optimizePipelineSchedule(
std::set<PipelineMove> visited;
std::set<TaskList> placements{placementIdentity(graph, initial)};
std::vector<size_t> beam{0};
auto neighborhoodStart = std::chrono::steady_clock::now();
auto proposals = pipelineNeighborhood(graph, executionModel, *groups, result, beam, visited, 2 * evaluationBudget);
result.neighborhoodSeconds = std::chrono::duration<double>(
std::chrono::steady_clock::now() - neighborhoodStart).count();
if (proposals.empty() || evaluationBudget == 1) {
result.termination = "one neighborhood schedule; queue comparison skipped";
return result;
@@ -1197,7 +1250,10 @@ mlir::FailureOr<PipelineSearchResult> optimizePipelineSchedule(
if (!error.empty()) return failure();
constexpr size_t maximumPackingAttempts = 4 * kPipelineQueueEvaluationBudget;
while (result.costs.size() < evaluationBudget && result.packingAttempts < maximumPackingAttempts) {
neighborhoodStart = std::chrono::steady_clock::now();
proposals = pipelineNeighborhood(graph, executionModel, *groups, result, beam, visited, 2 * evaluationBudget);
result.neighborhoodSeconds += std::chrono::duration<double>(
std::chrono::steady_clock::now() - neighborhoodStart).count();
if (proposals.empty()) break;
++result.rounds;
size_t evaluated = 0;
@@ -1211,17 +1267,25 @@ mlir::FailureOr<PipelineSearchResult> optimizePipelineSchedule(
++result.packingAttempts;
visited.emplace(proposal.parent, proposal.assignment.taskStages,
proposal.assignment.stageSizes, proposal.optimizeCores, proposal.adjustableStages);
auto packingStart = std::chrono::steady_clock::now();
auto assignment = proposal.assignment;
if (proposal.optimizeCores) {
auto allocated = allocatePipelineStageCores(graph, executionModel, assignment.taskStages,
pipelineStages, target, assignment.stageSizes, proposal.adjustableStages);
if (!allocated) continue;
if (!allocated) {
result.packingSeconds += std::chrono::duration<double>(
std::chrono::steady_clock::now() - packingStart).count();
continue;
}
assignment = std::move(*allocated);
}
MergeScheduleResult schedule;
std::string packingError;
if (failed(packPipelineAssignment(graph, model, assignment, schedule,
pipelineStages, target, packingError))) continue;
auto packed = packPipelineAssignment(graph, model, assignment, schedule,
pipelineStages, target, packingError);
result.packingSeconds += std::chrono::duration<double>(
std::chrono::steady_clock::now() - packingStart).count();
if (failed(packed)) continue;
if (!placements.insert(placementIdentity(graph, schedule)).second) continue;
auto estimate = cost(schedule);
if (!error.empty()) return failure();
@@ -1261,7 +1325,7 @@ mlir::LogicalResult packPipelineSchedule(const ComputeGraph &graph,
}
PipelineTaskModel model(graph, target);
return packPipelineAssignment(graph, model,
{{taskStages.begin(), taskStages.end()}, {stageSizes.begin(), stageSizes.end()}},
{{taskStages.begin(), taskStages.end()}, {stageSizes.begin(), stageSizes.end()}, {}},
schedule, stageSizes.size(), target, error);
}
@@ -114,6 +114,7 @@ struct PipelineSearchResult {
std::vector<WholeScheduleCost> costs;
std::vector<size_t> parents;
size_t rounds = 0, packingAttempts = 0, consideredAssignments = 0;
double neighborhoodSeconds = 0, packingSeconds = 0;
std::string termination;
};
@@ -11,15 +11,18 @@
namespace onnx_mlir::spatial {
// Greedy maximal prefixes minimize ordered pack count: both constraints are
// closed under taking subranges. The optimum cost cannot increase with packs.
inline Cost estimateOrderedPackCost(
// closed under taking subranges. Feasible optimum costs cannot increase with packs.
inline std::optional<Cost> estimateOrderedPackCost(
llvm::ArrayRef<Cost> taskCosts,
const std::vector<std::vector<size_t>> &taskWeightIds, size_t weightCount,
llvm::ArrayRef<size_t> tasks, size_t residentWeightCapacity,
size_t maximumPacks, std::vector<size_t> *taskPacks = nullptr) {
if (taskPacks) taskPacks->clear();
if (!maximumPacks) return std::nullopt;
Cost low = 0;
Cost high = 0;
for (size_t task : tasks) {
if (taskWeightIds[task].size() > residentWeightCapacity) return std::nullopt;
low = std::max(low, taskCosts[task]);
high = checkedAdd(high, taskCosts[task]);
}
@@ -65,6 +68,7 @@ inline Cost estimateOrderedPackCost(
}
return fits;
};
if (!pack(high, nullptr)) return std::nullopt;
while (low < high) {
Cost middle = low + (high - low) / 2;
if (pack(middle, nullptr))
@@ -73,9 +77,7 @@ inline Cost estimateOrderedPackCost(
low = middle + 1;
}
if (taskPacks) {
taskPacks->clear();
if (!pack(low, taskPacks))
taskPacks->clear();
pack(low, taskPacks);
}
return low;
}
@@ -113,6 +115,22 @@ struct IndexedResidentWeights {
return std::max<size_t>(!tasks.empty(), count / capacity + (count % capacity != 0));
}
std::optional<Cost> estimatePackCost(llvm::ArrayRef<size_t> tasks,
llvm::ArrayRef<Cost> costs, size_t capacity, size_t packCount,
std::vector<size_t> *taskPacks = nullptr) const {
if (taskPacks) taskPacks->clear();
if (!packCount) return std::nullopt;
if (auto ordered = estimateOrderedPackCost(costs, taskWeightIds,
weightCount, tasks, capacity, packCount, taskPacks)) return ordered;
auto assignment = pack(tasks, costs, capacity, packCount);
if (!assignment) return std::nullopt;
std::vector<Cost> loads(packCount);
for (auto [index, task] : llvm::enumerate(tasks))
loads[(*assignment)[index]] = checkedAdd(loads[(*assignment)[index]], costs[task]);
if (taskPacks) *taskPacks = std::move(*assignment);
return *llvm::max_element(loads);
}
// A constructive reuse-first best fit; failure is not a proof of infeasibility.
std::optional<std::vector<size_t>> pack(
llvm::ArrayRef<size_t> tasks, llvm::ArrayRef<Cost> costs,
@@ -67,6 +67,7 @@ struct WholeScheduleCost {
std::vector<double> processorTaskWorkNs, processorCriticalDelayNs;
struct AnalysisTiming {
double extractionSeconds = 0, expansionSeconds = 0, evaluationSeconds = 0;
double dependencySeconds = 0;
} timing;
std::optional<bool> convergenceProved() const;
std::optional<bool> loweringSupported() const;
+1
View File
@@ -69,6 +69,7 @@ add_pim_unittest(SpatialSchedulingTargetTest
add_pim_unittest(PeriodicDependenciesTest
PeriodicDependenciesTest.cpp
PeriodicDependenciesReference.cpp
LINK_LIBS PRIVATE
OMPimCompilerUtils
+142
View File
@@ -0,0 +1,142 @@
// Test-only reference from commit 3c7679594beb29ba9497aaf3e341cee74fa0be50.
#include "src/Accelerators/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PeriodicDependencies.hpp"
#include <limits>
namespace onnx_mlir::spatial {
mlir::FailureOr<PeriodicDependencyEstimate> analyzeFrozenPeriodicDependencies(
size_t nodeCount, llvm::ArrayRef<PeriodicDependency> edges,
std::string &error) {
std::vector<std::vector<size_t>> outgoing(nodeCount);
std::vector<size_t> indegree(nodeCount), recurrent;
Cost upper = 0;
for (auto [index, edge] : llvm::enumerate(edges)) {
if (edge.source >= nodeCount || edge.target >= nodeCount) {
error = "periodic dependency references an invalid event";
return mlir::failure();
}
if (edge.delayNs > std::numeric_limits<Cost>::max() - upper) {
error = "periodic delay sum exceeds the timing range";
return mlir::failure();
}
upper += edge.delayNs;
if (edge.distance) {
recurrent.push_back(index);
} else {
outgoing[edge.source].push_back(index);
++indegree[edge.target];
}
}
std::vector<size_t> order;
for (size_t node = 0; node < nodeCount; ++node)
if (!indegree[node])
order.push_back(node);
for (size_t cursor = 0; cursor < order.size(); ++cursor)
for (size_t index : outgoing[order[cursor]])
if (!--indegree[edges[index].target])
order.push_back(edges[index].target);
if (order.size() != nodeCount) {
error = "periodic distance-zero dependencies contain a cycle";
return mlir::failure();
}
PeriodicDependencyEstimate result;
std::vector<Cost> span(nodeCount);
for (size_t node : order) {
result.withinGenerationSpanNs =
std::max(result.withinGenerationSpanNs, span[node]);
for (size_t index : outgoing[node]) {
const auto &edge = edges[index];
span[edge.target] = std::max(span[edge.target],
checkedAdd(span[node], edge.delayNs));
}
}
if (recurrent.empty() || !upper)
return result;
const size_t absent = std::numeric_limits<size_t>::max();
std::vector<double> potential(nodeCount);
std::vector<size_t> predecessor(nodeCount, absent);
auto infeasible = [&](double interval) {
std::fill(potential.begin(), potential.end(), 0);
std::fill(predecessor.begin(), predecessor.end(), absent);
auto relax = [&](size_t index) {
const auto &edge = edges[index];
double value = potential[edge.source] + edge.delayNs
- interval * edge.distance;
if (value <= potential[edge.target] + 1e-7)
return false;
potential[edge.target] = value;
predecessor[edge.target] = index;
return true;
};
for (size_t node : order)
for (size_t index : outgoing[node])
relax(index);
// Eliminate distance-zero paths by a DAG sweep. A simple residual path
// visits at most R recurrent edges, so R+1 rounds detect a positive cycle.
for (size_t round = 0; round <= recurrent.size(); ++round) {
++result.relaxationRounds;
bool changed = false;
for (size_t index : recurrent)
changed |= relax(index);
for (size_t node : order)
for (size_t index : outgoing[node])
changed |= relax(index);
if (!changed)
return false;
}
return true;
};
result.recurrenceUpperNs = upper;
for (unsigned iteration = 0;
iteration < 64 && result.recurrenceUpperNs - result.recurrenceLowerNs > 1;
++iteration) {
double middle = (result.recurrenceLowerNs + result.recurrenceUpperNs) / 2;
if (middle == result.recurrenceLowerNs || middle == result.recurrenceUpperNs)
break;
if (infeasible(middle))
result.recurrenceLowerNs = middle;
else
result.recurrenceUpperNs = middle;
}
if (result.recurrenceUpperNs - result.recurrenceLowerNs > 1) {
error = "periodic timing exceeds one-nanosecond floating-point precision";
return mlir::failure();
}
if (!infeasible(result.recurrenceLowerNs))
return result;
// Recover witness cycles from the last longest-path predecessor forest.
std::vector<size_t> seen(nodeCount, absent);
double best = 0;
for (size_t start = 0; start < nodeCount; ++start) {
size_t node = start;
while (seen[node] == absent && predecessor[node] != absent) {
seen[node] = start;
node = edges[predecessor[node]].source;
}
if (seen[node] != start || predecessor[node] == absent)
continue;
size_t first = node;
Cost delay = 0, distance = 0;
std::vector<size_t> cycle;
do {
size_t index = predecessor[node];
cycle.push_back(index);
delay = checkedAdd(delay, edges[index].delayNs);
distance = checkedAdd(distance, static_cast<Cost>(edges[index].distance));
node = edges[index].source;
} while (node != first);
double ratio = distance ? static_cast<double>(delay) / distance : 0;
if (ratio > best) {
best = ratio;
std::reverse(cycle.begin(), cycle.end());
result.criticalCycle = std::move(cycle);
}
}
return result;
}
} // namespace onnx_mlir::spatial
+96
View File
@@ -5,11 +5,96 @@
#error "Periodic dependency tests require assertions"
#endif
#include <cmath>
#include <chrono>
#include <iostream>
#include <functional>
#include <random>
using namespace onnx_mlir::spatial;
namespace onnx_mlir::spatial {
mlir::FailureOr<PeriodicDependencyEstimate> analyzeFrozenPeriodicDependencies(
size_t, llvm::ArrayRef<PeriodicDependency>, std::string &);
}
static size_t differentialCases = 0, acceleratedRounds = 0, frozenRounds = 0;
static double acceleratedSeconds = 0, frozenSeconds = 0;
static void differential(size_t count, const std::vector<PeriodicDependency> &edges) {
std::string actualError, referenceError;
auto start = std::chrono::steady_clock::now();
auto actual = analyzePeriodicDependencies(count, edges, actualError);
acceleratedSeconds += std::chrono::duration<double>(std::chrono::steady_clock::now() - start).count();
start = std::chrono::steady_clock::now();
auto reference = analyzeFrozenPeriodicDependencies(count, edges, referenceError);
frozenSeconds += std::chrono::duration<double>(std::chrono::steady_clock::now() - start).count();
++differentialCases;
assert(mlir::succeeded(actual) == mlir::succeeded(reference));
assert(actualError == referenceError);
if (mlir::failed(actual)) return;
acceleratedRounds += actual->relaxationRounds;
frozenRounds += reference->relaxationRounds;
assert(actual->recurrenceLowerNs == reference->recurrenceLowerNs);
assert(actual->recurrenceUpperNs == reference->recurrenceUpperNs);
assert(actual->withinGenerationSpanNs == reference->withinGenerationSpanNs);
assert(actual->criticalCycle == reference->criticalCycle);
}
static void testFrozenDifferential() {
std::mt19937 random(171221);
for (size_t sample = 0; sample < 800; ++sample) {
const size_t count = sample < 600 ? 2 + random() % 31 : 64 + random() % 193;
std::vector<PeriodicDependency> edges;
for (size_t source = 0; source < count; ++source) {
const size_t component = source / 32;
for (size_t link = 0; link < 4; ++link) {
size_t target = random() % count;
if (sample % 3 == 0 && target / 32 != component) continue;
unsigned distance = target > source && link % 2 == 0 ? 0 : 1 + random() % 7;
edges.push_back({source, target, random() % 100001, distance});
}
}
differential(count, edges);
}
for (size_t sample = 0; sample < 100; ++sample) {
std::vector<PeriodicDependency> edges;
for (size_t node = 0; node < 32; ++node) {
edges.push_back({node, node, (Cost{1} << 40) + random() % 10001,
static_cast<unsigned>(1024 + random() % 7)});
edges.push_back({node, (node + 1) % 32, random() % 101, 1});
}
differential(32, edges);
}
differential(2, {{0,1,1,0}, {1,0,1000000000000ULL, std::numeric_limits<unsigned>::max()}});
for (size_t count : {2u, 64u, 1024u}) {
std::vector<PeriodicDependency> edges;
for (size_t node = 0; node < count; ++node) {
edges.push_back({node, node, 1000000 + node % 2, 3});
if (node + 1 < count) edges.push_back({node, node + 1, 1, 0});
}
differential(count, edges);
}
std::vector<PeriodicDependency> large;
constexpr size_t componentSize = 2048, components = 32;
for (size_t component = 0; component < components; ++component) {
size_t first = component * componentSize;
for (size_t node = first; node + 1 < first + componentSize; ++node)
large.push_back({node, node + 1, 1, 0});
large.push_back({first + componentSize - 1, first, 17 + component,
static_cast<unsigned>(1 + component % 7)});
}
differential(componentSize * components, large);
differential(0, {});
differential(2, {{0, 1, 0, 0}, {1, 0, 0, 0}});
differential(2, {{0, 1, 10, 0}, {1, 0, 3, 0}});
differential(1, {{0, 1, 1, 1}});
differential(1, {{0, 0, Cost{1} << 60, 1}});
differential(1, {{0, 0, std::numeric_limits<Cost>::max(), 1}, {0, 0, 1, 1}});
std::cout << "frozen_differential_cases=" << differentialCases
<< " accelerated_rounds=" << acceleratedRounds << " frozen_rounds=" << frozenRounds
<< " accelerated_seconds=" << acceleratedSeconds << " frozen_seconds=" << frozenSeconds << '\n';
}
static double enumerateCycles(size_t count,
const std::vector<PeriodicDependency> &edges) {
double maximum = 0;
@@ -59,11 +144,22 @@ static void check(size_t count, const std::vector<PeriodicDependency> &edges,
}
int main() {
testFrozenDifferential();
check(3, {{0, 1, 12, 0}, {1, 2, 7, 0}}, 0);
check(3, {{0, 1, 12, 1}, {1, 2, 7, 1}}, 0);
check(1, {{0, 0, 30, 2}}, 15);
// Independent core periods remain parallel.
check(2, {{0, 0, 10, 1}, {1, 1, 20, 1}}, 20);
// Positive-cycle checks must not change the optimum or the final witness.
std::vector<PeriodicDependency> independent;
for (size_t core = 0; core < 128; ++core)
independent.push_back({core, core, 1000 + core, 1});
check(128, independent, 1127);
std::string performanceError;
auto accelerated = analyzePeriodicDependencies(128, independent, performanceError);
assert(mlir::succeeded(accelerated));
assert(accelerated->criticalCycle == std::vector<size_t>{127});
assert(accelerated->relaxationRounds < 512);
// Equal serial demand, different cross-core blocking recurrences.
check(4, {{0, 1, 10, 0}, {2, 3, 10, 0},
{1, 0, 0, 1}, {3, 2, 0, 1}}, 10);
+54
View File
@@ -243,6 +243,17 @@ static void testSharedEngine(PimInstructionTiming target) {
actual = analyzePimExecution(abstract, target);
TEST_CHECK(actual.getInteger("global_read_requests") == 2);
TEST_CHECK(actual.getInteger("global_memory_service_ns") == 160);
// The construction limit applies to interpreted patterns, not just traffic.
// These zero-trip request bodies do not allocate a large event graph.
abstract[0] = {{E::Work, 4}, {E::Repeat, 1500000, 0, 0, 0,
{{E::Repeat, 0, 0, 0, 0, {{E::Load, 1}}}}}};
auto largerPattern = evaluatePimExecution(abstract, target);
TEST_CHECK(largerPattern.cost.available());
TEST_CHECK(largerPattern.report.getInteger("global_read_requests") == 0);
abstract[0] = {{E::Repeat, 2000001, 0, 0, 0, {{E::Load, 1}}}};
auto aboveBudget = evaluatePimExecution(abstract, target);
TEST_CHECK(!aboveBudget.cost.available());
TEST_CHECK(aboveBudget.cost.reason == "abstract repetition budget exceeded");
abstract[0] = {{E::Repeat, 1000000000000ull, 0, 0, 0, {{E::Load, 1}}}};
actual = analyzePimExecution(abstract, target);
TEST_CHECK(actual.getString("status") == "unavailable");
@@ -433,6 +444,48 @@ static void testDependencyTokens() {
TEST_CHECK(drained.empty() && drained.remaining().empty());
}
static void testSpatialExtractionBudget(PimInstructionTiming target) {
MLIRContext context;
context.loadDialect<arith::ArithDialect, func::FuncDialect, scf::SCFDialect,
tensor::TensorDialect, spatial::SpatialDialect>();
auto module = ModuleOp::create(UnknownLoc::get(&context));
OpBuilder builder(module.getBodyRegion());
auto loc = module.getLoc();
auto function = func::FuncOp::create(builder, loc, "test", builder.getFunctionType({}, {}));
builder.setInsertionPointToStart(function.addEntryBlock());
auto batch = spatial::SpatScheduledComputeBatch::create(builder, loc, TypeRange{},
builder.getI32IntegerAttr(2), ValueRange{}, ValueRange{});
batch->setAttr("coreIds", builder.getDenseI32ArrayAttr({0, 1}));
batch.getBody().push_back(new Block);
batch.getBody().front().addArgument(builder.getIndexType(), loc);
builder.setInsertionPointToStart(&batch.getBody().front());
Value zero = arith::ConstantIndexOp::create(builder, loc, 0);
Value upper = arith::ConstantIndexOp::create(builder, loc, 100000);
Value one = arith::ConstantIndexOp::create(builder, loc, 1);
auto type = RankedTensorType::get({16}, builder.getF32Type());
Value local = tensor::EmptyOp::create(builder, loc, type.getShape(), type.getElementType());
spatial::SpatReluOp::create(builder, loc, type, local);
auto loop = scf::ForOp::create(builder, loc, zero, upper, one);
builder.setInsertionPointToStart(loop.getBody());
for (int64_t i = 0; i < 5; ++i) arith::ConstantIndexOp::create(builder, loc, i);
// A zero-count wait is runtime-free, but requires indexed interpretation.
spatial::SpatWaitOp::create(builder, loc, zero, zero);
builder.setInsertionPointAfter(loop);
spatial::SpatYieldOp::create(builder, loc, ValueRange{});
builder.setInsertionPointAfter(batch);
func::ReturnOp::create(builder, loc);
TEST_CHECK(succeeded(verify(module)));
auto supported = evaluatePimSpatialSchedule(module, target);
TEST_CHECK(supported.report.getInteger("visited_operations") > 1000000);
TEST_CHECK(supported.cost.available());
builder.setInsertionPointToStart(loop.getBody());
for (int64_t i = 0; i < 5; ++i) arith::ConstantIndexOp::create(builder, loc, i);
auto unsupported = evaluatePimSpatialSchedule(module, target);
TEST_CHECK(!unsupported.cost.available());
TEST_CHECK(unsupported.cost.reason == "abstract extraction budget exceeded on physical core 0");
module.erase();
}
static void testReadOnlySpatial(PimInstructionTiming target, bool batched) {
MLIRContext context;
context.loadDialect<arith::ArithDialect, func::FuncDialect, scf::SCFDialect,
@@ -918,6 +971,7 @@ int main() {
testCapturedVectorWeights(hardware(period));
testPublicationAndOutput(hardware(period));
testUnrepresentableSpatial(hardware(period));
testSpatialExtractionBudget(hardware(period));
testReadOnlySpatial(hardware(period), false);
testReadOnlySpatial(hardware(period), true);
}
+250 -1
View File
@@ -3,6 +3,8 @@
#include "src/Accelerators/PIM/Compiler/PimInstructionTiming.hpp"
#include "src/Accelerators/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/PipelineScheduling.hpp"
#include "src/Accelerators/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/ScheduleEvaluation.hpp"
#include "src/Accelerators/PIM/Dialect/Spatial/Passes/Transforms/MergeComputeNodes/Scheduling/ResidentWeightPacking.hpp"
#include "src/Accelerators/PIM/Dialect/Spatial/SpatialOps.hpp"
#include <chrono>
#include <functional>
#include <map>
@@ -14,6 +16,76 @@ using namespace onnx_mlir::spatial;
using E = PimExecutionEvent;
static bool sawJointMove = false, sawSuccessiveMoves = false;
static void testPackingFeasibility() {
ComputeGraph graph;
for (size_t task = 0; task < 4; ++task) {
ResidentWeight weight;
weight.opaqueLane = task % 2;
graph.nodes.push_back({{nullptr, static_cast<uint32_t>(task), 1}, 10, {weight}, task});
}
IndexedResidentWeights weights(graph);
const std::vector<size_t> tasks{0, 1, 2, 3};
const std::vector<Cost> work{10, 10, 10, 10};
std::vector<size_t> assignment{99};
TEST_CHECK(!estimateOrderedPackCost(work, weights.taskWeightIds, weights.weightCount,
tasks, 1, 2, &assignment));
TEST_CHECK(assignment.empty());
TEST_CHECK(!estimateOrderedPackCost(work, weights.taskWeightIds, weights.weightCount,
tasks, 1, 0));
TEST_CHECK(weights.estimatePackCost(tasks, work, 1, 2) == 20);
auto placement = weights.pack(tasks, work, 1, 2);
TEST_CHECK(placement && (*placement)[0] == (*placement)[2]
&& (*placement)[1] == (*placement)[3] && (*placement)[0] != (*placement)[1]);
TEST_CHECK(!weights.estimatePackCost(tasks, work, 1, 1));
for (size_t cores = 4; cores <= 8; ++cores)
TEST_CHECK(estimateOrderedPackCost(work, weights.taskWeightIds, weights.weightCount,
tasks, 1, cores) == 10);
weights.taskWeightIds[0] = {0, 1};
TEST_CHECK(!estimateOrderedPackCost(work, weights.taskWeightIds, weights.weightCount,
tasks, 1, 4));
weights.taskWeightIds = {{0, 1}, {2}, {0, 1}, {2}};
weights.weightCount = 3;
TEST_CHECK(!estimateOrderedPackCost(work, weights.taskWeightIds, weights.weightCount,
tasks, 2, 2));
TEST_CHECK(weights.estimatePackCost(tasks, work, 2, 2) == 20);
}
static void testQuantizedFeasibility(uint32_t first) {
mlir::MLIRContext context;
context.loadDialect<spatial::SpatialDialect>();
auto module = mlir::ModuleOp::create(mlir::UnknownLoc::get(&context));
mlir::OpBuilder builder(module.getBodyRegion());
ComputeGraph graph;
graph.predecessors.resize(40); graph.successors.resize(40);
for (uint32_t group = 0; group < 2; ++group) {
const uint32_t lanes = group ? 40 - first : first;
auto batch = spatial::SpatGraphComputeBatch::create(builder, module.getLoc(),
mlir::TypeRange{}, builder.getI32IntegerAttr(lanes), mlir::ValueRange{}, mlir::ValueRange{});
for (uint32_t lane = 0; lane < lanes; ++lane) {
size_t task = graph.nodes.size();
ResidentWeight weight; weight.opaqueLane = task;
graph.nodes.push_back({{batch, lane, 1}, 10, {weight}, task});
if (group) {
graph.predecessors[task].push_back({0, {0, 0}});
graph.successors[0].push_back({task, {0, 0}});
}
}
}
SchedulingTarget target;
target.processorCount = 40; target.residentWeightCapacity = 1;
target.interProcessorLatencyNs.assign(1600, 0);
MergeScheduleResult schedule;
std::string error;
TEST_CHECK(mlir::succeeded(runPipelineScheduler(graph, schedule, 2, target, error)));
std::vector<size_t> budgets(2);
for (size_t stage : schedule.processorStages) ++budgets[stage];
TEST_CHECK(budgets == std::vector<size_t>({first, 40 - first}));
for (size_t task = 0; task < graph.nodes.size(); ++task)
TEST_CHECK(schedule.processorStages[schedule.computeToCpuMap.lookup(graph.nodes[task].instance)]
== (task < first ? 0 : 1));
module.erase();
}
// Test-only workloads use stage-boundary host transfers and local direct links.
// The same queue
// engine and complete-state evidence rank reference and searched schedules.
@@ -112,6 +184,176 @@ static PimInstructionTiming hardware(size_t profile) {
return target;
}
static void testInitializerPackingWitness() {
mlir::MLIRContext context;
context.loadDialect<spatial::SpatialDialect>();
auto module = mlir::ModuleOp::create(mlir::UnknownLoc::get(&context));
mlir::OpBuilder builder(module.getBodyRegion());
std::vector<mlir::Operation *> operations;
for (size_t i = 0; i < 3; ++i) operations.push_back(
spatial::SpatGraphComputeBatch::create(builder, module.getLoc(),
mlir::TypeRange{}, builder.getI32IntegerAttr(3), mlir::ValueRange{}, mlir::ValueRange{}));
const std::vector<std::vector<size_t>> resident{{2,1,5},{2,4},{0,2},{0},{4,6},{1},{2},{2},{1,4}};
const std::vector<Cost> costs{4,1,5,4,1,2,5,2,2};
ComputeGraph graph;
graph.predecessors.resize(9); graph.successors.resize(9);
for (uint32_t task = 0; task < 9; ++task) {
ResidentWeightSet weights;
for (size_t id : resident[task]) {
ResidentWeight w; w.opaqueLane = id; weights.push_back(w);
}
graph.nodes.push_back({{operations[task % 3], task / 3, 1}, costs[task], weights, task});
}
IndexedResidentWeights indexed(graph);
TEST_CHECK(!indexed.estimatePackCost({1,2,4,5,7,8}, costs, 3, 2));
std::vector<size_t> witness;
TEST_CHECK(indexed.estimatePackCost({1,4,7,2,5,8}, costs, 3, 2, &witness));
TEST_CHECK(witness.size() == 6);
auto target = hardware(1);
target.processorCount = 4; target.residentWeightCapacity = 3;
target.interProcessorLatencyNs.assign(16, 1);
MergeScheduleResult schedule;
std::string error;
TEST_CHECK(mlir::succeeded(runPipelineScheduler(graph, schedule, 2, target, error)));
std::vector<ResidentWeightSet> actual(4);
for (const auto &node : graph.nodes) {
size_t core = schedule.computeToCpuMap.lookup(node.instance);
TEST_CHECK(core < 4);
insertResidentWeights(actual[core], node.residentWeights);
TEST_CHECK(actual[core].size() <= 3);
}
// Exhaust all core assignments under the same stage map. The initialized
// assignment must belong to this physical-capacity reference set.
size_t feasible = 0;
for (size_t encoded = 0; encoded < (size_t{1} << 18); ++encoded) {
size_t choices = encoded;
std::vector<ResidentWeightSet> weights(4);
std::vector<size_t> groupStages(3, 2);
bool valid = true;
for (size_t task = 0; task < 9 && valid; ++task) {
size_t core = choices % 4; choices /= 4;
size_t stage = schedule.processorStages[core], group = task % 3;
if (groupStages[group] == 2) groupStages[group] = stage;
valid &= groupStages[group] == stage;
insertResidentWeights(weights[core], graph.nodes[task].residentWeights);
valid &= weights[core].size() <= 3;
}
feasible += valid;
}
TEST_CHECK(feasible > 0);
MergeScheduleResult alternative;
TEST_CHECK(mlir::succeeded(packPipelineSchedule(graph, std::vector<size_t>(9),
{3, 1}, target, alternative, error)));
std::cout << "initializer_physical_feasible_assignments=" << feasible << '\n';
module.erase();
}
static void testPhysicalReference() {
auto target = hardware(1);
target.processorCount = 3;
target.sharedMemoryRequestLatencyNs.resize(3);
target.sharedMemoryResponseLatencyNs.resize(3);
target.interProcessorLatencyNs = {0, 2, 11, 2, 0, 7, 11, 7, 0};
ComputeGraph graph;
graph.predecessors.resize(3); graph.successors.resize(3);
for (uint32_t task = 0; task < 3; ++task) {
ResidentWeight weight; weight.opaqueLane = task % 2;
graph.nodes.push_back({{nullptr, task, 1}, task == 1 ? 700u : 80u, {weight}, task});
if (task) {
graph.predecessors[task].push_back({task - 1, {64, 1}});
graph.successors[task - 1].push_back({task, {64, 1}});
}
}
double optimum = std::numeric_limits<double>::infinity();
size_t evaluations = 0;
for (size_t mask = 1; mask < 7; ++mask)
for (size_t placement = 0; placement < 27; ++placement) {
MergeScheduleResult schedule;
for (size_t core = 0; core < 3; ++core) schedule.processorStages.push_back((mask >> core) & 1);
size_t encoded = placement;
std::vector<size_t> stages, slots(3);
std::vector<ResidentWeightSet> resident(3);
bool valid = true;
for (const auto &node : graph.nodes) {
size_t core = encoded % 3; encoded /= 3;
schedule.computeToCpuMap[node.instance] = core;
schedule.computeToCpuSlotMap[node.instance] = slots[core]++;
stages.push_back(schedule.processorStages[core]);
insertResidentWeights(resident[core], node.residentWeights);
valid &= resident[core].size() <= target.residentWeightCapacity;
}
if (!valid || stages[0] != 0 || stages[1] < stages[0] || stages[2] < stages[1]) continue;
auto cost = queueCost(graph, schedule, target);
optimum = std::min(optimum, std::max(*cost.primary, *cost.alternative));
++evaluations;
}
std::string error;
MergeScheduleResult initial;
TEST_CHECK(mlir::succeeded(runPipelineScheduler(graph, initial, 2, target, error)));
auto evaluate = [&](const MergeScheduleResult &schedule) { return queueCost(graph, schedule, target); };
auto search = optimizePipelineSchedule(graph, initial, 2, target, evaluate, error);
TEST_CHECK(mlir::succeeded(search));
size_t winner = search->costs.empty() ? 0 : selectPipelineCandidate(search->costs, error);
auto selected = evaluate(search->schedules[winner]);
double value = std::max(*selected.primary, *selected.alternative);
TEST_CHECK(value + 1e-6 >= optimum);
std::cout << "physical_reference_ns=" << optimum << " selected_ns=" << value
<< " gap_percent=" << 100 * (value / optimum - 1)
<< " physical_evaluations=" << evaluations
<< " search_evaluations=" << search->costs.size()
<< " packing_attempts=" << search->packingAttempts << '\n';
}
static void testWorseningIntermediate() {
// A controlled cost oracle isolates the search's non-monotone landscape.
// Hardware queue objectives are covered separately by runReference().
auto target = hardware(0);
target.processorCount = 8;
target.interProcessorLatencyNs.assign(64, 0);
ComputeGraph graph;
graph.predecessors.resize(4); graph.successors.resize(4);
for (uint32_t task = 0; task < 4; ++task)
graph.nodes.push_back({{nullptr, task, 1}, 10, {}, task});
auto evaluate = [&](const MergeScheduleResult &schedule) {
bool firstStage = llvm::all_of(graph.nodes, [&](const auto &node) {
return schedule.processorStages[schedule.computeToCpuMap.lookup(node.instance)] == 0;
});
size_t cores = llvm::count(schedule.processorStages, 0);
double value = !firstStage ? 120 : cores == 1 ? 80 : cores >= 6 ? 100 : 110;
WholeScheduleCost cost;
cost.primary = cost.alternative = value;
cost.loweringSupport = StaticLoweringSupport::Supported;
cost.selectionMargin = kPipelineSelectionMargin;
for (auto &proof : cost.recurrence)
proof = {ScheduleRecurrenceEvidence::Proved, 1, value, {}};
return cost;
};
std::string error;
MergeScheduleResult initial, intermediate;
TEST_CHECK(mlir::succeeded(runPipelineScheduler(graph, initial, 2, target, error)));
TEST_CHECK(evaluate(initial).primary == 100);
const std::vector<size_t> stages(4);
TEST_CHECK(mlir::succeeded(packPipelineSchedule(graph, stages, {3, 5}, target, intermediate, error)));
TEST_CHECK(evaluate(intermediate).primary == 110);
double reference = std::numeric_limits<double>::infinity();
for (size_t cores = 1; cores < 8; ++cores) {
MergeScheduleResult schedule;
TEST_CHECK(mlir::succeeded(packPipelineSchedule(graph, stages, {cores, 8 - cores}, target, schedule, error)));
reference = std::min(reference, *evaluate(schedule).primary);
}
TEST_CHECK(reference == 80);
auto search = optimizePipelineSchedule(graph, initial, 2, target, evaluate, error);
TEST_CHECK(mlir::succeeded(search));
size_t winner = selectPipelineCandidate(search->costs, error);
double selected = *search->costs[winner].primary;
TEST_CHECK(selected >= reference && selected <= 100);
TEST_CHECK(search->costs.size() <= kPipelineQueueEvaluationBudget);
std::cout << "non_monotone_oracle_reference=" << reference << " selected=" << selected
<< " gap_percent=" << 100 * (selected / reference - 1)
<< " evaluations=" << search->costs.size()
<< " packing_attempts=" << search->packingAttempts << '\n';
}
static void runReference(size_t profile, size_t fixture) {
auto target = hardware(profile);
ComputeGraph graph;
@@ -217,8 +459,10 @@ static void runReference(size_t profile, size_t fixture) {
<< " gap_percent=" << 100 * (cost / reference - 1)
<< " best_evaluated_ns=" << bestEvaluated
<< " best_evaluated_gap_percent=" << 100 * (bestEvaluated / reference - 1)
<< " reference_scope=logical_with_deterministic_packing"
<< " reference_evaluations=" << referenceEvaluations
<< " search_evaluations=" << search->costs.size() << " refinement_depth=" << depth
<< " search_evaluations=" << search->costs.size()
<< " packing_attempts=" << search->packingAttempts << " refinement_depth=" << depth
<< " search_seconds=" << searchSeconds << " reference_seconds=" << referenceSeconds
<< " total_seconds=" << std::chrono::duration<double>(std::chrono::steady_clock::now() - begin).count()
<< '\n';
@@ -246,6 +490,11 @@ static void runReference(size_t profile, size_t fixture) {
}
int main() {
testInitializerPackingWitness();
testPackingFeasibility();
for (uint32_t first : {19u, 21u}) testQuantizedFeasibility(first);
testPhysicalReference();
testWorseningIntermediate();
for (size_t profile = 0; profile < 2; ++profile)
for (size_t fixture = 0; fixture < 6; ++fixture) runReference(profile, fixture);
TEST_CHECK(sawJointMove && sawSuccessiveMoves);
+6 -6
View File
@@ -1642,14 +1642,14 @@ static void testOrderedPackCostCurve() {
if (fits && packs <= cores) optimum = std::min(optimum, std::max(maximum, work));
}
std::vector<size_t> assignment;
Cost cost = estimateOrderedPackCost(costs, weights, 3, tasks, capacity, cores, &assignment);
auto cost = estimateOrderedPackCost(costs, weights, 3, tasks, capacity, cores, &assignment);
if (optimum == std::numeric_limits<Cost>::max()) {
assert(assignment.empty());
assert(!cost && assignment.empty());
continue;
}
assert(cost == optimum && cost <= previous && assignment.size() == tasks.size());
plateau |= cost == previous;
previous = cost;
assert(cost && *cost == optimum && *cost <= previous && assignment.size() == tasks.size());
plateau |= *cost == previous;
previous = *cost;
std::vector<Cost> loads(cores);
std::vector<llvm::SmallBitVector> resident(cores, llvm::SmallBitVector(3));
for (size_t task : tasks) {
@@ -1658,7 +1658,7 @@ static void testOrderedPackCostCurve() {
for (size_t weight : weights[task]) resident[assignment[task]].set(weight);
}
for (size_t core = 0; core < cores; ++core) {
assert(loads[core] <= cost && resident[core].count() <= capacity);
assert(loads[core] <= *cost && resident[core].count() <= capacity);
}
}
assert(plateau && previous == 11);
+1 -1
View File
@@ -78,7 +78,7 @@ def main():
parser.add_argument("--compiler", required=True)
args = parser.parse_args()
compiler = str(Path(args.compiler).resolve())
model = REPO / "validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx"
model = REPO / "validation/networks/pimcomp_models/googlenet/googlenet-12-no-lrn.onnx"
with tempfile.TemporaryDirectory(prefix="raptor-static-selection-") as temporary:
root = Path(temporary)
history = root / "validation/tools/pim/pimcomp/compare/results_comparison.csv"
+5 -12
View File
@@ -11,11 +11,11 @@ operations, standalone commands, configuration layout, and GA/OpenMP controls.
| Directory | Model / input | Source |
|---|---|---|
| `resnet18/` | ResNet-18 v1, `1x3x224x224` | [ONNX Model Zoo resnet18-v1-7](https://huggingface.co/onnxmodelzoo/resnet18-v1-7); symlink to `../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet18/` | ResNet-18 v1, `1x3x224x224` | [ONNX Model Zoo resnet18-v1-7](https://huggingface.co/onnxmodelzoo/resnet18-v1-7); symlink to `../../resnet18/depth_68/resnet18_depth_68.onnx`. |
| `resnet34/` | ResNet-34 v1, `1x3x224x224` | [ONNX Model Zoo resnet34-v1-7](https://huggingface.co/onnxmodelzoo/resnet34-v1-7); symbolic batch fixed to 1. |
| `googlenet/` | GoogLeNet, `1x3x224x224` | [ONNX Model Zoo googlenet-12](https://huggingface.co/onnxmodelzoo/googlenet-12), with only its two LRN nodes bypassed. |
| `vgg8/` | VGG-8 reconstruction, `1x1x28x28` | Six convolution and two fully connected layers with deterministic random weights. |
| `yolo11n/` | YOLO11n detection, `1x3x640x640` | Uses `../yolo11n/depth_51/yolo11n_depth_51.onnx`; this directory retains the historical Softmax-free variant. |
| `yolo11n/` | YOLO11n detection, `1x3x640x640` | [Ultralytics YOLO11n](https://docs.ultralytics.com/models/yolo11/); symlink to `../../yolo11n/depth_51/yolo11n_depth_51.onnx`. |
The paper's checkpoints were not published. The three Model Zoo networks
produce frontend graphs matching Pimcomp's bundled JSON graphs. VGG-8 is
@@ -29,15 +29,9 @@ Its terminal Softmax, output `prob_1`, weights, and other operations are retaine
The native reference uses this same graph. VGG8 and both ResNet graphs have no
removed operators; ResNet-34 fixes the symbolic batch size to one.
Historical performance models change the original computation:
- `googlenet/googlenet-12-pimsim-nn.onnx` removes two LRN nodes and terminal
Softmax, which Pimcomp does not schedule.
- `yolo11n/yolo11n-pimsim-nn.onnx` removes two internal Softmax nodes. It remains
available for reproducing historical measurements.
The current YOLO comparison uses `../yolo11n/depth_51/yolo11n_depth_51.onnx`,
The current YOLO comparison uses `yolo11n/yolo11n.onnx`,
including both Softmax operations. Its native reference uses the same full model.
The checked-in ONNX graph uses opset 18. The original exporter version and checkpoint hash are not recorded.
The functional simulator executes Softmax. Pimsim-nn skips `vsoftmax` before
fetch, and Raptor's PIM queue estimator omits the same instruction cost.
Timing and energy exclude Softmax work; they do not model hardware Softmax support.
@@ -50,9 +44,8 @@ Model SHA-256 checksums:
c3231061d081bdd47884137b02134f85142752a39e87263c529cd14ed242b096 resnet34/resnet34-v1-7.onnx
c99c507058eaf41de8723408fdda7db8325cb57f0a89f2ee07a716d6e963e14e googlenet/googlenet-12.onnx
c960c195953423388733cd0e60712cf425ca1f8917e264f5cf98e028752c61cf googlenet/googlenet-12-no-lrn.onnx
a26f9e33901c573e60c34a3f0abbb4744fff83e4e0f21b18fc66e20395e72982 googlenet/googlenet-12-pimsim-nn.onnx
396cdea21e5e7d02c3f26f14d22ef20975171702493f5c5e79b8e0d896e541ef vgg8/vgg8-mnist-reconstructed.onnx
229f3975af8933d39aee8d9031d969bff074b69c33e304a78abb35ff0c5f445f yolo11n/yolo11n-pimsim-nn.onnx
5e7bd85dc60b252674f8d7e1ad24f2317d00dfb8c524826536698462de04620d yolo11n/yolo11n.onnx
```
## Hardware profiles
@@ -0,0 +1 @@
../../yolo11n/depth_51/yolo11n_depth_51.onnx
@@ -11,7 +11,7 @@ MODELS = {
"resnet18": SUITE / "resnet18/resnet18-v1-7.onnx",
"resnet34": SUITE / "resnet34/resnet34-v1-7.onnx",
"googlenet": SUITE / "googlenet/googlenet-12-no-lrn.onnx",
"yolo11n": REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx",
"yolo11n": SUITE / "yolo11n/yolo11n.onnx",
}
MODEL_NAMES = tuple(MODELS)
DEFAULT_MODELS = MODEL_NAMES
+43 -1
View File
@@ -511,7 +511,9 @@ def test_comparison_uses_same_model_for_reference_and_execution(tmp_path, monkey
def test_yolo_uses_full_model_without_moving_comparison_artifacts():
assert suite.MODELS["yolo11n"] == suite.REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx"
assert suite.MODELS["yolo11n"] == suite.SUITE / "yolo11n/yolo11n.onnx"
assert suite.MODELS["yolo11n"].is_symlink()
assert suite.MODELS["yolo11n"].resolve() == suite.REPO / "validation/networks/yolo11n/depth_51/yolo11n_depth_51.onnx"
assert suite.model_dir(None, "yolo11n") == suite.SUITE / "yolo11n"
assert suite.common_dir(None, "yolo11n") == suite.SUITE / "yolo11n/artifacts/common-full-model"
@@ -756,3 +758,43 @@ def test_ablation_aggregate_keeps_pass_with_zero_performance(tmp_path):
assert rows[0]["functional_validation"] == "PASS"
assert rows[0]["baseline_functional_validation"] == "PASS"
assert all(rows[0][f"{metric}_gain_percent"] == "NA" for _, metric in ablation.PERCENTAGE_FIELDS)
def test_completed_no_sync_ablation_preserves_numerical_failure():
report = {"pimsim_mode": "throughput", "raptor_extra_args": ["--pim-disable-synchronization"],
"raptor_validation": {"status": "done", "passed": False, "max_diffs": {"output": 1}},
"raptor_performance": {"throughput": 100, "output_count": 8},
"steps": [{"name": "Functional validation Raptor", "status": "passed"}],
"failures": []}
assert suite.completed_synchronization_ablation(report)
assert not suite.comparison_passed(report)
assert suite.functional_validation_status(report["raptor_validation"]) == "FAIL"
for field, value in (("pimsim_mode", "latency"), ("raptor_extra_args", []),
("raptor_validation", {"status": "failed", "passed": False}),
("raptor_validation", {"status": "done", "passed": False}),
("raptor_performance", {"error": "SIGSEGV", "throughput": 100}),
("raptor_performance", {"throughput": 100, "output_count": 0}),
("steps", [{"status": "failed"}]),
("failures", [{"stage": "Compile Raptor Pim", "error": "failed"}])):
assert not suite.completed_synchronization_ablation({**report, field: value})
@pytest.mark.parametrize("process_failed", [False, True])
@pytest.mark.parametrize("variant,expected", [("no-sync", 0), ("no-plan", 1), ("baseline", 1)])
def test_ablation_runner_distinguishes_invalid_outputs_from_failed_execution(tmp_path, monkeypatch, variant, expected, process_failed):
report = {"pimsim_mode": "throughput", "raptor_extra_args": ["--pim-disable-synchronization"],
"raptor_validation": {"status": "done", "passed": False, "max_diffs": {"output": 1}},
"raptor_performance": {"throughput": 100, "output_count": 8},
"steps": [{"status": "passed"}], "failures": []}
output = suite.result_dir(tmp_path, "vgg8", "arch-a", "throughput", 4, variant) / "pimcomp/comparison_report.json"
output.parent.mkdir(parents=True)
output.write_text(json.dumps(report))
csv = tmp_path / "results.csv"
csv.write_text("summary\n")
monkeypatch.setattr(suite, "run", lambda *a, **k: 0)
monkeypatch.setattr(suite, "run_comparison_jobs", lambda *a: (["arch-a/vgg8/throughput/pipeline4"] if process_failed else [], 1))
monkeypatch.setattr(suite, "write_results_csv", lambda *a: csv)
monkeypatch.setattr(sys, "argv", ["run_pimcomp_models.py", "--out-dir", str(tmp_path),
"--models", "vgg8", "--archs", "arch-a", "--mode", "throughput", "--pipeline", "4",
"--raptor-only", "--ablation-variant", variant, "--jobs", "1"])
assert suite.main() == (1 if process_failed else expected)
+4 -2
View File
@@ -59,8 +59,10 @@ Both programs' functional statuses are recorded as `functional_validation`
and `baseline_functional_validation`. The latter belongs to the disabled feature.
Disabling synchronization removes host-buffer lifetime and pipeline restart
guarantees, so those variants may fail numerical validation and exit nonzero.
Baseline and `no-plan` must pass. Percentages involving failing programs remain
guarantees, so those variants may fail numerical validation. A completed
`no-sync` measurement retains `FAIL` in the CSV and is reported as diagnostic
in the runner summary. Compile, execution, missing-output, and measurement
failures still make the run fail. Baseline and `no-plan` must pass. Percentages involving failing programs remain
diagnostic comparisons, not valid speedups.
When fast-mode convergence fails, positive fallback throughput remains available.
@@ -1,65 +1,25 @@
variant,arch,model,mode,raptor_pipeline,functional_validation,latency,throughput,power,energy
baseline,arch-a,googlenet,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-a,googlenet,latency,1,PASS,+0.00%,NA,+0.00%,-0.00%
no-plan,arch-a,googlenet,latency,1,PASS,+11513.29%,NA,-69.42%,+3452.69%
no-sync-no-plan,arch-a,googlenet,latency,1,PASS,+11512.99%,NA,-69.42%,+3452.59%
baseline,arch-a,googlenet,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-a,googlenet,throughput,4,FAIL,+26.17%,-20.74%,+9.97%,+18.50%
no-plan,arch-a,googlenet,throughput,4,PASS,-100.00%,-100.00%,-62.73%,-100.00%
no-sync-no-plan,arch-a,googlenet,throughput,4,FAIL,+4094.63%,-97.62%,-59.86%,+1311.01%
baseline,arch-a,resnet18,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-a,resnet18,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-plan,arch-a,resnet18,latency,1,PASS,+3191.30%,NA,-60.97%,+1184.24%
no-sync-no-plan,arch-a,resnet18,latency,1,PASS,+3191.30%,NA,-60.97%,+1184.24%
baseline,arch-a,resnet18,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-a,resnet18,throughput,4,FAIL,+110.65%,-52.55%,+43.25%,+177.48%
no-plan,arch-a,resnet18,throughput,4,PASS,+1598.93%,-94.12%,-51.04%,+664.87%
no-sync-no-plan,arch-a,resnet18,throughput,4,FAIL,+1174.21%,-92.16%,-45.06%,+543.74%
baseline,arch-a,resnet34,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-a,resnet34,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-plan,arch-a,resnet34,latency,1,PASS,+2361.37%,NA,-56.53%,+969.99%
no-sync-no-plan,arch-a,resnet34,latency,1,PASS,+2361.37%,NA,-56.53%,+969.99%
baseline,arch-a,resnet34,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-a,resnet34,throughput,4,FAIL,+33.65%,-25.21%,+31.76%,+15.56%
no-plan,arch-a,resnet34,throughput,4,PASS,+1726.82%,-94.52%,-54.50%,+455.98%
no-sync-no-plan,arch-a,resnet34,throughput,4,FAIL,+813.41%,-89.04%,-49.77%,+206.84%
baseline,arch-a,vgg8,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-a,vgg8,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-plan,arch-a,vgg8,latency,1,PASS,+1800.66%,NA,-44.65%,+953.39%
no-sync-no-plan,arch-a,vgg8,latency,1,PASS,+1800.66%,NA,-44.65%,+953.39%
baseline,arch-a,vgg8,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-a,vgg8,throughput,4,PASS,+2.33%,-1.72%,+18.49%,-0.98%
no-plan,arch-a,vgg8,throughput,4,PASS,+1251.16%,-92.58%,-56.68%,+577.04%
no-sync-no-plan,arch-a,vgg8,throughput,4,PASS,+1260.47%,-92.63%,-53.36%,+467.83%
baseline,arch-b,googlenet,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-b,googlenet,latency,1,PASS,+0.00%,NA,+0.00%,-0.00%
no-plan,arch-b,googlenet,latency,1,PASS,+9849.64%,NA,-70.51%,+2833.89%
no-sync-no-plan,arch-b,googlenet,latency,1,PASS,+9849.55%,NA,-70.51%,+2833.89%
baseline,arch-b,googlenet,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-b,googlenet,throughput,4,FAIL,+11.45%,-10.29%,+21.82%,+19.27%
no-plan,arch-b,googlenet,throughput,4,PASS,-100.00%,-100.00%,-63.39%,-100.00%
no-sync-no-plan,arch-b,googlenet,throughput,4,FAIL,+3348.28%,-97.10%,-61.58%,+1026.09%
baseline,arch-b,resnet18,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-b,resnet18,latency,1,PASS,+0.00%,NA,+0.00%,-0.00%
no-plan,arch-b,resnet18,latency,1,PASS,+2935.63%,NA,-63.12%,+1019.81%
no-sync-no-plan,arch-b,resnet18,latency,1,PASS,+2936.09%,NA,-63.12%,+1019.97%
baseline,arch-b,resnet18,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-b,resnet18,throughput,4,FAIL,+8.03%,-7.44%,+47.59%,+40.61%
no-plan,arch-b,resnet18,throughput,4,PASS,+1511.85%,-93.80%,-52.93%,+560.80%
no-sync-no-plan,arch-b,resnet18,throughput,4,FAIL,+867.12%,-89.67%,-45.41%,+359.83%
baseline,arch-b,resnet34,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-b,resnet34,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-plan,arch-b,resnet34,latency,1,PASS,+2551.16%,NA,-62.51%,+893.70%
no-sync-no-plan,arch-b,resnet34,latency,1,PASS,+2550.91%,NA,-62.51%,+893.62%
baseline,arch-b,resnet34,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-b,resnet34,throughput,4,FAIL,+28.25%,-22.13%,+49.76%,+57.30%
no-plan,arch-b,resnet34,throughput,4,PASS,+2427.17%,-96.05%,-49.58%,+938.62%
no-sync-no-plan,arch-b,resnet34,throughput,4,FAIL,+531.79%,-84.19%,-42.32%,+197.08%
baseline,arch-b,vgg8,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-sync,arch-b,vgg8,latency,1,PASS,+0.00%,NA,+0.00%,+0.00%
no-plan,arch-b,vgg8,latency,1,PASS,+1784.97%,NA,-49.50%,+849.73%
no-sync-no-plan,arch-b,vgg8,latency,1,PASS,+1784.97%,NA,-49.50%,+849.73%
baseline,arch-b,vgg8,throughput,4,PASS,+0.00%,+0.00%,+0.00%,+0.00%
no-sync,arch-b,vgg8,throughput,4,PASS,+23.91%,-19.82%,+25.67%,+23.93%
no-plan,arch-b,vgg8,throughput,4,PASS,+1158.70%,-92.07%,-58.95%,+408.33%
no-sync-no-plan,arch-b,vgg8,throughput,4,PASS,+1172.83%,-92.17%,-55.83%,+327.00%
comparison,arch,model,mode,raptor_pipeline,baseline_variant,baseline_functional_validation,functional_validation,latency_gain_percent,throughput_gain_percent,power_gain_percent,energy_gain_percent
plan,arch-a,googlenet,latency,1,no-plan,PASS,PASS,+99.14%,NA,-226.98%,+97.19%
plan,arch-a,googlenet,throughput,4,no-plan,PASS,PASS,NA,NA,-281.83%,NA
sync,arch-a,googlenet,throughput,4,no-sync,FAIL,PASS,-5.17%,-4.96%,+12.23%,-1.65%
plan,arch-a,resnet18,latency,1,no-plan,PASS,PASS,+96.96%,NA,-156.23%,+92.21%
plan,arch-a,resnet18,throughput,4,no-plan,PASS,PASS,+95.17%,+1970.00%,-157.48%,+85.39%
sync,arch-a,resnet18,throughput,4,no-sync,FAIL,PASS,+7.15%,+7.81%,+14.70%,+7.92%
plan,arch-a,resnet34,latency,1,no-plan,PASS,PASS,+95.94%,NA,-130.05%,+90.65%
plan,arch-a,resnet34,throughput,4,no-plan,PASS,PASS,+96.00%,+2400.00%,-189.18%,+88.43%
sync,arch-a,resnet34,throughput,4,no-sync,FAIL,PASS,-9.11%,-8.42%,+7.32%,-1.02%
plan,arch-a,vgg8,latency,1,no-plan,PASS,PASS,+94.75%,NA,-80.65%,+90.51%
plan,arch-a,vgg8,throughput,4,no-plan,PASS,PASS,+92.77%,+1279.31%,-170.46%,+82.49%
sync,arch-a,vgg8,throughput,4,no-sync,FAIL,PASS,+19.18%,+24.03%,+21.29%,+18.62%
plan,arch-b,googlenet,latency,1,no-plan,PASS,PASS,+98.99%,NA,-239.10%,+96.59%
plan,arch-b,googlenet,throughput,4,no-plan,PASS,PASS,NA,NA,-306.99%,NA
sync,arch-b,googlenet,throughput,4,no-sync,FAIL,PASS,-1.28%,-1.63%,+9.99%,-4.29%
plan,arch-b,resnet18,latency,1,no-plan,PASS,PASS,+96.71%,NA,-171.16%,+91.07%
plan,arch-b,resnet18,throughput,4,no-plan,PASS,PASS,+95.46%,+2102.50%,-200.52%,+83.26%
sync,arch-b,resnet18,throughput,4,no-sync,FAIL,PASS,+2.22%,+2.32%,+8.26%,-7.88%
plan,arch-b,resnet34,latency,1,no-plan,PASS,PASS,+96.23%,NA,-166.76%,+89.94%
plan,arch-b,resnet34,throughput,4,no-plan,PASS,PASS,+97.58%,+4030.00%,-168.71%,+91.79%
sync,arch-b,resnet34,throughput,4,no-sync,FAIL,PASS,-0.80%,-0.96%,+10.61%,-13.51%
plan,arch-b,vgg8,latency,1,no-plan,PASS,PASS,+94.68%,NA,-98.04%,+89.47%
plan,arch-b,vgg8,throughput,4,no-plan,PASS,PASS,+93.70%,+1484.16%,-205.54%,+81.37%
sync,arch-b,vgg8,throughput,4,no-sync,FAIL,PASS,+17.07%,+20.30%,+19.03%,+9.63%
1 variant comparison arch model mode raptor_pipeline baseline_variant latency baseline_functional_validation throughput functional_validation power latency_gain_percent energy throughput_gain_percent power_gain_percent energy_gain_percent
2 baseline plan arch-a googlenet latency 1 no-plan +0.00% PASS NA PASS +0.00% +99.14% +0.00% NA -226.98% +97.19%
3 no-sync plan arch-a googlenet latency throughput 1 4 no-plan +0.00% PASS NA PASS +0.00% NA -0.00% NA -281.83% NA
4 no-plan sync arch-a googlenet latency throughput 1 4 no-sync +11513.29% FAIL NA PASS -69.42% -5.17% +3452.69% -4.96% +12.23% -1.65%
5 no-sync-no-plan plan arch-a googlenet resnet18 latency 1 no-plan +11512.99% PASS NA PASS -69.42% +96.96% +3452.59% NA -156.23% +92.21%
6 baseline plan arch-a googlenet resnet18 throughput 4 no-plan +0.00% PASS +0.00% PASS +0.00% +95.17% +0.00% +1970.00% -157.48% +85.39%
7 no-sync sync arch-a googlenet resnet18 throughput 4 no-sync +26.17% FAIL -20.74% FAIL PASS +9.97% +7.15% +18.50% +7.81% +14.70% +7.92%
8 no-plan plan arch-a googlenet resnet34 throughput latency 4 1 no-plan -100.00% PASS -100.00% PASS -62.73% +95.94% -100.00% NA -130.05% +90.65%
9 no-sync-no-plan plan arch-a googlenet resnet34 throughput 4 no-plan +4094.63% PASS -97.62% FAIL PASS -59.86% +96.00% +1311.01% +2400.00% -189.18% +88.43%
10 baseline sync arch-a resnet18 resnet34 latency throughput 1 4 no-sync +0.00% FAIL NA PASS +0.00% -9.11% +0.00% -8.42% +7.32% -1.02%
11 no-sync plan arch-a resnet18 vgg8 latency 1 no-plan +0.00% PASS NA PASS +0.00% +94.75% +0.00% NA -80.65% +90.51%
12 no-plan plan arch-a resnet18 vgg8 latency throughput 1 4 no-plan +3191.30% PASS NA PASS -60.97% +92.77% +1184.24% +1279.31% -170.46% +82.49%
13 no-sync-no-plan sync arch-a resnet18 vgg8 latency throughput 1 4 no-sync +3191.30% FAIL NA PASS -60.97% +19.18% +1184.24% +24.03% +21.29% +18.62%
14 baseline plan arch-a arch-b resnet18 googlenet throughput latency 4 1 no-plan +0.00% PASS +0.00% PASS +0.00% +98.99% +0.00% NA -239.10% +96.59%
15 no-sync plan arch-a arch-b resnet18 googlenet throughput 4 no-plan +110.65% PASS -52.55% FAIL PASS +43.25% NA +177.48% NA -306.99% NA
16 no-plan sync arch-a arch-b resnet18 googlenet throughput 4 no-sync +1598.93% FAIL -94.12% PASS -51.04% -1.28% +664.87% -1.63% +9.99% -4.29%
17 no-sync-no-plan plan arch-a arch-b resnet18 throughput latency 4 1 no-plan +1174.21% PASS -92.16% FAIL PASS -45.06% +96.71% +543.74% NA -171.16% +91.07%
18 baseline plan arch-a arch-b resnet34 resnet18 latency throughput 1 4 no-plan +0.00% PASS NA PASS +0.00% +95.46% +0.00% +2102.50% -200.52% +83.26%
19 no-sync sync arch-a arch-b resnet34 resnet18 latency throughput 1 4 no-sync +0.00% FAIL NA PASS +0.00% +2.22% +0.00% +2.32% +8.26% -7.88%
20 no-plan plan arch-a arch-b resnet34 latency 1 no-plan +2361.37% PASS NA PASS -56.53% +96.23% +969.99% NA -166.76% +89.94%
21 no-sync-no-plan plan arch-a arch-b resnet34 latency throughput 1 4 no-plan +2361.37% PASS NA PASS -56.53% +97.58% +969.99% +4030.00% -168.71% +91.79%
22 baseline sync arch-a arch-b resnet34 throughput 4 no-sync +0.00% FAIL +0.00% PASS +0.00% -0.80% +0.00% -0.96% +10.61% -13.51%
23 no-sync plan arch-a arch-b resnet34 vgg8 throughput latency 4 1 no-plan +33.65% PASS -25.21% FAIL PASS +31.76% +94.68% +15.56% NA -98.04% +89.47%
24 no-plan plan arch-a arch-b resnet34 vgg8 throughput 4 no-plan +1726.82% PASS -94.52% PASS -54.50% +93.70% +455.98% +1484.16% -205.54% +81.37%
25 no-sync-no-plan sync arch-a arch-b resnet34 vgg8 throughput 4 no-sync +813.41% FAIL -89.04% FAIL PASS -49.77% +17.07% +206.84% +20.30% +19.03% +9.63%
baseline arch-a vgg8 latency 1 +0.00% NA PASS +0.00% +0.00%
no-sync arch-a vgg8 latency 1 +0.00% NA PASS +0.00% +0.00%
no-plan arch-a vgg8 latency 1 +1800.66% NA PASS -44.65% +953.39%
no-sync-no-plan arch-a vgg8 latency 1 +1800.66% NA PASS -44.65% +953.39%
baseline arch-a vgg8 throughput 4 +0.00% +0.00% PASS +0.00% +0.00%
no-sync arch-a vgg8 throughput 4 +2.33% -1.72% PASS +18.49% -0.98%
no-plan arch-a vgg8 throughput 4 +1251.16% -92.58% PASS -56.68% +577.04%
no-sync-no-plan arch-a vgg8 throughput 4 +1260.47% -92.63% PASS -53.36% +467.83%
baseline arch-b googlenet latency 1 +0.00% NA PASS +0.00% +0.00%
no-sync arch-b googlenet latency 1 +0.00% NA PASS +0.00% -0.00%
no-plan arch-b googlenet latency 1 +9849.64% NA PASS -70.51% +2833.89%
no-sync-no-plan arch-b googlenet latency 1 +9849.55% NA PASS -70.51% +2833.89%
baseline arch-b googlenet throughput 4 +0.00% +0.00% PASS +0.00% +0.00%
no-sync arch-b googlenet throughput 4 +11.45% -10.29% FAIL +21.82% +19.27%
no-plan arch-b googlenet throughput 4 -100.00% -100.00% PASS -63.39% -100.00%
no-sync-no-plan arch-b googlenet throughput 4 +3348.28% -97.10% FAIL -61.58% +1026.09%
baseline arch-b resnet18 latency 1 +0.00% NA PASS +0.00% +0.00%
no-sync arch-b resnet18 latency 1 +0.00% NA PASS +0.00% -0.00%
no-plan arch-b resnet18 latency 1 +2935.63% NA PASS -63.12% +1019.81%
no-sync-no-plan arch-b resnet18 latency 1 +2936.09% NA PASS -63.12% +1019.97%
baseline arch-b resnet18 throughput 4 +0.00% +0.00% PASS +0.00% +0.00%
no-sync arch-b resnet18 throughput 4 +8.03% -7.44% FAIL +47.59% +40.61%
no-plan arch-b resnet18 throughput 4 +1511.85% -93.80% PASS -52.93% +560.80%
no-sync-no-plan arch-b resnet18 throughput 4 +867.12% -89.67% FAIL -45.41% +359.83%
baseline arch-b resnet34 latency 1 +0.00% NA PASS +0.00% +0.00%
no-sync arch-b resnet34 latency 1 +0.00% NA PASS +0.00% +0.00%
no-plan arch-b resnet34 latency 1 +2551.16% NA PASS -62.51% +893.70%
no-sync-no-plan arch-b resnet34 latency 1 +2550.91% NA PASS -62.51% +893.62%
baseline arch-b resnet34 throughput 4 +0.00% +0.00% PASS +0.00% +0.00%
no-sync arch-b resnet34 throughput 4 +28.25% -22.13% FAIL +49.76% +57.30%
no-plan arch-b resnet34 throughput 4 +2427.17% -96.05% PASS -49.58% +938.62%
no-sync-no-plan arch-b resnet34 throughput 4 +531.79% -84.19% FAIL -42.32% +197.08%
baseline arch-b vgg8 latency 1 +0.00% NA PASS +0.00% +0.00%
no-sync arch-b vgg8 latency 1 +0.00% NA PASS +0.00% +0.00%
no-plan arch-b vgg8 latency 1 +1784.97% NA PASS -49.50% +849.73%
no-sync-no-plan arch-b vgg8 latency 1 +1784.97% NA PASS -49.50% +849.73%
baseline arch-b vgg8 throughput 4 +0.00% +0.00% PASS +0.00% +0.00%
no-sync arch-b vgg8 throughput 4 +23.91% -19.82% PASS +25.67% +23.93%
no-plan arch-b vgg8 throughput 4 +1158.70% -92.07% PASS -58.95% +408.33%
no-sync-no-plan arch-b vgg8 throughput 4 +1172.83% -92.17% PASS -55.83% +327.00%
@@ -1,41 +1,41 @@
arch,model,mode,raptor_pipeline,pimcomp_pipeline,raptor_functional_validation,pimcomp_functional_validation,raptor_latency_ms,pimcomp_latency_ms,raptor_throughput_samples_s,pimcomp_throughput_samples_s,raptor_power_mw,pimcomp_power_mw,raptor_energy_pj,pimcomp_energy_pj,better_compiler,speedup
arch-a,vgg8,latency,1,element,PASS,PASS,1.51833,7.98507,NA,NA,321.046,200.111,4.87455e+08,1.5979e+09,raptor,5.26
arch-a,vgg8,throughput,2,batch,PASS,PASS,0.906321,1.0364,1100,965,429.311,482.597,4.07972e+08,5.41692e+08,raptor,1.14
arch-a,vgg8,throughput,4,batch,PASS,PASS,0.778242,1.0364,1280,965,457.039,482.597,4.15762e+08,5.41692e+08,raptor,1.33
arch-a,vgg8,throughput,8,batch,PASS,PASS,0.778094,1.0364,1290,965,464.071,482.597,3.96424e+08,5.41692e+08,raptor,1.34
arch-a,resnet18,latency,1,element,PASS,FAIL,14.0332,58.1775,NA,NA,455.67,238.389,6.3945e+09,1.38689e+10,raptor,4.15
arch-a,resnet18,throughput,2,batch,PASS,FAIL,11.7314,13.4585,85.2,74.3,503.111,481.604,6.62807e+09,6.56345e+09,raptor,1.15
arch-a,resnet18,throughput,4,batch,PASS,FAIL,19.2072,13.4585,52.1,74.3,380.1,481.604,7.95703e+09,6.56345e+09,pimcomp,1.43
arch-a,resnet18,throughput,8,batch,PASS,FAIL,14.1578,13.4585,70.6,74.3,453.416,481.604,6.77909e+09,6.56345e+09,pimcomp,1.05
arch-a,resnet34,latency,1,element,PASS,FAIL,29.7186,66.7114,NA,NA,411.806,277.485,1.22383e+10,1.85114e+10,raptor,2.24
arch-a,resnet34,throughput,2,batch,PASS,FAIL,23.2705,23.5817,43,42.4,471.717,515.68,1.3055e+10,1.22836e+10,raptor,1.01
arch-a,resnet34,throughput,4,batch,PASS,FAIL,27.468,23.5817,36.4,42.4,422.223,515.68,1.48991e+10,1.22836e+10,pimcomp,1.16
arch-a,resnet34,throughput,8,batch,PASS,FAIL,39.8392,23.5817,25.1,42.4,352.28,515.68,1.70627e+10,1.22836e+10,pimcomp,1.69
arch-a,googlenet,latency,1,element,PASS,PASS,9.77772,62.9248,NA,NA,563.864,231.193,5.5133e+09,1.45477e+10,raptor,6.44
arch-a,googlenet,throughput,2,batch,PASS,FAIL,8.41545,18.2643,119,54.8,610.746,435.107,5.77746e+09,8.16479e+09,raptor,2.17
arch-a,googlenet,throughput,4,batch,PASS,FAIL,10.7292,18.2643,93.2,54.8,528.026,435.107,6.59289e+09,8.16479e+09,raptor,1.70
arch-a,googlenet,throughput,8,batch,PASS,FAIL,10.1219,18.2643,98.8,54.8,523.808,435.107,6.52174e+09,8.16479e+09,raptor,1.80
arch-a,yolo11n,latency,1,element,PASS,FAIL,147.49,NA,NA,NA,290.174,NA,4.27977e+10,NA,NA,NA
arch-a,yolo11n,throughput,2,batch,SKIPPED,FAIL,102.337,NA,9.77,NA,337.882,NA,3.83081e+10,NA,NA,NA
arch-a,yolo11n,throughput,4,batch,SKIPPED,FAIL,87.9103,NA,11.4,NA,358.201,NA,3.87298e+10,NA,NA,NA
arch-a,yolo11n,throughput,8,batch,SKIPPED,FAIL,82.9508,NA,12.1,NA,366.725,NA,3.85347e+10,NA,NA,NA
arch-a,vgg8,throughput,2,batch,PASS,FAIL,0.625998,1.0364,1600,965,523.278,482.597,4.17419e+08,5.41692e+08,NA,1.66
arch-a,vgg8,throughput,4,batch,PASS,FAIL,0.625998,1.0364,1600,965,523.278,482.597,4.17419e+08,5.41692e+08,NA,1.66
arch-a,vgg8,throughput,8,batch,PASS,FAIL,0.62635,1.0364,1600,965,519.182,482.597,4.14242e+08,5.41692e+08,NA,1.66
arch-a,resnet18,latency,1,element,PASS,FAIL,14.0332,58.1775,NA,NA,455.67,238.389,6.3945e+09,1.38689e+10,NA,4.15
arch-a,resnet18,throughput,2,batch,PASS,FAIL,11.0904,13.4585,90.2,74.3,520.244,481.604,6.55167e+09,6.56345e+09,NA,1.21
arch-a,resnet18,throughput,4,batch,PASS,FAIL,12.0822,13.4585,82.8,74.3,487.576,481.604,6.91662e+09,6.56345e+09,NA,1.11
arch-a,resnet18,throughput,8,batch,PASS,FAIL,11.9279,13.4585,83.8,74.3,499.354,481.604,6.84729e+09,6.56345e+09,NA,1.13
arch-a,resnet34,latency,1,element,PASS,FAIL,29.7189,66.7114,NA,NA,411.804,277.485,1.22384e+10,1.85114e+10,NA,2.24
arch-a,resnet34,throughput,2,batch,PASS,FAIL,21.8196,23.5817,45.8,42.4,477.108,515.68,1.57899e+10,1.22836e+10,NA,1.08
arch-a,resnet34,throughput,4,batch,PASS,FAIL,20,23.5817,50,42.4,548.372,515.68,1.09674e+10,1.22836e+10,NA,1.18
arch-a,resnet34,throughput,8,batch,PASS,FAIL,20.6363,23.5817,48.5,42.4,504.854,515.68,1.23725e+10,1.22836e+10,NA,1.14
arch-a,googlenet,latency,1,element,PASS,FAIL,9.77731,62.9248,NA,NA,563.88,231.193,5.51323e+09,1.45477e+10,NA,6.44
arch-a,googlenet,throughput,2,batch,PASS,FAIL,7.34567,18.2643,136,54.8,679.351,435.107,5.5053e+09,8.16479e+09,NA,2.48
arch-a,googlenet,throughput,4,batch,PASS,FAIL,7.43728,18.2643,134,54.8,674.025,435.107,5.60809e+09,8.16479e+09,NA,2.45
arch-a,googlenet,throughput,8,batch,PASS,FAIL,9.01886,18.2643,111,54.8,595.582,435.107,5.71605e+09,8.16479e+09,NA,2.03
arch-a,yolo11n,latency,1,element,PASS,FAIL,149.46,NA,NA,NA,288.579,NA,4.31311e+10,NA,NA,NA
arch-a,yolo11n,throughput,2,batch,PASS,FAIL,94.946,NA,10.5,NA,346.466,NA,3.75539e+10,NA,NA,NA
arch-a,yolo11n,throughput,4,batch,PASS,FAIL,80.1193,NA,12.5,NA,377.408,NA,3.8811e+10,NA,NA,NA
arch-a,yolo11n,throughput,8,batch,PASS,FAIL,99.5488,NA,10,NA,339.331,NA,4.27113e+10,NA,NA,NA
arch-b,vgg8,latency,1,element,PASS,PASS,1.5333,7.15212,NA,NA,292.559,173.769,4.48582e+08,1.24281e+09,raptor,4.66
arch-b,vgg8,throughput,2,batch,PASS,PASS,0.685556,1.07282,1460,932,456.044,409.47,4.03694e+08,4.39289e+08,raptor,1.57
arch-b,vgg8,throughput,4,batch,PASS,PASS,0.673192,1.07282,1490,932,448.898,409.47,4.13527e+08,4.39289e+08,raptor,1.60
arch-b,vgg8,throughput,8,batch,PASS,PASS,0.751106,1.07282,1330,932,425.742,409.47,3.94458e+08,4.39289e+08,raptor,1.43
arch-b,resnet18,latency,1,element,PASS,FAIL,15.3758,64.3818,NA,NA,400.661,201.663,6.16048e+09,1.29834e+10,raptor,4.19
arch-b,resnet18,throughput,2,batch,PASS,FAIL,12.1214,13.434,82.5,74.4,461.407,484.804,6.29532e+09,6.56966e+09,raptor,1.11
arch-b,resnet18,throughput,4,batch,PASS,FAIL,11.3601,13.434,88,74.4,479.333,484.804,6.98745e+09,6.56966e+09,raptor,1.18
arch-b,resnet18,throughput,8,batch,PASS,FAIL,13.1216,13.434,76.2,74.4,441.248,484.804,6.38271e+09,6.56966e+09,raptor,1.02
arch-b,resnet34,latency,1,element,PASS,FAIL,27.9541,107.2,NA,NA,397.206,206.476,1.11035e+10,2.21342e+10,raptor,3.83
arch-b,resnet34,throughput,2,batch,PASS,FAIL,22.282,22.8454,44.9,43.8,438.238,493.332,1.42187e+10,1.13922e+10,raptor,1.03
arch-b,resnet34,throughput,4,batch,PASS,FAIL,29.0906,22.8454,34.4,43.8,364.956,493.332,1.63285e+10,1.13922e+10,pimcomp,1.27
arch-b,resnet34,throughput,8,batch,PASS,FAIL,47.68,22.8454,21,43.8,289.821,493.332,1.63464e+10,1.13922e+10,pimcomp,2.09
arch-b,googlenet,latency,1,element,PASS,PASS,11.2212,37.9367,NA,NA,483.266,242.037,5.42282e+09,9.18206e+09,raptor,3.38
arch-b,googlenet,throughput,2,batch,PASS,FAIL,9.21209,21.3902,109,46.8,545.488,417.866,5.5934e+09,9.01473e+09,raptor,2.33
arch-b,googlenet,throughput,4,batch,PASS,FAIL,8.56194,21.3902,117,46.8,555.264,417.866,5.86312e+09,9.01473e+09,raptor,2.50
arch-b,googlenet,throughput,8,batch,PASS,FAIL,10.8652,21.3902,92,46.8,483.477,417.866,6.12683e+09,9.01473e+09,raptor,1.97
arch-b,yolo11n,latency,1,element,PASS,FAIL,168.144,NA,NA,NA,245.314,NA,4.1248e+10,NA,NA,NA
arch-b,yolo11n,throughput,2,batch,SKIPPED,FAIL,104.494,NA,9.57,NA,302.121,NA,3.58015e+10,NA,NA,NA
arch-b,yolo11n,throughput,4,batch,SKIPPED,FAIL,108.247,NA,9.24,NA,297.484,NA,3.76297e+10,NA,NA,NA
arch-b,yolo11n,throughput,8,batch,SKIPPED,FAIL,100.129,NA,9.99,NA,308.833,NA,3.71808e+10,NA,NA,NA
arch-b,vgg8,throughput,2,batch,PASS,FAIL,0.624729,1.07282,1600,932,489.899,409.47,3.93036e+08,4.39289e+08,NA,1.72
arch-b,vgg8,throughput,4,batch,PASS,FAIL,0.62331,1.07282,1600,932,490.955,409.47,3.93418e+08,4.39289e+08,NA,1.72
arch-b,vgg8,throughput,8,batch,PASS,FAIL,0.62331,1.07282,1600,932,490.955,409.47,3.93418e+08,4.39289e+08,NA,1.72
arch-b,resnet18,latency,1,element,PASS,FAIL,15.3758,64.3818,NA,NA,400.661,201.663,6.16048e+09,1.29834e+10,NA,4.19
arch-b,resnet18,throughput,2,batch,PASS,FAIL,12.9113,13.434,77.5,74.4,438.931,484.804,6.47794e+09,6.56966e+09,NA,1.04
arch-b,resnet18,throughput,4,batch,PASS,FAIL,11.3505,13.434,88.1,74.4,478.606,484.804,6.66517e+09,6.56966e+09,NA,1.18
arch-b,resnet18,throughput,8,batch,PASS,FAIL,11.3685,13.434,88,74.4,478.33,484.804,6.51513e+09,6.56966e+09,NA,1.18
arch-b,resnet34,latency,1,element,PASS,FAIL,27.9541,107.2,NA,NA,397.206,206.476,1.11035e+10,2.21342e+10,NA,3.83
arch-b,resnet34,throughput,2,batch,PASS,FAIL,22.4967,22.8454,44.5,43.8,440.06,493.332,1.49623e+10,1.13922e+10,NA,1.02
arch-b,resnet34,throughput,4,batch,PASS,FAIL,24.1838,22.8454,41.3,43.8,426.129,493.332,1.30195e+10,1.13922e+10,NA,1.06
arch-b,resnet34,throughput,8,batch,PASS,FAIL,24.1516,22.8454,41.4,43.8,420.745,493.332,1.39184e+10,1.13922e+10,NA,1.06
arch-b,googlenet,latency,1,element,PASS,FAIL,11.2206,37.9367,NA,NA,483.283,242.037,5.42274e+09,9.18206e+09,NA,3.38
arch-b,googlenet,throughput,2,batch,PASS,FAIL,9.08396,21.3902,110,46.8,550.292,417.866,5.70509e+09,9.01473e+09,NA,2.35
arch-b,googlenet,throughput,4,batch,PASS,FAIL,8.23423,21.3902,121,46.8,597.327,417.866,5.62683e+09,9.01473e+09,NA,2.59
arch-b,googlenet,throughput,8,batch,PASS,FAIL,7.78944,21.3902,128,46.8,620.65,417.866,5.47039e+09,9.01473e+09,NA,2.74
arch-b,yolo11n,latency,1,element,PASS,FAIL,171.598,NA,NA,NA,243.177,NA,4.17288e+10,NA,NA,NA
arch-b,yolo11n,throughput,2,batch,PASS,FAIL,99.5318,NA,10,NA,309.102,NA,3.5219e+10,NA,NA,NA
arch-b,yolo11n,throughput,4,batch,PASS,FAIL,85.3497,NA,11.7,NA,326.712,NA,3.58691e+10,NA,NA,NA
arch-b,yolo11n,throughput,8,batch,PASS,FAIL,97.9941,NA,10.2,NA,308.041,NA,3.5011e+10,NA,NA,NA
1 arch model mode raptor_pipeline pimcomp_pipeline raptor_functional_validation pimcomp_functional_validation raptor_latency_ms pimcomp_latency_ms raptor_throughput_samples_s pimcomp_throughput_samples_s raptor_power_mw pimcomp_power_mw raptor_energy_pj pimcomp_energy_pj better_compiler speedup
2 arch-a vgg8 latency 1 element PASS PASS 1.51833 7.98507 NA NA 321.046 200.111 4.87455e+08 1.5979e+09 raptor 5.26
3 arch-a vgg8 throughput 2 batch PASS PASS FAIL 0.906321 0.625998 1.0364 1100 1600 965 429.311 523.278 482.597 4.07972e+08 4.17419e+08 5.41692e+08 raptor NA 1.14 1.66
4 arch-a vgg8 throughput 4 batch PASS PASS FAIL 0.778242 0.625998 1.0364 1280 1600 965 457.039 523.278 482.597 4.15762e+08 4.17419e+08 5.41692e+08 raptor NA 1.33 1.66
5 arch-a vgg8 throughput 8 batch PASS PASS FAIL 0.778094 0.62635 1.0364 1290 1600 965 464.071 519.182 482.597 3.96424e+08 4.14242e+08 5.41692e+08 raptor NA 1.34 1.66
6 arch-a resnet18 latency 1 element PASS FAIL 14.0332 58.1775 NA NA 455.67 238.389 6.3945e+09 1.38689e+10 raptor NA 4.15
7 arch-a resnet18 throughput 2 batch PASS FAIL 11.7314 11.0904 13.4585 85.2 90.2 74.3 503.111 520.244 481.604 6.62807e+09 6.55167e+09 6.56345e+09 raptor NA 1.15 1.21
8 arch-a resnet18 throughput 4 batch PASS FAIL 19.2072 12.0822 13.4585 52.1 82.8 74.3 380.1 487.576 481.604 7.95703e+09 6.91662e+09 6.56345e+09 pimcomp NA 1.43 1.11
9 arch-a resnet18 throughput 8 batch PASS FAIL 14.1578 11.9279 13.4585 70.6 83.8 74.3 453.416 499.354 481.604 6.77909e+09 6.84729e+09 6.56345e+09 pimcomp NA 1.05 1.13
10 arch-a resnet34 latency 1 element PASS FAIL 29.7186 29.7189 66.7114 NA NA 411.806 411.804 277.485 1.22383e+10 1.22384e+10 1.85114e+10 raptor NA 2.24
11 arch-a resnet34 throughput 2 batch PASS FAIL 23.2705 21.8196 23.5817 43 45.8 42.4 471.717 477.108 515.68 1.3055e+10 1.57899e+10 1.22836e+10 raptor NA 1.01 1.08
12 arch-a resnet34 throughput 4 batch PASS FAIL 27.468 20 23.5817 36.4 50 42.4 422.223 548.372 515.68 1.48991e+10 1.09674e+10 1.22836e+10 pimcomp NA 1.16 1.18
13 arch-a resnet34 throughput 8 batch PASS FAIL 39.8392 20.6363 23.5817 25.1 48.5 42.4 352.28 504.854 515.68 1.70627e+10 1.23725e+10 1.22836e+10 pimcomp NA 1.69 1.14
14 arch-a googlenet latency 1 element PASS PASS FAIL 9.77772 9.77731 62.9248 NA NA 563.864 563.88 231.193 5.5133e+09 5.51323e+09 1.45477e+10 raptor NA 6.44
15 arch-a googlenet throughput 2 batch PASS FAIL 8.41545 7.34567 18.2643 119 136 54.8 610.746 679.351 435.107 5.77746e+09 5.5053e+09 8.16479e+09 raptor NA 2.17 2.48
16 arch-a googlenet throughput 4 batch PASS FAIL 10.7292 7.43728 18.2643 93.2 134 54.8 528.026 674.025 435.107 6.59289e+09 5.60809e+09 8.16479e+09 raptor NA 1.70 2.45
17 arch-a googlenet throughput 8 batch PASS FAIL 10.1219 9.01886 18.2643 98.8 111 54.8 523.808 595.582 435.107 6.52174e+09 5.71605e+09 8.16479e+09 raptor NA 1.80 2.03
18 arch-a yolo11n latency 1 element PASS FAIL 147.49 149.46 NA NA NA 290.174 288.579 NA 4.27977e+10 4.31311e+10 NA NA NA
19 arch-a yolo11n throughput 2 batch SKIPPED PASS FAIL 102.337 94.946 NA 9.77 10.5 NA 337.882 346.466 NA 3.83081e+10 3.75539e+10 NA NA NA
20 arch-a yolo11n throughput 4 batch SKIPPED PASS FAIL 87.9103 80.1193 NA 11.4 12.5 NA 358.201 377.408 NA 3.87298e+10 3.8811e+10 NA NA NA
21 arch-a yolo11n throughput 8 batch SKIPPED PASS FAIL 82.9508 99.5488 NA 12.1 10 NA 366.725 339.331 NA 3.85347e+10 4.27113e+10 NA NA NA
22 arch-b vgg8 latency 1 element PASS PASS 1.5333 7.15212 NA NA 292.559 173.769 4.48582e+08 1.24281e+09 raptor 4.66
23 arch-b vgg8 throughput 2 batch PASS PASS FAIL 0.685556 0.624729 1.07282 1460 1600 932 456.044 489.899 409.47 4.03694e+08 3.93036e+08 4.39289e+08 raptor NA 1.57 1.72
24 arch-b vgg8 throughput 4 batch PASS PASS FAIL 0.673192 0.62331 1.07282 1490 1600 932 448.898 490.955 409.47 4.13527e+08 3.93418e+08 4.39289e+08 raptor NA 1.60 1.72
25 arch-b vgg8 throughput 8 batch PASS PASS FAIL 0.751106 0.62331 1.07282 1330 1600 932 425.742 490.955 409.47 3.94458e+08 3.93418e+08 4.39289e+08 raptor NA 1.43 1.72
26 arch-b resnet18 latency 1 element PASS FAIL 15.3758 64.3818 NA NA 400.661 201.663 6.16048e+09 1.29834e+10 raptor NA 4.19
27 arch-b resnet18 throughput 2 batch PASS FAIL 12.1214 12.9113 13.434 82.5 77.5 74.4 461.407 438.931 484.804 6.29532e+09 6.47794e+09 6.56966e+09 raptor NA 1.11 1.04
28 arch-b resnet18 throughput 4 batch PASS FAIL 11.3601 11.3505 13.434 88 88.1 74.4 479.333 478.606 484.804 6.98745e+09 6.66517e+09 6.56966e+09 raptor NA 1.18
29 arch-b resnet18 throughput 8 batch PASS FAIL 13.1216 11.3685 13.434 76.2 88 74.4 441.248 478.33 484.804 6.38271e+09 6.51513e+09 6.56966e+09 raptor NA 1.02 1.18
30 arch-b resnet34 latency 1 element PASS FAIL 27.9541 107.2 NA NA 397.206 206.476 1.11035e+10 2.21342e+10 raptor NA 3.83
31 arch-b resnet34 throughput 2 batch PASS FAIL 22.282 22.4967 22.8454 44.9 44.5 43.8 438.238 440.06 493.332 1.42187e+10 1.49623e+10 1.13922e+10 raptor NA 1.03 1.02
32 arch-b resnet34 throughput 4 batch PASS FAIL 29.0906 24.1838 22.8454 34.4 41.3 43.8 364.956 426.129 493.332 1.63285e+10 1.30195e+10 1.13922e+10 pimcomp NA 1.27 1.06
33 arch-b resnet34 throughput 8 batch PASS FAIL 47.68 24.1516 22.8454 21 41.4 43.8 289.821 420.745 493.332 1.63464e+10 1.39184e+10 1.13922e+10 pimcomp NA 2.09 1.06
34 arch-b googlenet latency 1 element PASS PASS FAIL 11.2212 11.2206 37.9367 NA NA 483.266 483.283 242.037 5.42282e+09 5.42274e+09 9.18206e+09 raptor NA 3.38
35 arch-b googlenet throughput 2 batch PASS FAIL 9.21209 9.08396 21.3902 109 110 46.8 545.488 550.292 417.866 5.5934e+09 5.70509e+09 9.01473e+09 raptor NA 2.33 2.35
36 arch-b googlenet throughput 4 batch PASS FAIL 8.56194 8.23423 21.3902 117 121 46.8 555.264 597.327 417.866 5.86312e+09 5.62683e+09 9.01473e+09 raptor NA 2.50 2.59
37 arch-b googlenet throughput 8 batch PASS FAIL 10.8652 7.78944 21.3902 92 128 46.8 483.477 620.65 417.866 6.12683e+09 5.47039e+09 9.01473e+09 raptor NA 1.97 2.74
38 arch-b yolo11n latency 1 element PASS FAIL 168.144 171.598 NA NA NA 245.314 243.177 NA 4.1248e+10 4.17288e+10 NA NA NA
39 arch-b yolo11n throughput 2 batch SKIPPED PASS FAIL 104.494 99.5318 NA 9.57 10 NA 302.121 309.102 NA 3.58015e+10 3.5219e+10 NA NA NA
40 arch-b yolo11n throughput 4 batch SKIPPED PASS FAIL 108.247 85.3497 NA 9.24 11.7 NA 297.484 326.712 NA 3.76297e+10 3.58691e+10 NA NA NA
41 arch-b yolo11n throughput 8 batch SKIPPED PASS FAIL 100.129 97.9941 NA 9.99 10.2 NA 308.833 308.041 NA 3.71808e+10 3.5011e+10 NA NA NA
@@ -282,6 +282,23 @@ def comparison_passed(report: dict) -> bool:
return result.get("status") == "done" and bool(result.get("passed"))
def completed_synchronization_ablation(report: dict) -> bool:
"""A completed no-sync measurement may have intentionally invalid outputs."""
validation = report.get("raptor_validation") or {}
performance = report.get("raptor_performance") or {}
return (
report.get("pimsim_mode") == "throughput"
and "--pim-disable-synchronization" in report.get("raptor_extra_args", [])
and validation.get("status") == "done" and validation.get("passed") is False
and bool(validation.get("max_diffs"))
and not report.get("failures")
and bool(report.get("steps"))
and all(step.get("status") == "passed" for step in report["steps"])
and not performance.get("error")
and performance_values(performance)["throughput"] is not None
)
def functional_validation_status(result: dict | None) -> str:
if result is None:
return "NA"
@@ -869,6 +886,7 @@ def main() -> int:
assert results_path is not None
print_stage(results_path.name, STAGE_COLORS["Compare outputs"])
print(results_path.read_text(encoding="utf-8"), end="")
diagnostic_failures = []
for arch in args.archs:
for name in args.models:
for mode, pipeline, _ in comparisons_by_arch[arch]:
@@ -876,15 +894,19 @@ def main() -> int:
report_path = result_dir(
out_dir, name, arch, mode, pipeline, args.ablation_variant
) / "pimcomp/comparison_report.json"
if not report_path.exists() or not comparison_passed(
json.loads(report_path.read_text(encoding="utf-8")),
):
if label not in failed:
report = json.loads(report_path.read_text(encoding="utf-8")) if report_path.exists() else {}
if not comparison_passed(report):
if (label not in failed and args.ablation_variant == "no-sync"
and completed_synchronization_ablation(report)):
diagnostic_failures.append(label)
elif label not in failed:
failed.append(label)
print(Style.BRIGHT + Fore.CYAN + "Summary" + Style.RESET_ALL)
total_jobs = sum(len(args.models) * len(comparisons) for comparisons in comparisons_by_arch.values())
print(Style.BRIGHT + f"Passed: {total_jobs - len(failed)}" + Style.RESET_ALL)
print(Style.BRIGHT + f"Passed: {total_jobs - len(failed) - len(diagnostic_failures)}" + Style.RESET_ALL)
print(Style.BRIGHT + f"Failed: {len(failed)}" + Style.RESET_ALL)
if diagnostic_failures:
print("No-sync numerical failures (diagnostic only): " + ", ".join(diagnostic_failures))
print(Style.BRIGHT + f"Results: {results_path}" + Style.RESET_ALL)
if failed:
print(
@@ -35,13 +35,15 @@ The corresponding scheduling call is disabled in
Rust validation starts those buffers at zero, exposing the missing computation.
```bash
.venv/bin/python validation/tools/pim/pimcomp/correctness/prefill/run_experiment.py
.venv/bin/python validation/tools/pim/pimcomp/correctness/prefill/run_experiment.py \
/path/to/comparison /path/to/matching-model.onnx
```
The default uses GoogLeNet Arch-A throughput/pipeline2. Optional positional
arguments are comparison directory, model, work directory, and shared reference
directory; the default work directory is `correctness/prefill/` below the
comparison artifacts.
The comparison directory and its matching model are required. Optional positional
arguments are work directory and shared reference directory. The default work
directory is `correctness/prefill/` below the comparison artifacts.
Historical Softmax-free models can be recovered from Git for historical artifacts.
Current GoogLeNet retains Softmax; prefill alone does not correct omitted Softmax execution.
The experiment compares the original memory image with one prefilled using
[`batch_memory.py`](prefill/batch_memory.py). In the recorded case, 20 of 39
@@ -16,8 +16,6 @@ from batch_memory import prefill_batch_memory
SCRIPT_DIR = Path(__file__).resolve().parent
REPO_ROOT = SCRIPT_DIR.parents[5]
DEFAULT_COMPARISON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/arch-a/throughput/pipeline2"
DEFAULT_MODEL = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/googlenet-12-pimsim-nn.onnx"
DEFAULT_COMMON_DIR = REPO_ROOT / "validation/networks/pimcomp_models/googlenet/artifacts/common"
SIMULATOR_MANIFEST = REPO_ROOT / "backend-simulators/pim/pim-simulator/Cargo.toml"
@@ -78,8 +76,8 @@ def compare_outputs(
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("comparison_dir", nargs="?", type=Path, default=DEFAULT_COMPARISON_DIR)
parser.add_argument("model", nargs="?", type=Path, default=DEFAULT_MODEL)
parser.add_argument("comparison_dir", type=Path)
parser.add_argument("model", type=Path, help="ONNX model used to generate the comparison artifacts")
parser.add_argument("work_dir", nargs="?", type=Path)
parser.add_argument("common_dir", nargs="?", type=Path, default=DEFAULT_COMMON_DIR)
args = parser.parse_args()