better throughput scheduling with steady state cost model, now limited to spatial

This commit is contained in:
NiccoloN
2026-10-26 12:07:01 +01:00
parent fd493ae6e8
commit 4b53f6cf10
62 changed files with 4293 additions and 871 deletions
+1 -1
View File
@@ -17,7 +17,7 @@ Boolean switches default to off unless stated otherwise.
| `--pim-only-codegen` | Run the codegen tail on bufferized Pim IR. |
| `--pim-emit-json` | Also emit per-core JSON instructions. |
| `--pim-dump-ir` | Write diagnostic IR snapshots with elided large constants; default: off. |
| `--pim-periodic-diagnostics` | Write the optional pre-codegen recurrence report; default: off. |
| `--pim-periodic-diagnostics` | Write optional recurrence and Spatial execution-work reports; default: off. |
| `--pim-export-spatial-dataflow=none\|spatial1\|spatial2\|spatial3\|spatial4\|all` | Export graph, trivially merged, scheduled, and realized CSV snapshots under `reports/`; default: `none`. |
| `--pim-conv-lowering=STRATEGY` | `auto` (default), `legacy`, `depthwise`, `packed-im2col`, `streamed-patch`, `streamed-packed`, `output-channel-tiled`, `input-k-tiled`, or `tiled-2d`. |
| `--pim-conv-im2col-max-elements=N` | Global im2col limit before streaming; default: 1048576. |
+139
View File
@@ -72,6 +72,59 @@ byte-sized elements. Blueprint assembly charges per fragment. A full fragment
receive directly into assembly storage omits duplicate endpoint-memory edge
cost; later copy elimination can leave this estimate conservative.
`CopyAccessPlan` shares ordered contiguous-run geometry between projection
costing and PIM copy normalization. Its inputs are a logical copy shape and
proven source and destination strides, in elements. The immutable result keeps
outer dimensions, run count, run length, and relative offsets compact. Building
the plan takes linear time and space in rank, independent of loop trip counts.
It creates no IR and does not prove aliasing, storage ownership, or copy legality.
PIM bufferization still owns those decisions and the mapping to byte accesses.
The plan alone does not describe a complete scheduled memory-request stream.
Construction rejects an unrepresentable maximum relative element offset,
including the last element of each run. Callers still check byte scaling and
base-address addition.
Projection costing uses checked row-major strides. Unsupported flattened
offsets or strides produce at most one warning per graph cost query. The query
uses one separately rounded request per element as a conservative service bound.
It does not reject otherwise valid Spatial IR. Target lowering retains its own
address and layout legality checks. Unsupported estimates and malformed IR are
different conditions.
The PIM adapter's `planComputeInputMaterialization` reads scheduled Spatial
regions without changing them. It returns `Unused`, `ForwardConstant`,
`ForwardFunctionInput`, or `CopyAtEntry`. Function-input normalization forwards
host inputs into compute bodies before entry-copy materialization.
Single-core lowering forwards used arithmetic constants. Batch lowering copies
remaining inputs at body entry, including constants. Lowering and costing
consume the same decisions. The query has constant time and space per
input and does not expand lanes or loops. Abstract target work extraction uses
the same query. Subsequent canonicalization can remove uses.
`getReusableOutputTensor` shares read-only destination selection with vector
lowering. It identifies a tensor operand or requires fresh storage. It does not
prove a physical alias, assign an address, or replace one-shot bufferization.
The result is valid only for the current SSA graph. Changes to uses, operation
order, loop yields, or operands require recomputation. Lowering consumes this
decision immediately; it does not cache values across rewrites. Current PIM
bufferization consumes the explicit destination ties introduced by target
lowering. Output-copy forwarding also consumes the resulting allocation,
subview, copy, and dominance facts. An early destination choice alone cannot
describe those physical aliases or the final storage lifetimes.
Logical storage identity and physical address equality are different facts.
Local-memory planning can assign the same address to distinct allocations whose
lifetimes do not overlap. Scalar register setup therefore needs placement and
instruction-order information in addition to logical alias relationships.
Abstract cost estimates need not reconstruct these addresses.
This input plan does not count physical constant loads. A forwarded host
constant can need a new local buffer at each use. Bufferization, host folding,
alias analysis, and memory normalization determine the resulting requests.
`isZeroBitPatternSplat` shares constant-content classification between graph
accounting and code generation. PIM replaces eligible positive-zero and integer-zero
loads with local fills. Negative floating zero retains its distinct bit pattern.
Constant content is candidate-independent; input placement and buffer reuse
depend on the realized candidate. Production uses these shared decisions when
extracting Spatial execution events for the queue estimate.
Execution transpose costs share `getTransposeRun` with lowering: round memory
reads/writes per run and add vector execution when source stride is not one.
The [Pimsim vector unit](../../backend-simulators/pim/pimsim-nn/src/core/component/VectorUnit.cpp)
@@ -118,3 +171,89 @@ scheduling overview. Cached PEFT center distances avoid a topology scan per task
projection and priority analyses share/cache work rather than revisiting every
template body per chunk. Ready-transfer signature searches remain another
potential compile-time hotspot.
### Spatial execution costs
`SchedulingTarget::estimateRealizedScheduleCost` is a read-only query on realized Spatial IR.
The PIM adapter extracts compact ordered execution events without constructing PIM operations.
It uses physical core assignments, communication, input-materialization decisions, and shared copy geometry.
No physical buffer addresses are required.
The shared engine models controller arbitration, source FIFOs, synchronization, and repeated generations.
Both arbitration policies are sensitivity checks, not guaranteed bounds.
The engine returns typed intervals and recurrence evidence before diagnostic serialization.
The target adapter adds static lowering support. Spatial owns the selection policy margin.
JSON fields are report output and do not control availability or eligibility.
Typed evaluation completes before reporting. Serialization never runs queue analysis.
Only optional diagnostics construct fingerprints, traces, and per-core resource reports.
Normal compilation extracts costs for multiple candidates and lowers only the selected schedule.
A single candidate bypasses the cost query entirely.
Candidate summaries distinguish available timing, static lowering support, and proved modeled recurrence.
Unknown support remains explicit. All candidates must satisfy these checks before comparison.
`Unsupported` identifies a known static violation, including invalid or repeated physical-core assignments.
`Unknown` identifies missing evidence or an unmodeled form, rather than a proven backend rejection.
`Supported` covers checked static forms and does not prove final allocation success.
A finite interval alone does not establish eligibility.
Preflight does not repeat bufferization or allocation.
The backend owns final legality and signed 32-bit local-memory address validation.
Resource failures marked by that owner permit an explicit retry from the original checkpoint.
Unclassified failures stop compilation. Every attempted schedule and cause appears in the selection report.
### Complete-state recurrence
Queue analysis checks translated states at one fixed core's body entry.
Snapshots contain completion counts relative to core entry, ready-input counts, and busy state for every event.
Run-length encoding keeps these records compact.
Snapshots also contain pending events in arbitration order, remaining FIFO tokens, controller availability, and the tie policy.
One pending firing per event makes node IDs sufficient to resolve simultaneous-event ties.
The graph and target parameters remain fixed within each query.
Time offsets use checked unsigned differences.
Equal states require equal positive generation progress on every core.
This proves recurrence for the deterministic abstract model, not physical execution-time accuracy.
Reporting counters and analysis budgets are not hardware state.
Token initialization distances and graph edges remain immutable within a query.
Consumption clears a drained queue and resets its head immediately.
Drained and never-populated queues therefore have the same readiness state.
Consumed prefixes, capacities, and compaction indices cannot affect future timing or FIFO contents.
An idle controller's past completion cannot delay future arrivals.
These normalizations require causal, monotonic transitions, which the proof checks.
Proof construction uses at most 256 generations and 16 MiB of snapshot storage.
Each checkpoint scans events, dependencies, and remaining tokens, then orders a copy of the pending frontier.
Exact comparisons use bounded history; hashes only filter comparisons.
Unproved recurrence remains explicit and cannot guide multi-candidate selection.
When recurrence is proved, its interval is authoritative for that arbitration policy and its report.
Proof-mode reports retain interval-window results under `legacy_interval_window`.
Normal costing stops once recurrence is proved. Detailed diagnostics can continue to collect interval-window evidence.
A legacy event or generation limit cannot override a later complete-state proof.
Budget limits, lack of ready events, and unproved recurrence remain separate evidence.
Internal diagnostic queries can cap interval-window work without reducing the recurrence proof budget.
The standalone instruction diagnostic keeps its interval-window behavior when recurrence is not requested.
### Approximation and diagnostics
Scalar setup, copy forwarding, local reuse, and host-slice folding can change request readiness.
These effects remain prediction limitations. Physical address reuse itself adds no execution work.
Direct returns use the same return-operand test as lowering and preserve output multiplicity.
Captured weights used by vector operations require host loads.
Publication-only slices of compact results use `CopyAccessPlan` runs without an intermediate copy.
Matrix writeback requires complete aligned array groups.
Partial output groups, encoded layouts, unavailable geometry, and indirect output collection produce unavailable costs.
`--pim-periodic-diagnostics` adds resource summaries and ordered-request fingerprints.
Normal compilation creates no fingerprints, detailed traces, or sensitivity runs.
Optional local-copy sensitivity removes modeled copy work without changing generated code.
It does not describe a legal transformed program or an interval bound.
The standalone instruction diagnostic remains available for offline timing validation.
Checked arithmetic rejects unrepresentable sizes, loop bounds, repeated work, and service durations.
Work-only repeats remain compact. Immutable summaries are cached within a query.
Ordered memory and communication occurrences expand within the event budget.
For `P` compact records, `R` expanded events, and `D` dependencies, construction costs `O(P+R+D)` time.
With `B` processed events and `T` token propagations, queue evaluation costs `O(B log R+T)` time.
Storage costs `O(P+R+G D)` for bounded generation count `G`, plus bounded recurrence history.
Reports separate extraction, expansion, and queue-analysis times.
Fingerprints compare memory and blocking-operation order, without addresses or work between requests.
Equal fingerprints therefore do not prove equal request readiness or timing.
+73 -103
View File
@@ -136,7 +136,8 @@ introduced after deferred realization, including final output collection, is
also absent. Instruction issue, local contention, network injection queues, and
memory arbitration remain unmodeled. Missing effects are listed in the report;
`selection_eligible` remains false. This plan-level diagnostic does not rank
production candidates. Production uses the instruction-level queue model below.
production candidates. Production extracts execution events from realized Spatial
and evaluates them with the shared queue engine.
For `V` events, `E` edges, and `R` recurrence edges, construction requires
`O(T + F + S log S)` time and `O(V + E + S)` storage, for existing compute
@@ -236,7 +237,8 @@ generations. The maximum request wait includes bootstrap. Event-index order
resolves simultaneous readiness. A second run reverses that order and reports
`alternative_tie_interval_ns`. These two orders are sensitivity checks, not
bounds over every possible tie order. Clock phases and frontend overlap remain
approximate. The standalone report remains read-only. Production selection uses the same queue model through its C++ API.
approximate. The standalone report remains read-only.
Production evaluates Spatial-derived events with the shared queue engine.
For `R` global requests, the queue mode adds `O(R)` events and dependencies.
For `V` resulting events, `E` dependencies, and generation bound `K`, execution
@@ -250,120 +252,88 @@ The standalone frontend retains the recurrence witness. Neither analysis changes
## Default throughput selection
Full PIM code generation performs these steps for `--pipeline > 1`.
`--pim-only-codegen` retains the physical schedule already present in its input.
Full PIM code generation uses Spatial-only queue-aware selection for `--pipeline > 1`.
Latency PEFT and `--pim-only-codegen` retain their existing behavior.
Intermediate emission retains the original schedule and stops at its requested stage.
Its schedule can differ from the final program.
The normal compilation workflow is:
The compiler performs these steps:
1. Run the ONNX frontend, layout planning, common lowering, and graph preparation once.
2. Generate at most eleven verified physical schedules from the existing searches.
3. Remove duplicate placements, including their core order and stage identities.
4. Clone the prepared module for each candidate using MLIR's lossless `IRMapping`.
5. Remap each schedule's compute identities to the clone.
6. Realize communication and lower the clone through verification and instruction selection.
7. Generate binary instruction records in memory, without weights, exports, or candidate files.
8. Run both static controller-arbitration tie policies on that instruction program.
9. Select an eligible candidate and emit only its normal program.
1. Run the frontend, layout planning, and graph preparation once.
2. Generate at most eleven legal schedules from the existing searches.
3. Remove duplicate physical placements, including stage identities and core order.
4. Realize each candidate in an independent clone of the prepared checkpoint.
5. Query the target adapter over realized Spatial IR, then release the clone.
6. Require supported static forms, available costs, and proved modeled recurrence for every candidate.
7. Select a candidate with the existing 2.1% selection margin.
8. Realize the selected schedule again from the unchanged checkpoint.
9. Lower, allocate, verify, and emit only the selected program.
Immutable attributes, including constants and weights, remain shared in the MLIR
context. Candidate operations, values, communication plans, and synchronization
state remain independent. Trials never mutate the prepared checkpoint. Candidates
run sequentially. At most one best lowered module survives between trials.
The retained module follows the conservative selection rule throughout evaluation.
A close alternative does not evict candidate zero. Unsupported timing contracts
and synchronization ablation generate only the original schedule and lower it
once. The compiler checks this before collecting alternatives. The frontend is
not repeated.
A single candidate skips event extraction, queue analysis, recurrence proof, and comparison.
It is realized and lowered once, even when optional timing is unavailable.
No rejected candidate enters PIM IR or instruction generation during cost evaluation.
Constants and weights remain shared immutable attributes in the MLIR context.
Candidate operations, communication plans, and synchronization state remain independent.
No full candidate compilation directories are created.
Intermediate targets (`--EmitSpatial`, `--EmitPim`, and `--EmitPimBufferized`)
retain the original schedule. They do not perform queue-aware selection or run
transformations beyond the requested stage. Their schedule can therefore differ
from final code generation. Full PIM code generation uses queue-aware selection.
`SchedulingTarget::estimateRealizedScheduleCost` is the read-only whole-schedule query.
Its typed result carries both intervals, recurrence evidence, static lowering support,
failure reasons, and the selection margin. Unknown support remains explicit.
The target supplies model evidence. Spatial applies the 2.1% selection policy margin.
Selection does not read JSON diagnostics. Reports serialize the typed results.
Spatial owns candidate generation, legality, placement, and target-neutral comparison.
The PIM adapter owns execution-event extraction, instruction timing, and resource contention.
The shared queue engine models controller arrival arbitration and cross-generation dependencies.
The compiler does not execute candidate programs or invoke a simulator.
The PIM compiler owns instruction timing and controller behavior. Spatial owns
legal candidate generation and schedule identity remapping. `PimPeriodicAnalysis`
is the single instruction-level timing implementation. `raptor-periodic` is its
read-only diagnostic frontend. The compiler invokes neither that executable nor
an external selection script.
Both arbitration policies must establish complete-state recurrence within bounded analysis budgets.
Their estimates are sensitivity checks, not formal bounds.
The candidate with the lowest larger estimate must separate from the original schedule.
Comparison uses `[min(ties)/1.021, max(ties)/0.979]`.
Close alternatives retain candidate zero. The 2.1% margin is a policy heuristic.
Spatial timing predictions have no formal accuracy bound.
The search finds the best distinguishable candidate in its bounded set, without global optimality.
`SchedulingTarget::prepareScheduleCost` is an optional whole-schedule cost query.
It accepts a verified target-lowered module and returns an opaque program identity
and a deferred static evaluation. Spatial compares the resulting interval
estimates and applies the target's reported uncertainty. Missing uncertainty
makes an estimate ineligible. The target owns instruction extraction,
protocol timing, contention, and unavailable-result reasons. Preparation and
evaluation do not change the candidate. Equal program identities avoid repeated
queue analysis. An absent query retains the original schedule.
If generation yields one distinct schedule, no comparison is required. The
compiler realizes and lowers it once, without a trial clone, duplicate
instruction generation, or queue analysis. The selection report records this
case separately from unavailable estimates. Final code generation is unchanged.
Unavailable timing, unknown static support, or unproved recurrence stops a multi-candidate comparison.
The diagnostic identifies each affected candidate and property.
The compiler does not silently exclude unknown candidates or use instruction-level fallback selection.
PIM transport details reside in `PimInstructionTiming`. The optional plan-level
diagnostic also belongs to the PIM compiler. Spatial supplies its physical plans
through an injected reporting callback; it does not encode the PIM handshake.
The instruction emitter remains the source of register setup, address-dependent
instruction suppression, loop expansion, and physical array-group identity.
Candidate evaluation uses this emitter in memory after local-memory planning.
It does not reconstruct these rules from scalar Spatial operation costs.
Static preflight does not prove final backend legality.
The selected candidate runs through normal bufferization, allocation, and backend verification.
A backend owner can mark a resource-limit diagnostic as candidate-specific.
Currently, the local-memory allocator marks signed 32-bit address-space exhaustion.
The compiler discards that failed clone, explicitly records its rejection, and selects again from the remaining estimated schedules.
The first remaining schedule becomes the reference for the same selection policy.
Each schedule is attempted at most once. No graph or placement constraints are relaxed.
Unclassified or systemic backend failures stop compilation immediately.
Code-generation failures also stop compilation when recovery cannot be justified.
If every candidate fails, one diagnostic lists all attempted candidates and causes.
Production projects native fp32 transfer sizes and vector widths to the existing
oracle int8 timing contract. It preserves instruction counts, matrix precision,
vector element counts, synchronization, and transfer identities. This projection
does not change the emitted native program. Unsupported widths, targets without
complete timing, and unsupported instruction semantics produce explicit fallback
reasons. Synchronization ablation retains the original schedule.
Both arbitration tie runs must converge within the existing generation and event
budgets. Their estimates are sensitivity checks, not formal bounds. Duplicate
instruction programs are analyzed once. The candidate with the lowest larger
estimate must separate from the original schedule. Close alternatives do not
cancel a gain that separates from the original. Comparison uses `[min(ties)/1.021, max(ties)/0.979]`. The fixed 2.1% prediction uncertainty
covers offline validation error with upward rounding. It is a selection margin,
not a fitted instruction latency or a guaranteed error bound. Overlapping ranges
with the original retain candidate zero. Missing original estimates also retain candidate zero.
Unavailable alternatives cannot displace an eligible original. This search finds
the best distinguishable candidate in its bounded set; it does not establish
global optimality.
`reports/pipeline_selection.json` records candidate identities, estimates,
selection reasons, and generation, lowering, instruction-generation, queue-analysis,
and finalization times. The selected verified lowered module and its captured
schedule report are reused without repeating materialization or communication
realization. Memory reports come from normal final code generation.
With `--enable-timing`, candidate summaries also contain per-pass lowering times
and operation/value counts before bufferization and instruction generation.
Instruction-generation summaries separate preparation from per-core emission.
Explicit Spatial exports, IR snapshots,
and plan-level periodic diagnostics reconstruct the selected Spatial schedule
because those outputs require pre-lowering operations. This optional work does
not change the retained instruction program. Candidate instructions and rejected
IR are released immediately. No temporary compilation directories are created.
`--pim-dump-ir` explicitly enables diagnostic IR snapshots;
these snapshots elide large constants and cannot serve as lossless checkpoints.
`reports/pipeline_selection.json` records Spatial costs, eligibility, selection, and backend attempts.
It also records candidate generation, realization, extraction, expansion, queue, lowering, and finalization times.
`--enable-timing` adds per-pass lowering times and IR operation/value counts.
Final instruction generation uses the normal compiler timing report.
Explicit Spatial exports and IR dumps reconstruct pre-lowering artifacts for the selected schedule.
They do not change the emitted program. Diagnostic MLIR dumps elide large constants and are not lossless checkpoints.
### Static analysis, diagnostics, and external validation
Production decisions use only the input program, compiler options, candidate
instructions and plans, and configured target resources and timing. The static
model executes deterministic abstract events. It does not execute the generated
program. It reads no benchmark CSV, profiling trace, measured latency cache, or
reference artifact. Identical static inputs produce the same decision regardless
of historical measurements. Throughput deferred-demand ordering uses IR order,
rather than allocation addresses. Latency PEFT and its demand ordering are unchanged.
Production decisions use the input graph, compiler options, physical plans, and configured target parameters only.
Identical static inputs produce the same decision regardless of historical measurements.
There are no measured-latency caches, workload lookup tables, or fitted timing corrections.
Optional diagnostics expose recurrence witnesses, instruction timing, and resource
service. Offline Pimsim fast-mode measurements validate these predictions and
candidate rankings. Simulator results never enter production selection. Timing
corrections must follow verified hardware behavior or configured parameters.
`raptor-periodic` retains the instruction-derived analyzer as a standalone validation frontend.
It shares the same timing definitions and queue engine with Spatial event evaluation.
Optional diagnostics expose resource work, recurrence, and ordered-request fingerprints.
Offline Pimsim fast-mode runs validate predictions and emitted programs.
Simulator measurements never enter production selection.
For `B <= 11` schedules, `T` compute instances, and `P` cores, captured schedules
use `O(B(T+P))` storage. Placement deduplication takes `O(B²T)` time. Each distinct
finalist requires downstream lowering and one instruction scan. Queue execution
uses the bounded complexity stated above. Peak storage includes the prepared
module, one best lowered module, one trial, and one queue graph. It does not scale
with retained complete candidate programs. Maximum-cycle-ratio diagnostics do not
run in the candidate evaluation loop.
For `C <= 11` candidates, scheduling metadata uses `O(C(T+P))` storage for `T` compute instances and `P` cores.
Placement deduplication costs `O(C²T)` time.
Candidate realization and extraction are sequential; complete candidate modules are not retained.
Normal successful compilation performs one candidate-specific backend lowering.
Recovery performs at most `C` lowerings. Retry selection costs `O(C²)` time and `O(C)` metadata.
The bounded queue-analysis costs are described in the implementation notes.
## Capacity and recovery