Raptor/docs/compiler/scheduling.md

Spatial scheduling

prepareSpatialSchedules constructs the shared ComputeGraph. Latency mode uses PEFT. For --pipeline > 1, the PIM compiler selects among bounded pipeline candidates with static queue-aware instruction analysis. Pipeline mode does not run PEFT.

See implementation notes for fragment-view eligibility, receive forwarding, transfer rounding, cost-model boundaries, and recovery limits.

Ownership

Responsibility	Owner
Instances, dependencies, execution/transfer costs, resident weights	ComputeGraph, ComputeInstanceUtils
Target resources and physical link costs	Compiler target adapter via SchedulingTarget
Latency priorities, reservations, placement, per-core order	PeftScheduler
Capacity-driven workload splitting	PipelineWorkloadPreparation
Groups, stage partitioning, core budgets	buildPipelineGroups, partitionPipelineGroups, assignPipelineStages
Topology clustering and stage-local packing	PipelineCoreLayout, packPipelineStages, IndexedResidentWeights
Timing evaluation and publication equivalence	evaluateSchedule, populateScheduleEquivalenceClasses
Selected execution order and homogeneous runs	Scheduled compute planning/materialization
Routing, forwarding, blocking communication order	Deferred transfer planning and communication scheduling
READY/FREE counts, registers, overwrite/restart boundaries	Deferred boundary planning/realization

Both strategies produce MergeScheduleResult: core assignments, contiguous per-core slots, estimated starts, last-task markers, equivalence classes, and cost summaries. Pipeline results also assign each core a stage. Materialization uses verified slots for execution order and merges only consecutive compatible instances. Graph/class lane identity does not imply core identity or execution order.

Dependencies and transfer costs share getGraphBatchPublicationMap. Statically proven physical-slot projections support many-to-many edges; unproven projections retain complete producer dependencies, including region captures. Malformed publication metadata fails graph construction.

Normative contracts: graph fragments, pipeline stages, target boundary, synchronization, and performance.

Costs and interpretation

Execution and fixed transfer costs use integer nanoseconds, converted from target cycles. Bytes, elements, flits, weight slots, and lanes remain resource quantities. Memory work is rounded at its actual fragment/run granularity; runtime-free views cost zero. Transposes charge memory traffic and strided vector work; concatenation charges per-input blocks per outer row. Direct receives into assembly storage avoid duplicate endpoint-memory charges. Sub-byte storage is analyzable in Spatial but rejected by byte-addressed PIM communication lowering.

The evaluator separates serial work, per-core work, dependency critical path, summed transfer work, single-inference completion, and stage spans. Dependency paths exclude core-order constraints; completion includes them. Stage spans exclude cross-stage routing, restart/host synchronization, and contention. Simulator latency and measured throughput remain the performance gates.

Pipeline candidate generation retains the original ordered-packing/dependency-tail heuristic and the memory-aware and sharing-aware searches. The partition DP keeps at most eight compute/memory trade-offs per state. Four final alternatives and the initial partition undergo core allocation, topology placement, and capacity verification. The previous compute/memory guards identify candidate zero, the original fallback. They do not reject legal alternatives before queue analysis. All candidates preserve candidate zero's non-empty pipeline stages. PEFT is unchanged.

ComputeGraph retains physical publication and consumer identities separately from aggregated dependency edges. Cross-stage demands charge one store per shared physical publication and one load per consumer transfer. Publication identity includes the immutable result generation, exact payload type, and physical source fragment. Scalar projections retain separate stores conservatively. Same-stage local/direct transfers incur no host traffic. Function inputs, constants, and outputs supply a separate baseline estimate. The target adapter supplies access width and read/write service times; request rounding precedes aggregation. Serialized service is shared by all stages and is not added to individual stage spans. Baseline loads remain approximate before placement, bufferization, aliasing, and load folding. Scalar publication stores round each contiguous source run separately. Consumer load counts can remain conservative before load folding. The scheduler evaluates boundary demand again after physical placement. Full graph fragments resolve to unique physical producers, so their sharing does not depend on reader placement. Deferred planning uses the same identity rule and retains separate loads, including repeated loads on one reader core. Placement ranks maximum per-core compute/incident-transfer demand, then stage completion and compute load; within 1% demand, stage span breaks the tie. These are placement heuristics, not a steady-state timing model.

Periodic dependency diagnostics

--pim-periodic-diagnostics writes reports/pipeline_periodic.json after deferred boundary planning. This diagnostic does not change schedule selection or IR. It consumes the cached compute-instance costs, materialized core order, physical transfer families, publication store flags, and allocated READY/FREE registers. Compute runs remain atomic; their lanes and loop iterations are not expanded. Compact runs require one physical core; multi-core classes use per-lane steps. One event sequence represents each physical core. The model retains one planned load per physical consumer transfer and one store per shared publication. Later load folding or strided transfer splitting can change request granularity.

Edges express t(v,k) >= t(u,k-d) + w. Within-generation edges have distance zero. Each core's completion-to-entry edge has distance one. Reader-entry FREE signals therefore release the preceding generation through that edge. Signals join their exact planned WAIT count. Dissemination barriers, restart permissions, and the downstream release tree retain their core identities and order. The diagnostic checks its event-register summary against the emitted program in the synchronization tests; it does not replace deadlock verification.

The injected target describes direct-transfer completion and link, local-memory, shared-memory, and signal timing. The current adapter specifies peer permission before a source read. SEND completion follows the read and payload enqueue; RECV completion also requires payload delivery and a destination write. Unsupported or incomplete target timing produces an unavailable report.

The analysis brackets the maximum cycle ratio within one nanosecond. It tests edge delays w - II*d, sweeping the distance-zero DAG between recurrence relaxations. At most R+1 sweeps detect a positive cycle, where R counts positive-distance edges. Bisection uses at most 64 tests. Timing outside the supported numeric range produces an explicit unavailable result. A predecessor cycle supplies a witness, including delay, iteration distance, cores, and links. Detailed witness output is limited to 128 edges; totals cover the entire cycle.

The reported interval estimate is the maximum of the modeled recurrence, per-core compute work, and aggregate serialized shared-memory service. Memory service is never assigned a fixed global request order. Taking the maximum does not add the same service twice. The within-generation dependency span is reported separately and is not an initiation interval.

These are conditional estimates for the abstract execution model, not exact simulator predictions or guaranteed physical lower bounds. Serial compute costs can overestimate overlapping work. Baseline memory accesses retain their aggregate service bound, but their positions are not modeled. Communication introduced after deferred realization, including final output collection, is also absent. Instruction issue, local contention, network injection queues, and memory arbitration remain unmodeled. Missing effects are listed in the report; selection_eligible remains false. This plan-level diagnostic does not rank production candidates. Production uses the instruction-level queue model below.

For V events, E edges, and R recurrence edges, construction requires O(T + F + S log S) time and O(V + E + S) storage, for existing compute instances T, physical transfers F, and synchronization events S. Analysis requires O(L (R+1) (V+E)) time and O(V+E) storage, where L <= 64. Here R is at most the number of active physical cores. There is no dense all-core dependency matrix or inference-generation unrolling. For C analyzed candidates, time scales by C; scratch storage can be reused. The current diagnostic analyzes only the selected schedule (C = 1).

Post-codegen instruction diagnostic

raptor-periodic provides an explicit, read-only diagnostic over an emitted binary artifact. It accepts native or oracle-exported artifacts. Use the oracle-exported artifact when comparing against Pimsim performance results:

build_release/Release/bin/raptor-periodic /tmp/model/pimsim_nn \
  --pim-target-config=validation/pimsim_configs/pimcomp/arch-a/throughput_config_1000ms.json \
  --core-count=168 --crossbar-count=96 --crossbar-size=128 > /tmp/periodic.json

The diagnostic uses physical instruction files and their array-group map. Each file owns one physical core's work, including all lanes and static loop iterations already emitted there. It does not expand compiler loops or batches. It replaces Spatial compute costs rather than adding instruction costs to them. The lightweight compiler report remains separate and does not invoke this tool.

The supported execution contract requires ROB capacity one. Non-scalar instructions therefore retain their serial execution order. Local compute and blocking memory runs become weighted chunks between communication events. Every emitted global request contributes its actual size, rounding, and count. Inputs, constants, split transfers, folded loads, and output collection are included through their emitted instructions. Their positions relative to communication remain explicit. The global service bound remains separate; the model does not impose an order on different cores' global requests.

Timing queries reuse the injected memory, link, and matrix definitions. PIM instruction semantics belong to the compiler target adapter. Vector writeback follows the oracle's actual width rules. Matrix writeback uses the physical array-group width. Local-memory timing uses its own configured period. The oracle passes vector_latency_cycle directly to a nanosecond wait. This parameter does not scale with the processor period; no separate vector clock exists in the supported oracle configuration. In the current oracle, global loads return zero network payload bytes. Global stores retain their outgoing payload cost.

The source switch has one unbounded FIFO for outgoing requests. Enqueue does not wait for earlier requests. FIFO dispatch waits for their completion. Direct SEND requires permission delivery before the source read. Its payload enqueue event coincides with source-read completion. SEND completes after enqueue, independently of FIFO dispatch or payload delivery. RECV requires delivery and destination write. Permission responses join both peer readiness and the receiver's outgoing FIFO. SYNC delivery can precede its local completion. The model retains WAIT counts, restart events, and cross-generation FIFO state. It does not replace the compiler's synchronization or deadlock verification.

--model-network-queues=false and --model-issue=false support timing comparisons. The latter omits scalar issue and retirement delays while retaining SYNC's instruction-defined execution time. Completion notifications can join the current edge before the clock process runs. The model uses minimum retirement delays; it does not reproduce SystemC clock-phase ordering. Scalar issue is charged serially, so these estimates are not guaranteed physical lower bounds. The base recurrence estimate omits global arbitration, frontend buffering, and initial bootstrap. Production does not rank candidates with this recurrence-only estimate. Reports with at most 128 events include timing_graph for focused timing checks. Its dependencies distinguish payload enqueue from FIFO dispatch and delivery. The instruction test accepts --pimsim /path/to/ChipTest for optional fast-mode timing comparisons. These cover both hardware profiles and distinct processor, local-memory, and vector latency settings.

For N emitted instructions, F direct transfers, S synchronization events, and P active cores, construction takes O(N + (F+S) log(F+S)) time. The indexed graph uses O(F+S+P) space and at most 2P recurrence edges. Global requests are counted during the same scan without per-request graph nodes. One instruction file is memory-mapped at a time. Maximum-cycle-ratio analysis uses the same bounded algorithm as the lightweight diagnostic.

--model-memory-controller adds a separate memory_controller estimate. Each global request retains its readiness, transport, service, and completion dependencies. The controller serves requests in predicted arrival order. Read and write requests share that FIFO. Zero-size requests also join it. Source switches remain occupied until transport completion. The model does not assign a fixed global request order or consume simulator trace order.

The analysis executes the indexed periodic graph with timestamped tokens. FIFO tokens retain dependencies across generations; the graph is not copied. A request arrival reserves the controller after earlier arrivals, and its completion enables dependent events. Independent cores and switches remain concurrent. The maximum-cycle-ratio solver and its report remain unchanged.

The queue estimate compares adjacent two-generation windows with the oracle's one-percent convergence threshold. It reports the mean interval across both windows. Per-core queue waits use the corresponding four completed body generations. The maximum request wait includes bootstrap. Event-index order resolves simultaneous readiness. A second run reverses that order and reports alternative_tie_interval_ns. These two orders are sensitivity checks, not bounds over every possible tie order. Clock phases and frontend overlap remain approximate. The standalone report remains read-only. Production selection uses the same queue model through its C++ API.

For R global requests, the queue mode adds O(R) events and dependencies. For V resulting events, E dependencies, and generation bound K, execution takes O(K(E + V log V)) time. Storage is O(V + E + B), where B counts pending tokens. The event heap contains at most one event per node. FIFO token storage retains pending generations, rather than instruction-stream copies. The diagnostic limits each event to 64 generations and processed events to 16(V + E). It reports unavailable if either bound prevents convergence. The two tie-order runs reuse the same graph. Production uses this queue execution directly and omits maximum-cycle-ratio analysis. The standalone frontend retains the recurrence witness. Neither analysis changes latency PEFT.

Default throughput selection

Full PIM code generation performs these steps for --pipeline > 1. --pim-only-codegen retains the physical schedule already present in its input.

The normal compilation workflow is:

1. Run the ONNX frontend, layout planning, common lowering, and graph preparation once.
2. Generate at most eleven verified physical schedules from the existing searches.
3. Remove duplicate placements, including their core order and stage identities.
4. Clone the prepared module for each candidate using MLIR's lossless IRMapping.
5. Remap each schedule's compute identities to the clone.
6. Realize communication and lower the clone through verification and instruction selection.
7. Generate binary instruction records in memory, without weights, exports, or candidate files.
8. Run both static controller-arbitration tie policies on that instruction program.
9. Select an eligible candidate and emit only its normal program.

Immutable attributes, including constants and weights, remain shared in the MLIR context. Candidate operations, values, communication plans, and synchronization state remain independent. Trials never mutate the prepared checkpoint. Candidates run sequentially. At most one best lowered module survives between trials. The retained module follows the conservative selection rule throughout evaluation. A close alternative does not evict candidate zero. Unsupported timing contracts and synchronization ablation generate only the original schedule and lower it once. The compiler checks this before collecting alternatives. The frontend is not repeated.

Intermediate targets (--EmitSpatial, --EmitPim, and --EmitPimBufferized) retain the original schedule. They do not perform queue-aware selection or run transformations beyond the requested stage. Their schedule can therefore differ from final code generation. Full PIM code generation uses queue-aware selection.

The PIM compiler owns instruction timing and controller behavior. Spatial owns legal candidate generation and schedule identity remapping. PimPeriodicAnalysis is the single instruction-level timing implementation. raptor-periodic is its read-only diagnostic frontend. The compiler invokes neither that executable nor an external selection script.

SchedulingTarget::prepareScheduleCost is an optional whole-schedule cost query. It accepts a verified target-lowered module and returns an opaque program identity and a deferred static evaluation. Spatial compares the resulting interval estimates and applies the target's reported uncertainty. Missing uncertainty makes an estimate ineligible. The target owns instruction extraction, protocol timing, contention, and unavailable-result reasons. Preparation and evaluation do not change the candidate. Equal program identities avoid repeated queue analysis. An absent query retains the original schedule. If generation yields one distinct schedule, no comparison is required. The compiler realizes and lowers it once, without a trial clone, duplicate instruction generation, or queue analysis. The selection report records this case separately from unavailable estimates. Final code generation is unchanged.

PIM transport details reside in PimInstructionTiming. The optional plan-level diagnostic also belongs to the PIM compiler. Spatial supplies its physical plans through an injected reporting callback; it does not encode the PIM handshake. The instruction emitter remains the source of register setup, address-dependent instruction suppression, loop expansion, and physical array-group identity. Candidate evaluation uses this emitter in memory after local-memory planning. It does not reconstruct these rules from scalar Spatial operation costs.

Production projects native fp32 transfer sizes and vector widths to the existing oracle int8 timing contract. It preserves instruction counts, matrix precision, vector element counts, synchronization, and transfer identities. This projection does not change the emitted native program. Unsupported widths, targets without complete timing, and unsupported instruction semantics produce explicit fallback reasons. Synchronization ablation retains the original schedule.

Both arbitration tie runs must converge within the existing generation and event budgets. Their estimates are sensitivity checks, not formal bounds. Duplicate instruction programs are analyzed once. The candidate with the lowest larger estimate must separate from the original schedule. Close alternatives do not cancel a gain that separates from the original. Comparison uses [min(ties)/1.021, max(ties)/0.979]. The fixed 2.1% prediction uncertainty covers offline validation error with upward rounding. It is a selection margin, not a fitted instruction latency or a guaranteed error bound. Overlapping ranges with the original retain candidate zero. Missing original estimates also retain candidate zero. Unavailable alternatives cannot displace an eligible original. This search finds the best distinguishable candidate in its bounded set; it does not establish global optimality.

reports/pipeline_selection.json records candidate identities, estimates, selection reasons, and generation, lowering, instruction-generation, queue-analysis, and finalization times. The selected verified lowered module and its captured schedule report are reused without repeating materialization or communication realization. Memory reports come from normal final code generation. With --enable-timing, candidate summaries also contain per-pass lowering times and operation/value counts before bufferization and instruction generation. Instruction-generation summaries separate preparation from per-core emission. Explicit Spatial exports, IR snapshots, and plan-level periodic diagnostics reconstruct the selected Spatial schedule because those outputs require pre-lowering operations. This optional work does not change the retained instruction program. Candidate instructions and rejected IR are released immediately. No temporary compilation directories are created. --pim-dump-ir explicitly enables diagnostic IR snapshots; these snapshots elide large constants and cannot serve as lossless checkpoints.

Static analysis, diagnostics, and external validation

Production decisions use only the input program, compiler options, candidate instructions and plans, and configured target resources and timing. The static model executes deterministic abstract events. It does not execute the generated program. It reads no benchmark CSV, profiling trace, measured latency cache, or reference artifact. Identical static inputs produce the same decision regardless of historical measurements. Throughput deferred-demand ordering uses IR order, rather than allocation addresses. Latency PEFT and its demand ordering are unchanged.

Optional diagnostics expose recurrence witnesses, instruction timing, and resource service. Offline Pimsim fast-mode measurements validate these predictions and candidate rankings. Simulator results never enter production selection. Timing corrections must follow verified hardware behavior or configured parameters.

For B <= 11 schedules, T compute instances, and P cores, captured schedules use O(B(T+P)) storage. Placement deduplication takes O(B²T) time. Each distinct finalist requires downstream lowering and one instruction scan. Queue execution uses the bounded complexity stated above. Peak storage includes the prepared module, one best lowered module, one trial, and one queue graph. It does not scale with retained complete candidate programs. Maximum-cycle-ratio diagnostics do not run in the candidate evaluation loop.

Capacity and recovery

Distinct-weight union / per-core capacity is a lower bound; an oversized task is infeasible. Ordered pack counts estimate budgets, and failed heuristic packing does not prove infeasibility. Exhausted budgets restart from lower bounds and constructive packing establishes feasibility.

Latency reservations try load/communication-aware placement, then largest-weight-first and shared reuse-first best fit on failure. Pipeline packing compares arrival/load-aware candidates and uses constructive recovery. The shared constructor retries in reverse order, then at most 16 single-choice deviations within 65,536 core-candidate checks. Successful placements are preserved. Exhausted search produces a bounded diagnostic, not a claim of proved hardware infeasibility.

Complexity and limits

Let T be tasks, E dependencies, P cores, G groups, S stages, R weight references, W distinct weights, C modeled work, F physical fragment/consumer demands, and bounded frontier size K = 8.

Analysis	Time / storage
Schedule evaluation	O((T + E) log T + P) / O(T + E + P)
Weight indexing	Up to O(R W) equality comparisons
Weight lower bounds	O(R + W) / O(W) bits
Constructive best fit	O(T log T + P R + P W) / O(P W) bits plus O(T + P); bounded retries only on failure
Partition DP	Up to O(S G² ((T + R) log C + W + K³)) plus dependency traversal / O(S G² + S G K); at most five additional physical placements per memory objective
Cut memory service	Expected O(F + G) time / O(F + G) scratch and retained demands
Topology clustering / cached PEFT distances	O(P²) time
Concat cost extraction	O(A D) for A inputs of rank D; constant auxiliary space
Cached pipeline priorities	O(T + D) time / O(T + G) space for D template-body operations
Exact projections	O(Q + L) time for Q slot visits and L producer lanes; repeated demands aggregate by projection and fragment
Conservative producer tracing	Linear in the input projection DAG, including captures

Latency insertion-gap searches can remain quadratic. Issue overhead, eliminated assembly copies, contention, and pipeline synchronization remain cost approximations. Changes require matched execution measurements. Loop-invariant projections use trip-count multiplication. Loop-invariant resident weights are collected once. Loop-dependent projections and weights can still require iteration visits; retained projection demands scale with distinct physical fragments instead of repeated visits.