Raptor/docs/compiler/scheduling.md

Spatial scheduling

prepareSpatialSchedules constructs the shared ComputeGraph. Latency mode uses PEFT. For --pipeline > 1, the PIM compiler selects among bounded pipeline candidates with static queue-aware instruction analysis. Pipeline mode does not run PEFT.

See implementation notes for fragment-view eligibility, receive forwarding, transfer rounding, cost-model boundaries, and recovery limits.

Ownership

Responsibility	Owner
Instances, dependencies, execution/transfer costs, resident weights	ComputeGraph, ComputeInstanceUtils
Target resources and physical link costs	Compiler target adapter via SchedulingTarget
Latency priorities, reservations, placement, per-core order	PeftScheduler
Capacity-driven workload splitting	PipelineWorkloadPreparation
Groups, stage partitioning, core budgets	buildPipelineGroups, partitionPipelineGroups, assignPipelineStages
Topology clustering and stage-local packing	PipelineCoreLayout, packPipelineStages, IndexedResidentWeights
Timing evaluation and publication equivalence	evaluateSchedule, populateScheduleEquivalenceClasses
Selected execution order and homogeneous runs	Scheduled compute planning/materialization
Routing, forwarding, blocking communication order	Deferred transfer planning and communication scheduling
READY/FREE counts, registers, overwrite/restart boundaries	Deferred boundary planning/realization

Both strategies produce MergeScheduleResult: core assignments, contiguous per-core slots, estimated starts, last-task markers, equivalence classes, and cost summaries. Pipeline results also assign each core a stage. Materialization uses verified slots for execution order and merges only consecutive compatible instances. Graph/class lane identity does not imply core identity or execution order.

Dependencies and transfer costs share getGraphBatchPublicationMap. Statically proven physical-slot projections support many-to-many edges; unproven projections retain complete producer dependencies, including region captures. Malformed publication metadata fails graph construction.

Normative contracts: graph fragments, pipeline stages, target boundary, synchronization, and performance.

Costs and interpretation

Execution and fixed transfer costs use integer nanoseconds, converted from target cycles. Bytes, elements, flits, weight slots, and lanes remain resource quantities. Memory work is rounded at its actual fragment/run granularity; runtime-free views cost zero. Transposes charge memory traffic and strided vector work; concatenation charges per-input blocks per outer row. Direct receives into assembly storage avoid duplicate endpoint-memory charges. Sub-byte storage is analyzable in Spatial but rejected by byte-addressed PIM communication lowering.

The evaluator separates serial work, per-core work, dependency critical path, summed transfer work, single-inference completion, and stage spans. Dependency paths exclude core-order constraints; completion includes them. Stage spans exclude cross-stage routing, restart/host synchronization, and contention. Simulator latency and measured throughput remain the performance gates.

Pipeline candidate generation retains the original ordered-packing/dependency-tail heuristic and the memory-aware and sharing-aware searches. The partition DP keeps at most eight compute/memory trade-offs per state. Four final alternatives and the initial partition undergo core allocation, topology placement, and capacity verification. The previous compute/memory guards identify candidate zero, the original fallback. They do not reject legal alternatives before queue analysis. All candidates preserve candidate zero's non-empty pipeline stages. PEFT is unchanged.

ComputeGraph retains physical publication and consumer identities separately from aggregated dependency edges. Cross-stage demands charge one store per shared physical publication and one load per consumer transfer. Publication identity includes the immutable result generation, exact payload type, and physical source fragment. Scalar projections retain separate stores conservatively. Same-stage local/direct transfers incur no host traffic. Function inputs, constants, and outputs supply a separate baseline estimate. The target adapter supplies access width and read/write service times; request rounding precedes aggregation. Serialized service is shared by all stages and is not added to individual stage spans. Baseline loads remain approximate before placement, bufferization, aliasing, and load folding. Scalar publication stores round each contiguous source run separately. Consumer load counts can remain conservative before load folding. The scheduler evaluates boundary demand again after physical placement. Full graph fragments resolve to unique physical producers, so their sharing does not depend on reader placement. Deferred planning uses the same identity rule and retains separate loads, including repeated loads on one reader core. Placement ranks maximum per-core compute/incident-transfer demand, then stage completion and compute load; within 1% demand, stage span breaks the tie. These are placement heuristics, not a steady-state timing model.

Periodic dependency diagnostics

--pim-periodic-diagnostics writes reports/pipeline_periodic.json after deferred boundary planning. This diagnostic does not change schedule selection or IR. It consumes the cached compute-instance costs, materialized core order, physical transfer families, publication store flags, and allocated READY/FREE registers. Compute runs remain atomic; their lanes and loop iterations are not expanded. Compact runs require one physical core; multi-core classes use per-lane steps. One event sequence represents each physical core. The model retains one planned load per physical consumer transfer and one store per shared publication. Later load folding or strided transfer splitting can change request granularity.

Edges express t(v,k) >= t(u,k-d) + w. Within-generation edges have distance zero. Each core's completion-to-entry edge has distance one. Reader-entry FREE signals therefore release the preceding generation through that edge. Signals join their exact planned WAIT count. Dissemination barriers, restart permissions, and the downstream release tree retain their core identities and order. The diagnostic checks its event-register summary against the emitted program in the synchronization tests; it does not replace deadlock verification.

The injected target describes direct-transfer completion and link, local-memory, shared-memory, and signal timing. The current adapter specifies peer permission before a source read. SEND completion follows the read and payload enqueue; RECV completion also requires payload delivery and a destination write. Unsupported or incomplete target timing produces an unavailable report.

The analysis brackets the maximum cycle ratio within one nanosecond. It tests edge delays w - II*d, sweeping the distance-zero DAG between recurrence relaxations. At most R+1 sweeps detect a positive cycle, where R counts positive-distance edges. Bisection uses at most 64 tests. Timing outside the supported numeric range produces an explicit unavailable result. A predecessor cycle supplies a witness, including delay, iteration distance, cores, and links. Detailed witness output is limited to 128 edges; totals cover the entire cycle.

The reported interval estimate is the maximum of the modeled recurrence, per-core compute work, and aggregate serialized shared-memory service. Memory service is never assigned a fixed global request order. Taking the maximum does not add the same service twice. The within-generation dependency span is reported separately and is not an initiation interval.

These are conditional estimates for the abstract execution model, not exact simulator predictions or guaranteed physical lower bounds. Serial compute costs can overestimate overlapping work. Baseline memory accesses retain their aggregate service bound, but their positions are not modeled. Communication introduced after deferred realization, including final output collection, is also absent. Instruction issue, local contention, network injection queues, and memory arbitration remain unmodeled. Missing effects are listed in the report; selection_eligible remains false. This plan-level diagnostic does not rank production candidates. Production extracts execution events from realized Spatial and evaluates them with the shared queue engine.

For V events, E edges, and R recurrence edges, construction requires O(T + F + S log S) time and O(V + E + S) storage, for existing compute instances T, physical transfers F, and synchronization events S. Analysis requires O(L (R+1) (V+E)) time and O(V+E) storage, where L <= 64. Here R is at most the number of active physical cores. There is no dense all-core dependency matrix or inference-generation unrolling. For C analyzed candidates, time scales by C; scratch storage can be reused. The current diagnostic analyzes only the selected schedule (C = 1).

Post-codegen instruction diagnostic

raptor-periodic provides an explicit, read-only diagnostic over an emitted binary artifact. It accepts native or oracle-exported artifacts. Use the oracle-exported artifact when comparing against Pimsim performance results:

build_release/Release/bin/raptor-periodic /tmp/model/pimsim_nn \
  --pim-target-config=validation/pimsim_configs/pimcomp/arch-a/throughput_config_1000ms.json \
  --core-count=168 --crossbar-count=96 --crossbar-size=128 > /tmp/periodic.json

The diagnostic uses physical instruction files and their array-group map. Each file owns one physical core's work, including all lanes and static loop iterations already emitted there. It does not expand compiler loops or batches. It replaces Spatial compute costs rather than adding instruction costs to them. The lightweight compiler report remains separate and does not invoke this tool.

The supported execution contract requires ROB capacity one. Non-scalar instructions therefore retain their serial execution order. Local compute and blocking memory runs become weighted chunks between communication events. Every emitted global request contributes its actual size, rounding, and count. Inputs, constants, split transfers, folded loads, and output collection are included through their emitted instructions. Their positions relative to communication remain explicit. The global service bound remains separate; the model does not impose an order on different cores' global requests.

Timing queries reuse the injected memory, link, and matrix definitions. PIM instruction semantics belong to the compiler target adapter. Vector writeback follows the oracle's actual width rules. Matrix writeback uses the physical array-group width. Local-memory timing uses its own configured period. The oracle passes vector_latency_cycle directly to a nanosecond wait. This parameter does not scale with the processor period; no separate vector clock exists in the supported oracle configuration. In the current oracle, global loads return zero network payload bytes. Global stores retain their outgoing payload cost.

The source switch has one unbounded FIFO for outgoing requests. Enqueue does not wait for earlier requests. FIFO dispatch waits for their completion. Direct SEND requires permission delivery before the source read. Its payload enqueue event coincides with source-read completion. SEND completes after enqueue, independently of FIFO dispatch or payload delivery. RECV requires delivery and destination write. Permission responses join both peer readiness and the receiver's outgoing FIFO. SYNC delivery can precede its local completion. The model retains WAIT counts, restart events, and cross-generation FIFO state. It does not replace the compiler's synchronization or deadlock verification.

--model-network-queues=false and --model-issue=false support timing comparisons. The latter omits scalar issue and retirement delays while retaining SYNC's instruction-defined execution time. Completion notifications can join the current edge before the clock process runs. The model uses minimum retirement delays; it does not reproduce SystemC clock-phase ordering. Scalar issue is charged serially, so these estimates are not guaranteed physical lower bounds. The base recurrence estimate omits global arbitration, frontend buffering, and initial bootstrap. Production does not rank candidates with this recurrence-only estimate. Reports with at most 128 events include timing_graph for focused timing checks. Its dependencies distinguish payload enqueue from FIFO dispatch and delivery. The instruction test accepts --pimsim /path/to/ChipTest for optional fast-mode timing comparisons. These cover both hardware profiles and distinct processor, local-memory, and vector latency settings.

For N emitted instructions, F direct transfers, S synchronization events, and P active cores, construction takes O(N + (F+S) log(F+S)) time. The indexed graph uses O(F+S+P) space and at most 2P recurrence edges. Global requests are counted during the same scan without per-request graph nodes. One instruction file is memory-mapped at a time. Maximum-cycle-ratio analysis uses the same bounded algorithm as the lightweight diagnostic.

--model-memory-controller adds a separate memory_controller estimate. Each global request retains its readiness, transport, service, and completion dependencies. The controller serves requests in predicted arrival order. Read and write requests share that FIFO. Zero-size requests also join it. Source switches remain occupied until transport completion. The model does not assign a fixed global request order or consume simulator trace order.

The analysis executes the indexed periodic graph with timestamped tokens. FIFO tokens retain dependencies across generations; the graph is not copied. A request arrival reserves the controller after earlier arrivals, and its completion enables dependent events. Independent cores and switches remain concurrent. The maximum-cycle-ratio solver and its report remain unchanged.

The queue estimate compares adjacent two-generation windows with the oracle's one-percent convergence threshold. It reports the mean interval across both windows. Per-core queue waits use the corresponding four completed body generations. The maximum request wait includes bootstrap. Event-index order resolves simultaneous readiness. A second run reverses that order and reports alternative_tie_interval_ns. These two orders are sensitivity checks, not bounds over every possible tie order. Clock phases and frontend overlap remain approximate. The standalone report remains read-only. Production evaluates Spatial-derived events with the shared queue engine.

For R global requests, the queue mode adds O(R) events and dependencies. For V resulting events, E dependencies, and generation bound K, execution takes O(K(E + V log V)) time. Storage is O(V + E + B), where B counts pending tokens. The event heap contains at most one event per node. FIFO token storage retains pending generations, rather than instruction-stream copies. The diagnostic limits each event to 64 generations and processed events to 16(V + E). It reports unavailable if either bound prevents convergence. The two tie-order runs reuse the same graph. Production uses this queue execution directly and omits maximum-cycle-ratio analysis. The standalone frontend retains the recurrence witness. Neither analysis changes latency PEFT.

Default throughput selection

Full PIM code generation uses Spatial-only queue-aware selection for --pipeline > 1. Latency PEFT and --pim-only-codegen retain their existing behavior. Intermediate emission retains the original schedule and stops at its requested stage. Its schedule can differ from the final program.

The compiler performs these steps:

1. Run the frontend, layout planning, and graph preparation once.
2. Generate at most eleven legal schedules from the existing searches.
3. Remove duplicate physical placements, including stage identities and core order.
4. Realize each candidate in an independent clone of the prepared checkpoint.
5. Query the target adapter over realized Spatial IR, then release the clone.
6. Require supported static forms, available costs, and proved modeled recurrence for every candidate.
7. Select a candidate with the existing 2.1% selection margin.
8. Realize the selected schedule again from the unchanged checkpoint.
9. Lower, allocate, verify, and emit only the selected program.

A single candidate skips event extraction, queue analysis, recurrence proof, and comparison. It is realized and lowered once, even when optional timing is unavailable. No rejected candidate enters PIM IR or instruction generation during cost evaluation. Constants and weights remain shared immutable attributes in the MLIR context. Candidate operations, communication plans, and synchronization state remain independent. No full candidate compilation directories are created.

SchedulingTarget::estimateRealizedScheduleCost is the read-only whole-schedule query. Its typed result carries both intervals, recurrence evidence, static lowering support, failure reasons, and the selection margin. Unknown support remains explicit. The target supplies model evidence. Spatial applies the 2.1% selection policy margin. Selection does not read JSON diagnostics. Reports serialize the typed results. Spatial owns candidate generation, legality, placement, and target-neutral comparison. The PIM adapter owns execution-event extraction, instruction timing, and resource contention. The shared queue engine models controller arrival arbitration and cross-generation dependencies. The compiler does not execute candidate programs or invoke a simulator.

Both arbitration policies must establish complete-state recurrence within bounded analysis budgets. Their estimates are sensitivity checks, not formal bounds. The candidate with the lowest larger estimate must separate from the original schedule. Comparison uses [min(ties)/1.021, max(ties)/0.979]. Close alternatives retain candidate zero. The 2.1% margin is a policy heuristic. Spatial timing predictions have no formal accuracy bound. The search finds the best distinguishable candidate in its bounded set, without global optimality.

Unavailable timing, unknown static support, or unproved recurrence stops a multi-candidate comparison. The diagnostic identifies each affected candidate and property. The compiler does not silently exclude unknown candidates or use instruction-level fallback selection.

Static preflight does not prove final backend legality. The selected candidate runs through normal bufferization, allocation, and backend verification. A backend owner can mark a resource-limit diagnostic as candidate-specific. Currently, the local-memory allocator marks signed 32-bit address-space exhaustion. The compiler discards that failed clone, explicitly records its rejection, and selects again from the remaining estimated schedules. The first remaining schedule becomes the reference for the same selection policy. Each schedule is attempted at most once. No graph or placement constraints are relaxed. Unclassified or systemic backend failures stop compilation immediately. Code-generation failures also stop compilation when recovery cannot be justified. If every candidate fails, one diagnostic lists all attempted candidates and causes.

reports/pipeline_selection.json records Spatial costs, eligibility, selection, and backend attempts. It also records candidate generation, realization, extraction, expansion, queue, lowering, and finalization times. --enable-timing adds per-pass lowering times and IR operation/value counts. Final instruction generation uses the normal compiler timing report. Explicit Spatial exports and IR dumps reconstruct pre-lowering artifacts for the selected schedule. They do not change the emitted program. Diagnostic MLIR dumps elide large constants and are not lossless checkpoints.

Static analysis, diagnostics, and external validation

Production decisions use the input graph, compiler options, physical plans, and configured target parameters only. Identical static inputs produce the same decision regardless of historical measurements. There are no measured-latency caches, workload lookup tables, or fitted timing corrections.

raptor-periodic retains the instruction-derived analyzer as a standalone validation frontend. It shares the same timing definitions and queue engine with Spatial event evaluation. Optional diagnostics expose resource work, recurrence, and ordered-request fingerprints. Offline Pimsim fast-mode runs validate predictions and emitted programs. Simulator measurements never enter production selection.

For C <= 11 candidates, scheduling metadata uses O(C(T+P)) storage for T compute instances and P cores. Placement deduplication costs O(C²T) time. Candidate realization and extraction are sequential; complete candidate modules are not retained. Normal successful compilation performs one candidate-specific backend lowering. Recovery performs at most C lowerings. Retry selection costs O(C²) time and O(C) metadata. The bounded queue-analysis costs are described in the implementation notes.

Capacity and recovery

Distinct-weight union / per-core capacity is a lower bound; an oversized task is infeasible. Ordered pack counts estimate budgets, and failed heuristic packing does not prove infeasibility. Exhausted budgets restart from lower bounds and constructive packing establishes feasibility.

Latency reservations try load/communication-aware placement, then largest-weight-first and shared reuse-first best fit on failure. Pipeline packing compares arrival/load-aware candidates and uses constructive recovery. The shared constructor retries in reverse order, then at most 16 single-choice deviations within 65,536 core-candidate checks. Successful placements are preserved. Exhausted search produces a bounded diagnostic, not a claim of proved hardware infeasibility.

Complexity and limits

Let T be tasks, E dependencies, P cores, G groups, S stages, R weight references, W distinct weights, C modeled work, F physical fragment/consumer demands, and bounded frontier size K = 8.

Analysis	Time / storage
Schedule evaluation	O((T + E) log T + P) / O(T + E + P)
Weight indexing	Up to O(R W) equality comparisons
Weight lower bounds	O(R + W) / O(W) bits
Constructive best fit	O(T log T + P R + P W) / O(P W) bits plus O(T + P); bounded retries only on failure
Partition DP	Up to O(S G² ((T + R) log C + W + K³)) plus dependency traversal / O(S G² + S G K); at most five additional physical placements per memory objective
Cut memory service	Expected O(F + G) time / O(F + G) scratch and retained demands
Topology clustering / cached PEFT distances	O(P²) time
Concat cost extraction	O(A D) for A inputs of rank D; constant auxiliary space
Cached pipeline priorities	O(T + D) time / O(T + G) space for D template-body operations
Exact projections	O(Q + L) time for Q slot visits and L producer lanes; repeated demands aggregate by projection and fragment
Conservative producer tracing	Linear in the input projection DAG, including captures

Latency insertion-gap searches can remain quadratic. Issue overhead, eliminated assembly copies, contention, and pipeline synchronization remain cost approximations. Changes require matched execution measurements. Loop-invariant projections use trip-count multiplication. Loop-invariant resident weights are collected once. Loop-dependent projections and weights can still require iteration visits; retained projection demands scale with distinct physical fragments instead of repeated visits.