Raptor/docs/compiler/scheduling.md

# Spatial scheduling

`prepareSpatialSchedules` constructs the shared `ComputeGraph`. Latency mode
uses PEFT. For `--pipeline > 1`, the PIM compiler selects among bounded pipeline
candidates with static queue-aware instruction analysis. Pipeline mode does not run PEFT.

See [implementation notes](scheduling-implementation.md) for fragment-view eligibility,
receive forwarding, transfer rounding, cost-model boundaries, and recovery limits.

## Ownership

| Responsibility | Owner |
|---|---|
| Instances, dependencies, execution/transfer costs, resident weights | `ComputeGraph`, `ComputeInstanceUtils` |
| Target resources and physical link costs | Compiler target adapter via `SchedulingTarget` |
| Latency priorities, reservations, placement, per-core order | `PeftScheduler` |
| Capacity-driven workload splitting | `PipelineWorkloadPreparation` |
| Groups, stage partitioning, core budgets | `buildPipelineGroups`, `partitionPipelineGroups`, `assignPipelineStages` |
| Topology clustering and stage-local packing | `PipelineCoreLayout`, `packPipelineStages`, `IndexedResidentWeights` |
| Timing evaluation and publication equivalence | `evaluateSchedule`, `populateScheduleEquivalenceClasses` |
| Selected execution order and homogeneous runs | Scheduled compute planning/materialization |
| Routing, forwarding, blocking communication order | Deferred transfer planning and communication scheduling |
| READY/FREE counts, registers, overwrite/restart boundaries | Deferred boundary planning/realization |

Both strategies produce `MergeScheduleResult`: core assignments, contiguous
per-core slots, estimated starts, last-task markers, equivalence classes, and
cost summaries. Pipeline results also assign each core a stage. Materialization
uses verified slots for execution order and merges only consecutive compatible
instances. Graph/class lane identity does not imply core identity or execution
order.

Dependencies and transfer costs share `getGraphBatchPublicationMap`.
Statically proven physical-slot projections support many-to-many edges;
unproven projections retain complete producer dependencies, including region
captures. Malformed publication metadata fails graph construction.

Normative contracts:
[graph fragments](../../.agents/invariants/GRAPH_COMPUTE_BATCH_INVARIANT.md),
[pipeline stages](../../.agents/invariants/PIPELINE_SCHEDULING_INVARIANT.md),
[target boundary](../../.agents/invariants/SPATIAL_TARGET_GENERALITY_INVARIANT.md),
[synchronization](../../.agents/invariants/PIM_SYNCHRONIZATION_INVARIANT.md),
and [performance](../../.agents/invariants/PERFORMANCE_OPTIMIZATION_INVARIANT.md).

## Costs and interpretation

Execution and fixed transfer costs use integer nanoseconds, converted from
target cycles. Bytes, elements, flits, weight slots, and lanes remain resource
quantities. Memory work is rounded at its actual fragment/run granularity;
runtime-free views cost zero. Transposes charge memory traffic and strided
vector work; concatenation charges per-input blocks per outer row. Direct
receives into assembly storage avoid duplicate endpoint-memory charges.
Sub-byte storage is analyzable in Spatial but rejected by byte-addressed PIM
communication lowering.

The evaluator separates serial work, per-core work, dependency critical path,
summed transfer work, single-inference completion, and stage spans. Dependency
paths exclude core-order constraints; completion includes them. Stage spans
exclude cross-stage routing, restart/host synchronization, and contention.
Simulator latency and measured throughput remain the performance gates.

Pipeline candidate generation retains the original ordered-packing/dependency-tail
heuristic and the memory-aware and sharing-aware searches. The partition DP
keeps at most eight compute/memory trade-offs per state. Four final alternatives
and the initial partition undergo core allocation, topology placement, and
capacity verification. The previous compute/memory guards identify candidate
zero, the original fallback. They do not reject legal alternatives before queue
analysis. All candidates preserve candidate zero's non-empty pipeline stages.
PEFT is unchanged.

`ComputeGraph` retains physical publication and consumer identities separately
from aggregated dependency edges. Cross-stage demands charge one store per
shared physical publication and one load per consumer transfer. Publication
identity includes the immutable result generation, exact payload type, and
physical source fragment. Scalar projections retain separate stores
conservatively. Same-stage local/direct transfers incur no host traffic.
Function inputs, constants, and outputs supply a separate baseline estimate.
The target adapter supplies access width and read/write service times; request
rounding precedes aggregation. Serialized service is shared by all stages and
is not added to individual stage spans. Baseline loads remain approximate
before placement, bufferization, aliasing, and load folding.
Scalar publication stores round each contiguous source run separately.
Consumer load counts can remain conservative before load folding.
The scheduler evaluates boundary demand again after physical placement.
Full graph fragments resolve to unique physical producers, so their sharing
does not depend on reader placement. Deferred planning uses the same identity
rule and retains separate loads, including repeated loads on one reader core.
Placement ranks maximum per-core compute/incident-transfer demand, then stage
completion and compute load; within 1% demand, stage span breaks the tie.
These are placement heuristics, not a steady-state timing model.

## Periodic dependency diagnostics

`--pim-periodic-diagnostics` writes `reports/pipeline_periodic.json` after deferred
boundary planning. This diagnostic does not change schedule selection or IR.
It consumes the cached compute-instance costs, materialized core order, physical
transfer families, publication store flags, and allocated READY/FREE registers.
Compute runs remain atomic; their lanes and loop iterations are not expanded.
Compact runs require one physical core; multi-core classes use per-lane steps.
One event sequence represents each physical core. The model retains one planned
load per physical consumer transfer and one store per shared publication.
Later load folding or strided transfer splitting can change request granularity.

Edges express `t(v,k) >= t(u,k-d) + w`. Within-generation edges have distance
zero. Each core's completion-to-entry edge has distance one. Reader-entry FREE
signals therefore release the preceding generation through that edge. Signals
join their exact planned WAIT count. Dissemination barriers, restart permissions,
and the downstream release tree retain their core identities and order.
The diagnostic checks its event-register summary against the emitted program
in the synchronization tests; it does not replace deadlock verification.

The injected target describes direct-transfer completion and link, local-memory,
shared-memory, and signal timing. The current adapter specifies peer permission
before a source read. SEND completion follows the read and payload enqueue;
RECV completion also requires payload delivery and a destination write.
Unsupported or incomplete target timing produces an `unavailable` report.

The analysis brackets the maximum cycle ratio within one nanosecond. It tests
edge delays `w - II*d`, sweeping the distance-zero DAG between recurrence
relaxations. At most `R+1` sweeps detect a positive cycle, where `R` counts
positive-distance edges. Bisection uses at most 64 tests. Timing outside the
supported numeric range produces an explicit unavailable result. A predecessor
cycle supplies a witness, including delay, iteration distance, cores, and links.
Detailed witness output is limited to 128 edges; totals cover the entire cycle.

The reported interval estimate is the maximum of the modeled recurrence,
per-core compute work, and aggregate serialized shared-memory service. Memory
service is never assigned a fixed global request order. Taking the maximum
does not add the same service twice. The within-generation dependency span is
reported separately and is not an initiation interval.

These are conditional estimates for the abstract execution model, not exact
simulator predictions or guaranteed physical lower bounds. Serial compute costs
can overestimate overlapping work. Baseline memory accesses retain their
aggregate service bound, but their positions are not modeled. Communication
introduced after deferred realization, including final output collection, is
also absent. Instruction issue, local contention, network injection queues, and
memory arbitration remain unmodeled. Missing effects are listed in the report;
`selection_eligible` remains false. This plan-level diagnostic does not rank
production candidates. Production extracts execution events from realized Spatial
and evaluates them with the shared queue engine.

For `V` events, `E` edges, and `R` recurrence edges, construction requires
`O(T + F + S log S)` time and `O(V + E + S)` storage, for existing compute
instances `T`, physical transfers `F`, and synchronization events `S`.
Analysis requires `O(L (R+1) (V+E))` time and `O(V+E)` storage, where `L <= 64`.
Here `R` is at most the number of active physical cores. There is no dense
all-core dependency matrix or inference-generation unrolling.
For `C` analyzed candidates, time scales by `C`; scratch storage can be reused.
The current diagnostic analyzes only the selected schedule (`C = 1`).

### Post-codegen instruction diagnostic

`raptor-periodic` provides an explicit, read-only diagnostic over an emitted
binary artifact. It accepts native or oracle-exported artifacts. Use the
oracle-exported artifact when comparing against Pimsim performance results:

```bash
build_release/Release/bin/raptor-periodic /tmp/model/pimsim_nn \
  --pim-target-config=validation/pimsim_configs/pimcomp/arch-a/throughput_config_1000ms.json \
  --core-count=168 --crossbar-count=96 --crossbar-size=128 > /tmp/periodic.json
```

The diagnostic uses physical instruction files and their array-group map.
Each file owns one physical core's work, including all lanes and static loop
iterations already emitted there. It does not expand compiler loops or batches.
It replaces Spatial compute costs rather than adding instruction costs to them.
The lightweight compiler report remains separate and does not invoke this tool.

The supported execution contract requires ROB capacity one. Non-scalar
instructions therefore retain their serial execution order. Local compute and
blocking memory runs become weighted chunks between communication events.
Every emitted global request contributes its actual size, rounding, and count.
Inputs, constants, split transfers, folded loads, and output collection are
included through their emitted instructions. Their positions relative to
communication remain explicit. The global service bound remains separate;
the model does not impose an order on different cores' global requests.

Timing queries reuse the injected memory, link, and matrix definitions.
PIM instruction semantics belong to the compiler target adapter. Vector
writeback follows the oracle's actual width rules. Matrix writeback uses the
physical array-group width. Local-memory timing uses its own configured period.
The oracle passes `vector_latency_cycle` directly to a nanosecond wait.
This parameter does not scale with the processor period; no separate vector
clock exists in the supported oracle configuration.
In the current oracle, global loads return zero network payload bytes.
Global stores retain their outgoing payload cost.

The source switch has one unbounded FIFO for outgoing requests. Enqueue does
not wait for earlier requests. FIFO dispatch waits for their completion.
Direct SEND requires permission delivery before the source read. Its payload
enqueue event coincides with source-read completion. SEND completes after
enqueue, independently of FIFO dispatch or payload delivery. RECV requires delivery and
destination write. Permission responses join both peer readiness and the
receiver's outgoing FIFO. SYNC delivery can precede its local completion.
The model retains WAIT counts, restart events, and cross-generation FIFO state.
It does not replace the compiler's synchronization or deadlock verification.

`--model-network-queues=false` and `--model-issue=false` support timing
comparisons. The latter omits scalar issue and retirement delays while retaining
SYNC's instruction-defined execution time. Completion notifications can join
the current edge before the clock process runs. The model uses minimum
retirement delays; it does not reproduce SystemC clock-phase ordering.
Scalar issue is charged serially, so these estimates are not guaranteed
physical lower bounds. The base recurrence estimate omits global arbitration,
frontend buffering, and initial bootstrap. Production does not rank candidates
with this recurrence-only estimate.
Reports with at most 128 events include `timing_graph` for focused timing checks.
Its dependencies distinguish payload enqueue from FIFO dispatch and delivery.
The instruction test accepts `--pimsim /path/to/ChipTest` for optional fast-mode
timing comparisons. These cover both hardware profiles and distinct processor,
local-memory, and vector latency settings.

For `N` emitted instructions, `F` direct transfers, `S` synchronization events,
and `P` active cores, construction takes `O(N + (F+S) log(F+S))` time.
The indexed graph uses `O(F+S+P)` space and at most `2P` recurrence edges.
Global requests are counted during the same scan without per-request graph
nodes. One instruction file is memory-mapped at a time. Maximum-cycle-ratio
analysis uses the same bounded algorithm as the lightweight diagnostic.

`--model-memory-controller` adds a separate `memory_controller` estimate.
Each global request retains its readiness, transport, service, and completion
dependencies. The controller serves requests in predicted arrival order.
Read and write requests share that FIFO. Zero-size requests also join it.
Source switches remain occupied until transport completion. The model does not
assign a fixed global request order or consume simulator trace order.

The analysis executes the indexed periodic graph with timestamped tokens.
FIFO tokens retain dependencies across generations; the graph is not copied.
A request arrival reserves the controller after earlier arrivals, and its
completion enables dependent events. Independent cores and switches remain
concurrent. The maximum-cycle-ratio solver and its report remain unchanged.

The queue estimate compares adjacent two-generation windows with the oracle's
one-percent convergence threshold. It reports the mean interval across both
windows. Per-core queue waits use the corresponding four completed body
generations. The maximum request wait includes bootstrap. Event-index order
resolves simultaneous readiness. A second run reverses that order and reports
`alternative_tie_interval_ns`. These two orders are sensitivity checks, not
bounds over every possible tie order. Clock phases and frontend overlap remain
approximate. The standalone report remains read-only.
Production evaluates Spatial-derived events with the shared queue engine.

For `R` global requests, the queue mode adds `O(R)` events and dependencies.
For `V` resulting events, `E` dependencies, and generation bound `K`, execution
takes `O(K(E + V log V))` time. Storage is `O(V + E + B)`, where `B` counts
pending tokens. The event heap contains at most one event per node. FIFO token
storage retains pending generations, rather than instruction-stream copies.
The diagnostic limits each event to 64 generations and processed events to
`16(V + E)`. It reports `unavailable` if either bound prevents convergence.
The two tie-order runs reuse the same graph. Production uses this queue execution directly and omits maximum-cycle-ratio analysis.
The standalone frontend retains the recurrence witness. Neither analysis changes latency PEFT.

## Default throughput selection

Full PIM code generation uses Spatial-only queue-aware selection for `--pipeline > 1`.
Latency PEFT and `--pim-only-codegen` retain their existing behavior.
Intermediate emission retains the original schedule and stops at its requested stage.
Its schedule can differ from the final program.

The compiler performs these steps:

1. Run the frontend, layout planning, and graph preparation once.
2. Generate at most eleven legal schedules from the existing searches.
3. Remove duplicate physical placements, including stage identities and core order.
4. Realize each candidate in an independent clone of the prepared checkpoint.
5. Query the target adapter over realized Spatial IR, then release the clone.
6. Require supported static forms, available costs, and proved modeled recurrence for every candidate.
7. Select a candidate with the existing 2.1% selection margin.
8. Realize the selected schedule again from the unchanged checkpoint.
9. Lower, allocate, verify, and emit only the selected program.

A single candidate skips event extraction, queue analysis, recurrence proof, and comparison.
It is realized and lowered once, even when optional timing is unavailable.
No rejected candidate enters PIM IR or instruction generation during cost evaluation.
Constants and weights remain shared immutable attributes in the MLIR context.
Candidate operations, communication plans, and synchronization state remain independent.
No full candidate compilation directories are created.

`SchedulingTarget::estimateRealizedScheduleCost` is the read-only whole-schedule query.
Its typed result carries both intervals, recurrence evidence, static lowering support,
failure reasons, and the selection margin. Unknown support remains explicit.
The target supplies model evidence. Spatial applies the 2.1% selection policy margin.
Selection does not read JSON diagnostics. Reports serialize the typed results.
Spatial owns candidate generation, legality, placement, and target-neutral comparison.
The PIM adapter owns execution-event extraction, instruction timing, and resource contention.
The shared queue engine models controller arrival arbitration and cross-generation dependencies.
The compiler does not execute candidate programs or invoke a simulator.

Both arbitration policies must establish complete-state recurrence within bounded analysis budgets.
Their estimates are sensitivity checks, not formal bounds.
The candidate with the lowest larger estimate must separate from the original schedule.
Comparison uses `[min(ties)/1.021, max(ties)/0.979]`.
Close alternatives retain candidate zero. The 2.1% margin is a policy heuristic.
Spatial timing predictions have no formal accuracy bound.
The search finds the best distinguishable candidate in its bounded set, without global optimality.

Unavailable timing, unknown static support, or unproved recurrence stops a multi-candidate comparison.
The diagnostic identifies each affected candidate and property.
The compiler does not silently exclude unknown candidates or use instruction-level fallback selection.

Static preflight does not prove final backend legality.
The selected candidate runs through normal bufferization, allocation, and backend verification.
A backend owner can mark a resource-limit diagnostic as candidate-specific.
Currently, the local-memory allocator marks signed 32-bit address-space exhaustion.
The compiler discards that failed clone, explicitly records its rejection, and selects again from the remaining estimated schedules.
The first remaining schedule becomes the reference for the same selection policy.
Each schedule is attempted at most once. No graph or placement constraints are relaxed.
Unclassified or systemic backend failures stop compilation immediately.
Code-generation failures also stop compilation when recovery cannot be justified.
If every candidate fails, one diagnostic lists all attempted candidates and causes.

`reports/pipeline_selection.json` records Spatial costs, eligibility, selection, and backend attempts.
It also records candidate generation, realization, extraction, expansion, queue, lowering, and finalization times.
`--enable-timing` adds per-pass lowering times and IR operation/value counts.
Final instruction generation uses the normal compiler timing report.
Explicit Spatial exports and IR dumps reconstruct pre-lowering artifacts for the selected schedule.
They do not change the emitted program. Diagnostic MLIR dumps elide large constants and are not lossless checkpoints.

### Static analysis, diagnostics, and external validation

Production decisions use the input graph, compiler options, physical plans, and configured target parameters only.
Identical static inputs produce the same decision regardless of historical measurements.
There are no measured-latency caches, workload lookup tables, or fitted timing corrections.

`raptor-periodic` retains the instruction-derived analyzer as a standalone validation frontend.
It shares the same timing definitions and queue engine with Spatial event evaluation.
Optional diagnostics expose resource work, recurrence, and ordered-request fingerprints.
Offline Pimsim fast-mode runs validate predictions and emitted programs.
Simulator measurements never enter production selection.

For `C <= 11` candidates, scheduling metadata uses `O(C(T+P))` storage for `T` compute instances and `P` cores.
Placement deduplication costs `O(C²T)` time.
Candidate realization and extraction are sequential; complete candidate modules are not retained.
Normal successful compilation performs one candidate-specific backend lowering.
Recovery performs at most `C` lowerings. Retry selection costs `O(C²)` time and `O(C)` metadata.
The bounded queue-analysis costs are described in the implementation notes.

## Capacity and recovery

Distinct-weight union / per-core capacity is a lower bound; an oversized task
is infeasible. Ordered pack counts estimate budgets, and failed heuristic
packing does not prove infeasibility. Exhausted budgets restart from lower
bounds and constructive packing establishes feasibility.

Latency reservations try load/communication-aware placement, then
largest-weight-first and shared reuse-first best fit on failure. Pipeline
packing compares arrival/load-aware candidates and uses constructive recovery.
The shared constructor retries in reverse order, then at most 16 single-choice
deviations within 65,536 core-candidate checks. Successful placements are
preserved. Exhausted search produces a bounded diagnostic, not a claim of
proved hardware infeasibility.

## Complexity and limits

Let `T` be tasks, `E` dependencies, `P` cores, `G` groups, `S` stages,
`R` weight references, `W` distinct weights, `C` modeled work, `F` physical
fragment/consumer demands, and bounded frontier size `K = 8`.

| Analysis | Time / storage |
|---|---|
| Schedule evaluation | `O((T + E) log T + P)` / `O(T + E + P)` |
| Weight indexing | Up to `O(R W)` equality comparisons |
| Weight lower bounds | `O(R + W)` / `O(W)` bits |
| Constructive best fit | `O(T log T + P R + P W)` / `O(P W)` bits plus `O(T + P)`; bounded retries only on failure |
| Partition DP | Up to `O(S G² ((T + R) log C + W + K³))` plus dependency traversal / `O(S G² + S G K)`; at most five additional physical placements per memory objective |
| Cut memory service | Expected `O(F + G)` time / `O(F + G)` scratch and retained demands |
| Topology clustering / cached PEFT distances | `O(P²)` time |
| Concat cost extraction | `O(A D)` for `A` inputs of rank `D`; constant auxiliary space |
| Cached pipeline priorities | `O(T + D)` time / `O(T + G)` space for `D` template-body operations |
| Exact projections | `O(Q + L)` time for `Q` slot visits and `L` producer lanes; repeated demands aggregate by projection and fragment |
| Conservative producer tracing | Linear in the input projection DAG, including captures |

Latency insertion-gap searches can remain quadratic. Issue overhead,
eliminated assembly copies, contention, and pipeline synchronization remain
cost approximations. Changes require matched execution measurements.
Loop-invariant projections use trip-count multiplication. Loop-invariant
resident weights are collected once. Loop-dependent projections and weights
can still require iteration visits; retained projection demands scale with
distinct physical fragments instead of repeated visits.