Raptor/docs/compiler/scheduling.md

# Spatial scheduling

`prepareSpatialSchedules` constructs the shared `ComputeGraph`. Latency mode
uses PEFT. For `--pipeline > 1`, the PIM compiler selects among bounded pipeline
candidates with static queue-aware instruction analysis. Pipeline mode does not run PEFT.

See [implementation notes](scheduling-implementation.md) for fragment-view eligibility,
receive forwarding, transfer rounding, cost-model boundaries, and recovery limits.

## Ownership

| Responsibility | Owner |
|---|---|
| Instances, dependencies, execution/transfer costs, resident weights | `ComputeGraph`, `ComputeInstanceUtils` |
| Target resources and physical link costs | Compiler target adapter via `SchedulingTarget` |
| Latency priorities, reservations, placement, per-core order | `PeftScheduler` |
| Capacity-driven workload splitting | `PipelineWorkloadPreparation` |
| Groups, stage partitioning, core budgets | `buildPipelineGroups`, `partitionPipelineGroups`, `assignPipelineStages` |
| Topology clustering and stage-local packing | `PipelineCoreLayout`, `packPipelineStages`, `IndexedResidentWeights` |
| Timing evaluation and publication equivalence | `evaluateSchedule`, `populateScheduleEquivalenceClasses` |
| Selected execution order and homogeneous runs | Scheduled compute planning/materialization |
| Routing, forwarding, blocking communication order | Deferred transfer planning and communication scheduling |
| READY/FREE counts, registers, overwrite/restart boundaries | Deferred boundary planning/realization |

Both strategies produce `MergeScheduleResult`: core assignments, contiguous
per-core slots, estimated starts, last-task markers, equivalence classes, and
cost summaries. Pipeline results also assign each core a stage. Materialization
uses verified slots for execution order and merges only consecutive compatible
instances. Graph/class lane identity does not imply core identity or execution
order.

Dependencies and transfer costs share `getGraphBatchPublicationMap`.
Statically proven physical-slot projections support many-to-many edges;
unproven projections retain complete producer dependencies, including region
captures. Malformed publication metadata fails graph construction.

Normative contracts:
[graph fragments](../../.agents/invariants/GRAPH_COMPUTE_BATCH_INVARIANT.md),
[pipeline stages](../../.agents/invariants/PIPELINE_SCHEDULING_INVARIANT.md),
[target boundary](../../.agents/invariants/SPATIAL_TARGET_GENERALITY_INVARIANT.md),
[synchronization](../../.agents/invariants/PIM_SYNCHRONIZATION_INVARIANT.md),
and [performance](../../.agents/invariants/PERFORMANCE_OPTIMIZATION_INVARIANT.md).

## Costs and interpretation

Execution and fixed transfer costs use integer nanoseconds, converted from
target cycles. Bytes, elements, flits, weight slots, and lanes remain resource
quantities. Memory work is rounded at its actual fragment/run granularity;
runtime-free views cost zero. Transposes charge memory traffic and strided
vector work; concatenation charges per-input blocks per outer row. Direct
receives into assembly storage avoid duplicate endpoint-memory charges.
Sub-byte storage is analyzable in Spatial but rejected by byte-addressed PIM
communication lowering.

The evaluator separates serial work, per-core work, dependency critical path,
summed transfer work, single-inference completion, and stage spans. Dependency
paths exclude core-order constraints; completion includes them. Stage spans
exclude cross-stage routing, restart/host synchronization, and contention.
Simulator latency and measured throughput remain the performance gates.

Pipeline candidate generation retains the original ordered-packing/dependency-tail
heuristic and the memory-aware and sharing-aware searches. The partition DP
keeps at most eight compute/memory trade-offs per state. Four final alternatives
and the initial partition undergo core allocation, topology placement, and
capacity verification. The previous compute/memory guards identify candidate
zero, the original fallback. They do not reject legal alternatives before queue
analysis. All candidates preserve candidate zero's non-empty pipeline stages.
PEFT is unchanged.

`ComputeGraph` retains physical publication and consumer identities separately
from aggregated dependency edges. Cross-stage demands charge one store per
shared physical publication and one load per consumer transfer. Publication
identity includes the immutable result generation, exact payload type, and
physical source fragment. Scalar projections retain separate stores
conservatively. Same-stage local/direct transfers incur no host traffic.
Function inputs, constants, and outputs supply a separate baseline estimate.
The target adapter supplies access width and read/write service times; request
rounding precedes aggregation. Serialized service is shared by all stages and
is not added to individual stage spans. Baseline loads remain approximate
before placement, bufferization, aliasing, and load folding.
Scalar publication stores round each contiguous source run separately.
Consumer load counts can remain conservative before load folding.
The scheduler evaluates boundary demand again after physical placement.
Full graph fragments resolve to unique physical producers, so their sharing
does not depend on reader placement. Deferred planning uses the same identity
rule and retains separate loads, including repeated loads on one reader core.
Placement ranks maximum per-core compute/incident-transfer demand, then stage
completion and compute load; within 1% demand, stage span breaks the tie.
These are placement heuristics, not a steady-state timing model.

## Periodic dependency diagnostics

`--pim-periodic-diagnostics` writes `reports/pipeline_periodic.json` after deferred
boundary planning. This diagnostic does not change schedule selection or IR.
It consumes the cached compute-instance costs, materialized core order, physical
transfer families, publication store flags, and allocated READY/FREE registers.
Compute runs remain atomic; their lanes and loop iterations are not expanded.
Compact runs require one physical core; multi-core classes use per-lane steps.
One event sequence represents each physical core. The model retains one planned
load per physical consumer transfer and one store per shared publication.
Later load folding or strided transfer splitting can change request granularity.

Edges express `t(v,k) >= t(u,k-d) + w`. Within-generation edges have distance
zero. Each core's completion-to-entry edge has distance one. Reader-entry FREE
signals therefore release the preceding generation through that edge. Signals
join their exact planned WAIT count. Dissemination barriers, restart permissions,
and the downstream release tree retain their core identities and order.
The diagnostic checks its event-register summary against the emitted program
in the synchronization tests; it does not replace deadlock verification.

The injected target describes direct-transfer completion and link, local-memory,
shared-memory, and signal timing. The current adapter specifies peer permission
before a source read. SEND completion follows the read and payload enqueue;
RECV completion also requires payload delivery and a destination write.
Unsupported or incomplete target timing produces an `unavailable` report.

The analysis brackets the maximum cycle ratio within one nanosecond. It tests
edge delays `w - II*d`, sweeping the distance-zero DAG between recurrence
relaxations. At most `R+1` sweeps detect a positive cycle, where `R` counts
positive-distance edges. Bisection uses at most 64 tests. Timing outside the
supported numeric range produces an explicit unavailable result. A predecessor
cycle supplies a witness, including delay, iteration distance, cores, and links.
Detailed witness output is limited to 128 edges; totals cover the entire cycle.

The reported interval estimate is the maximum of the modeled recurrence,
per-core compute work, and aggregate serialized shared-memory service. Memory
service is never assigned a fixed global request order. Taking the maximum
does not add the same service twice. The within-generation dependency span is
reported separately and is not an initiation interval.

These are conditional estimates for the abstract execution model, not exact
simulator predictions or guaranteed physical lower bounds. Serial compute costs
can overestimate overlapping work. Baseline memory accesses retain their
aggregate service bound, but their positions are not modeled. Communication
introduced after deferred realization, including final output collection, is
also absent. Instruction issue, local contention, network injection queues, and
memory arbitration remain unmodeled. Missing effects are listed in the report;
`selection_eligible` remains false. This plan-level diagnostic does not rank
production candidates. Production uses the instruction-level queue model below.

For `V` events, `E` edges, and `R` recurrence edges, construction requires
`O(T + F + S log S)` time and `O(V + E + S)` storage, for existing compute
instances `T`, physical transfers `F`, and synchronization events `S`.
Analysis requires `O(L (R+1) (V+E))` time and `O(V+E)` storage, where `L <= 64`.
Here `R` is at most the number of active physical cores. There is no dense
all-core dependency matrix or inference-generation unrolling.
For `C` analyzed candidates, time scales by `C`; scratch storage can be reused.
The current diagnostic analyzes only the selected schedule (`C = 1`).

### Post-codegen instruction diagnostic

`raptor-periodic` provides an explicit, read-only diagnostic over an emitted
binary artifact. It accepts native or oracle-exported artifacts. Use the
oracle-exported artifact when comparing against Pimsim performance results:

```bash
build_release/Release/bin/raptor-periodic /tmp/model/pimsim_nn \
  --pim-target-config=validation/pimsim_configs/pimcomp/arch-a/throughput_config_1000ms.json \
  --core-count=168 --crossbar-count=96 --crossbar-size=128 > /tmp/periodic.json
```

The diagnostic uses physical instruction files and their array-group map.
Each file owns one physical core's work, including all lanes and static loop
iterations already emitted there. It does not expand compiler loops or batches.
It replaces Spatial compute costs rather than adding instruction costs to them.
The lightweight compiler report remains separate and does not invoke this tool.

The supported execution contract requires ROB capacity one. Non-scalar
instructions therefore retain their serial execution order. Local compute and
blocking memory runs become weighted chunks between communication events.
Every emitted global request contributes its actual size, rounding, and count.
Inputs, constants, split transfers, folded loads, and output collection are
included through their emitted instructions. Their positions relative to
communication remain explicit. The global service bound remains separate;
the model does not impose an order on different cores' global requests.

Timing queries reuse the injected memory, link, and matrix definitions.
PIM instruction semantics belong to the compiler target adapter. Vector
writeback follows the oracle's actual width rules. Matrix writeback uses the
physical array-group width. Local-memory timing uses its own configured period.
The oracle passes `vector_latency_cycle` directly to a nanosecond wait.
This parameter does not scale with the processor period; no separate vector
clock exists in the supported oracle configuration.
In the current oracle, global loads return zero network payload bytes.
Global stores retain their outgoing payload cost.

The source switch has one unbounded FIFO for outgoing requests. Enqueue does
not wait for earlier requests. FIFO dispatch waits for their completion.
Direct SEND requires permission delivery before the source read. Its payload
enqueue event coincides with source-read completion. SEND completes after
enqueue, independently of FIFO dispatch or payload delivery. RECV requires delivery and
destination write. Permission responses join both peer readiness and the
receiver's outgoing FIFO. SYNC delivery can precede its local completion.
The model retains WAIT counts, restart events, and cross-generation FIFO state.
It does not replace the compiler's synchronization or deadlock verification.

`--model-network-queues=false` and `--model-issue=false` support timing
comparisons. The latter omits scalar issue and retirement delays while retaining
SYNC's instruction-defined execution time. Completion notifications can join
the current edge before the clock process runs. The model uses minimum
retirement delays; it does not reproduce SystemC clock-phase ordering.
Scalar issue is charged serially, so these estimates are not guaranteed
physical lower bounds. The base recurrence estimate omits global arbitration,
frontend buffering, and initial bootstrap. Production does not rank candidates
with this recurrence-only estimate.
Reports with at most 128 events include `timing_graph` for focused timing checks.
Its dependencies distinguish payload enqueue from FIFO dispatch and delivery.
The instruction test accepts `--pimsim /path/to/ChipTest` for optional fast-mode
timing comparisons. These cover both hardware profiles and distinct processor,
local-memory, and vector latency settings.

For `N` emitted instructions, `F` direct transfers, `S` synchronization events,
and `P` active cores, construction takes `O(N + (F+S) log(F+S))` time.
The indexed graph uses `O(F+S+P)` space and at most `2P` recurrence edges.
Global requests are counted during the same scan without per-request graph
nodes. One instruction file is memory-mapped at a time. Maximum-cycle-ratio
analysis uses the same bounded algorithm as the lightweight diagnostic.

`--model-memory-controller` adds a separate `memory_controller` estimate.
Each global request retains its readiness, transport, service, and completion
dependencies. The controller serves requests in predicted arrival order.
Read and write requests share that FIFO. Zero-size requests also join it.
Source switches remain occupied until transport completion. The model does not
assign a fixed global request order or consume simulator trace order.

The analysis executes the indexed periodic graph with timestamped tokens.
FIFO tokens retain dependencies across generations; the graph is not copied.
A request arrival reserves the controller after earlier arrivals, and its
completion enables dependent events. Independent cores and switches remain
concurrent. The maximum-cycle-ratio solver and its report remain unchanged.

The queue estimate compares adjacent two-generation windows with the oracle's
one-percent convergence threshold. It reports the mean interval across both
windows. Per-core queue waits use the corresponding four completed body
generations. The maximum request wait includes bootstrap. Event-index order
resolves simultaneous readiness. A second run reverses that order and reports
`alternative_tie_interval_ns`. These two orders are sensitivity checks, not
bounds over every possible tie order. Clock phases and frontend overlap remain
approximate. The standalone report remains read-only. Production selection uses the same queue model through its C++ API.

For `R` global requests, the queue mode adds `O(R)` events and dependencies.
For `V` resulting events, `E` dependencies, and generation bound `K`, execution
takes `O(K(E + V log V))` time. Storage is `O(V + E + B)`, where `B` counts
pending tokens. The event heap contains at most one event per node. FIFO token
storage retains pending generations, rather than instruction-stream copies.
The diagnostic limits each event to 64 generations and processed events to
`16(V + E)`. It reports `unavailable` if either bound prevents convergence.
The two tie-order runs reuse the same graph. Production uses this queue execution directly and omits maximum-cycle-ratio analysis.
The standalone frontend retains the recurrence witness. Neither analysis changes latency PEFT.

## Default throughput selection

Full PIM code generation performs these steps for `--pipeline > 1`.
`--pim-only-codegen` retains the physical schedule already present in its input.

The normal compilation workflow is:

1. Run the ONNX frontend, layout planning, common lowering, and graph preparation once.
2. Generate at most eleven verified physical schedules from the existing searches.
3. Remove duplicate placements, including their core order and stage identities.
4. Clone the prepared module for each candidate using MLIR's lossless `IRMapping`.
5. Remap each schedule's compute identities to the clone.
6. Realize communication and lower the clone through verification and instruction selection.
7. Generate binary instruction records in memory, without weights, exports, or candidate files.
8. Run both static controller-arbitration tie policies on that instruction program.
9. Select an eligible candidate and emit only its normal program.

Immutable attributes, including constants and weights, remain shared in the MLIR
context. Candidate operations, values, communication plans, and synchronization
state remain independent. Trials never mutate the prepared checkpoint. Candidates
run sequentially. At most one best lowered module survives between trials.
The retained module follows the conservative selection rule throughout evaluation.
A close alternative does not evict candidate zero. Unsupported timing contracts
and synchronization ablation generate only the original schedule and lower it
once. The compiler checks this before collecting alternatives. The frontend is
not repeated.

Intermediate targets (`--EmitSpatial`, `--EmitPim`, and `--EmitPimBufferized`)
retain the original schedule. They do not perform queue-aware selection or run
transformations beyond the requested stage. Their schedule can therefore differ
from final code generation. Full PIM code generation uses queue-aware selection.

The PIM compiler owns instruction timing and controller behavior. Spatial owns
legal candidate generation and schedule identity remapping. `PimPeriodicAnalysis`
is the single instruction-level timing implementation. `raptor-periodic` is its
read-only diagnostic frontend. The compiler invokes neither that executable nor
an external selection script.

`SchedulingTarget::prepareScheduleCost` is an optional whole-schedule cost query.
It accepts a verified target-lowered module and returns an opaque program identity
and a deferred static evaluation. Spatial compares the resulting interval
estimates and applies the target's reported uncertainty. Missing uncertainty
makes an estimate ineligible. The target owns instruction extraction,
protocol timing, contention, and unavailable-result reasons. Preparation and
evaluation do not change the candidate. Equal program identities avoid repeated
queue analysis. An absent query retains the original schedule.
If generation yields one distinct schedule, no comparison is required. The
compiler realizes and lowers it once, without a trial clone, duplicate
instruction generation, or queue analysis. The selection report records this
case separately from unavailable estimates. Final code generation is unchanged.

PIM transport details reside in `PimInstructionTiming`. The optional plan-level
diagnostic also belongs to the PIM compiler. Spatial supplies its physical plans
through an injected reporting callback; it does not encode the PIM handshake.
The instruction emitter remains the source of register setup, address-dependent
instruction suppression, loop expansion, and physical array-group identity.
Candidate evaluation uses this emitter in memory after local-memory planning.
It does not reconstruct these rules from scalar Spatial operation costs.

Production projects native fp32 transfer sizes and vector widths to the existing
oracle int8 timing contract. It preserves instruction counts, matrix precision,
vector element counts, synchronization, and transfer identities. This projection
does not change the emitted native program. Unsupported widths, targets without
complete timing, and unsupported instruction semantics produce explicit fallback
reasons. Synchronization ablation retains the original schedule.

Both arbitration tie runs must converge within the existing generation and event
budgets. Their estimates are sensitivity checks, not formal bounds. Duplicate
instruction programs are analyzed once. The candidate with the lowest larger
estimate must separate from the original schedule. Close alternatives do not
cancel a gain that separates from the original. Comparison uses `[min(ties)/1.021, max(ties)/0.979]`. The fixed 2.1% prediction uncertainty
covers offline validation error with upward rounding. It is a selection margin,
not a fitted instruction latency or a guaranteed error bound. Overlapping ranges
with the original retain candidate zero. Missing original estimates also retain candidate zero.
Unavailable alternatives cannot displace an eligible original. This search finds
the best distinguishable candidate in its bounded set; it does not establish
global optimality.

`reports/pipeline_selection.json` records candidate identities, estimates,
selection reasons, and generation, lowering, instruction-generation, queue-analysis,
and finalization times. The selected verified lowered module and its captured
schedule report are reused without repeating materialization or communication
realization. Memory reports come from normal final code generation.
With `--enable-timing`, candidate summaries also contain per-pass lowering times
and operation/value counts before bufferization and instruction generation.
Instruction-generation summaries separate preparation from per-core emission.
Explicit Spatial exports, IR snapshots,
and plan-level periodic diagnostics reconstruct the selected Spatial schedule
because those outputs require pre-lowering operations. This optional work does
not change the retained instruction program. Candidate instructions and rejected
IR are released immediately. No temporary compilation directories are created.
`--pim-dump-ir` explicitly enables diagnostic IR snapshots;
these snapshots elide large constants and cannot serve as lossless checkpoints.

### Static analysis, diagnostics, and external validation

Production decisions use only the input program, compiler options, candidate
instructions and plans, and configured target resources and timing. The static
model executes deterministic abstract events. It does not execute the generated
program. It reads no benchmark CSV, profiling trace, measured latency cache, or
reference artifact. Identical static inputs produce the same decision regardless
of historical measurements. Throughput deferred-demand ordering uses IR order,
rather than allocation addresses. Latency PEFT and its demand ordering are unchanged.

Optional diagnostics expose recurrence witnesses, instruction timing, and resource
service. Offline Pimsim fast-mode measurements validate these predictions and
candidate rankings. Simulator results never enter production selection. Timing
corrections must follow verified hardware behavior or configured parameters.

For `B <= 11` schedules, `T` compute instances, and `P` cores, captured schedules
use `O(B(T+P))` storage. Placement deduplication takes `O(B²T)` time. Each distinct
finalist requires downstream lowering and one instruction scan. Queue execution
uses the bounded complexity stated above. Peak storage includes the prepared
module, one best lowered module, one trial, and one queue graph. It does not scale
with retained complete candidate programs. Maximum-cycle-ratio diagnostics do not
run in the candidate evaluation loop.

## Capacity and recovery

Distinct-weight union / per-core capacity is a lower bound; an oversized task
is infeasible. Ordered pack counts estimate budgets, and failed heuristic
packing does not prove infeasibility. Exhausted budgets restart from lower
bounds and constructive packing establishes feasibility.

Latency reservations try load/communication-aware placement, then
largest-weight-first and shared reuse-first best fit on failure. Pipeline
packing compares arrival/load-aware candidates and uses constructive recovery.
The shared constructor retries in reverse order, then at most 16 single-choice
deviations within 65,536 core-candidate checks. Successful placements are
preserved. Exhausted search produces a bounded diagnostic, not a claim of
proved hardware infeasibility.

## Complexity and limits

Let `T` be tasks, `E` dependencies, `P` cores, `G` groups, `S` stages,
`R` weight references, `W` distinct weights, `C` modeled work, `F` physical
fragment/consumer demands, and bounded frontier size `K = 8`.

| Analysis | Time / storage |
|---|---|
| Schedule evaluation | `O((T + E) log T + P)` / `O(T + E + P)` |
| Weight indexing | Up to `O(R W)` equality comparisons |
| Weight lower bounds | `O(R + W)` / `O(W)` bits |
| Constructive best fit | `O(T log T + P R + P W)` / `O(P W)` bits plus `O(T + P)`; bounded retries only on failure |
| Partition DP | Up to `O(S G² ((T + R) log C + W + K³))` plus dependency traversal / `O(S G² + S G K)`; at most five additional physical placements per memory objective |
| Cut memory service | Expected `O(F + G)` time / `O(F + G)` scratch and retained demands |
| Topology clustering / cached PEFT distances | `O(P²)` time |
| Concat cost extraction | `O(A D)` for `A` inputs of rank `D`; constant auxiliary space |
| Cached pipeline priorities | `O(T + D)` time / `O(T + G)` space for `D` template-body operations |
| Exact projections | `O(Q + L)` time for `Q` slot visits and `L` producer lanes; repeated demands aggregate by projection and fragment |
| Conservative producer tracing | Linear in the input projection DAG, including captures |

Latency insertion-gap searches can remain quadratic. Issue overhead,
eliminated assembly copies, contention, and pipeline synchronization remain
cost approximations. Changes require matched execution measurements.
Loop-invariant projections use trip-count multiplication. Loop-invariant
resident weights are collected once. Loop-dependent projections and weights
can still require iteration visits; retained projection demands scale with
distinct physical fragments instead of repeated visits.