rework documentation
Validate Operations / validate-operations (push) Failing after 3h10m55s

add plot generation scripts
adjust validation tools
add up-to-date results
This commit is contained in:
NiccoloN
2026-10-19 16:34:01 +02:00
parent 9053cc5e8b
commit 034c197260
33 changed files with 3077 additions and 1798 deletions
@@ -1,177 +1,9 @@
# Spatial scheduling
`ScheduleAndRealizeSpatial` selects one scheduling strategy after constructing
the shared `ComputeGraph`. Pipeline scheduling does not run, require, or repair
a latency PEFT placement.
The [scheduling overview](../../../../../../../../docs/compiler/scheduling.md)
documents ownership, costs, capacity recovery, complexity, and links to the
normative invariants. See [implementation details](../../../../../../../../docs/compiler/scheduling-implementation.md)
for fragment views, receive forwarding, transfer rounding, and placement models.
## Ownership and flow
| Decision | Owner |
| --- | --- |
| Compute instances, dependency extraction, execution/transfer costs, resident-weight identities | `ComputeGraph` and `ComputeInstanceUtils` |
| Target configuration and physical link costs | Compiler target adapter, exposed through `SchedulingTarget` |
| Latency priorities, reservations, physical assignment, insertion gaps, per-core order | `PeftScheduler` |
| Capacity-driven additive workload splitting | `PipelineWorkloadPreparation` |
| Operation groups, stage adjacency and input constraints | `buildPipelineGroups` |
| Stage partitioning | `partitionPipelineGroups` |
| Stage core budgets | `assignPipelineStages` |
| Physical topology clustering | `PipelineCoreLayout` |
| Stage-local placement and resident-weight packing | `packPipelineStages` and `IndexedResidentWeights` |
| Schedule timing evaluation | `evaluateSchedule` |
| Publication-compatible core equivalence classes | `populateScheduleEquivalenceClasses` |
| Emission of the selected core order and homogeneous runs | Scheduled compute planning/materialization |
| Routing, local forwarding and transfer families | Deferred transfer planning |
| Deadlock-free blocking communication order | Deferred communication scheduling |
| READY/FREE counts, registers, overwrite boundaries and restart synchronization | Deferred boundary planning |
| Explicit communication and synchronization operations | Deferred boundary realization |
Both strategies produce `MergeScheduleResult`: physical core assignment,
contiguous per-core execution slots, estimated starts, last-task markers,
equivalence classes and cost summaries. Pipeline schedules also provide the
stage of every physical core. Graph identity identifies publications; it does
not order execution. Materialization sorts only by the verified scheduler slots
and combines only consecutive compatible graph instances.
Class lanes keep graph encounter order independently of physical core IDs,
preserving regular source-lane mappings without changing any core's execution
order. Complete singleton fragment publications use reshape views only with
matching static shapes, zero offsets, complete sizes and unit strides, including
scalar fragments. Other insertions retain their slice semantics. Receive forwarding
runs after copy normalization and can cross lookup-table loads, while writes
and synchronization remain barriers.
Physical fragment slots are resolved through `getGraphBatchPublicationMap`.
Direct statically evaluable batch projections can have many-to-many edges.
Unproven projections retain the complete producer dependency. Blueprint
dependencies belong to the shared graph, rather than a separate pipeline-only
dependency graph.
Sources reached through intermediate projections retain conservative producer
dependencies, including region captures. Graph construction reports malformed
publication metadata through normal pass failure.
Projected communication costs count the selected physical fragments through
the same publication map; graph-range overlap is not a transfer-volume estimate.
Conservative costs scale payload bytes before rounding memory/packet costs, so
small fragments retain endpoint and packet overhead. Conservative producer-chunk
payloads scale elements/bits before byte rounding, preserving one aggregate
chunk estimate; projected publication costs retain their per-fragment granularity.
Spatial sub-byte types reach analysis, while the current PIM byte-addressed
communication lowering rejects non-byte-sized element storage.
## Cost dimensions
Execution times and fixed transfer costs use integer nanoseconds. Hardware
cycles are converted using the corresponding target period. Bytes, elements,
flits, resident-weight slots and lane counts remain resource quantities.
Blueprint reconstruction charges memory work separately for each fragment;
small projected slices are not assumed to require a single copy. Full receives
that can write directly into assembly storage do not charge endpoint memory work
again on their edges. Later copy elimination can make this assembly estimate
conservative. Blueprint reconstruction uses target memory read/write costs; element counts
and cycles are not added to nanoseconds. Runtime-free views cost zero.
Runtime transposes charge target memory read/write traffic rather than generic
nested-operation fallback cost. Their shared lowering/run analysis charges rounded
memory accesses per run and vector execution for strided runs. The matched oracle
charges vector payload bytes, not the full strided address footprint. Instruction
setup/issue time and contention remain approximation boundaries.
Concatenation charges each input block separately
for each outer row, including memory-width rounding. Axis, size and target-period
scaling have focused coverage.
The evaluator distinguishes total serial work, maximum per-core work,
dependency critical path, summed edge transfer latency, single-inference
completion and individual stage spans. The dependency path excludes core-order
constraints; completion includes them. Summed communication is a work metric,
not elapsed time. Stage spans exclude cross-stage routing, restart/host-buffer
synchronization and network contention. They are scheduling proxies, not an
authoritative prediction of steady-state throughput. Simulator latency and
measured throughput period remain the performance gates.
Partitioning retains its successful ordered-packing plus dependency-tail
priority. This deliberately biases cuts away from long dependent groups and
is not reported as predicted execution time. Pipeline partitioning and packing
use nominal volume priorities: transpose traffic is weighted as a bulk movement,
preserving the measured heuristic without pretending that individual runs are
free. The per-operation priority adjustment is cached and scaled by chunk lanes.
PEFT and final schedule evaluation always use physical execution estimates.
Pipeline candidate placement compares
maximum per-core serial compute/incident-transfer demand, then stage completion
and compute load. Incident transfer latency is charged to each participating
core as a placement proxy; it is not added twice to predicted elapsed time.
This preserves useful parallelism that an arrival-time-only ranking can lose.
The final reported compute/incident-transfer demand uses physical execution
estimates, separately from the priority scores used to select the placement.
When serial-demand estimates are within 1%, stage span breaks the near tie;
this is a heuristic resolution rule, not a pipeline timing model. Core budgets
come from stage allocation and capacity recovery. A proposed additional one-core
load refinement was removed after a matched throughput regression; improving
the placement proxy did not establish a better execution period.
## Capacity and recovery
The distinct-weight union divided by per-core capacity is a provable lower
bound; an oversized task is individually infeasible. Ordered pack counts are
heuristic core-budget estimates for unrestricted placement. Their failure does
not prove hardware infeasibility. When these estimates exhaust the fleet, core
allocation restarts from the lower bounds and actual packing establishes
feasibility.
Latency reservations prioritize physical communication/load, or reuse under
high weight pressure. If graph-order reservations fail, a largest-weight-first
retry retains load-aware placement before using the shared reuse-first best-fit
constructor. The constructor establishes feasibility; its concentrated assignment
does not replace a successful load-aware reservation. Pipeline packing compares arrival-aware and load-aware placements;
ordered packing and the same shared constructor supply constructive recovery
assignments. Failed search budgets guide bounded core reallocation, but are not
resource lower bounds. Diagnostics distinguish a proved capacity limit from an
unsuccessful heuristic search. No exact combinatorial solver is used.
The shared constructor retries once in reverse order after greedy failure.
Only if both orders fail, it replays at most16 single-choice deviations with a
shared budget of65536 core-candidate checks, reusing its scratch buffers.
Successful existing assignments are preserved; exhausted repair is not proof of
infeasibility. PEFT propagates reservation failures to the scheduling pass for
a bounded diagnostic and normal pass failure.
## Complexity
Let `T` be tasks, `E` dependencies, `P` processors, `G` operation groups, `S`
stages, `R` resident-weight references, `W` distinct weights and `C` total modeled
work.
- Schedule evaluation takes `O((T + E) log T + P)` time with deterministic ready
ordering, and `O(T + E + P)` space.
- Resident-weight indexing retains `O(R W)` worst-case equality comparisons.
Lower bounds take `O(R + W)` time and `O(W)` bits. Constructive best fit takes
`O(T log T + P R + P W)` time, with `O(P W)` bits and `O(T + P)` additional
storage. It runs on failure paths rather than replacing successful placement.
Recovery adds at most17 constructions with the same complexity (one reverse
traversal plus16 single-choice replays); repair candidate checks are also
capped at65536. Scratch storage is reused, with one extra `O(T)` failed-assignment
snapshot only after both original attempts fail.
- The partition DP remains superlinear: up to
`O(S G² ((T + R) log C + W))` time for ordered segment estimates, plus group
dependency traversal, and `O(S G²)` cache space. This bounded existing search
is retained rather than replaced with global placement optimization.
- Physical topology clustering and cached PEFT center distances take `O(P²)`
time. Center lookups are constant time instead of repeating a full physical
topology scan for each task/candidate processor.
- Concatenation cost extraction takes `O(A D)` time and constant auxiliary space
for `A` inputs of rank `D`; it does not expand the outer rows or copy blocks.
- Graph execution costs are scaled by chunk lane counts. Pipeline priority
analysis adds two cost traversals per operation template and caches the
adjustment, taking `O(T + D)` time and `O(T + G)` space for `D` total template
body operations. It does not revisit template bodies for every chunk.
- Exact projection analysis visits each selected physical slot once per
consumer chunk and input use, with a producer-lane bitset for deduplication.
If `L` is producer lanes and `Q` is projected slot visits, it takes
`O(Q + L)` time and `O(L + Q)` temporary space; publication maps are cached
across all chunks. Cost extraction shares this traversal with dependencies.
- Conservative producer tracing visits each value/projection once per input,
including captured region operands, with linear time and space in that input's
projection DAG. Direct physical-batch edges retain their existing fast path.
- Pipeline mode removes a complete PEFT run, including its reservation and
insertion-gap searches. Latency mode retains those algorithms and their
existing potentially quadratic gap traversal.
Remaining approximation boundaries include operation issue overhead, eliminated
assembly copies, communication contention and pipeline synchronization. Weight
indexing and ready-transfer signature searches remain possible compiler
hotspots. Changes to these require matched execution measurements.
Implementation sources remain in this directory; project-wide compiler
navigation starts at [docs/README.md](../../../../../../../../docs/README.md).