NiccoloN
93 lines
3.7 KiB
Markdown
93 lines
3.7 KiB
Markdown
# Compiler Performance Optimization Invariant
|
|
|
|
## Scope
|
|
|
|
This invariant applies to:
|
|
|
|
- ONNX-to-Spatial lowering;
|
|
- graph and trivial merge;
|
|
- PEFT scheduling;
|
|
- scheduled materialization;
|
|
- deferred communication;
|
|
- Spatial-to-PIM lowering;
|
|
- bufferization;
|
|
- liveness planning;
|
|
- PIM code generation.
|
|
|
|
## Invariant
|
|
|
|
A compiler compile-time or memory optimization must not reduce available
|
|
hardware parallelism or worsen theoretical execution time. In the absence of a
|
|
precise runtime model, uncertainty is not permission to accept a possible
|
|
runtime regression. It is forbidden to serialize independent work, reduce
|
|
the number of simultaneously schedulable compute instances, replace parallel
|
|
graph or core batches with sequential loops, introduce recomputation, add
|
|
runtime copies, increase communication or instruction count, or lengthen the
|
|
schedule critical path merely to reduce compiler time or memory usage.
|
|
|
|
Compiler representations may share analysis, planning, or code-generation
|
|
work only when the emitted execution retains the same independent work,
|
|
schedule semantics, scheduling granularity, and available parallelism.
|
|
|
|
Static runtime proxies are useful for screening changes, but they are not the
|
|
latency authority. When comparable validation results are available for the
|
|
same model, inputs, hardware configuration, and non-functional simulation
|
|
configuration, the `Latency` reported by `validation/validate.py` is
|
|
authoritative. An optimization is acceptable only when its static runtime
|
|
proxies are equal or better, unless such a matched validation comparison
|
|
demonstrates equal or better latency.
|
|
|
|
## Asymptotic cost
|
|
|
|
For each new or materially changed compiler algorithm, identify the relevant
|
|
input size and target linear or sublinear time and space. Avoid repeated full-IR
|
|
walks, nested scans, and per-operation recomputation when indexing, caching, or
|
|
a single traversal can express the same behavior.
|
|
|
|
When linear-or-better complexity is not possible, use the lowest justified
|
|
complexity and report the actual time and space Big-O, the input variable, and
|
|
why a lower bound is not practical. Include that cost in the final report; do
|
|
not hide a superlinear path behind small current test sizes.
|
|
|
|
## Forbidden optimization trades
|
|
|
|
Do not:
|
|
|
|
- use fewer parallel lanes or cores to simplify IR;
|
|
- coarsen lane, core, batch, or scheduling granularity when it may reduce
|
|
concurrency;
|
|
- scalarize a valid graph or core batch;
|
|
- replace independent operations with one sequential loop;
|
|
- recompute values to reduce storage;
|
|
- add device/device or host/device copies to reduce peak memory;
|
|
- change hardware parameters to make a benchmark easier;
|
|
- reduce communication concurrency by imposing a global serial order;
|
|
- increase emitted instruction count to reduce compiler analysis;
|
|
- move PIM work to the host.
|
|
|
|
## Required proof
|
|
|
|
Every performance change must report before and after:
|
|
|
|
- matched validation `Latency` when the changed execution can be simulated;
|
|
- compiler wall time;
|
|
- compiler peak RSS;
|
|
- IR operation and value counts at the changed stage;
|
|
- logical compute-instance count;
|
|
- graph compute batch and scheduled batch lane counts;
|
|
- PEFT class and core assignment;
|
|
- maximum scheduled critical-path or step count;
|
|
- MVM/VMM and vector instruction counts;
|
|
- send and receive counts and bytes;
|
|
- total emitted instruction count;
|
|
- maximum instructions assigned to one core;
|
|
- number and size of runtime memory copies;
|
|
- final host, weights, logical-core, physical-core, and maximum-core memory;
|
|
- numerical validation.
|
|
|
|
## Stop rule
|
|
|
|
If compile time or memory improves but a runtime proxy worsens, stop and reject
|
|
the change unless a matched validation comparison demonstrates equal or better
|
|
`Latency`. Proxy improvements do not override a validation latency regression.
|