# Graph Explorer architecture and API

The [README](README.md) covers installation, input headers, commands, and tests.
This reference describes the contracts maintained by the implementation.

## Ingestion and identity

[`discovery.py`](raptor_graph_explorer/discovery.py) recursively pairs reports
by their full filename prefix, including ZIP inputs, and rejects unsafe archive
paths. That prefix is `report_id`; unrelated reports never share endpoints.

[`ingest.py`](raptor_graph_explorer/ingest.py) streams CSVs in batches of 2,000
rows through temporary SQLite staging tables. Node IDs, operation IDs, lanes,
cores, and channels remain separate fields. Known node prefixes are `gc`,
`gcb`, `sc`, `scb`; unknown ones retain `unknown:<prefix>`. Explicit operation
and lane columns are authoritative. Blank operation IDs, and blank batch lanes,
may use a parseable source ID with a warning. Explicit edge lanes override
endpoint lanes, with contradictory values diagnosed. Missing numbers stay
SQL NULL; malformed rows are diagnosed and omitted, never coerced to zero.
Extra columns survive in `attributes_json`.

Duplicate node IDs retain the first source row and produce diagnostics. Bulk
joins resolve edges within their report; unknown endpoints are omitted with
bounded examples. Staging tables are dropped even on error. The ingestion
connection closes before indexes, aggregates, motifs, and diagnostics are
built. Secondary indexes are created after raw insertion. CSVs without extra
columns reuse an empty JSON representation.

Raw edge IDs use report ID plus original CSV row number; aggregate IDs use
report ID and grouping keys. Input filesystem paths and browser layout do not
define identity. The manifest stores versions and counts, not duplicate raw
rows or machine-specific input paths. Database schema compatibility is checked
before serving; incompatible outputs must be rebuilt.

## Aggregation and exact lane mappings

Operation nodes group by `(report_id, op_id)`, retaining kind counts, lane/core
cardinalities and ranges, and SSA summaries. Operation edges group by the
source/target operation pair, retaining counts, weights, tensor types, channels,
and exact mapping classification. Core aggregation has a distinct `unassigned`
group for missing cores. Node-kind aggregation is a small diagnostic view.

[`mappings.py`](raptor_graph_explorer/mappings.py) uses all normalized lane
pairs, in the following precedence order:

| Class | Meaning |
|---|---|
| `scalar_to_scalar` | Both lanes absent for every pair. |
| `scalar_to_batch` / `batch_to_scalar` | One side entirely absent, the other entirely present; range, cardinality, and contiguity recorded. |
| `identity` | One-to-one relation with equal lanes. |
| `constant_offset` | One-to-one with one nonzero target-minus-source offset. |
| `permutation` | Remaining one-to-one relation with equal cardinalities. |
| `broadcast` / `fan_in` | One source to many targets / many sources to one target. |
| `stencil` | Multiple sources share exactly the same target-offset set, containing 2–64 offsets. |
| `irregular` | Complete lane data matching none of the preceding forms. |
| `unknown` | Empty or partially missing relation outside the scalar/batch cases. |

SQL computes degrees/cardinalities; stencil comparison streams one operation
pair at a time. No sampling determines classification. Mapping rows remain
paginated; matrices show exact sparse points up to the configured threshold,
then bounded density bins.

## Display projection and layout

[`projection.py`](raptor_graph_explorer/projection.py) rebuilds the entire
display from the current expanded-operation set. Expanding replaces an
operation aggregate with its raw nodes, including isolated nodes. An aggregate
edge survives only while both endpoint operations are collapsed. Otherwise its
raw edges replace it: each endpoint is raw if its operation is expanded and
aggregate if collapsed. The complete set determines endpoints, so expansion
order cannot leave stale endpoints or duplicate aggregate/raw representations.
Raw view expands every operation, preserving each valid source edge once.

Operation ranks use the complete operation graph, including isolated nodes.
Collapsed operations occupy one row; expanded lanes use compact rows in numeric
order, followed by deterministic lane-less rows. Sparse lane numbers leave no
empty rows, and nodes sharing a lane get centered deterministic horizontal
offsets. Operation/raw views use stable server coordinates restored by Reset
layout. Core/node-kind views use ELK. Views flow top-to-bottom.

The [viewer](raptor_graph_explorer/static/index.html) first requests an
operation aggregate before applying the report's view selection. Raw selections
use raw details rather than aggregate-only endpoints. Operation labels use the
first SSA name, falling back to source identity; label collision handling and
bounded node sizes keep dense views usable. Dependencies are pinned in that
file (Graphology, Sigma.js, ELK); first browser use requires CDN access.

## Structural motifs

[`motifs.py`](raptor_graph_explorer/motifs.py) loads only the operation aggregate
into NetworkX. It condenses SCCs into a DAG, attaches synthetic source/sink
nodes, and computes dominators and reverse post-dominators. Branching entries
are paired with their immediate post-dominator; regions retain nodes reachable
before that exit which can reach it, rejecting external entrances/exits.
One shortest representative path per branch avoids enumerating all paths.

`exact_diamond` has two direct one-node branches; `rhomboid` has disjoint
branch interiors; `split_rejoin` covers other valid single-entry/single-exit
regions. Identical memberships are deduplicated; nested motifs remain.
SCC members map back to operation IDs, with deterministic representative
operations for multi-operation SCC endpoints/paths and `contains_cycle` markers.
Screen geometry never determines motif membership.

## Read-only API

All routes below use GET; see [`api.py`](raptor_graph_explorer/api.py).
Responses include `Cache-Control: no-store` to avoid mixing viewer revisions.

| Route | Result / relevant query parameters |
|---|---|
| `/api/manifest` | Tool/schema versions and report counts. |
| `/api/reports` | Available reports. |
| `/api/reports/{report_id}/summary` | Counts, mapping classes, diagnostics. |
| `/api/reports/{report_id}/graph/{level}` | Complete `operation`, `core`, or `node_kind` aggregate. |
| `/api/reports/{report_id}/display-graph` | Deterministic projection; `view=operation` or `raw`, repeated `expanded_op_id`, `expand_all`. |
| `/api/reports/{report_id}/motifs` | Structural motifs. |
| `/api/aggregate-nodes/{aggregate_id}` | Aggregate details; operation SSA names capped at 100. |
| `/api/aggregate-edges/{aggregate_edge_id}` | Aggregate edge details. |
| `/api/aggregate-edges/{aggregate_edge_id}/mapping` | Exact raw lane rows; `limit`, `offset`. Operation edges only. |
| `/api/aggregate-edges/{aggregate_edge_id}/matrix` | Exact sparse points or density bins. Operation edges only. |
| `/api/reports/{report_id}/raw-nodes` | Raw node page; `limit`, `offset`. |
| `/api/reports/{report_id}/raw-edges` | Raw edge page; `limit`, `offset`. |
| `/api/reports/{report_id}/subgraph` | `aggregate_id`, `direction=in/out/both`, `depth`, optional `expand_lanes`. |

`create_app` defaults: page size 100 with a maximum of 500; expansion cap 5,000;
exact matrix threshold 2,000 points; density grid 64×64. Subgraph depth defaults
to 1 and caps at 5. Oversized raw/expanded projections return HTTP 400 without a
partial graph. The projection cap counts raw nodes plus incident raw edges;
collapsed aggregate nodes/edges are additional. Aggregate endpoints have no
pagination or expansion cap, so very large operation graphs remain a scaling
limit. Raw CSV graphs are never loaded wholesale into NetworkX.