finally fast googlenet with correct latency artifacts for fair comparison
Validate Operations / validate-operations (push) Has been cancelled
Validate Operations / validate-operations (push) Has been cancelled
This commit is contained in:
@@ -8,7 +8,7 @@ model it can:
|
||||
3. compile PIM artifacts with Raptor;
|
||||
4. run the reference implementation and functional PIM simulator;
|
||||
5. compare their outputs;
|
||||
6. run `pimsim-nn` to report latency and power.
|
||||
6. run `pimsim-nn` to report latency, power, and energy.
|
||||
|
||||
Run the script from the repository root with the repository Python environment.
|
||||
|
||||
@@ -65,7 +65,7 @@ The script discovers them recursively.
|
||||
The PIMCOMP paper-model suite has a one-command Arch-A comparison:
|
||||
|
||||
```bash
|
||||
.venv/bin/python validation/tools/run_pimcomp_paper_latency.py
|
||||
.venv/bin/python validation/tools/pimcomp/run_pimcomp_paper_latency.py
|
||||
```
|
||||
|
||||
The runner verifies PIMCOMP's population-200, 1000-iteration GA settings,
|
||||
@@ -134,7 +134,7 @@ count with `-j` or `--jobs`:
|
||||
| `--simulator-dir PATH` | Functional `pim-simulator` crate directory. Defaults to the in-tree simulator. |
|
||||
| `--non-functional-simulator-build-dir PATH` | `pimsim-nn` build directory. Defaults to the in-tree build. |
|
||||
| `--pimcomp-config {arch-a,arch-b,arch-c}` | Non-functional hardware/timing profile. Defaults to `arch-a`. |
|
||||
| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency and power measurement. |
|
||||
| `--skip-non-functional-simulation` | Skip `pimsim-nn` latency, power, and energy measurement. |
|
||||
| `--threshold FLOAT` | Absolute output-comparison tolerance. Defaults to `1e-3`. |
|
||||
| `--relative-threshold FLOAT` | Relative output-comparison tolerance. Defaults to `1e-5`. |
|
||||
| `--seed INT` | Seed for generated inputs. Defaults to `0`. |
|
||||
@@ -165,17 +165,17 @@ prints the incompatible values; functional validation still runs.
|
||||
The checked-in profiles are under
|
||||
`validation/pimsim_configs/pimcomp/<profile>/latency_config.json`.
|
||||
|
||||
Use `--skip-non-functional-simulation` when latency and power are not required.
|
||||
Use `--skip-non-functional-simulation` when latency, power, and energy are not required.
|
||||
The summary reports non-functional results as measured, failed, unsupported, or
|
||||
skipped.
|
||||
|
||||
Overall PASS/FAIL is determined by compilation and functional output
|
||||
comparison. A non-functional simulation failure remains visible as `ERROR` in
|
||||
the latency and power columns but does not change a functional PASS.
|
||||
the latency, power, and energy columns but does not change a functional PASS.
|
||||
|
||||
`pimsim-nn` does not currently implement the `vsoftmax` instruction. When its
|
||||
explicit unsupported-op diagnostic is encountered, Softmax validations retain
|
||||
their functional PASS and show `UNSUPPORTED` in both non-functional columns.
|
||||
their functional PASS and show `UNSUPPORTED` in the non-functional columns.
|
||||
Other `pimsim-nn` failures remain `ERROR`.
|
||||
|
||||
## Generated artifacts
|
||||
@@ -186,7 +186,7 @@ Artifacts are written beside each model:
|
||||
|---|---|
|
||||
| `inputs/` | Generated input CSV files. |
|
||||
| `outputs/` | ONNX-MLIR reference output CSV files. |
|
||||
| `raptor/` | Exported MLIR, dialect snapshots, reports, and final `pim/` artifacts. |
|
||||
| `raptor/` | Exported MLIR, dialect snapshots, reports, final FP32 `pim/` artifacts, and the int8-equivalent `pimsim_nn/` latency view. |
|
||||
| `runner/` | Generated reference runner source, build tree, and shared library. |
|
||||
| `simulation/` | Functional simulator outputs used for numerical comparison. |
|
||||
| `pimcomp/` | PIMCOMP graph, instruction, simulator, and comparison-report artifacts. |
|
||||
|
||||
Reference in New Issue
Block a user