NiccoloN
5f2db946f0
Validate Operations / validate-operations (push) Has been cancelled
fix failing no-plan throughput ablation runs update results
2.1 KiB
2.1 KiB
| 1 | comparison | arch | model | mode | raptor_pipeline | baseline_variant | baseline_functional_validation | functional_validation | latency_gain_percent | throughput_gain_percent | power_gain_percent | energy_gain_percent |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | plan | arch-a | googlenet | latency | 1 | no-plan | PASS | PASS | +99.14% | NA | -226.98% | +97.19% |
| 3 | plan | arch-a | googlenet | throughput | 4 | no-plan | PASS | PASS | NA | NA | -281.83% | NA |
| 4 | sync | arch-a | googlenet | throughput | 4 | no-sync | FAIL | PASS | -5.17% | -4.96% | +12.23% | -1.65% |
| 5 | plan | arch-a | resnet18 | latency | 1 | no-plan | PASS | PASS | +96.96% | NA | -156.23% | +92.21% |
| 6 | plan | arch-a | resnet18 | throughput | 4 | no-plan | PASS | PASS | +95.17% | +1970.00% | -157.48% | +85.39% |
| 7 | sync | arch-a | resnet18 | throughput | 4 | no-sync | FAIL | PASS | +7.15% | +7.81% | +14.70% | +7.92% |
| 8 | plan | arch-a | resnet34 | latency | 1 | no-plan | PASS | PASS | +95.94% | NA | -130.05% | +90.65% |
| 9 | plan | arch-a | resnet34 | throughput | 4 | no-plan | PASS | PASS | +96.00% | +2400.00% | -189.18% | +88.43% |
| 10 | sync | arch-a | resnet34 | throughput | 4 | no-sync | FAIL | PASS | -9.11% | -8.42% | +7.32% | -1.02% |
| 11 | plan | arch-a | vgg8 | latency | 1 | no-plan | PASS | PASS | +94.75% | NA | -80.65% | +90.51% |
| 12 | plan | arch-a | vgg8 | throughput | 4 | no-plan | PASS | PASS | +92.77% | +1279.31% | -170.46% | +82.49% |
| 13 | sync | arch-a | vgg8 | throughput | 4 | no-sync | FAIL | PASS | +19.18% | +24.03% | +21.29% | +18.62% |
| 14 | plan | arch-b | googlenet | latency | 1 | no-plan | PASS | PASS | +98.99% | NA | -239.10% | +96.59% |
| 15 | plan | arch-b | googlenet | throughput | 4 | no-plan | PASS | PASS | NA | NA | -306.99% | NA |
| 16 | sync | arch-b | googlenet | throughput | 4 | no-sync | FAIL | PASS | -1.28% | -1.63% | +9.99% | -4.29% |
| 17 | plan | arch-b | resnet18 | latency | 1 | no-plan | PASS | PASS | +96.71% | NA | -171.16% | +91.07% |
| 18 | plan | arch-b | resnet18 | throughput | 4 | no-plan | PASS | PASS | +95.46% | +2102.50% | -200.52% | +83.26% |
| 19 | sync | arch-b | resnet18 | throughput | 4 | no-sync | FAIL | PASS | +2.22% | +2.32% | +8.26% | -7.88% |
| 20 | plan | arch-b | resnet34 | latency | 1 | no-plan | PASS | PASS | +96.23% | NA | -166.76% | +89.94% |
| 21 | plan | arch-b | resnet34 | throughput | 4 | no-plan | PASS | PASS | +97.58% | +4030.00% | -168.71% | +91.79% |
| 22 | sync | arch-b | resnet34 | throughput | 4 | no-sync | FAIL | PASS | -0.80% | -0.96% | +10.61% | -13.51% |
| 23 | plan | arch-b | vgg8 | latency | 1 | no-plan | PASS | PASS | +94.68% | NA | -98.04% | +89.47% |
| 24 | plan | arch-b | vgg8 | throughput | 4 | no-plan | PASS | PASS | +93.70% | +1484.16% | -205.54% | +81.37% |
| 25 | sync | arch-b | vgg8 | throughput | 4 | no-sync | FAIL | PASS | +17.07% | +20.30% | +19.03% | +9.63% |