NiccoloN
034c197260
Validate Operations / validate-operations (push) Failing after 3h10m55s
add plot generation scripts adjust validation tools add up-to-date results
4.8 KiB
4.8 KiB
| 1 | arch | model | mode | raptor_pipeline | pimcomp_pipeline | raptor_functional_validation | pimcomp_functional_validation | raptor_latency_ms | pimcomp_latency_ms | raptor_throughput_samples_s | pimcomp_throughput_samples_s | raptor_power_mw | pimcomp_power_mw | raptor_energy_pj | pimcomp_energy_pj | better_compiler | speedup |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | arch-a | vgg8 | latency | 1 | element | PASS | PASS | 1.52 | 7.99 | NA | NA | 321.05 | 200.11 | 487455139.04 | 1597904071.12 | raptor | 5.26 |
| 3 | arch-a | vgg8 | throughput | 2 | batch | PASS | PASS | 0.91 | 1.04 | 1100.00 | 965.00 | 423.80 | 482.60 | 431626597.00 | 541691691.00 | raptor | 1.14 |
| 4 | arch-a | vgg8 | throughput | 4 | batch | PASS | PASS | 0.86 | 1.04 | 1160.00 | 965.00 | 432.99 | 482.60 | 416224904.20 | 541691691.00 | raptor | 1.20 |
| 5 | arch-a | vgg8 | throughput | 8 | batch | PASS | PASS | 1.63 | 1.04 | 615.00 | 965.00 | 308.12 | 482.60 | 616026711.90 | 541691691.00 | pimcomp | 1.57 |
| 6 | arch-a | resnet18 | latency | 1 | element | PASS | FAIL | 14.03 | 58.18 | NA | NA | 455.67 | 238.39 | 6394498463.12 | 13868866808.12 | raptor | 4.15 |
| 7 | arch-a | resnet18 | throughput | 2 | batch | PASS | FAIL | 11.73 | 13.46 | 85.20 | 74.30 | 503.11 | 481.60 | 6628073894.00 | 6563451004.00 | raptor | 1.15 |
| 8 | arch-a | resnet18 | throughput | 4 | batch | PASS | FAIL | 19.62 | 13.46 | 51.00 | 74.30 | 376.10 | 481.60 | 8024732847.00 | 6563451004.00 | pimcomp | 1.46 |
| 9 | arch-a | resnet18 | throughput | 8 | batch | PASS | FAIL | 19.92 | 13.46 | 50.20 | 74.30 | 368.24 | 481.60 | 9023016592.00 | 6563451004.00 | pimcomp | 1.48 |
| 10 | arch-a | resnet34 | latency | 1 | element | PASS | FAIL | 29.72 | 66.71 | NA | NA | 411.81 | 277.49 | 12238321706.68 | 18511425563.68 | raptor | 2.24 |
| 11 | arch-a | resnet34 | throughput | 2 | batch | PASS | FAIL | 22.26 | 23.58 | 44.90 | 42.40 | 473.84 | 515.68 | 16417303180.00 | 12283627410.00 | raptor | 1.06 |
| 12 | arch-a | resnet34 | throughput | 4 | batch | PASS | FAIL | 27.37 | 23.58 | 36.50 | 42.40 | 415.90 | 515.68 | 17019112320.00 | 12283627410.00 | pimcomp | 1.16 |
| 13 | arch-a | resnet34 | throughput | 8 | batch | PASS | FAIL | 41.04 | 23.58 | 24.40 | 42.40 | 346.15 | 515.68 | 17351854700.00 | 12283627410.00 | pimcomp | 1.74 |
| 14 | arch-a | googlenet | latency | 1 | element | PASS | PASS | 9.78 | 62.92 | NA | NA | 563.88 | 231.19 | 5513234559.92 | 14547744283.24 | raptor | 6.44 |
| 15 | arch-a | googlenet | throughput | 2 | batch | PASS | FAIL | 10.16 | 18.26 | 98.50 | 54.80 | 544.21 | 435.11 | 6029999329.00 | 8164787803.00 | raptor | 1.80 |
| 16 | arch-a | googlenet | throughput | 4 | batch | PASS | FAIL | 11.96 | 18.26 | 83.60 | 54.80 | 473.16 | 435.11 | 6741923526.00 | 8164787803.00 | raptor | 1.53 |
| 17 | arch-a | googlenet | throughput | 8 | batch | PASS | FAIL | 20.66 | 18.26 | 48.40 | 54.80 | 348.76 | 435.11 | 8421537146.00 | 8164787803.00 | pimcomp | 1.13 |
| 18 | arch-a | yolo11n | latency | 1 | element | PASS | FAIL | 147.49 | NA | NA | NA | 290.17 | NA | 42797722129.00 | NA | NA | NA |
| 19 | arch-a | yolo11n | throughput | 2 | batch | PASS | FAIL | 120.93 | NA | 8.27 | NA | 313.20 | NA | 41830757130.00 | NA | NA | NA |
| 20 | arch-a | yolo11n | throughput | 4 | batch | PASS | FAIL | 92.06 | NA | 10.90 | NA | 351.03 | NA | 39451980450.00 | NA | NA | NA |
| 21 | arch-a | yolo11n | throughput | 8 | batch | PASS | FAIL | 104.07 | NA | 9.61 | NA | 330.76 | NA | 44002712680.00 | NA | NA | NA |
| 22 | arch-b | vgg8 | latency | 1 | element | PASS | PASS | 1.53 | 7.15 | NA | NA | 292.56 | 173.77 | 448581679.04 | 1242814988.12 | raptor | 4.66 |
| 23 | arch-b | vgg8 | throughput | 2 | batch | PASS | PASS | 1.18 | 1.07 | 846.00 | 932.00 | 327.05 | 409.47 | 429183159.10 | 439288835.80 | pimcomp | 1.10 |
| 24 | arch-b | vgg8 | throughput | 4 | batch | PASS | PASS | 0.92 | 1.07 | 1090.00 | 932.00 | 383.82 | 409.47 | 464794094.50 | 439288835.80 | raptor | 1.17 |
| 25 | arch-b | vgg8 | throughput | 8 | batch | PASS | PASS | 1.62 | 1.07 | 616.00 | 932.00 | 276.49 | 409.47 | 524048148.70 | 439288835.80 | pimcomp | 1.51 |
| 26 | arch-b | resnet18 | latency | 1 | element | PASS | FAIL | 15.38 | 64.38 | NA | NA | 400.67 | 201.66 | 6160441550.12 | 12983432281.12 | raptor | 4.19 |
| 27 | arch-b | resnet18 | throughput | 2 | batch | PASS | FAIL | 13.08 | 13.43 | 76.40 | 74.40 | 440.79 | 484.80 | 6399942695.00 | 6569655551.00 | raptor | 1.03 |
| 28 | arch-b | resnet18 | throughput | 4 | batch | PASS | FAIL | 20.68 | 13.43 | 48.40 | 74.40 | 327.57 | 484.80 | 7777606323.00 | 6569655551.00 | pimcomp | 1.54 |
| 29 | arch-b | resnet18 | throughput | 8 | batch | PASS | FAIL | 21.17 | 13.43 | 47.20 | 74.40 | 317.64 | 484.80 | 8897039159.00 | 6569655551.00 | pimcomp | 1.58 |
| 30 | arch-b | resnet34 | latency | 1 | element | PASS | FAIL | 27.95 | 107.20 | NA | NA | 397.21 | 206.48 | 11103517952.68 | 22134174654.68 | raptor | 3.83 |
| 31 | arch-b | resnet34 | throughput | 2 | batch | PASS | FAIL | 22.28 | 22.85 | 44.90 | 43.80 | 438.24 | 493.33 | 14218683390.00 | 11392215660.00 | raptor | 1.03 |
| 32 | arch-b | resnet34 | throughput | 4 | batch | PASS | FAIL | 39.61 | 22.85 | 25.20 | 43.80 | 315.01 | 493.33 | 14629161660.00 | 11392215660.00 | pimcomp | 1.74 |
| 33 | arch-b | resnet34 | throughput | 8 | batch | PASS | FAIL | 49.34 | 22.85 | 20.30 | 43.80 | 285.76 | 493.33 | 16591020470.00 | 11392215660.00 | pimcomp | 2.16 |
| 34 | arch-b | googlenet | latency | 1 | element | PASS | PASS | 11.22 | 37.94 | NA | NA | 483.27 | 242.04 | 5422799605.92 | 9182062336.24 | raptor | 3.38 |
| 35 | arch-b | googlenet | throughput | 2 | batch | PASS | FAIL | 9.83 | 21.39 | 102.00 | 46.80 | 520.79 | 417.87 | 5669213312.00 | 9014726090.00 | raptor | 2.18 |
| 36 | arch-b | googlenet | throughput | 4 | batch | PASS | FAIL | 14.47 | 21.39 | 69.10 | 46.80 | 401.01 | 417.87 | 6835460477.00 | 9014726090.00 | raptor | 1.48 |
| 37 | arch-b | googlenet | throughput | 8 | batch | PASS | FAIL | 37.28 | 21.39 | 26.80 | 46.80 | 245.38 | 417.87 | 11453848400.00 | 9014726090.00 | pimcomp | 1.75 |
| 38 | arch-b | yolo11n | latency | 1 | element | PASS | FAIL | 168.14 | NA | NA | NA | 245.31 | NA | 41248047514.00 | NA | NA | NA |
| 39 | arch-b | yolo11n | throughput | 2 | batch | PASS | FAIL | 118.15 | NA | 8.46 | NA | 287.59 | NA | 37062981240.00 | NA | NA | NA |
| 40 | arch-b | yolo11n | throughput | 4 | batch | PASS | FAIL | 94.18 | NA | 10.60 | NA | 316.45 | NA | 37424491250.00 | NA | NA | NA |
| 41 | arch-b | yolo11n | throughput | 8 | batch | PASS | FAIL | 142.86 | NA | 7.00 | NA | 290.68 | NA | 41525990280.00 | NA | NA | NA |