NiccoloN
6.2 KiB
6.2 KiB
| 1 | arch | model | mode | raptor_pipeline | pimcomp_pipeline | raptor_functional_validation | pimcomp_functional_validation | raptor_latency_ms | pimcomp_latency_ms | raptor_throughput_samples_s | pimcomp_throughput_samples_s | raptor_power_mw | pimcomp_power_mw | raptor_energy_pj | pimcomp_energy_pj | better_compiler | speedup |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | arch-a | vgg8 | latency | 1 | element | PASS | PASS | 1.47 | 7.99 | NA | NA | 325.67 | 200.11 | 477232782.04 | 1597904071.12 | raptor | 5.45 |
| 3 | arch-a | vgg8 | throughput | 2 | batch | PASS | PASS | 0.94 | 1.04 | 1060.00 | 965.00 | 377.54 | 482.60 | 474325940.70 | 541691691.00 | raptor | 1.10 |
| 4 | arch-a | vgg8 | throughput | 4 | batch | PASS | PASS | 1.19 | 1.04 | 839.00 | 965.00 | 310.32 | 482.60 | 500148858.20 | 541691691.00 | pimcomp | 1.15 |
| 5 | arch-a | vgg8 | throughput | 8 | batch | PASS | PASS | 1.52 | 1.04 | 658.00 | 965.00 | 279.37 | 482.60 | 554348483.90 | 541691691.00 | pimcomp | 1.47 |
| 6 | arch-a | resnet18 | latency | 1 | element | PASS | FAIL | 15.65 | 58.18 | NA | NA | 425.87 | 238.39 | 6666000553.12 | 13869146503.12 | raptor | 3.72 |
| 7 | arch-a | resnet18 | throughput | 2 | batch | PASS | FAIL | 30.55 | 13.46 | 32.70 | 74.30 | 295.90 | 481.60 | 10218790630.00 | 6562535800.00 | pimcomp | 2.27 |
| 8 | arch-a | resnet18 | throughput | 4 | batch | PASS | FAIL | 26.60 | 13.46 | 37.60 | 74.30 | 313.14 | 481.60 | 10993189350.00 | 6562535800.00 | pimcomp | 1.98 |
| 9 | arch-a | resnet18 | throughput | 8 | batch | PASS | FAIL | 35.51 | 13.46 | 28.20 | 74.30 | 275.95 | 481.60 | 12844245960.00 | 6562535800.00 | pimcomp | 2.63 |
| 10 | arch-a | resnet34 | latency | 1 | element | PASS | FAIL | 32.37 | 66.71 | NA | NA | 391.82 | 277.48 | 12684227802.68 | 18511540483.68 | raptor | 2.06 |
| 11 | arch-a | resnet34 | throughput | 2 | batch | FAIL | FAIL | NA | 23.58 | NA | 42.40 | NA | 515.66 | NA | 12277699310.00 | NA | NA |
| 12 | arch-a | resnet34 | throughput | 4 | batch | FAIL | FAIL | NA | 23.58 | NA | 42.40 | NA | 515.66 | NA | 12277699310.00 | NA | NA |
| 13 | arch-a | resnet34 | throughput | 8 | batch | PASS | FAIL | 115.11 | 23.58 | 8.69 | 42.40 | 229.85 | 515.66 | 29881854960.00 | 12277699310.00 | pimcomp | 4.88 |
| 14 | arch-a | googlenet | latency | 1 | element | PASS | PASS | 13.03 | 62.92 | NA | NA | 465.11 | 231.19 | 6060646341.92 | 14547510894.24 | raptor | 4.83 |
| 15 | arch-a | googlenet | throughput | 2 | batch | PASS | FAIL | 24.01 | 18.27 | 41.70 | 54.70 | 328.17 | 435.11 | 8529823770.00 | 8164469406.00 | pimcomp | 1.31 |
| 16 | arch-a | googlenet | throughput | 4 | batch | PASS | FAIL | 27.01 | 18.27 | 37.00 | 54.70 | 305.68 | 435.11 | 10498372450.00 | 8164469406.00 | pimcomp | 1.48 |
| 17 | arch-a | googlenet | throughput | 8 | batch | PASS | FAIL | 40.03 | 18.27 | 25.00 | 54.70 | 268.61 | 435.11 | 15290418970.00 | 8164469406.00 | pimcomp | 2.19 |
| 18 | arch-a | yolo11n | latency | 1 | element | PASS | FAIL | 267.73 | NA | NA | NA | 235.73 | NA | 63111612279.00 | NA | NA | NA |
| 19 | arch-a | yolo11n | throughput | 2 | batch | PASS | FAIL | 500.00 | NA | 2.00 | NA | 227.74 | NA | 113867559100.00 | NA | NA | NA |
| 20 | arch-a | yolo11n | throughput | 4 | batch | PASS | FAIL | 500.00 | NA | 2.00 | NA | 233.16 | NA | 116581237400.00 | NA | NA | NA |
| 21 | arch-a | yolo11n | throughput | 8 | batch | PASS | FAIL | 500.00 | NA | 2.00 | NA | 253.59 | NA | 126797224900.00 | NA | NA | NA |
| 22 | arch-b | vgg8 | latency | 1 | element | PASS | PASS | 1.44 | 7.15 | NA | NA | 304.72 | 173.77 | 438329594.04 | 1242814988.12 | raptor | 4.97 |
| 23 | arch-b | vgg8 | throughput | 2 | batch | PASS | PASS | 1.56 | 1.07 | 640.00 | 932.00 | 287.50 | 409.47 | 478592086.00 | 439288835.80 | pimcomp | 1.46 |
| 24 | arch-b | vgg8 | throughput | 4 | batch | PASS | PASS | 1.14 | 1.07 | 874.00 | 932.00 | 323.79 | 409.47 | 457617420.30 | 439288835.80 | pimcomp | 1.07 |
| 25 | arch-b | vgg8 | throughput | 8 | batch | PASS | PASS | 1.48 | 1.07 | 676.00 | 932.00 | 282.34 | 409.47 | 437422264.40 | 439288835.80 | pimcomp | 1.38 |
| 26 | arch-b | resnet18 | latency | 1 | element | PASS | FAIL | 17.98 | 64.38 | NA | NA | 362.63 | 201.66 | 6520518736.12 | 12983607699.12 | raptor | 3.58 |
| 27 | arch-b | resnet18 | throughput | 2 | batch | PASS | FAIL | 33.84 | 13.43 | 29.60 | 74.40 | 253.32 | 484.80 | 9423636948.00 | 6569655551.00 | pimcomp | 2.51 |
| 28 | arch-b | resnet18 | throughput | 4 | batch | PASS | FAIL | 26.14 | 13.43 | 38.30 | 74.40 | 286.86 | 484.80 | 9493311384.00 | 6569655551.00 | pimcomp | 1.94 |
| 29 | arch-b | resnet18 | throughput | 8 | batch | PASS | FAIL | 28.29 | 13.43 | 35.30 | 74.40 | 272.66 | 484.80 | 11314980340.00 | 6569655551.00 | pimcomp | 2.11 |
| 30 | arch-b | resnet34 | latency | 1 | element | PASS | FAIL | 49.51 | 107.20 | NA | NA | 284.72 | 206.47 | 14097763205.68 | 22134333948.68 | raptor | 2.17 |
| 31 | arch-b | resnet34 | throughput | 2 | batch | PASS | FAIL | 93.13 | 22.80 | 10.70 | 43.90 | 215.38 | 493.39 | 21278771500.00 | 11381213070.00 | pimcomp | 4.10 |
| 32 | arch-b | resnet34 | throughput | 4 | batch | PASS | FAIL | 67.55 | 22.80 | 14.80 | 43.90 | 237.97 | 493.39 | 19346594270.00 | 11381213070.00 | pimcomp | 2.97 |
| 33 | arch-b | resnet34 | throughput | 8 | batch | PASS | FAIL | 115.01 | 22.80 | 8.70 | 43.90 | 197.27 | 493.39 | 26678372930.00 | 11381213070.00 | pimcomp | 5.05 |
| 34 | arch-b | googlenet | latency | 1 | element | PASS | PASS | 16.08 | 37.94 | NA | NA | 378.99 | 242.04 | 6095729659.92 | 9181933205.24 | raptor | 2.36 |
| 35 | arch-b | googlenet | throughput | 2 | batch | PASS | FAIL | 28.37 | 21.39 | 35.20 | 46.70 | 267.15 | 417.83 | 8684222064.00 | 9014655684.00 | pimcomp | 1.33 |
| 36 | arch-b | googlenet | throughput | 4 | batch | PASS | FAIL | 26.16 | 21.39 | 38.20 | 46.70 | 280.21 | 417.83 | 9300527914.00 | 9014655684.00 | pimcomp | 1.22 |
| 37 | arch-b | googlenet | throughput | 8 | batch | PASS | FAIL | 46.45 | 21.39 | 21.50 | 46.70 | 237.73 | 417.83 | 14078049200.00 | 9014655684.00 | pimcomp | 2.17 |
| 38 | arch-b | yolo11n | latency | 1 | element | PASS | FAIL | 316.70 | NA | NA | NA | 195.43 | NA | 61892225295.00 | NA | NA | NA |
| 39 | arch-b | yolo11n | throughput | 2 | batch | PASS | FAIL | 333.33 | NA | 3.00 | NA | 214.01 | NA | 71336401290.00 | NA | NA | NA |
| 40 | arch-b | yolo11n | throughput | 4 | batch | PASS | FAIL | 333.33 | NA | 3.00 | NA | 216.24 | NA | 72079194110.00 | NA | NA | NA |
| 41 | arch-b | yolo11n | throughput | 8 | batch | PASS | FAIL | 500.00 | NA | 2.00 | NA | 218.42 | NA | 109211119900.00 | NA | NA | NA |
| 42 | arch-c | vgg8 | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 43 | arch-c | vgg8 | throughput | 2 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 44 | arch-c | vgg8 | throughput | 4 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 45 | arch-c | vgg8 | throughput | 8 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 46 | arch-c | resnet18 | latency | 1 | element | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 47 | arch-c | resnet18 | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 48 | arch-c | resnet18 | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 49 | arch-c | resnet18 | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 50 | arch-c | resnet34 | latency | 1 | element | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 51 | arch-c | resnet34 | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 52 | arch-c | resnet34 | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 53 | arch-c | resnet34 | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 54 | arch-c | googlenet | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 55 | arch-c | googlenet | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 56 | arch-c | googlenet | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 57 | arch-c | googlenet | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 58 | arch-c | yolo11n | latency | 1 | element | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 59 | arch-c | yolo11n | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 60 | arch-c | yolo11n | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 61 | arch-c | yolo11n | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |