NiccoloN
2e76164aed
Validate Operations / validate-operations (push) Waiting to run
update pimsim-nn submodule
4.9 KiB
4.9 KiB
| 1 | model | arch | mode | raptor_pipeline | pimcomp_pipeline | raptor_functional_validation | pimcomp_functional_validation | raptor_throughput_samples_s | pimcomp_throughput_samples_s | raptor_latency_ms | pimcomp_latency_ms | raptor_power_mw | pimcomp_power_mw | raptor_energy_pj | pimcomp_energy_pj | better_compiler | speedup |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | vgg8 | arch-a | latency | 1 | element | PASS | PASS | NA | NA | 1.465778 | 7.985074 | 325.627854 | 200.111367 | 477298145.040001 | 1597904071.120000 | raptor | 5.45 |
| 3 | vgg8 | arch-b | latency | 1 | element | PASS | PASS | NA | NA | 1.438869 | 7.152125 | 304.673458 | 173.768633 | 438385194.040001 | 1242814988.120001 | raptor | 4.97 |
| 4 | vgg8 | arch-c | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 5 | vgg8 | arch-a | throughput | 2 | batch | PASS | PASS | 1480.000000 | 1380.000000 | 0.674197 | 0.725548 | 456.064662 | 475.694103 | 307477506.200000 | 345138738.300000 | raptor | 1.07 |
| 6 | vgg8 | arch-b | throughput | 2 | batch | PASS | PASS | 1160.000000 | 1080.000000 | 0.861527 | 0.921878 | 333.045167 | 408.540967 | 286927542.600000 | 376624916.800000 | raptor | 1.07 |
| 7 | vgg8 | arch-c | throughput | 2 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 8 | vgg8 | arch-a | throughput | 4 | batch | PASS | PASS | 2160.000000 | 1380.000000 | 0.462342 | 0.725548 | 446.528150 | 475.694103 | 206448818.800000 | 345138738.300000 | raptor | 1.57 |
| 9 | vgg8 | arch-c | throughput | 4 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 10 | vgg8 | arch-a | throughput | 8 | batch | PASS | PASS | 831.000000 | 1380.000000 | 1.202894 | 0.725548 | 331.341726 | 475.694103 | 398569128.700000 | 345138738.300000 | pimcomp | 1.66 |
| 11 | vgg8 | arch-c | throughput | 8 | batch | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 12 | resnet18 | arch-a | latency | 1 | element | PASS | PASS | NA | NA | 28.099951 | 58.855175 | 312.513413 | 237.590194 | 8781611597.119984 | 13983412446.119972 | raptor | 2.09 |
| 13 | resnet18 | arch-b | latency | 1 | element | PASS | PASS | NA | NA | 34.938563 | 65.084209 | 254.439797 | 200.986958 | 8889760875.119959 | 13081077194.119965 | raptor | 1.86 |
| 14 | resnet18 | arch-c | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 15 | resnet18 | arch-a | throughput | 2 | batch | PASS | FAIL | 20.000000 | 76.000000 | 50.000000 | 13.149606 | 263.148477 | 479.788072 | 13157423870.000000 | 6309024249.000000 | pimcomp | 3.80 |
| 16 | resnet18 | arch-b | throughput | 2 | batch | PASS | FAIL | 26.300000 | 83.200000 | 38.016529 | 12.020906 | 250.553072 | 483.029728 | 9525158116.000000 | 5806454916.000000 | pimcomp | 3.16 |
| 17 | resnet18 | arch-c | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 18 | resnet18 | arch-a | throughput | 4 | batch | PASS | FAIL | 31.700000 | 76.000000 | 31.578947 | 13.149606 | 336.949886 | 479.788072 | 10640522720.000000 | 6309024249.000000 | pimcomp | 2.40 |
| 19 | resnet18 | arch-c | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 20 | resnet18 | arch-a | throughput | 8 | batch | PASS | FAIL | 46.400000 | 76.000000 | 21.566110 | 13.149606 | 319.292391 | 479.788072 | 6885894954.000000 | 6309024249.000000 | pimcomp | 1.64 |
| 21 | resnet18 | arch-c | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 22 | resnet34 | arch-a | latency | 1 | element | PASS | PASS | NA | NA | 45.781484 | 91.608751 | 326.833876 | 248.044564 | 14962939889.679951 | 22723052668.680016 | raptor | 2.00 |
| 23 | resnet34 | arch-b | latency | 1 | element | PASS | PASS | NA | NA | 72.119607 | 94.519582 | 239.192092 | 215.513439 | 17250439680.679901 | 20370240175.680019 | raptor | 1.31 |
| 24 | resnet34 | arch-c | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 25 | resnet34 | arch-a | throughput | 2 | batch | FAIL | FAIL | NA | 40.800000 | NA | 24.522761 | NA | 506.131154 | NA | 12411733160.000000 | NA | NA |
| 26 | resnet34 | arch-b | throughput | 2 | batch | PASS | FAIL | 11.600000 | 43.300000 | 86.250000 | 23.076923 | 260.049856 | 489.500435 | 22429300100.000000 | 11296163880.000000 | pimcomp | 3.73 |
| 27 | resnet34 | arch-c | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 28 | resnet34 | arch-a | throughput | 4 | batch | FAIL | FAIL | NA | 40.800000 | NA | 24.522761 | NA | 506.131154 | NA | 12411733160.000000 | NA | NA |
| 29 | resnet34 | arch-c | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 30 | resnet34 | arch-a | throughput | 8 | batch | PASS | FAIL | 24.300000 | 40.800000 | 41.176471 | 24.522761 | 271.814969 | 506.131154 | 11192381060.000000 | 12411733160.000000 | pimcomp | 1.68 |
| 31 | resnet34 | arch-c | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 32 | googlenet | arch-a | latency | 1 | element | PASS | PASS | NA | NA | 13.032305 | 62.923369 | 465.072002 | 231.194088 | 6060960174.919998 | 14547510894.240002 | raptor | 4.83 |
| 33 | googlenet | arch-b | latency | 1 | element | PASS | PASS | NA | NA | 16.086822 | 37.935747 | 378.951815 | 242.039077 | 6096130396.919978 | 9181933205.239973 | raptor | 2.36 |
| 34 | googlenet | arch-c | latency | 1 | element | FAIL | PASS | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 35 | googlenet | arch-a | throughput | 2 | batch | PASS | FAIL | 55.000000 | 66.200000 | 18.181818 | 15.094340 | 346.696726 | 434.959232 | 6303576840.000000 | 6565422375.000000 | pimcomp | 1.20 |
| 36 | googlenet | arch-b | throughput | 2 | batch | PASS | FAIL | 55.000000 | 72.000000 | 18.181818 | 13.888889 | 314.645370 | 417.992095 | 5720824912.000000 | 5805445765.000000 | pimcomp | 1.31 |
| 37 | googlenet | arch-c | throughput | 2 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 38 | googlenet | arch-a | throughput | 4 | batch | PASS | FAIL | 111.000000 | 66.200000 | 9.012016 | 15.094340 | 373.195227 | 434.959232 | 3363241363.000000 | 6565422375.000000 | raptor | 1.68 |
| 39 | googlenet | arch-c | throughput | 4 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
| 40 | googlenet | arch-a | throughput | 8 | batch | PASS | FAIL | 61.300000 | 66.200000 | 16.310680 | 15.094340 | 341.592923 | 434.959232 | 5571612719.000000 | 6565422375.000000 | pimcomp | 1.08 |
| 41 | googlenet | arch-c | throughput | 8 | batch | FAIL | FAIL | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |