Ciro Santilli
OurBigBook.commlcommons.org/en/ Their homepage is not amazingly organized, but it does the job.
Benchmark focused on deep learning. It has two parts:Furthermore, a specific network model is specified for each benchmark in the closed category: so it goes beyond just specifying the dataset.
Results can be seen e.g. at:
Those URLs broke as of 2025 of course, now you have to click on their Tableau down to the 2.1 round and there's no fixed URL for it:
And there are also separate repositories for each:
E.g. on mlcommons.org/en/training-normal-21/ we can see what the the benchmarks are:
| Dataset | Model |
|---|---|
| ImageNet | ResNet |
| KiTS19 | 3D U-Net |
| OpenImages | RetinaNet |
| COCO dataset | Mask R-CNN |
| LibriSpeech | RNN-T |
| Wikipedia | BERT |
| 1TB Clickthrough | DLRM |
| Go | MiniGo |