Life would be so much easier if only we had the source code...
Home -> Publications
Home
  Publications
    
edited volumes
  Awards
  Research
  Teaching
  Miscellaneous
  Full CV [pdf]
  BLOG






  Events








  Past Events





Publications of Torsten Hoefler
Torsten Hoefler, Roberto Belli:

 Scientific Benchmarking of Parallel Computing Systems

(presented in Austin, TX, USA, pages 73:1--73:12, ACM, ISBN: 978-1-4503-3723-6, Nov. 2015, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC15) )

Abstract

Measuring and reporting performance of parallel computers constitutes the basis for scientific advancement of high-performance computing (HPC). Most scientific reports show performance improvements of new techniques and are thus obliged to ensure reproducibility or at least interpretability. Our investigation of a stratified sample of 120 papers across three top conferences in the field shows that the state of the practice is not sufficient. For example, it is often unclear if reported improvements are in the noise or observed by chance. In addition to distilling best practices from existing work, we propose statistically sound analysis and reporting techniques and simple guidelines for experimental design in parallel computing. We aim to improve the standards of reporting research results and initiate a discussion in the HPC field. A wide adoption of this minimal set of rules will lead to better reproducibility and interpretability of performance results and improve the scientific culture around HPC.

Documents

download article:
download slides:


Recorded talk (best effort)

 

BibTeX

@inproceedings{benchmarking,
  author={Torsten Hoefler and Roberto Belli},
  title={{Scientific Benchmarking of Parallel Computing Systems}},
  year={2015},
  month={Nov.},
  pages={73:1--73:12},
  location={Austin, TX, USA},
  publisher={ACM},
  isbn={978-1-4503-3723-6},
  note={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC15)},
  source={http://www.unixer.de/~htor/publications/},
}


serving: 34.239.153.44:55508© Torsten Hoefler