Discamus continentiam augere, luxuriam coercere
Home -> Publications
edited volumes
  Full CV [pdf]


  Past Events

Publications of Torsten Hoefler
Andrei Ivanov, Nikoli Dryden, Torsten Hoefler:

 STen: An Interface for Efficient Sparsity in PyTorch

(In Sparsity in Neural Networks workshop, Jul. 2022)


As deep learning models grow, sparsity is becoming an increasingly critical component of deep neural networks, enabling improved performance and reduced storage. However, existing frameworks offer poor support for sparsity. They primarily focus on sparse tensors in classical formats such as COO and CSR, which are not well suited to the sparsity regimes typical of deep learning, and neglect the broader sparsification pipeline necessary for using sparse models. To address this, we propose a new sparsity interface for PyTorch, STen, that incorporates sparsity layouts for tensors (including parameters and transients, e.g., activations), sparsity-aware operators, and sparsifiers, which define how a tensor is sparsified, and supports virtually all sparsification methods. STen can enable better sparse performance and simplify building sparse models, helping to make sparsity easily accessible.


download article:
download attachment:


  author={Andrei Ivanov and Nikoli Dryden and Torsten Hoefler},
  title={{STen: An Interface for Efficient Sparsity in PyTorch}},
  booktitle={Sparsity in Neural Networks workshop},

serving:© Torsten Hoefler