Ali Hassani

Research Scientist at NVIDIA

I am a Research Scientist at NVIDIA's Deep Imagination Research Group. My primary research area is High-Performance AI Architecture, which involves making fast and efficient AI, from neural network architecture down to computer architecture.

I've been working on Attention and Sparse Attention since about 2022. A large part of my work is available through NATTEN, an open-source project for fast multi-dimensional sparse attention (see my presentation on GPU MODE). In our most recent paper, Generalized Neighborhood Attention, we discuss outstanding challenges in sparse attention infrastructure, and our proposed solutions, which involve both an analytical study, and a fast implementation achieving the maximum speedup theoretically possible.

GitHub / Google Scholar / Twitter / LinkedIn / CV