I am a computer science PhD student at University of Oregon, and am advised by Prof. Humphrey Shi. My research interests are attention-based models, representation learning, and self-supervision. During my undergrad, I did research under the supervision of Prof. Abbas Salemi at the Mahani Mathematical Research Center.
Select publications & preprints
I've been working on attention-based models for vision (i.e. vision transformers) and investigating data-efficiency. We recently released NAT, a model based on Neighborhood Attention, which competes with state-of-the-art attention-based models (Swin) and CNNs (ConvNeXt). Our first work, CCT extended vision transformers to data-limited regimes, and competed with state-of-the-art CNNs on datasets such as CIFAR-10. It also became the new state of the art on Flowers-102.
2019 to 2021
Mahani Mathematical Research Center.
My research consisted of clustering and feature extraction using matrix factorization.