I am a first year computer science graduate student at University of Oregon, and am advised by Prof. Humphrey Shi. I work at SHI Lab @ UO as a graduate researcher.
My research mostly involves computer vision and language processing. My research interests are transformer-based models, vision transformers and self-supervision.
During my undergrad years, I did research under the supervision of Prof. Abbas Salemi at the Mahani Mathematical Research Center.
Top undergraduate science major researcher in the Province of Kerman in 2020
Top undergraduate researcher in the College of Math. and CS & the CS Dept., University of Kerman in 2020
Distinguished undergraduate researcher in the College of Math. and CS, University of Kerman in 2019
CIS 322 (Introduction to Software Engineering) lab instructor and teaching assistant.
Coordinated and held the workshop online, gave lectures and wrote programming examples on data representation and feature engineering, introduction to PyTorch, classical learning algorithms, support vector machines, neural networks, convolutional networks and recurrent networks.
TA'd advanced programming, internet engineering and linear algebra.
https://arxiv.org/abs/2104.05704
https://raw.githubusercontent.com/SHI-Labs/Compact-Transformers/main/images/model_sym.png
With the rise of Transformers as the standard for language processing, and their advancements in computer vi-sion, along with their unprecedented size and amounts of training data, many have come to believe that they are not suitable for small sets of data. This trend leads to great concerns, including but not limited to: limited availability of data in certain scientific domains and the exclusion ofthose with limited resource from research in the field. In this paper, we dispel the myth that transformers are “data-hungry” and therefore can only be applied to large sets of data. We show for the first time that with the right size and tokenization, transformers can perform head-to-head with state-of-the-art CNNs on small datasets. Our model eliminates the requirement for class token and positional embed-dings through a novel sequence pooling strategy and the use of convolutions. We show that compared to CNNs, our compact transformers have fewer parameters and MACs,while obtaining similar accuracies. Our method is flexible in terms of model size, and can have as little as 0.28M parameters and achieve reasonable results. It can reach an ac-curacy of 94.72% when training from scratch on CIFAR-10,which is comparable with modern CNN based approaches,and a significant improvement over previous Transformer based models. Our simple and compact design democratizes transformers by making them accessible to those equipped with basic computing resources and/or dealing with important small datasets.
arXiv GitHub