Tam Minh Nguyen

Email: Minh.Tam.Nguyen@rice.edu, CV

Education

Brief Biography

I am currently a Ph.D. student in Electrical and Computer Engineering at Rice University, where I am advised by Dr. Richard G. Baraniuk. Prior to my time at Rice University, I participated in the FPT AI Residency program, under the mentorship of Dr. Tan Nguyen and Dr. Nhat Ho. I hold a B.S. in International Economics from the University of Economics and Business, Vietnam National University, Ha Noi, which I completed in January 2018.

Research Interests

My research aims at understanding and advancing self-attention mechanisms in transformers. Adopting a mathematical approach, I prove that self-attention has connections with various established and well-developed techniques, from probabilistic clustering and non-parametric regression to primal-dual optimization and image denoising. These connections reveal the inherent properties and limitations of self-attention while providing principled frameworks to further develop transformers for real-world applications.

Conference Publications

• Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals. Conference on Neural Information Processing Systems (NeurIPS), 2023.
  Tam Nguyen, Tan M. Nguyen, Richard G. Baraniuk.

• A Primal-Dual Framework for Transformers and Neural Networks. International Conference on Learning Representations (ICLR) (notable-top-25%), 2023.
  Tan M. Nguyen*, Tam Nguyen*, Nhat Ho, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher.

• FourierFormer: Transformer Meets Generalized Fourier Integral Theorem. Conference on Neural Information Processing Systems (NeurIPS), 2022.
  Tan M. Nguyen*, Minh Pham*, Tam Nguyen, Khai Nguyen, Stanley J. Osher, Nhat Ho.

• Improving Transformer with an Admixture of Attention Heads. Conference on Neural Information Processing Systems (NeurIPS), 2022.
  Tan M. Nguyen*, Tam Nguyen*, Hai Do, Khai Nguyen, Vishwanath Saragadam, Minh Pham, Khuong Nguyen, Nhat Ho**, Stanley J. Osher**.

• Probabilistic Framework for Pruning Transformers via a Finite Admixture of Keys. International Conference on Acoustics, Speech, and Signal Processing (ICASSP, notable-top-3%), 2023.
  Tan M. Nguyen*, Tam Nguyen*, Long Bui*, Hai Do, Dung Le, Hung Tran-The, Khuong Nguyen, Richard G. Baraniuk, Nhat Ho**, Stanley J. Osher**.

• Improving Transformers with Probabilistic Attention Keys. International Conference on Machine Learning (ICML), 2022.
  Tam Nguyen*, Tan M. Nguyen*, Dung Le, Khuong Nguyen, Anh Tran, Richard G. Baraniuk, Nhat Ho**, Stanley J. Osher**.