Tianyu Pang

Tianyu Pang

Master student @ Nanjing Univeristy

Hi, I’m Tianyu. I am a master student at Department of Mathematics, Nanjing University. I have the great honor of being advised by Prof. Yaoqing Yang at Dartmouth College.

I am passionate about mechanistic interpretability and model diagnostics . My current research is focused on

  • understanding the mechanisms, dynamics and generalization of ML models from the perspective of random matrix theory, high-dimensional statistics and loss landscape;
  • leveraging model/data diagnostics and interpretability insights to improve the transparency, robustness and efficiency of (scientific) machine learning.

📖 Educations

  • 2022.09 - present, M.S. in Mathematics, Department of Mathematics, Nanjing University.
  • 2018.09 - 2022.06, B.S. in Statistics, Kuang Yaming Honors School, Nanjing University.

🔥 News

  • 2024.10:  😁 Completing a wonderful three-month visiting at Dartmouth College.
  • 2024.09:  🎉🎉 Excited to share that our work “Model Balancing Helps Low-data Training and Fine-tuning” is accepted by EMNLP 2024 as Oral Presentation!
  • 2023.09:  🎉🎉 Excited to share that our work “Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training” is accepted by NeurIPS 2023 as Spotlight!

📝 Publications

（# denotes equal contribution）

EMNLP 2024
sym

Model Balancing Helps Low-data Training and Fine-tuning

Zihang Liu#, Yuanzhe Hu#, Tianyu Pang, Yefan Zhou, Pu Ren, Yaoqing Yang

Code|Paper|Video

EMNLP 2024 Oral

NeurIPS 2023
sym

Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training

Yefan Zhou#, Tianyu Pang#, Keqin Liu, Charles H. Martin, Michael Mahoney, Yaoqing Yang

Code|Paper|Video

NeurIPS 2023 Spotlight

Preprint
sym

Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise

Vignesh Kothapalli, Tianyu Pang, Shenyang Deng, Zongmin Liu, Yaoqing Yang

Code|Paper

Preprint