I am a 2nd year PhD student at the University of Edinburgh (start from Sept. 2023), a member of EdinburghNLP, supervised by Pasquale Minervini and Mirella Lapata. My research interests lie in foundation model pre-training and scalable mechanism interpretability. I am currently working to âopen the black boxâ to enable more efficient LLMs pre-training and inference.
đ I am now looking for internship opportunities
Recent Works
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini
arXiv 2024
Analysing the Residual Stream of Language Models Under Knowledge Conflicts
Yu Zhao, Xiaotang Du, Giwon Hong, Aryo Pradipta Gema, Alessio Devoto, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini
MINT @ NeurIPS 2024
A Simple and Effective Norm-Based Strategy for KV Cache Compression
Alessio Devoto*, Yu Zhao*, Simone Scardapane, Pasquale Minervini
EMNLP 2024, Oral
Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr MiĆoĆ, Yuxiang Wu, Pasquale Minervini
ACL 2024, Oral
Check out all my publications in Google Scholar or Semantic Scholar.