I am a 2nd year PhD student at the University of Edinburgh (start from Sept. 2023), a member of EdinburghNLP, supervised by Pasquale Minervini and Mirella Lapata. My research interests lie in foundation model pre-training and scalable mechanism interpretability. I am currently working to “open the black box” to enable more efficient LLMs pre-training and inference.

📌 I am now looking for internship opportunities

Recent Works

Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini
arXiv 2024

Analysing the Residual Stream of Language Models Under Knowledge Conflicts
Yu Zhao, Xiaotang Du, Giwon Hong, Aryo Pradipta Gema, Alessio Devoto, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini
MINT @ NeurIPS 2024

A Simple and Effective Norm-Based Strategy for KV Cache Compression
Alessio Devoto*, Yu Zhao*, Simone Scardapane, Pasquale Minervini
EMNLP 2024, Oral

Analysing The Impact of Sequence Composition on Language Model Pre-Training
Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr MiƂoƛ, Yuxiang Wu, Pasquale Minervini
ACL 2024, Oral


Check out all my publications in Google Scholar or Semantic Scholar.