Publications

Ph.D. Dissertation

• Improving the Accuracy and Inference Efficiency for Low-resource Automatic Speech Recognition
  

2026

• Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach
  Junkun Chen, Jian Xue, Ming Tang, Abdel Heba, Hoda Gholami, Ruchao Fan, and Jinyu Li
  in arXiv preprint arXiv:2607.13158
  

• Compress the Cache, Not the Speech Embedding: KV Compression for Efficient Speech LLMs
  Ke-Han Lu, Keqi Deng, Ruchao Fan, Rui Zhao, and Jinyu Li
  in arXiv preprint arXiv:2607.06827
  

• Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving
  Ruchao Fan, Yiming Wang, Rui Zhao, Liliang Ren, Keqi Deng, Xiaoyang Chen, Ali Zare, Bo Ren, Yuxuan Hu, Junkun Chen, Yan Huang, Yelong Shen, and Jinyu Li
  in arXiv preprint arXiv:2607.01733
  

• Preserving Speech-to-Text LLM Capabilities in Speech-to-Speech Generation
  Yuxuan Hu, Heng Lu, Ruchao Fan, Yao Qian, Xiaofei Wang, Jian Xue, Heming Wang, Shuohang Wang, Young Jin Kim, Yelong Shen, and Jinyu Li
  in arXiv preprint arXiv:2606.30944
  

• LLM can Read Spectrogram: Encoder-free Speech-Language Modeling
  Ruchao Fan, Yiming Wang, Yuxuan Hu, Bo Ren, Yufei Xia, Xiaofei Wang, Yao Qian, Shujie Liu, and Jinyu Li
  in arXiv preprint arXiv:2606.10231
  

• UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training
  Keqi Deng, Shaoshi Ling, Ruchao Fan, and Jinyu Li
  in arXiv preprint arXiv:2605.27740
  

• Speech LLMs are Contextual Reasoning Transcribers
  Keqi Deng, Ruchao Fan, Bo Ren, Yiming Wang, and Jinyu Li
  in arXiv preprint arXiv:2604.00610
  

• Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio
  Mohan Shi, Xiong Xiao, Ruchao Fan, Shaoshi Ling, and Jinyu Li
  in ICASSP 2026, IEEE.
  

• RLBR: Reinforcement Learning with Biasing Rewards for Contextual Speech Large Language Models
  Bo Ren, Ruchao Fan, Yelong Shen, Weizhu Chen, and Jinyu Li
  in ICASSP 2026, IEEE.
  

2025

• OleSpeech-IV: A Large-Scale Multispeaker and Multilingual Conversational Speech Dataset with Diverse Topics
  Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Xavier Menendez-Pidal, Ruchao Fan, Chenfeng Miao, Chanwoo Kim, Bhiksha Raj, and Rita Singh
  in arXiv preprint arXiv:2509.04702
  

• AlignFormer: Modality Matching Can Achieve Better Zero-shot Instruction-Following Speech-LLM
  Ruchao Fan, Bo Ren, Yuxuan Hu, Rui Zhao, Shujie Liu, and Jinyu Li
  in IEEE Journal of Selected Topics in Signal Processing (JSTSP), 2025.
  

• Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model
  Haibin Wu, Yuxuan Hu, Ruchao Fan, Xiaofei Wang, Kenichi Kumatani, Bo Ren, Jianwei Yu, Heng Lu, Lijuan Wang, Yao Qian, and Jinyu Li
  in ASRU 2025, IEEE.
  

• SLM-S2ST: A Multimodal Language Model for Direct Speech-to-Speech Translation
  Yuxuan Hu, Haibin Wu, Ruchao Fan, Xiaofei Wang, Heng Lu, Yao Qian, and Jinyu Li
  in ASRU 2025, IEEE.
  

• Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
  Microsoft et al.
  in arXiv preprint arXiv:2503.01743
  

2024

• CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
  Rui Zhao, Jinyu Li, Ruchao Fan, and Matt Post
  in SLT 2024.
  

• Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models
  Ruchao Fan, Natarajan Balaji Shankar, and Abeer Alwan
  in Interspeech 2024.
  

• SOA: Reducing Domain Mismatch in SSL Pipeline by Speech Only Adaptation for Low Resource ASR
  Natarajan Balaji Shankar, Ruchao Fan, and Abeer Alwan
  in ICASSP 2024 SASB Workshop.
  

• UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models
  Ruchao Fan, Natarajan Balaji Shankar, and Abeer Alwan
  in IEEE Singal Processing Letters (SPL).
  

2023

• A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition
  Ruchao Fan, Wei Chu, Peng Chang, and Abeer Alwan
  in IEEE Transactions on Audio, Speech, and Language Processing (TASLP).
  

• CTCBERT: Advancing Hidden-unit BERT with CTC Objectives [Poster]
  Ruchao Fan, Yiming Wang, Yashesh Gaur, and Jinyu Li
  in ICASSP 2023, IEEE.
  

2022

• Acoustic-aware Non-autoregressive Spell Correction with Mask Sample Decoding
  Ruchao Fan, Guoli Ye, Yashesh Gaur, Jinyu Li
  in arXiv preprint arXiv:2210.08665
  

• Towards Better Domain Adaptation for Self-Supervised Models: A Case Study of Child ASR [Poster]
  Ruchao Fan, Yunzheng Zhu, Jinhan Wang, and Abeer Alwan
  in Journal of Selected Topics in Signal Processing (JSTSP), IEEE.
  

• DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASR [Poster]
  Ruchao Fan, and Abeer Alwan
  in Interspeech 2022.
  

• Towards Better Meta-Initialization with Task Augmentation for Kindergarten-aged Speech Recognition
  Yunzheng Zhu, Ruchao Fan, and Abeer Alwan
  in ICASSP 2022, IEEE.
  

• LPC Augment: An LPC-Based ASR Data Augmentation Algorithm for Low and Zero-Resource Children's Dialects
  Alexander Johnson, Ruchao Fan, Robin Morris, and Abeer Alwan
  in ICASSP 2022, IEEE.
  

2021

• Fundamental frequency feature warping for frequency normalization and data augmentation in child automatic speech recognition
  Gary Yeung, Ruchao Fan, and Abeer Alwan
  in Speech Communication 2021.
  

• An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition [Poster]
  Ruchao Fan, Wei Chu, Peng Chang, Jing Xiao and Abeer Alwan
  in Interspeech 2021, pp. 3715-3719.
  

• Low Resource German ASR with Untranscribed Data Spoken by Non-native Children – INTERSPEECH 2021 Shared Task SPAPL System
  Jinhan Wang, Yunzheng Zhu, Ruchao Fan, Wei Chu and Abeer Alwan
  in Interspeech 2021, pp. 1279-1283.
  

• CASS-NAT: CTC Alignment-based Single Step Non-autoregressive Transformer for Speech Recognition [Slides], [Poster]
  Ruchao Fan, Wei Chu, Peng Chang, and Jing Xiao
  in ICASSP 2021, IEEE, pp. 5889–5893.
  

• Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-training and Its Application to Children's ASR [Slides], [Poster]
  Ruchao Fan, Amber Afshan, and Abeer Alwan
  in ICASSP 2021, IEEE, pp. 7023–7027.
  

• Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognition
  Gary Yeung, Ruchao Fan, and Abeer Alwan
  in ICASSP 2021, IEEE, pp. 6993-6997.
  

2020

• Exploring the Use of an Unsupervised Autoregressive Model as a Shared Encoder for Text-Dependent Speaker Verification
  Vijay Ravi, Ruchao Fan, Amber Afshan, Huanhua Lu, and Abeer Alwan
  in Proc. Interspeech 2020, 2020.
  

2019

• Improving Generalization of Transformer for Speech Recognition with Parallel Schedule Sampling and Relative Positional Embedding
  Pan Zhou, Ruchao Fan, Wei Chen, and Jia Jia
  in arXiv preprint arXiv:1911.00203.
  

• An Online Attention-based Model for Speech Recognition
  Ruchao Fan, Pan Zhou, Wei Chen, Jia Jia, and Gang Liu
  in Proc. Interspeech 2019, 2019, pp. 4390-4394.
  

2018

• CNN-based Audio Front End Processing on Speech Recognition [Code]
  Ruchao Fan and Gang Liu
  in 2018 International Conference on Audio, Language and Image Processing (ICALIP). IEEE, 2018, pp. 349–354.
  

2017

• Frequency Warping Based on Two Factor Weighted Fusion
  Gang Liu, Haobin Chen, and Ruchao Fan
  in 2017 4th International Conference on Systems and Informatics (ICSAI). IEEE, 2017, pp. 1082–1086.
  

• Multi-parameter Frequency Warping Based on LDA
  Gang Liu, Boce Chu, and Ruchao Fan
  in 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). IEEE, 2017, pp. 2710–2714.
  

• An Improved Locality Preserving Projection for Feature Transform in Speech Recognition.
  Gang Liu, Boce Chu, and Ruchao Fan
  in 2017 International Conference on Machine Learning and Cybernetics (ICMLC). IEEE, 2017, vol. 1, pp. 77–82.