Ruchao Fan

Ruchao Fan, Ph.D. Applied Scientist at Microsoft Email: fanruchao at g dot ucla dot edu Department of Electrical and Computer Engineering University of California, Los Angeles Ph.D. Advisor: Abeer Alwan Lab: Speech Processing and Auditory Perception Laboratory

Bio

Ruchao Fan is currently a Senior Applied Scientist at Microsoft Azure AI Speech Team. He obtained his Ph.D. degree from the Department of Electrical and Computer Engineering, UCLA. His research interests are multi-modal representation learning, speech and language modeling, and so on.

During the Ph.D., Rucho Fan worked with Prof. Abeer Alwan on problems of speech processing. His work focused on children's speech recognition from a low-resource perspective and end-to-end speech recognition. Prior to that, He received his Master's and Bachelor's degree from Beijing University of Posts and Telecommunications (BUPT) in 2019 and 2016, respectively.

Ruchao has interned at Sogou Inc. (2018-2019), PAII Inc. (2020), and Microsoft Corporation (2021, 2022), Amazon Web Service (2023), conducting research on speech recognition and language modeling. He was awarded UCLA-Amazon Fellowship during 2021-2022, and was honored as Amazon Fellow.

News

• [Jun 2024] Check out our paper on bechmarking children's ASR with speech foundation models. The paper was accpeted to Interspeech 2024.

  • Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models

  • Code, models are released in Huggingface!

• [May 2024] I joined Microsoft Azure Speech Team as an Applied Scientist.

• [Mar 2024] I successfully defended my Ph.D. thesis and graduated from UCLA.

  • Improving the Accuracy and Inference Efficiency for Low-resource Automatic Speech Recognition

• [Feb 2024] The Last work for CASS-NAT seris was accepted to IEEE Signal Processing Letters. Check out the paper here.

  • UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models

• [Jun 2023] Summer intern at Amazon Web Service, working on long-context large language modeling.

• [Mar 2023] My second Journal Paper was accepted to IEEE Transactions on Audio, Speech,and Language Processing (TASLP), and is available for early access:

  • A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition

  • Codes for our CASS-NAT work are available at CASS-NAT Code

• [Feb 2023] My summer intern work CTCBERT was accepted to ICASSP 2023!

  • CTCBERT: Advancing Hidden-unit BERT with CTC Objectives

  • ICASSP Poster Presentation

• [Oct 2022] My first Journal paper was accpeted to JSTSP (IEEE Journal of Selected Topics in Signal Processing)!

  • Towards Better Domain Adaptation for Self-Supervised Models: A Case Study of Child ASR

  • ICASSP Presentation of Journal Papers

  • Code for the DRAFT!

• [Jun 2022] A wonderful summer intern @Microsoft, working with Dr. Yiming Wang, Dr. Yashesh Gaur, Dr. Guoli Ye, and Dr. Jinyu Li. This is also my second intern at Microsoft.

• [Jun 2022] One paper has been accepted by Interspeech 2022!

  • DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASR

• [Feb 2022] I was awarded with UCLA-Amazon Fellowship!

• [Jun 2022] Two co-authored papers have been accepted by ICASSP 2022!

  • Towards Better Meta-Initialization with Task Augmentation for Kindergarten-aged Speech Recognition

  • LPC Augment: An LPC-Based ASR Data Augmentation Algorithm for Low and Zero-Resource Children's Dialects

• [Dec 2021] I passed my Oral Qualifying Exam and became a Ph.D. Candiate.

• [Dec 2021] Our journal paper Fundamental frequency feature warping for frequency normalization and data augmentation in child automatic speech recognition has been published on Speech Communication.

• [Jun 2021] I started my summer intern @Microsoft, working with Dr. Guoli Ye and Dr. Jinyu Li.

• [Jun 2021] Two papers have been accepted by Interspeech 2021!

  • An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition

  • Low Resource German ASR with Untranscribed Data Spoken by Non-native Children – INTERSPEECH 2021 Shared Task SPAPL System

• [Jan 2021] Three papers have been accepted by ICASSP 2021!

  • CASS-NAT: CTC Alignment-based Single Step Non-autoregressive Transformer for Speech Recognition

  • Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-training and Its Application to Children's ASR

  • Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognition

• [Sep 2020] I had a fruitful summer @PAII, working with Dr. Wei Chu and Dr. Peng Chang.

• [Jul 2020] Our challenge paper Exploring the Use of an Unsupervised Autoregressive Model as a Shared Encoder for Text-Dependent Speaker Verification has been accepted by Interspeech 2020.

• [Sep 2019] I started my Ph.D. at UCLA.

• [Jun 2019] I got my Master's degree in Beijing University of Posts and Telecommunications.

• [Jun 2019] Our Paper An Online Attention-based Model for Speech Recognition has been accepted by Interspeech 2019.

Peer Review

• Top speech conference and journal

  • Interspeech, ICASSP

  • IEEE Signal Processing Letter

  • IEEE Journal of Selected Topics in Signal Processing, JSTSP

  • IEEE Transactionon Audio, Speech, and Language Processing, TASLP

• Others

  • IEEE Transactions on Pattern Analysis and Machine Intelligence, TPAMI

  • NeurIPS 2024

  • ICLR 2023, 2024

  • IEEE International Conference on Multimedia and Expo, ICME (2021)

  • Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC (2020, 2021)

Education

• Ph.D. University of California, Los Angeles, Los Angeles, CA, U.S.A., Sept. 2019 - Mar. 2024

• M.S. & B.E., Beijing University of Posts and Telecommunications, Beijing, China

  • M.S., Sept. 2016 - Jun. 2019

  • B.E., Sept. 2012 - Jun. 2016