Alert button
Picture for Yanni Chen

Yanni Chen

Alert button

Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Add code
Bookmark button
Alert button
Jan 29, 2023
Xian Shi, Yanni Chen, Shiliang Zhang, Zhijie Yan

Figure 1 for Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model
Figure 2 for Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model
Figure 3 for Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model
Figure 4 for Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability. This paper proposes to perform timestamp prediction~(TP) while recognizing by utilizing continuous integrate-and-fire~(CIF) mechanism in non-autoregressive ASR model - Paraformer. Foucing on the fire place bias issue of CIF, we conduct post-processing strategies including fire-delay and silence insertion. Besides, we propose to use scaled-CIF to smooth the weights of CIF output, which is proved beneficial for both ASR and TP task. Accumulated averaging shift~(AAS) and diarization error rate~(DER) are adopted to measure the quality of timestamps and we compare these metrics of proposed system and conventional hybrid force-alignment system. The experiment results over manually-marked timestamps testset show that the proposed optimization methods significantly improve the accuracy of CIF timestamps, reducing 66.7\% and 82.1\% of AAS and DER respectively. Comparing to Kaldi force-alignment trained with the same data, optimized CIF timestamps achieved 12.3\% relative AAS reduction.

Viaarxiv icon
Github IconAccess Paper or Ask Questions
Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon