RVTS: retrieval-based text-to-speech

Jin Yang; Xuesong Su; Wenke Xu

doi:10.1117/12.3032936

5 July 2024 RVTS: retrieval-based text-to-speech

Jin Yang, Xuesong Su, Wenke Xu

Proceedings Volume 13184, Third International Conference on Electronic Information Engineering and Data Processing (EIEDP 2024); 131841L (2024) https://doi.org/10.1117/12.3032936
Event: 3rd International Conference on Electronic Information Engineering and Data Processing (EIEDP 2024), 2024, Kuala Lumpur, Malaysia

Abstract

The single-stage text-to-speech model has made significant advancements in recent years. A comparison between two prominent models, VITS and VITS2, reveals that training with the same individual's voice dataset led to the voiceprint model generated by VITS closely aligning with the original voiceprint; however, its segmentation and fluency were subpar. On the other hand, VITS2 showed the opposite characteristics. Through the implementation of Retrieval-based Speech (RVTS), involving the extraction of intonation and fragmented sentences from the VITS2 model followed by code utilization, we were able to achieve speech synthesis. Our experimental findings demonstrate that our approach surpasses both VITS and VITS2, leveraging the strengths of both models.

Citation Download Citation

Jin Yang, Xuesong Su, and Wenke Xu "RVTS: retrieval-based text-to-speech", Proc. SPIE 13184, Third International Conference on Electronic Information Engineering and Data Processing (EIEDP 2024), 131841L (5 July 2024); https://doi.org/10.1117/12.3032936

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available