This study investigates whether phonological features can be applied in text-to-speech systems to generate native and non-native speech in English and Mandarin. We present a mapping of ARPABET/pinyin to SAMPA/SAMPA-SC and then to phonological features. We tested whether this mapping could lead to the successful generation of native, non-native, and code-switched speech in the two languages. We ran two experiments, one with a small dataset and one with a larger dataset. The results proved that phonological features could be used as a feasible input system, although further investigation is needed to improve model performance. The accented output generated by the TTS models also helps with understanding human second language acquisition processes.
This study investigates whether phonological features can be applied in text-to-speech systems to generate native and non-native speech. We present a mapping between ARPABET/pinyin->SAMPA/SAMPA-SC->phonological features in this paper, and tested whether native, non-native, and code-switched speech could be successfully generated using this mapping. We ran two experiments, one with a small dataset and one with a larger dataset. The results proved that phonological features can be a feasible input system, although it needs further investigation to improve model performance. The accented output generated by the TTS models also helps with understanding human second language acquisition processes.
@misc{zhang2021applying,presentation={https://congzhang365.github.io/feature_tts/},title={{Applying Phonological Features in Multilingual Text-To-Speech}},author={Zhang, Cong and Zeng, Huinan and Liu, Huang and Zheng, Jiewen},journal={journal},year={2021}}
Related talks:
speech tech
Featurally Underspecified Lexicon (FUL) Model: Evidence from multilingual Text-to-Speech
Cong Zhang, Huinan Zeng, Huang Liu, and Jiewen Zheng
The 18th Conference on Laboratory Phonology Online, 23-25 jun 2022
This study investigates whether the phonological features derived from the Featurally Underspecified Lexicon model can be applied in text-to-speech systems to generate native and non-native speech in English and Mandarin. The results supported that phonological features could be used as a feasible input system for languages in or not in the train data. The results lend support to FUL by presenting successfully synthesised output, and by having the output carrying a source-language accent when synthesising a language not in the training data. The TTS process stimulated second language acquisition process and thus also confirm FUL’s ability to account for acquisition.
@conference{zhang2022featurally-c,author={Zhang, Cong and Zeng, Huinan and Liu, Huang and Zheng, Jiewen},title={Featurally Underspecified Lexicon (FUL) Model: Evidence from multilingual Text-to-Speech},booktitle={The 18th Conference on Laboratory Phonology},year={2022},month={23-25 Jun},location={Online}}
Phone & word alignments for 1300 hours of open-source Mandarin speech datasets. Automatically aligned with our own Charsiu Forced Aligner.
@misc{zhang2021TTS,author={Zhang, Cong and Zeng, Huinan},title={{Phonological feature mapping for FeatureTTS}},year={2021},category={dataset},doi={10.5281/ZENODO.5553685}}