Messages
In our previous work, we proposed the CampNet for end-to-end text-based speech editing, which can delete, replace and insert the speech at the word level by editing the transcription.
In this work, we propose Emo-CampNet, which can provide emotional options for generated speech on the basis of text-based speech editing, so as to make generated speech more expressive. This page shows the samples in the paper "Emotion Selectable End-to-End Text-based Speech Editing".
Experiments were based on VCTK and ESD . All test samples have not appeared in the training set and validation set.
Arxiv: https://arxiv.org/abs/2212.10191
Content
1: Insertion and replacement of emotion selectable text-based speech editing based on Emo-CampNet
speaker | Original | Emotion | Replace some words with new words | Insert some words | p362 (female) seen speaker |
The rainbow is a division of white light into many beautiful colors. |
Neutral |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
---|---|---|---|---|
Happy |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Sad |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Angry |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Surprise |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
p225 (female) unseen speaker |
The rainbow is a division of white light into many beautiful colors. |
Neutral |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
Happy |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Sad |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Angry |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
||
Surprise |
The rainbow is a division of black light ( |
The rainbow is colorful light and a division of white light into many beautiful colors. |
p226 (male) unseen speaker |
Others have tried to explain the phenomenon physically. |
Neutral |
Others have tried to explain the strange thing ( |
Others have tried to explain the very very weird phenomenon physically. |
Happy |
Others have tried to explain the strange thing ( |
Others have tried to explain the very very weird phenomenon physically. |
||
Sad |
Others have tried to explain the strange thing ( |
Others have tried to explain the very very weird phenomenon physically. |
||
Angry |
Others have tried to explain the strange thing ( |
Others have tried to explain the very very weird phenomenon physically. |
||
Surprise |
Others have tried to explain the strange thing ( |
Others have tried to explain the very very weird phenomenon physically. |
2: Comparison with baseline systems (Insert operation)
a: Seen speaker
Emotion | Original | p-w/o-eda | p-w/o-nda | p-w/o-dis | Emo-CampNet |
---|---|---|---|---|---|
Neutral |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Happy |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Sad |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Angry |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Surprise |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
b: Unseen speaker
Emotion | Original | p-w/o-eda | p-w/o-nda | p-w/o-dis | Emo-CampNet |
---|---|---|---|---|---|
Neutral |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Happy |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Sad |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Angry |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Surprise |
Others have tried to explain the phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
Others have tried to explain the very very weird phenomenon physically. |
3: Examples of F0 disturbances for training dataset
Original | shift semitones -5 | shift semitones -4 | shift semitones -3 | shift semitones -2 | shift semitones -1 | shift semitones 1 | shift semitones 2 | shift semitones 3 | shift semitones 4 | shift semitones 5 |
---|---|---|---|---|---|---|---|---|---|---|
What are you waiting for? man. |
||||||||||
Another one, ten thousand dollars. |