Shouts, non recognized words

[hanz-s]

Hello,

is there a way to increase recognition of shouts, or voice that is'nt necessarily a word, to not get filtered when they are not recognized as a word. so one can get the timing of those shouts in the ultrastar text file? would i need a language model trained for this, oder can i add an additional model somehow which increases supports this anlong with the normal language models?

[rakuri255]

Its currently a difficult task, since the llm models only give timestamps for words. I think when you find an model on huggingface which is trained for singing, that would be the best chance.

[rakuri255]

I have been thinking about it again. Since we have the voice separated, we could analyze the volume. Then we could say that if there is no word/timestamp, then there is some kind of "lalala".
We need this approach anyway to eliminate the silent pause in the word itself.

[hanz-s]

nice! a placeholder would be worth gold. maybe one could set a param with a min volume / length of human voice to be recognized for a placeholder position.
thank your for your feedback to my question!

[rakuri255]

We have now some spectrograms to play with.