Get Lyrics and Timing from Youtube Subtitles

[mARTin-B78]

Hey guys,
would it be possible to get the Lyrics and timing straight from the YouTube subtitles?
Guess that's easier and more reliable than speech recognition.

[mARTin-B78]

Ok, I started working on this.
Here is a little script that Downloads the .vtt track in German, and if not present in English, and if not present the Transcript from every Video in a Youtube Playlist.

It could be useful as a source for AI to correct the Karaoke File generated by UltraSinger with Speech Recognition.
Thinking of letting the AI compare each single syllable of the Speech Recognizer Lyrics to the syllable of the Captions from YouTube.
If the syllable matches, fine, go to the next. If not, check it and replace it with the syllable from the Caption File.

In that case, the timing and tonality of the syllable can be used from the Speech recognition but the wording is used from the Caption (or any other source if you come up with a good one)

What do you guys think?

download captions from YouTube Playlist.py.txt

download captions from YouTube Playlist.py
`import os
from pytube import Playlist
import yt_dlp

Get playlist URL from user

playlist_url = input("Enter the YouTube playlist URL: ")

Create a Playlist object

playlist = Playlist(playlist_url)

Create a directory to save subtitles

if not os.path.exists('subtitles'):
os.makedirs('subtitles')

Function to download subtitles

def download_subtitles(video_url):
ydl_opts = {
'skip_download': True,
'writesubtitles': True,
'writeautomaticsub': True, # Use auto-generated subtitles if no regular ones
'subtitleslangs': ['de', 'en'], # Prefer German, fallback to English
'outtmpl': 'subtitles/%(title)s.%(ext)s',
'ignoreerrors': True, # Continue on download errors
'nocheckcertificate': True, # Ignore SSL certificate errors
}
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
ydl.download([video_url])

Iterate over all videos in the playlist

for video in playlist.video_urls:
try:
download_subtitles(video)
except Exception as e:
print(f"Error downloading subtitles for {video}: {e}")

print("Subtitle extraction complete.")
`

[mARTin-B78]

The following steps and technologies could be considered to automate such corrections and adaptations:
Natural Language Processing (NLP): libraries such as NLTK, spaCy or Stanford CoreNLP for text analysis and correction.
Machine Learning (ML): Train models to recognise and predict patterns in syllable assignment and line break positioning.
Audio analysis tools: Tools such as Librosa or PyAudioAnalysis could be used for accurate timestamp determination (:-lines).