Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show transcript for lectures with subtitles #1369

Open
carlobortolan opened this issue Aug 30, 2024 · 3 comments · May be fixed by #1403
Open

Show transcript for lectures with subtitles #1369

carlobortolan opened this issue Aug 30, 2024 · 3 comments · May be fixed by #1403

Comments

@carlobortolan
Copy link
Member

Is your feature request related to a problem? Please describe.
-

Describe the solution you'd like

  • Add the option to see a searchable transcript next to the lecture video if subtitles are available.
  • Transcript should be synchronized with the current progress of the video (and, for example, also show the current video timestamp - see example screenshot from YouTube below).

Describe alternatives you've considered
Download option for transcripts as a text file.

Additional context

  1. I'd work on this issue as part of the Open Source Lab
2. YouTube transcript function

image

@joschahenningsen
Copy link
Member

probably duplicate of #1364 :)
Functionality is implemented (see https://github.com/TUM-Dev/TUM-Live-Voice-Service), we never got enough GPU capacity to enable the feature. We're actively investigating buying hardware privately.

This lecture already has subtitles:
grafik
https://tum.live/w/cit-gr/26188

@carlobortolan
Copy link
Member Author

carlobortolan commented Aug 30, 2024

probably duplicate of #1364 :) Functionality is implemented (see https://github.com/TUM-Dev/TUM-Live-Voice-Service)

Ah sorry for the confusion, besides searching for certain keywords, I was thinking more of a feature that lets you read and scroll through the entire transcript of a lecture.
And while it probably won't be a major feature, I'd guess that it would still be an interesting functionality (especially for those who might be in a hurry and just want to quickly recap / scroll through the content of a certain lecture).

we never got enough GPU capacity to enable the feature. We're actively investigating buying hardware privately.

iirc, in a recent tumlive meeting, there was the idea of using some unused resources (e.g., the Worker VMs at night or during the semester breaks) to transcribe past VODs - even if the VMs don't have powerful GPUs, it still should be enough to transcribe VODs. So it might even be that next semester there will be more lecture VODs with subtitles 👀

@CommanderStorm
Copy link
Member

we never got enough GPU capacity

This actually can't happen that we don't communicate such stuff.
I have put a 3090 in "our the usual place". That should be enough and a big upgrade to the 1660 Super.

With algorithmic improvements like https://github.com/SYSTRAN/faster-whisper I don't see a reason why we could not have semi-realrime transcription if we are smart about this.

@joschahenningsen could you please install the GPU?

@carlobortolan carlobortolan linked a pull request Nov 12, 2024 that will close this issue
@carlobortolan carlobortolan linked a pull request Nov 12, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants