Karaoke Films

Two projects cut from found karaoke videos

Derrick Schultz · 2026

The pipeline

Karaoke videos

Found on YouTube

→

Isolated vocals

Singer without the backing track

→

CREPE pitch track

One estimate every 10 ms

→

Note segments

Split on a 50-cent change or silence

→

Pitch database

24,239 segments from 34 videos

Karaoke videos are full of people singing, but nothing in the file says which note is being sung at which moment. To cut them by pitch, I first needed an index: every sung note, in every video, with a start and end time. This is how that index gets built. The inputs are karaoke videos downloaded from YouTube, mostly nineties love songs. The vocals are isolated first, so the pitch tracker hears the singer and not the backing track. CREPE, a neural pitch tracker, then runs on the audio in ten-millisecond steps, using its full-size model with smoothing. The script starts a new segment when the pitch moves more than fifty cents from the segment’s median, or when the audio goes quiet, and drops anything shorter than a tenth of a second. Every segment goes into one database with its note, its confidence, its loudness and its frame range. The January database holds 24,239 segments from 34 videos. From there I can pull out every instance of one note, which is Middle C, or rebuild a melody from notes sung by many different singers, which is Love Song.

Output · 2026

Middle C

Clips the database labels C4, longest first. A 12-minute film of karaoke singers on one note.

C4 is the third most common note in the database: 1,874 of 24,239 segments.

Middle C · process

Pitch database

24,239 segments from 34 videos

→

Group by note

Longest clips first

→

One folder per note

Cut with ffmpeg at one size and frame rate

→

C4 film

12 minutes

Output · Always

Matching a melody

764 guide segments: 534 notes and 230 rests
Every note matched exactly; none transposed
4,744 source segments used

This is Always, from the Love Song EP. Love Song runs in the other direction from Middle C. I start with a MIDI file of the song, pick the melody channel, and convert it into a guide: a list of notes and rests with exact times. For each note in the guide, the matcher looks for a sung segment with the same MIDI note. It scores the candidates on how well the duration fits, how confident CREPE was, and how steady the segment is, weighted thirty, forty and thirty percent. If a note is missing, it can transpose a nearby one by up to twelve semitones. The numbers on this slide come from Rick Roll, the one match plan still on this drive. Its guide had 764 segments, 534 notes and 230 rests. Every note found an exact match, so nothing was transposed, and the finished edit uses 4,744 different source segments.

Output · Love Song Sing-Along (BBVDAY2026)

The Kiss Me · Duet · Ride It. Six songs, 3 to 6 minutes each.

Output · Fantasy/Monstrosity

2026, 6 minutes. Found karaoke and 90s R&B video from YouTube.

Love Song Sing-Along · process

Pitch database

24,239 segments from 34 videos

→

MIDI melody guide

Notes and rests with exact times

→

Match each note

Same note first, else transpose

→

Rendered video

Plus an edit list for the editor

What I’d DO next

• Web Player?
• AI MIDI generator? MIDI 2 Viz?
• ????