[🟣 SDK] Bring your own API key to generate captions

well i think the best is to use assembly ai, since it cheaper that way + gemini llm is expensive than assembly and wih that we can use something like word level timing.

let me share a test version=>

import { AssemblyAI } from 'assemblyai';
import fs from 'fs';

const client = new AssemblyAI({
  apiKey: 'reducted - api key- here',
});

const audioUrl =
  'https://assembly.ai/sports_injuries.mp3';
// const audioUrl= './path/to/file.mp3';

const run = async () => {
  const transcript = await client.transcripts.transcribe({ audio: audioUrl });

  const vttLines = ['WEBVTT\n'];

  transcript.words.forEach((word) => {
    const start = msToVtt(word.start);
    const end = msToVtt(word.end);
    vttLines.push(`${start} --> ${end}`);
    vttLines.push(word.text);
    vttLines.push('');
  });

  const vttContent = vttLines.join('\n');
  const fileName = `transcript_${transcript.id}.vtt`;
  fs.writeFileSync(fileName, vttContent);
  console.log(`Saved: ${fileName}`);
};

function msToVtt(ms) {
  const h = Math.floor(ms / 3600000);
  const m = Math.floor((ms % 3600000) / 60000);
  const s = ((ms % 60000) / 1000).toFixed(3);
  return `${String(h).padStart(2, '0')}:${String(m).padStart(2, '0')}:${s.padStart(6, '0')}`;
}

run();

this is an example and is more precise ( especially if you are trying to generate caption from voice overs )

then in you can group the words into sentences/phrases ( let say 5 words per segment) - this way the word is not highlighted before it is said in the voiceover