Klippy

Transcript editor

The transcript panel provides word-level editing, AI-assisted clean-up, and quick b-roll insertion. It connects to the timeline, captions, and export stack through the shared Redux store. This guide covers the data model, panel anatomy, and advanced tooling available inside the transcript workflow.

Data model

  • transcriptsSlice.ts stores transcript words keyed by media ID plus global preferences (virtualization, follow playhead, committed removes).
  • Each word entry contains start, end, text, confidence, and optional speaker metadata (TranscriptWord in app/types/transcript.ts).
  • Undo/redo is wired through unifiedHistoryMiddleware; transcript changes appear as semantic history entries alongside timeline edits.
  • Imports (SRT/VTT) and automatic transcription populate the slice, while exports keep transcript data bundled in project saves and .klippy.bundle.json files.

Panel layout

Header & modes

  • TranscriptHeader controls follow-playhead, virtualization, and preview toggles.
  • Mode switches between Simple cleanup, Advanced, and AI assists (see components under transcript/ui).
  • Shortcuts popover (ShortcutHelpPopover) documents navigation (J/K/L), selection, and range commands.

Word display

  • TranscriptDisplay.tsx virtualises long transcripts, highlights the active word, and supports drag-selection to define ranges.
  • Ranges snap to phrase boundaries when the option is enabled (see snapRangeToPhrase in utils/detectionAlgorithms.ts).
  • Context menu (TranscriptContextMenu) exposes edit operations, quick inserts, and timeline mapping.

Editing & clean-up

The transcript panel can commit timeline edits, preview clean speech, and batch operations:

  • Use the Edit actions toolbar to add Remove, Mute, or Silence ranges. Actions dispatch to the edits slice and mirror on the timeline preview lane.
  • Preview clean speech runs temporary cuts without committing history entries, letting you audition edits before applying.
  • Suggested edits (fillers, breaths, pauses) live in AssistedEditsPanel.tsx; applying them batches multiple ranges through runInBatch so undo remains concise.
  • The Preview/Commit panel summarises pending ranges and provides a single-click “Apply all” for recommendations.

AI tooling

  • Transcription: useTranscription posts audio to /api/transcription (Cloudflare Whisper or custom endpoint). Progress displays in the panel, and results automatically populate the active clip.
  • Summaries & chapters: useTranscriptAI (consumed by SummaryPanel and SuggestedClipsPanel) calls /api/ai to draft chapters, video summaries, and highlight clips. Suggestions feed back into projects state via setSuggestedClips.
  • Quick insert: QuickInsertPanel searches Tenor for b‑roll, GIFs, and sound effects scoped to the selected words; drops media directly onto the timeline with timing aligned to the transcript range.

AI requests respect the secure proxy configuration described in Backend routes. Ensure AI_REQUIRE_CLIENT_KEY is set appropriately before enabling in production.

Timeline & captions

  • Time mapping functions (mapTranscriptToTimelineForClip, mapTimeThroughRemoves) keep transcript selections in sync with clip trims and remove ranges.
  • Caption actions funnel through addCaptionTrack / addCaption in projectSlice, enabling SRT export or in-editor styling (Captioning).
  • Transcript edits coexist with timeline history; applying removes updates both transcript previews and timeline audio waveforms.

Performance tips

  • Enable Virtualize transcript for hour-long recordings to render only visible rows. Follow playhead automatically scrolls when virtualization is disabled.
  • Large remove previews can be merged with Merge adjacent deletes to reduce history size and keep mapping accurate.
  • Turn off live preview when auditioning large compilations; timeline updates become cheaper when the transcript panel is collapsed.

Troubleshooting

  • Transcript out of sync: re-run Map to timeline from the panel header or clear preview ranges; check clip trims were applied after transcription.
  • API errors: verify /api/transcription and /api/ai environment variables (tokens, allowed origins). Errors surface in the status bar with a retry shortcut.
  • Scroll jank: disable follow playhead or virtualization when working on short clips to keep the DOM small.