Whisper API Guide — Accurate Speech-to-Text with OpenAI and Local Whisper

Introduction

Why This Matters

Accurate transcription used to require expensive specialized services or unreliable open-source models. Whisper changed that in 2022, and its API matured into a production-grade service by 2024. At $0.006 per minute, transcribing an hour of audio costs$ 0.36 — orders of magnitude cheaper than human transcription and comparable in accuracy for most use cases. For developers building meeting summarization tools, podcast platforms, accessibility features, voice-driven interfaces, or customer call analysis, Whisper is the starting point for any speech-to-text pipeline.

OpenAI Whisper API — Basic Integration

from openai import OpenAI
from pathlib import Path
import os
 
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
 
def transcribe_file(audio_path: str, language: str = None) -> str:
    """Transcribe an audio file and return the text."""
    with open(audio_path, "rb") as audio_file:
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            language=language,  # Optional: ISO 639-1 code, e.g., "en", "fr"
            response_format="text"
        )
    return transcript
 
# Basic usage
text = transcribe_file("meeting.mp3")
print(text)

Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, webm

File size limit: 25 MB — files larger than this must be split before sending.

Getting Timestamps

For applications that need word-level or segment-level timing (subtitle generation, searchable transcripts, audio navigation):

def transcribe_with_timestamps(audio_path: str) -> dict:
    """Transcribe with segment-level timestamps."""
    with open(audio_path, "rb") as audio_file:
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            response_format="verbose_json",  # Returns timing data
            timestamp_granularities=["segment", "word"]
        )
    return transcript
 
result = transcribe_with_timestamps("interview.mp3")
 
# Access segments
for segment in result.segments:
    start = segment["start"]
    end = segment["end"]
    text = segment["text"]
    print(f"[{start:.1f}s - {end:.1f}s] {text}")
 
# Access word-level timestamps
for word in result.words:
    print(f"{word['word']} ({word['start']:.2f}s - {word['end']:.2f}s)")

Generating SRT Subtitles

Request SRT format directly from the API:

def generate_subtitles(audio_path: str, output_path: str) -> None:
    """Generate an SRT subtitle file from audio."""
    with open(audio_path, "rb") as audio_file:
        srt_content = client.audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            response_format="srt"  # Direct SRT output
        )
    
    with open(output_path, "w", encoding="utf-8") as f:
        f.write(srt_content)
 
generate_subtitles("lecture.mp4", "lecture.srt")

Handling Long Audio Files

The 25 MB limit means most audio needs splitting. Use pydub to chunk audio:

from pydub import AudioSegment
import math
import io
 
def transcribe_long_audio(audio_path: str, chunk_minutes: int = 10) -> str:
    """Transcribe audio files longer than 25MB by splitting into chunks."""
    audio = AudioSegment.from_file(audio_path)
    chunk_ms = chunk_minutes * 60 * 1000
    num_chunks = math.ceil(len(audio) / chunk_ms)
    
    transcripts = []
    
    for i in range(num_chunks):
        start = i * chunk_ms
        end = min((i + 1) * chunk_ms, len(audio))
        chunk = audio[start:end]
        
        # Export chunk to bytes
        buffer = io.BytesIO()
        chunk.export(buffer, format="mp3")
        buffer.seek(0)
        buffer.name = f"chunk_{i}.mp3"  # Whisper uses filename for format detection
        
        transcript = client.audio.transcriptions.create(
            model="whisper-1",
            file=buffer,
            response_format="text"
        )
        transcripts.append(transcript)
        print(f"Chunk {i+1}/{num_chunks} transcribed")
    
    return " ".join(transcripts)

Translation

Whisper can transcribe and translate to English in one API call:

def translate_to_english(audio_path: str) -> str:
    """Transcribe audio in any language and translate to English."""
    with open(audio_path, "rb") as audio_file:
        translation = client.audio.translations.create(
            model="whisper-1",
            file=audio_file,
            response_format="text"
        )
    return translation

Running Whisper Locally

For privacy-sensitive content (legal proceedings, medical consultations, confidential meetings), run Whisper locally using faster-whisper — a CTranslate2-optimized version that is 4x faster than the original with lower memory usage:

pip install faster-whisper

from faster_whisper import WhisperModel
 
# Load model once at startup
# Model sizes: tiny, base, small, medium, large-v3
model = WhisperModel(
    "large-v3",
    device="cuda",        # or "cpu"
    compute_type="float16" # float32 for CPU
)
 
def local_transcribe(audio_path: str, language: str = None) -> str:
    segments, info = model.transcribe(
        audio_path,
        language=language,
        beam_size=5,
        vad_filter=True,   # Remove silence
        vad_parameters={"min_silence_duration_ms": 500}
    )
    
    return " ".join(segment.text for segment in segments)
 
# With timestamps
def local_transcribe_timed(audio_path: str) -> list[dict]:
    segments, _ = model.transcribe(audio_path, word_timestamps=True)
    
    result = []
    for segment in segments:
        result.append({
            "start": segment.start,
            "end": segment.end,
            "text": segment.text.strip(),
            "words": [
                {"word": w.word, "start": w.start, "end": w.end}
                for w in (segment.words or [])
            ]
        })
    return result

Improving Accuracy with Prompts

The Whisper API accepts an optional prompt parameter — text that precedes the transcription and provides context. Use it to:

Improve transcription of domain-specific vocabulary
Preserve specific spelling of product names or technical terms
Set the correct language register

transcript = client.audio.transcriptions.create(
    model="whisper-1",
    file=audio_file,
    prompt=(
        "This is a software engineering podcast discussing Kubernetes, "
        "Terraform, gRPC, and microservices. "
        "Proper nouns include: Helm, Istio, Argo CD."
    )
)

Common Mistakes

Not specifying language for single-language audio: Language auto-detection is accurate but adds 1-2 seconds of latency. Specify language="en" when you know the language.
Ignoring the file size limit: Files over 25 MB fail silently or with an opaque error. Always check file size before sending.
Not trimming silence: Long silences in audio inflate processing time and cost. Use VAD (voice activity detection) to strip silence before transcription.
Not caching transcripts: Transcribing the same audio file multiple times wastes API budget. Cache by file hash.

Best Practices

Use faster-whisper locally for privacy-sensitive audio — it matches API accuracy with no data leaving your infrastructure
Specify language explicitly for a 10-20% accuracy improvement on non-English audio
Use the prompt parameter to pre-supply technical vocabulary and proper nouns in your domain
Cache transcription results keyed by audio file hash — reprocessing the same content is unnecessary cost
For long-form audio, chunk at natural boundaries (sentence breaks, pauses) rather than fixed time intervals

Key Takeaways

Whisper API costs $0.006 per minute — transcribing one hour of audio costs$ 0.36
The 25 MB file size limit requires splitting long audio before sending; pydub handles this reliably
verbose_json response format provides segment and word-level timestamps for subtitle generation and audio navigation
The translations endpoint transcribes audio in any supported language and returns English text in one step
faster-whisper (CTranslate2-optimized) runs locally at 4x the speed of the original model with lower VRAM usage
The prompt parameter significantly improves transcription of technical vocabulary, product names, and domain jargon
VAD (voice activity detection) filtering removes silence before processing, reducing cost and improving accuracy
For privacy-sensitive content, local Whisper deployment processes audio entirely on your own hardware with no external API calls

Introduction#

Why This Matters#

OpenAI Whisper API — Basic Integration#

Getting Timestamps#

Generating SRT Subtitles#

Handling Long Audio Files#

Translation#

Running Whisper Locally#

Improving Accuracy with Prompts#

Common Mistakes#

Best Practices#

Key Takeaways#