Engineering

Cracking the Whisper Repetition Loop: Live N-Gram Collapsing in Local Speech Pipelines

One of the primary pillars of the Maple user experience is automated meeting note-taking. To achieve the absolute level of privacy that modern enterprises demand, our product operates a local-first speech processing pipeline. We run optimized speech-to-text transformer models natively on our users' desktop environments. This means your audio never travels to external APIs or third-party servers.

"When acoustic signals drop during silences, the model begins feeding on its own output history. A single repetitive token can trigger a cascade of infinite loops."

However, running small, quantized speech models on consumer-grade CPUs and GPUs introduces a critical algorithmic vulnerability: the autoregressive repetition loop. During long pauses, silences, or segments with quiet background humming, the attention mechanism of autoregressive decoders tends to loop over its own history, hallucinating the same phrase or tokens indefinitely (e.g., repeating "you you you"). Under standard setups, this results in bloated text logs and high processor utilization.

Autoregressive repetition loops: why they happen

Whisper decoders generate text token-by-token. During inference, the next token's probability distribution is conditioned on the previous sequence of tokens. In ideal conditions, acoustic feature cross-attention provides a strong signal that guides the decoder. But when silence or uniform background noise occurs, the acoustic signal diminishes, leaving the self-attention weights to dominate the prediction. If the model predicts a repetitive sequence once, that pattern enters the context history, heavily biasing future self-attention outputs towards continuing that identical sequence.

Figure 1: Live Local Transcription & Filtering Pipeline
WebSocket Float32 Audio RMS Gate Silence Detection Local Whisper Autoregressive STT N-Gram Filter Repetition Collapse

Performance metrics & validation

To demonstrate the effectiveness of this architecture, we benchmarked the live N-gram filtering pipeline against a standard Whisper deployment under a simulated 60-second study featuring 15 seconds of clean speech followed by 45 seconds of continuous white noise and ambient typing drafts. The result is summarized below:

Metric Assessed Standard Local Whisper Maple Live Filter Pipeline Architectural Impact
Silence CPU Usage 94% (Full Thread Lock) 4% (Idle Sleep state) Avoids heat throttle & system drain
Hallucinated Words (60s) 142 duplicate sentences 0 duplicate sentences Crisp, clean transcription blocks
Gating Threshold latency 0 ms < 15 ms Imperceptible real-time feedback
Token consumption 2,450 redundant tokens 148 clean tokens 93.9% reduction in downstream context load

How we filter repetitions in real time

Our core live N-gram filter runs in three consecutive phases. First, we perform consecutive single-word token squashing. Second, we run a sliding N-gram loop that searches for matching blocks of words (from size 1 to 12) repeating consecutively, and collapses them. Here is a simplified code review of our repetition-collapsing routine:

def remove_repetitive_words(text: str) -> str:
    words = text.split()
    if len(words) < 4:
        return text
    cleaned_words = []
    consecutive_count = 0
    last_word = None
    for word in words:
        clean_word = "".join(c for c in word.lower() if c.isalnum())
        if clean_word == last_word:
            consecutive_count += 1
        else:
            consecutive_count = 1
            last_word = clean_word
        if consecutive_count <= 2:
            cleaned_words.append(word)
    n = len(cleaned_words)
    i = 0
    final_words = []
    while i < n:
        matched_k = None
        matched_repeats = 0
        for k in range(1, min(12, (n - i) // 2 + 1)):
            pattern = cleaned_words[i : i + k]
            repeats = 1
            while i + (repeats + 1) * k <= n:
                next_block = cleaned_words[i + repeats * k : i + (repeats + 1) * k]
                if [w.lower().strip(".,?!:") for w in pattern] == [w.lower().strip(".,?!:") for w in next_block]:
                    repeats += 1
                else:
                    break
            if (k == 1 and repeats >= 3) or (k >= 2 and repeats >= 2):
                matched_k = k
                matched_repeats = repeats
                break
        if matched_k is not None:
            final_words.extend(cleaned_words[i : i + matched_k])
            i += matched_repeats * matched_k
        else:
            final_words.append(cleaned_words[i])
            i += 1
    return " ".join(final_words)