One of the primary pillars of the Maple user experience is automated meeting note-taking. To achieve the absolute level of privacy that modern enterprises demand, our product operates a local-first speech processing pipeline. We run optimized speech-to-text transformer models natively on our users' desktop environments. This means your audio never travels to external APIs or third-party servers.
However, running small, quantized speech models on consumer-grade CPUs and GPUs introduces a critical algorithmic vulnerability: the autoregressive repetition loop. During long pauses, silences, or segments with quiet background humming, the attention mechanism of autoregressive decoders tends to loop over its own history, hallucinating the same phrase or tokens indefinitely (e.g., repeating "you you you"). Under standard setups, this results in bloated text logs and high processor utilization.
Autoregressive repetition loops: why they happen
Whisper decoders generate text token-by-token. During inference, the next token's probability distribution is conditioned on the previous sequence of tokens. In ideal conditions, acoustic feature cross-attention provides a strong signal that guides the decoder. But when silence or uniform background noise occurs, the acoustic signal diminishes, leaving the self-attention weights to dominate the prediction. If the model predicts a repetitive sequence once, that pattern enters the context history, heavily biasing future self-attention outputs towards continuing that identical sequence.
Performance metrics & validation
To demonstrate the effectiveness of this architecture, we benchmarked the live N-gram filtering pipeline against a standard Whisper deployment under a simulated 60-second study featuring 15 seconds of clean speech followed by 45 seconds of continuous white noise and ambient typing drafts. The result is summarized below:
| Metric Assessed | Standard Local Whisper | Maple Live Filter Pipeline | Architectural Impact |
|---|---|---|---|
| Silence CPU Usage | 94% (Full Thread Lock) | 4% (Idle Sleep state) | Avoids heat throttle & system drain |
| Hallucinated Words (60s) | 142 duplicate sentences | 0 duplicate sentences | Crisp, clean transcription blocks |
| Gating Threshold latency | 0 ms | < 15 ms | Imperceptible real-time feedback |
| Token consumption | 2,450 redundant tokens | 148 clean tokens | 93.9% reduction in downstream context load |
How we filter repetitions in real time
Our core live N-gram filter runs in three consecutive phases. First, we perform consecutive single-word token squashing. Second, we run a sliding N-gram loop that searches for matching blocks of words (from size 1 to 12) repeating consecutively, and collapses them. Here is a simplified code review of our repetition-collapsing routine:
def remove_repetitive_words(text: str) -> str:
words = text.split()
if len(words) < 4:
return text
cleaned_words = []
consecutive_count = 0
last_word = None
for word in words:
clean_word = "".join(c for c in word.lower() if c.isalnum())
if clean_word == last_word:
consecutive_count += 1
else:
consecutive_count = 1
last_word = clean_word
if consecutive_count <= 2:
cleaned_words.append(word)
n = len(cleaned_words)
i = 0
final_words = []
while i < n:
matched_k = None
matched_repeats = 0
for k in range(1, min(12, (n - i) // 2 + 1)):
pattern = cleaned_words[i : i + k]
repeats = 1
while i + (repeats + 1) * k <= n:
next_block = cleaned_words[i + repeats * k : i + (repeats + 1) * k]
if [w.lower().strip(".,?!:") for w in pattern] == [w.lower().strip(".,?!:") for w in next_block]:
repeats += 1
else:
break
if (k == 1 and repeats >= 3) or (k >= 2 and repeats >= 2):
matched_k = k
matched_repeats = repeats
break
if matched_k is not None:
final_words.extend(cleaned_words[i : i + matched_k])
i += matched_repeats * matched_k
else:
final_words.append(cleaned_words[i])
i += 1
return " ".join(final_words)