Voice input on Cloudflare's free tier: three silent failures, all HTTP 200

A Linux voice input method, a daily allowance of free GPU inference, and about forty lines of TypeScript in between. Hold the right Alt key, say a sentence, and Whisper transcribes it while a language model strips out the disfluencies before the cleaned-up text lands in whatever window you were typing into.

The working feature took an afternoon. What took considerably longer were three failures that all looked like success. Every request returned HTTP 200. Nothing timed out, nothing threw, and every log line looked healthy — the text just quietly came out unprocessed.

Here is the part that makes it worth the trouble. Spoken Chinese, full of the verbal noise everyone produces and nobody wants transcribed:

Raw: 呃那个我们今天就是说要把这个 k8s 的部署搞一下然后呢那个 post gre sql 的迁移也要考虑就是说这两个事情都比较急

>

Polished: 今天我们计划部署 k8s，同时需要考虑 PostgreSQL 的迁移。这两项任务都比较紧急。

Note that post gre sql — which is what Whisper hears when someone says "PostgreSQL" out loud — comes back spelled correctly. That's the language model doing work the transcriber can't.

Versions, since all of this is version-sensitive: fcitx5-vinput 2.3.5, sherpa-onnx 1.13.5, wrangler 4.111, on Arch. The bridge Worker is on GitHub as vinput-cf-bridge, MIT licensed.

Is it actually free?

Workers AI gives you 10,000 Neurons per day, reset at 00:00 UTC. Speech models consume them at wildly different rates:

Three and a half hours of audio a day, free. For voice input — a few seconds at a time, a few dozen times a day — that is not a limit you will reach. At an average of eight seconds per utterance it works out to over 1,600 dictations daily.

whisper-large-v3-turbo is the one to use. It's the turbo variant of large-v3, its Chinese is far ahead of a local 130 MB model, and it costs essentially the same Neurons as plain whisper. The Deepgram models are good but more than ten times the price, which turns the daily allowance into about twenty minutes.

The Worker doing the bridging is also free: 100,000 requests a day on the free plan.

Why you can't just point it at Cloudflare

vinput ships a provider called openai-compatible/batch, which sounds exactly like something you could hand a URL. Reading its entry.py, it:

1. reads raw PCM from stdin (S16_LE, mono, 16kHz)
2. wraps it into a WAV with the wave module
3. POSTs it as `multipart/form-data` to /v1/audio/transcriptions, file field named file
4. expects a JSON response with a top-level text field

Cloudflare's OpenAI-compatible surface covers /v1/chat/completions and /v1/embeddings. It does not include `/v1/audio/transcriptions`. Speech models are reachable only through the native REST endpoint:

POST https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/openai/whisper-large-v3-turbo

which takes a raw binary body or base64 inside JSON rather than multipart, and answers {"result": {"text": "..."}, "success": true} — with text buried inside result instead of at the top level.

Three incompatibilities: path, request encoding, response shape. Pointing VINPUT_ASR_URL at Cloudflare cannot work, and no amount of configuration will change that.

So the bridge exists to translate. It also solves a credentials problem, which turned out to be the better reason: a Worker calling Workers AI through the `AI` binding needs no API token at all — the platform handles authorisation. The only secret on the local machine is one I generated myself, not an account-level Cloudflare token. The polishing step could have talked to Cloudflare's native chat endpoint directly, but that would have meant putting an account token in a local config file, so it went through the same Worker.

① transcription
  mic ──→ vinput ──→ entry.py ──✗02──→ CF edge ──→ Worker ──→ Whisper
                                                                 │
                                                    transcript   │
       ┌─────────────────────────────────────────────────────────┘
       ▼
② polishing
  scene polish ──→ Worker chat ──→ qwen3-30b ──✗03──→ vinput parses ──→ on screen

Where the two silent failures sit on that path matters:

• Wall 02 stops the request before it reaches the Worker. No amount of editing Worker code helps, and nothing appears in its logs.
• Wall 03 happens after a correct response has already been generated. The model was right; the wrapping made it unparseable to the caller.

Three walls

Numbered in the order I hit them. They were genuinely serial — each one had to be cleared before the next became visible — and none of the three reported an error.

Wall 01 — the local model is missing bpe.vocab

offline-model-config.cc:Validate:114 bpe_vocab: '.../bpe.vocab' does not exist
vinput-daemon: async ASR backend reload failed after 0 ms

The vinput-registry manifest declares modeling_unit: "bpe" and bpe_vocab: "bpe.vocab" for the X-ASR models, but k2-fsa's official release tarballs contain only bpe.model. There has never been a bpe.vocab in them. Both the offline and streaming entries are affected — I listed the tar contents with curl -sL <url> | tar -tjf - to be sure rather than assuming.

The intuitive fix — blanking bpe_vocab in the config — does not work, and it's worth seeing why:

if (!modeling_unit.empty() &&
    (modeling_unit == "bpe" || modeling_unit == "cjkchar+bpe" ||
     modeling_unit == "bbpe")) {
  if (!FileExists(bpe_vocab)) {   // blanked: FileExists("") is false too
    return false;
  }
}

The gate is modeling_unit, not whether bpe_vocab is set. online-model-config.cc:164 has character-for-character the same logic, so the streaming model fails identically. The check also runs unconditionally, even though sherpa-onnx's own flag documentation says bpe_vocab is only used when you supply hotwords — so people who never touch hotwords are blocked by a file they don't need.

bpe.vocab can be regenerated losslessly from bpe.model, which fixes it without touching any config — meaning upgrades won't conflict:

uv run --with sentencepiece python -c '
import sys, sentencepiece as spm
sp = spm.SentencePieceProcessor(); sp.load(sys.argv[1])
open(sys.argv[2],"w").writelines(
    f"{sp.id_to_piece(i)}\t{sp.get_score(i)}\n" for i in range(sp.get_piece_size()))
' ~/.local/share/vinput/models/sherpa-onnx/x-asr-*/bpe.model \
  ~/.local/share/vinput/models/sherpa-onnx/x-asr-*/bpe.vocab
systemctl --user restart vinput-daemon.service

There's no python-sentencepiece in the Arch repos, hence the throwaway uv run --with environment. The output should be 5000 lines, matching tokens.txt. Only about 984 of them have a non-zero score — those are the English BPE fragments; Chinese runs character-by-character at score 0. That is correct output, not a failed export.

Filed upstream: vinput-registry#18.

Wall 02 — Cloudflare's edge bans the Python-urllib user agent

curl against the Worker worked perfectly. vinput's provider script got a 403. The request never reached the Worker — its logs were empty.

HTTP 403  error_code: 1010
error_name: browser_signature_banned
detail: The site owner has blocked access based on your browser's signature.

entry.py uses urllib, which sends a default user agent, and Cloudflare's bot protection rejects it at the edge before origin. Tested side by side:

It is that exact string that's banned. Any other value — including no user agent at all — passes. The fix is one header in ~/.local/share/vinput/providers/openai-compatible/batch:

headers={
    "Authorization": f"Bearer {api_key}",
    "Content-Type": f"multipart/form-data; boundary={boundary}",
    "Accept": "application/json, text/plain;q=0.9, */*;q=0.8",
    # Cloudflare (and other WAFs) reject the default Python-urllib
    # signature with HTTP 403 error 1010 before the request reaches
    # the origin, so send an explicit User-Agent.
    "User-Agent": "vinput-asr/1.0",
},

Wall 03 — the polished text arrives inside a markdown fence

Polishing was enabled and the raw, unpolished transcript kept appearing on screen. The only trace was a single daemon log line — no error, no timeout, just a silent fallback:

vinput-daemon: LLM response from .../v1/chat/completions returned no valid candidates

vinput appends its own format contract to whatever scene prompt you write, and sends response_format: {"type":"json_object"}, stream: false, temperature: 0.2:

## Constraints
- Return only the JSON object described below.
- Each candidate must contain only the final rewritten text.
- Do not include explanations, Markdown fences, or extra keys.

## Format
Return EXACTLY 1 candidate(s) in a JSON object:
{"candidates": ["<string>", "<string>"]}

What the model returned:

```json
{"candidates": ["效果测试。"]}
```

The JSON is perfectly correct. It's just wrapped in a fence — the one thing the prompt explicitly forbade. vinput doesn't strip fences, so it saw no candidates and fell back to the raw transcript.

The fix belongs in the bridge, not in the prompt. Real OpenAI guarantees a bare JSON body when response_format is json_object, and callers parse it directly on that basis; my Worker was ignoring the field entirely.

/**
 * Real OpenAI guarantees a bare JSON body when `response_format` is
 * `json_object`, and callers parse it directly. Workers AI text models honour
 * the instruction but still wrap the object in a Markdown fence, so unwrap it
 * here rather than making every caller cope.
 */
function unwrapJsonObject(text: string): string {
  let t = text.trim();

  const fenced = t.match(/^```(?:json)?\s*([\s\S]*?)\s*```$/i);
  if (fenced) t = fenced[1].trim();

  if (!t.startsWith("{") && !t.startsWith("[")) {
    const start = t.indexOf("{");
    const end = t.lastIndexOf("}");
    if (start !== -1 && end > start) t = t.slice(start, end + 1);
  }

  try {
    JSON.parse(t);
    return t;
  } catch {
    // Not valid JSON after unwrapping; hand back a well-formed object so the
    // caller sees the text rather than a parse failure.
    return JSON.stringify({ candidates: [text.trim()] });
  }
}

Prompting harder would have been the wrong instinct. The model was already obeying as well as it was going to; the contract it was being held to is one the platform doesn't actually enforce, and the layer that knows about the discrepancy is the one that should absorb it.

I guessed wrong twice

Wall 03 deserves its own account of the debugging, because the first two theories were both wrong, and both were wrong in a way that felt right.

First guess: the response structure was off — something about how choices was being read. Changed it. No effect.

Second guess: the SSE frame ordering. I had the client packing role and content into a single frame, where the convention is a role-only frame followed by content deltas, which would make a spec-compliant parser drop the content. The reasoning chain was complete, self-consistent, and entirely plausible. Changed it. Still nothing.

Then I captured the traffic. A recording reverse proxy between vinput and the Worker, logging actual bytes. The answer was immediate: the request had stream: false. The SSE path was never being taken at all. My second theory was a detailed, coherent explanation of a code path that never executed.

The lesson I'd like to remember: with a silent failure, the second failed guess is the signal to stop guessing and capture. Each of those two changes cost a deploy plus a round of speaking into a microphone to test. The proxy took under five minutes to write and produced deterministic evidence instead of another hypothesis.

It paid off twice, too — captured requests can be replayed verbatim. Once the fix was in, confirming it took seconds of firing the recorded JSON back at the Worker, rather than asking a human to talk to a computer again.

The proxy itself promptly hit Wall 02: it forwarded with urllib, and Cloudflare handed it an HTML block page on the very first run.

Verification tricks worth stealing

Use a sample whose answer you already know. The sherpa models ship test_wavs/0.wav with a documented correct transcription (昨天是 Monday, today is 礼拜二……). Far better than recording something fresh: it exercises authentication, response shape and mixed Chinese/English at once, and the result is comparable across runs.

Use memory to tell which path is live. The daemon holds about 410 MB with a local model loaded and about 4 MB when transcription goes to the cloud. One glance at RSS answers "is this actually hitting the network" without reading a single log line.

Log structure, never content. During debugging the Worker logged key names, roles and character counts — not what was said. Enough to locate a parsing fault, without writing anyone's dictation into a cloud logging pipeline.

Fail closed by construction. The Worker was deployed *before* its secret was set, so every request returned 401 until AUTH_TOKEN existed. The window between deploy and configuration failed in the safe direction.

Two more, found while open-sourcing it

Tidying the Worker into a proper project — strict TypeScript, tests, CI — surfaced two more problems that only show up when you actually run things.

The AI binding is a proxy, and run can't leave it

Refactoring pulled env.AI.run out into a variable to inject it. Instant 500:

workers-ai: Cannot set properties of undefined (setting '#options')

The obvious diagnosis is a lost this, so the obvious fix is ai.run.bind(ai). That fails too. The binding is a Proxy, and bind sets this to the proxy rather than the underlying object, which breaks its private field access just as thoroughly. Nothing short of a genuine method call works:

const binding = ai as unknown as { run: AiRunMethod };
return (model, input) => binding.run(model, input);

Same shape as Wall 03: the first fix was wrong, and it was wrong in the way that looks most like being right. I added a regression test — and then checked that the test had teeth by reverting the code to the broken version and confirming it actually failed. Worth doing deliberately: the fake binding in the test has to use a real private field (#calls), because a stub without one cannot reproduce the bug and the test would pass against broken code.

vitest-pool-workers wasn't usable

Cloudflare's own Workers test pool crashes outright on Node 26 — vm._setUnsafeEval has been removed. And it wants to open a remote session for the ai binding, which for a public repository means CI would need Cloudflare credentials just to run unit tests.

The way out was to have the handler accept an injected model runner, so tests run on plain Node against a stub: 42 tests in 400 ms, no credentials, no billed inference calls. The only Workers API actually missing under Node was crypto.subtle.timingSafeEqual, filled in with Node's own crypto.timingSafeEqual — a real constant-time implementation, not a stub that returns true and quietly voids the auth test.

Choosing the polishing model

Same spoken text, same prompt, one run each:

Both correctly reconstruct post gre sql into PostgreSQL. The one with "flash" in its name is the slow one — model names are marketing, not measurements.

That was enough to pick qwen3 and move on, and it was also the weakest measurement in this whole project: one sample each, of a thing I already knew to be variable.

Measuring it properly

Nine runs each, same prompt, through the deployed bridge:

My original 2.3s for qwen3 was a single sample that happened to land near the floor; the honest median is 3.4s. And the interesting difference isn't the median at all — the fastest runs are a tie, 2.52s against 2.53s. The win is entirely in the tail. qwen3 occasionally wanders off for six seconds; DeepSeek tops out at 4.25s with half the standard deviation.

For dictation that distinction matters more than the average, because you are sitting there waiting for it every single time. An occasional six-second stall is far more noticeable than a consistent three.

The model you name may not be the model that answers

My first attempt at that comparison produced "identical performance", which should have been a bigger surprise than it was.

The bridge resolves short model names through an alias map and falls back to the configured default whenever it doesn't recognise one. deepseek-v4-flash wasn't in that map yet. So I had benchmarked qwen3 against qwen3: HTTP 200 both times, valid JSON, plausible Chinese, no warning anywhere.

The only thing that gives it away is the model field in the reply, which reports what actually ran rather than what you asked for:

requested: deepseek-v4-flash
served by: @cf/qwen/qwen3-30b-a3b-fp8

Same family as the three walls above. Nothing errors; you just get a confident answer to a question you didn't ask.

The thinking switch was in the wrong language

The bridge sends enable_thinking: false with every request. That is Qwen's parameter. DeepSeek-family models take reasoning_effort instead — so the bridge had been politely asking a DeepSeek model not to think, in a language it doesn't speak, and paying for the reasoning anyway.

Workers AI documents the parameter as "string | null" and does not enumerate the values, so I measured them. Five runs each:

All six were accepted. Leaving it unset behaves like `medium` — reasoning was on the whole time. Asking for none takes about a third off.

One confound worth ruling out: DeepSeek bills cached input separately, so a warming prompt cache could have produced a downward drift that I'd misread as a real effect. The run order was none → minimal → low → medium → high, and the later values came out *slower*. A warming cache can't produce that shape.

End to end, a polish request now returns in 1.41s median, against 2.86s when I started this comparison.

What it costs

deepseek-v4-flash is 120,000 Neurons per million output tokens against qwen3's 30,475 — 3.9× the price for a job that is almost entirely output. It also requires a paid billing method on Workers AI, which means it is not available on the free allowance this article is otherwise about.

So this is an upgrade, not a correction. Everything described here works on the free tier with qwen3-30b-a3b-fp8, which is what I'd still recommend starting with. DeepSeek buys you a tighter tail and roughly half the latency, and you pay for both.

Either way the reasoning has to be dealt with. Both models think by default; the polished text goes straight into whatever document you were typing in, so leaking a reasoning trace into it would be worse than not polishing at all. The Worker suppresses reasoning through whichever parameter the model actually honours, and keeps stripReasoning() behind that as a backstop.

What you give up

Losing hotwords is the only regression you actually feel. The obvious compensation is initial_prompt in the Worker, biasing Whisper toward the terms you use:

initial_prompt: "以下是技术讨论，可能包含 Kubernetes、PostgreSQL、Neovim、gRPC、Wayland 等术语。"

Softer than explicit hotwords, and — as it turned out — not free.

Here is the sequence, because the order I learned it in is the interesting part. On the sherpa model's own test sample, the local 130 MB model produced better capitalisation and punctuation than cloud Whisper:

local X-ASR:   昨天是 Monday, today is 礼拜二， the day after tomorrow 是星期三。
cloud Whisper: 昨天是monday。today is 礼拜二。the day after tomorrow 是星期三。

A 130 MB model beating large-v3-turbo was surprising enough that I filed it under "biased benchmark — it's their own test set" and moved on.

It wasn't the benchmark. It was my own compensation. Clearing ASR_PROMPT — dropping the terminology bias entirely — and rerunning the same sample:

cloud Whisper: 昨天是Monday. Today is 礼拜二.

Correct capitalisation, correct punctuation. A Chinese initial_prompt appears to pull Whisper toward all-Chinese writing conventions, full-width punctuation included, across the whole transcript — not just toward the terms in it.

So the compensation for lost hotwords costs something real, and I had been measuring the cloud model with its hands tied. Which of the two is genuinely better still takes days of ordinary use to know; what's settled is that the earlier comparison wasn't measuring what I thought it was.

Configuration reference

Worker routes:

POST /                        → Whisper transcription (vinput's ASR provider points here)
POST /v1/audio/transcriptions → same
GET  /v1/models               → model list (probed by vinput `llm test`)
POST /v1/chat/completions     → polishing

Deployment:

# key fields in wrangler.jsonc
#   "ai": { "binding": "AI" }
#   "account_id": "<your account id>"

wrangler deploy
wrangler secret put AUTH_TOKEN < .auth-token   # from a file, not a command-line argument

A wrangler trap: write operations require a real TTY. Running wrangler login or deploy from a non-interactive shell triggers an OAuth reauthorisation that times out and fails — and it can destroy your existing login state in the process. Do the login from an actual terminal.

Wiring it into vinput:

# ASR — note this wants the short ID (oai-batch), not the full one
vinput provider add oai-batch
vinput config set /asr/providers/1/env/VINPUT_ASR_URL "https://<your worker>"
vinput config set /asr/providers/1/env/VINPUT_ASR_MODEL "whisper-large-v3-turbo"
tr -d '\n' < .auth-token | vinput config set /asr/providers/1/env/VINPUT_ASR_API_KEY -i
vinput provider use oai-batch

# polishing
vinput llm add cf -u "https://<your worker>/v1" -k "<token>"
vinput llm test cf
vinput scene add --id polish -l "润色" -p cf -m qwen3-30b -c 1 \
  --timeout 15000 --context-lines 0 -t '<prompt, with {{asr}} as the placeholder>'
vinput scene use polish

-m takes whatever short names the bridge's alias map defines, or a full @cf/…

name. Anything else resolves to the bridge's configured default **without

complaining**, so after changing it, check the model field in a reply rather

than trusting that the name stuck.

Switching. The two layers are independent, so "local transcription plus cloud polishing" is a valid combination:

vinput provider use sherpa-onnx   # ASR back to local (hot reload, no daemon restart)
vinput provider use oai-batch     # ASR back to the cloud
vinput scene use __raw__          # polishing off
vinput scene use polish           # polishing on

Switching back to the cloud does not unload an already-loaded local model — only a daemon restart drops it back to 4 MB. That's a feature rather than a leak: flipping back and forth is instant. There are no shipped keybindings for provider switching; the CLI above is the verified path.

Three CLI traps:

• vinput provider add wants the short ID (oai-batch), not provider.openai-compatible.batch.
• vinput config set only modifies existing keys and cannot create them. A provider's env has exactly the three slots the registry declares (API_KEY / URL / MODEL); anything else, like LANGUAGE, means editing config.json directly. The defaults in entry.py are good enough (response_format=json, timeout=60s), and the Worker defaults to zh when no language is sent.
• A scene's --timeout defaults to 4000 ms, which cloud APIs exceed easily — and exceeding it is another silent fallback to the raw text. Use 10000–15000.

If you just want free cloud Whisper

Then don't do any of this. Groq is dramatically simpler — it serves a genuinely OpenAI-compatible /v1/audio/transcriptions, so no Worker is required:

VINPUT_ASR_API_KEY=gsk_your_groq_key
VINPUT_ASR_URL=https://api.groq.com/openai/v1/audio/transcriptions
VINPUT_ASR_MODEL=whisper-large-v3-turbo
VINPUT_ASR_LANGUAGE=zh

Three minutes, also free. The Cloudflare route earns its extra complexity on two counts: the allowance is precisely quantified (10,000 Neurons a day converts exactly into minutes of audio), and owning the middle layer means you can set your own prompts and parameters, unwrap malformed responses, and add multi-model fallback. That last one is what turned Wall 03 from a dead end into a nine-line function.

What gets overwritten

Two of the three fixes live in files that a reinstall will replace: