A personal project to make phone calls easier for an elderly, non English-speaking parent that works entirely offline, and the long road through Whisper models, sherpa-onnx, frame-size crashes, and Android's scoped storage pitfalls before it finally worked with zipformer.

12 min read · Android · AI models · Bengali · ASR · On-device AI

I wanted to build a simple tool for my elderly, non-English-speaking mother: an Android voice dialer that would let her say a Bengali nickname—like "মিলি" or "ছোটদি"—and immediately call the right person.
So, the plan was to build a fully offline app where she could just say "মাকে ফোন করো" (Call mom) or use nicknames, and the phone would dial.

If you are trying to deploy on-device AI for low-resource languages, or just wrestling with mobile machine learning runtimes, here is the honest post-mortem of how I eventually built a tiny app with real time conversational voice dialer that runs on cpu.

I tested it on a Pixel 10 tensor G5 chip, Android 16.

Step 1 — Testing Whisper Models on Mac

Before writing any Android code I spent time on testing which Whisper model could handle Bengali. All tests were on a MacBook using whisper.cpp.

whisper-small (~244 MB)

The small model heard Bengali phonemes and wrote them in Devanagari script — essentially the Hindi alphabet rendering of what it heard. "মাকে ফোন করো" came out as something like "माकी फोन करो". Phonetically close, but in the wrong script entirely. Not useful for matching Bengali contact aliases.

whisper-medium (~769 MB)

Medium was actually worse. It produced fragmented output — random punctuation, occasional English words in the middle of Bengali sentences, inconsistent script. I am not sure why medium performs worse than small for Bengali short-form audio, but it does. The training data distribution at that model size seems to work against low-resource languages.

whisper-medium with a prompt

Adding an explicit transcription prompt ("Transcribe the following Bengali speech:") improved medium noticeably. It started producing more consistent Bengali or Devanagari output. But it was still 769 MB and inconsistent enough that I kept looking.

whisper-large-v3-turbo — the one that worked on Mac

The turbo variant of large-v3 was the best of the bunch. Most of the time it produced clean output, either in Bengali script or at least phonetically correct Devanagari. I decided this was the one to take to Android. The GGML q5_0 quantised version is around 900 MB.

Step 2 — whisper.cpp on Android. It Took Forever.

I ported the whisper.cpp JNI integration to Android. The setup was standard: CMake build, C++ library, Kotlin calling JNI. I pushed the turbo model to the device and ran a test.
The first transcription took 80 seconds for a three-second voice clip.
The reason is architectural. On six CPU threads with q5_0 quantised weights, the encoder pass alone takes 60–80 seconds. The decoder is fast. The encoder is the entire problem.
I tuned three parameters to reduce the window:

params.single_segment = true; // force a single decode pass
params.audio_ctx = 512; // process ~16s instead of 30s
params.n_max_text_ctx = 64; // limit decoder iterations

It helped at the margins. The decoder sped up. The encoder was still 55–60 seconds. I also attempted to wire in the OpenCL GPU backend , but it was a dead end.
At this point it was clear that whisper.cpp could not deliver a usable latency for this use case on mobile CPU. I needed a different runtime.

Step 3 — sherpa-onnx + Whisper turbo

I rewrote the Android project from scratch using sherpa-onnx — a production ASR framework from the k2/Kaldi team that ships as an Android AAR and uses ONNX Runtime internally. The same turbo model files, different runtime.

First breakthrough: about 2 seconds. Same model, 40× faster.

Getting the model working required three things that are not obvious from the documentation:

`featureDim = 128 // turbo uses 128 mel bins — v1/v2 used 80
tailPaddings = -1 // auto-detect; value 0 silently produces empty output
language = "" // this one became its own saga

The language parameter problem

Whisper uses special prefix tokens to steer the decoder toward a language. Setting language="bn" in sherpa-onnx v1.13.7 produces completely silent and empty output — no error, no warning, nothing transcribed. I found GitHub issues confirming this is a known bug with the ISO code format in this version.
I tried the two alternative formats suggested in those issues:

• language="bengali" — hard crash on first transcription: Invalid language: bengali
• language="<|bn|>" — hard crash: Invalid language: <|bn|>

The only value that worked was language="" — auto-detect. The problem with auto-detect is that for short Bengali utterances, Whisper's internal language classifier randomly picks Hindi, English, and occasionally Spanish. The transcript script varied per utterance. I wrote a toBengaliDisplay() mapper to normalise Devanagari and romanised Latin back to Bengali script, and extended the intent parser to match all three variants. It worked. But it was fragile and felt like the wrong solution.

NNAPI on Whisper turbo

I wired in the ONNX Runtime NNAPI execution provider and bumped minSdk to 27. The app reported Ready (NNAPI). Logcat confirmed initialisation with provider=nnapi. Transcription was still ~2 seconds — because ORT silently fell back to CPU for the Whisper weights. The screen label was misleading; actual execution was CPU all along.

Step 4 — A Bengali-Specific Model Changes Everything

While researching the language detection problem I came across a model published in the sherpa-onnx ASR release assets in February 2026.

sherpa-onnx-streaming-zipformer-bn-vosk-2026-02-09.tar.bz2 (83 MB)

It originates from the Vosk project's Bengali model (alphacep/vosk-model-small-streaming-bn), converted to ONNX for sherpa-onnx. Bengali-only. No language detection parameter. No language configuration of any kind. It just speaks Bengali.

Getting the model onto the device — the scoped storage trap

This took longer than it should have. On Android 11+, running adb push to Android/data//files/ technically succeeds but the files land with shell:shell ownership. The app's File.exists() returns false. No error anywhere. The files are simply invisible to the app.
The cause is that Android's FUSE layer enforces group membership, and the shell user is not in the app's storage group. The fix is straightforward once you understand it:

The correct sequence for pushing model files via adb

Launch the app first so it calls getExternalFilesDir(). This creates the directory owned by the app user with the setgid bit set. Then run adb push. Files pushed into a setgid directory inherit the correct group automatically.

For the Zipformer model I ended up using a debuggable build and adb shell run-as to copy files from /data/local/tmp/ as the app user — the most reliable approach when the directory ownership is already wrong.

The frame size crash

Once the model loaded, the first transcription attempt crashed with:

W sherpa-onnx: features.cc:GetFrames:188
W sherpa-onnx: 0 + 77 > 1

The streaming encoder requires at least decode_chunk_len=64 frames per decode() call. My initial implementation fed the audio in 320-sample chunks (20ms, about 2 frames) — far below the minimum the model needs to start producing output. The fix was to feed the entire recorded buffer at once and let sherpa-onnx handle its internal chunking via the isReady() loop:

kotlin
stream.acceptWaveform(samples = data, sampleRate = 16000)
stream.inputFinished()
while (recognizer.isReady(stream)) {
recognizer.decode(stream)
}
val text = recognizer.getResult(stream).text.trim()


The result

Bengali speech → মাকে ফোন করো. Pure Bengali script. Every single time. No Devanagari, no Latin, no language detection lottery. The toBengaliDisplay() mapper became a no-op. The intent parser matched immediately.

NNAPI on Zipformer — and why it stopped mattering

I tested NNAPI on the Zipformer model as well, mostly out of curiosity. ORT's NNAPI execution provider fell back to CPU again with the same internal quirk about API level reporting. But I stopped chasing it at that point. The Zipformer model transcribes a three-second Bengali command in well under a second on CPU on the Pixel 10. There was nothing left to optimise for this use case.

What I Learned

The runtime matters as much as the model

whisper.cpp and sherpa-onnx ran the same Whisper turbo weights with a 40× latency difference. ONNX Runtime's int8 ARM NEON kernels are not in the same category as ggml's q5_0 path on mobile CPU.

Language-specific models exist and they are worth looking for.

Whisper carries 99 languages in 988 MB. The Bengali Zipformer model carries one language in 90 MB and is faster, more accurate, and completely deterministic on script output. Before defaulting to Whisper for a single-language use case, check the sherpa-onnx and Vosk model zoos.

There are models for many Indian languages that are far better suited to mobile deployment.

Android scoped storage will silently mislead you.

adb push reporting success is not the same as the app being able to read the files. Let the app create its own directories first.

NNAPI is not always necessary.

A well-quantised 90 MB model runs fast enough on CPU on a current-generation phone. Reach for NNAPI only when you have measured that CPU latency is the actual bottleneck.

The app works. I can say "মাকে ফোন করো" and it dials. The next step is replacing the regex-based intent parser with Gemini Nano on-device — to handle the natural variations in how she actually phrases things, like "মাকে একটু ফোন করো না" or calling my sister by three different pet names depending on the day. That will be the next post.
I will be happy if this blog can help you in any ways and feel free to comment.

Android #Bengali #SpeechRecognition #Whisper #sherpa-onnx #On-device #AI #ONNX #Zipformer #Vosk #Indian Languages