Executive summary
Crane AI Labs fine-tuned NVIDIA Nemotron 3.5 ASR Streaming 0.6B for Luganda, Shona and Swahili, while maintaining English capability with a measured word-error-rate trade-off. We refer to the adaptation in this article as Crane’s Nemotron-based ASR v1.2. Its existing public repository remains CraneAILabs/crane-nemo-asr-1.2; “v1.2” is Crane’s release number, not NVIDIA’s base-model generation. The adaptation uses the speech components of the NVIDIA NeMo Framework.
On a frozen evaluation set of 50 clips per language, the unadapted checkpoint produced approximately 100% word error rate (WER) on the three target languages. The adapted model recorded 4.3% character error rate (CER) / 17.9% WER on Luganda, 5.9% / 26.6% on Shona, and 6.8% / 22.3% on Swahili. These are small-sample benchmark results, not field-performance measurements.
The speech model is now the front end of an agentic financial workflow with SocialPay, a Ugandan fintech serving savings groups and communities. The milestone is a controlled live-partner integration, not a production rollout. In the captured run, eight financial read tool calls were executed against SocialPay’s live APIs. Two of eight tool-backed model responses passed grounding, both English text; six were handled by deterministic fallback. In one case, the model generated an unsupported financial amount and the grounding layer rejected it before delivery. The design principle is simple: the model should never guess your money. The captured run shows that safeguard being exercised, alongside the system’s remaining limitations.
1. The problem: the interface can exclude the user
Across Uganda, people manage money through savings groups, SACCOs, small businesses and digital platforms. Voice and local languages are important parts of the financial interactions this work is intended to support. The question is whether an interface based on English, typing or structured commands can create avoidable friction for those users.
SocialPay had already built the financial platform, customer workflows and WhatsApp-based service layer. Crane’s role was not to replace that infrastructure, but to explore whether local-language speech and an adapted Nemotron stack could become another interface to it. The controlled work therefore focused on a narrow question: can a member ask naturally about financial information while SocialPay’s systems remain the source of truth?
Financial services should not become harder to access because the interface assumes English or typing. Many of the communities we serve communicate naturally by voice and in local languages. The opportunity is to make the technology meet people where they already are.
— Nicholas Kamanzi, Co-Founder, SocialPay
2. Step 1 — Making the speech model understand the target languages
The starting point was nvidia/nemotron-3.5-asr-streaming-0.6b, a cache-aware streaming speech model. We selected it for an interactive voice-agent workflow, the ability to host open weights in-region, and its adaptation path through NVIDIA NeMo Framework.
The comparison below uses a frozen set of 50 clips per language: Luganda and Shona from WAXAL, and Swahili and English from FLEURS. Both checkpoints were evaluated using the same cache-aware streaming configuration and text-normalization procedure. The split documentation and its speaker-separation limitation are described in the technical documentation; speaker-disjoint evaluation is not asserted here.
English baseline correction
After removing language-ID tags from the unadapted checkpoint’s output, base English CER is 6.1% and WER is 10.4%, replacing the earlier unnormalized figures of 10.7% CER and 19.3% WER. This corrects evaluation preprocessing; the fine-tuned model’s reported results are unchanged.
English capability is maintained, with a measured trade-off. On this evaluation, v1.2 has higher English WER than the normalized base, 11.9% versus 10.4%, while CER is lower, 5.5% versus 6.1%. An earlier Crane release, v1.1, recorded 9.1% English WER. Maintaining English performance remains an explicit optimization target.
We report both CER and WER rather than treating either as a complete measure of user experience. A supplementary text-only LLM evaluation provides an exploratory signal of meaning preservation, not a human-validated measure of usability. Its protocol and limitations are retained in the technical documentation.
The main adaptation lessons concerned training configuration and data preparation, including recovering usable segments from long recordings already available to the team. The complete training recipe, data composition and negative experimental results are preserved in the technical documentation rather than interrupting the customer story.
The biggest lesson for me was that low-resource adaptation is rarely about a single clever technique. It’s about getting the basics right: a clean recipe, good segmentation, and actually using the speech you already have. Once we fixed the training pipeline and recovered the dropped hours, the model caught up fast — more a course correction than a breakthrough.
— Caleb Lwanga, AI Engineer, Crane AI Labs, who led the ASR adaptation
3. Step 2 — From transcript to institution-backed information
A transcript alone does not answer “Ssente zange ziri meka?” The voice workflow connects:
WhatsApp voice → Crane’s Nemotron-based ASR v1.2 → Crane Nemotron 4B → financial read tools → SocialPay’s systems → localized reply.
Text requests enter the workflow without the speech-transcription step.
Crane Nemotron 4B is our experimental adaptation of nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16, a NemotronH hybrid Mamba-2 and attention model. The published Crane weights are BF16. The model was adapted to six African languages and English through continued pretraining, supervised fine-tuning and GRPO. In the captured integration it was served through NVIDIA NIM 1.15.0 with the vLLM backend on an A100 80 GB environment, following NVIDIA engineering guidance for this interactive, low-concurrency workload.
The safety design separates language generation from financial authority. Financial facts must come from the institution’s systems. The grounding layer is designed to reject model-generated amounts that lack tool support and route the response to deterministic fallback. The AI is the interface and reasoning layer; SocialPay remains the source of truth.
4. Step 3 — What the controlled live integration showed
In August 2026, working alongside SocialPay’s technical team, we moved from synthetic financial personas into a controlled exercise against live partner APIs. This was not a production rollout, production validation or member-scale deployment.
- 12 requests were captured; 8 financial read tool calls were executed against partner APIs for balance, loan balance and recent activity, using the captured read-oriented tool surface.
- 2 of 8 tool-backed model responses passed the grounding validator, both English text. Six were handled by deterministic fallback; the model-repair loop succeeded in 0 of 6 attempts in this run.
- One model generation contained an unsupported financial amount. The grounding layer rejected it before delivery. This was the same failure class the earlier real-channel testing had exposed; that testing recorded seven failure classes across channel, parsing, language, TTS and grounding.
- No Luganda or voice model-generated response passed grounding in this captured run. Both remain validation targets. A fallback response is not counted as a successful grounded model generation.
- Native tool-call emission was not observed in this NIM/vLLM configuration. All eight executed tool calls were recovered from model text and required argument repair before dispatch.
5. SocialPay as the end customer
For SocialPay, the value of the collaboration is not simply adding another AI model. It is testing whether the financial workflows the company already operates can become easier to access through local-language voice while preserving the institution’s control over the underlying information. The controlled integration gave both teams a first view of that architecture against live partner APIs; the next phase is to determine whether that interface improves real user access and task completion.
We had already been experimenting with hosted AI models, but as usage grows, the cost of relying entirely on API-based inference becomes an important consideration. What interested us about the Nemotron approach was the possibility of adapting a model to our local context and eventually running more of that intelligence on infrastructure and costs we control.
— Isaac Kyuka, Co-Founder, SocialPay
6. The human story: into the field
The next layer documents what happens when the technology leaves the controlled environment: a field story focused on women-led micro and small businesses — shops, markets and savings circles — with participant selection and consent handled separately.
The field question is what changes when a shop owner or savings-group member can interact with financial systems by speaking naturally. The story will not assume the answer. It will look for where voice genuinely reduces friction and where the institution, a deterministic system or a human must remain in control.
7. What we validate next
The next validation targets are code-switching between Luganda or Swahili and English; WhatsApp Opus compression and handset variability; environmental noise and far-field speech; financial-domain vocabulary; entity and numeric accuracy through ASR and reasoning; and end-to-end task success. Grounded local-language and voice model generation remain explicit targets, neither having passed in the captured run.
8. Closing: the next interface may be a conversation
This began as a language-adaptation problem: can an open speech model understand languages on which the unadapted checkpoint performs poorly? The harder question follows: can the adapted model sit inside a financial workflow where mistakes have consequences, the institution remains the source of truth, and the interface feels natural?
The controlled integration does not answer that completely. It shows a progression from open model → adapted model → governed workflow, with the remaining failure modes exposed rather than hidden. The field phase is where technical performance must become human and business impact.
The ASR adaptation was led by Caleb Lwanga. Crane Nemotron 4B was built by Bronson Bakunga, who guided the technical work overall. The work uses NVIDIA Nemotron models, NVIDIA NeMo Framework and NVIDIA NIM.
Crane AI Labs and SocialPay are members of the NVIDIA Inception program for startups—a free program that guides AI startups through the NVIDIA platform and ecosystem.
