Automatic Speech Recognition (ASR) 2026: India’s Reality

TL;DR

Automatic speech recognition (ASR) is AI technology that converts spoken language into written text. It’s the entry point of every voice AI system, and when it gets words wrong, everything downstream fails. ASR accuracy varies wildly by language: US English hits 4-6% word error rate, while Hinglish code-switching sits at 15-22% in good conditions and much worse in production. For Indian enterprises, evaluating ASR means testing on your actual audio, not trusting benchmarks run on clean datasets.

What Is Automatic Speech Recognition (ASR)?

Automatic speech recognition, commonly abbreviated ASR, is AI technology that converts spoken language into written text by analyzing audio waveforms and identifying patterns in human speech. You’ll also see it called speech-to-text (STT) or computer speech recognition. All three terms describe the same core function.

Think of ASR as the “ears” of any voice AI system. When a customer calls a bank to ask about their EMI due date, ASR is the component that listens to the raw audio and produces a text transcript. That transcript then feeds into natural language understanding (NLU), which figures out what the customer actually wants.

Here’s the critical thing about ASR that doesn’t get enough attention: if the transcription is wrong, everything downstream is wrong. Sentiment analysis, intent classification, compliance detection, quality assurance scoring, all of it depends on accurate text from the ASR layer. A substitution error that changes “return” to “retain” will cause the intent classifier to route the call incorrectly, even if every downstream component is perfect.

The global ASR software market is projected to grow from $17.91 billion in 2026 to $29.42 billion by 2035. The broader speech and voice recognition market tells an even bigger story, expected to reach $104 billion by 2034 at a 20.3% CAGR.

If you’re exploring how voice AI fits into banking operations, the AI for banking glossary provides useful context alongside this guide.

How ASR Works: A Four-Step Pipeline

Modern ASR systems follow a pipeline that can be broken down into four stages:

1. Audio capture. A microphone or telephony system captures sound waves. In contact centers, this typically means 8kHz telephony audio, which is lower quality than the 16kHz+ audio most ASR benchmarks use.

2. Feature extraction. The raw audio waveform gets converted into spectral features, most commonly mel-spectrograms. These are mathematical representations that capture the frequency patterns in speech, essentially translating sound into numbers a neural network can process.

3. Model inference. A neural network maps those spectral features to text tokens. Modern systems use end-to-end architectures (typically transformer-based) that handle this in a single model rather than chaining multiple specialized components together.

4. Language decoding. A language model refines the raw output into coherent text. This step handles things like choosing between homophones (“there” vs. “their”) and smoothing out recognition uncertainties.

A Brief History of ASR

The story starts in 1952, when Bell Labs built “Audrey,” a system that could recognize spoken digits. For most of the next fifty years, speech recognition relied on classical machine learning, primarily Hidden Markov Models (HMMs).

The breakthrough came in 2014, when Baidu published Deep Speech. By replacing the hand-built pipeline with a single neural network trained on large amounts of data, it hit a 16% error rate on a standard benchmark and kicked off the modern ASR era.

Today, ASR is dominated by end-to-end foundation models trained on millions of hours of audio. The transformation from GMM-HMMs and DNN-HMMs to these architectures has been driven entirely by advances in deep learning.

Where ASR Fits in the Voice AI Stack

A voice AI system is a four-part pipeline:

1. ASR / STT (speech to text) converts raw audio into text
2. NLU / LLM (natural language understanding) interprets intent and meaning
3. TTS (text to speech) generates spoken responses
4. Orchestration layer manages flow, context, integrations, and handoffs

ASR is the entry point. It sits at the very front of the pipeline, and its accuracy determines the ceiling for everything that follows. An NLU model that’s been fine-tuned for financial conversations can’t compensate for garbled input from a poor ASR system.

This is why teams building conversational AI for contact centers obsess over ASR quality. It’s not just a nice-to-have. It’s the foundation.

How ASR Accuracy Is Measured

Word Error Rate (WER)

Word Error Rate is the standard metric for evaluating ASR performance. The formula is straightforward:

WER = (S + D + I) / N × 100

Where:

• S = Substitutions (wrong word)
• D = Deletions (missed word)
• I = Insertions (extra word)
• N = Total words in the reference transcript

A WER of 10% means roughly 1 in 10 words is wrong. Lower is better.

Why WER Alone Isn’t Enough

WER dominates the research community. An informal survey of Interspeech papers published between 2023 and 2025 found that 86.6% used WER as an evaluation metric, while fewer than 40% considered anything else. But WER has serious blind spots.

The biggest problem: WER treats all errors equally. A missed “um” counts the same as a wrong phone number, even though only one of those breaks a downstream workflow. In a collections call, transcribing “₹5,000” as “₹50,000” is catastrophic. Transcribing “basically” as “basically, um” is irrelevant.

Production voice AI teams increasingly track paired metrics. WER captures transcription quality. Task completion rate captures whether the errors actually matter. Some teams also track entity error rate (how often names, numbers, and dates are wrong) and semantic WER (whether the meaning was preserved even if exact words differ).

Real-World ASR Accuracy by Language

This is where things get interesting for Indian enterprises. The clean WER numbers you see on vendor websites rarely match what happens in production.

Research on agricultural field recordings in India found that Hindi achieved the best overall performance at 16.2% WER, while Odia presented the greatest challenges with a best WER of 35.1%.

On real Hindi-English code-mixed telephony audio in noisy conditions, global models typically achieve 14 to 16% WER. Leading homegrown Indic models built specifically for this audio bring that down to 11 to 14%.

The critical takeaway: in 2026, the best ASR systems rival human transcriptionists on clean English speech. But real-world Indian audio, with its mix of languages, accents, telephony compression, and background noise, remains significantly harder. For a detailed look at accuracy benchmarks across Indian languages, see this guide on voice agent accuracy metrics.

The Code-Switching Problem in Indian ASR

Five of the top ten search results for ASR-related queries focus on code-switching. There’s a reason for that. It’s the single biggest challenge in making ASR work for Indian enterprises.

What Is Code-Switching?

Code-switching (or code-mixing) is the practice of alternating between two or more languages within a single conversation, often within the same sentence. It’s not broken English or poor language fluency. It’s a sophisticated, rule-governed behavior exhibited by highly proficient multilingual speakers.

In India, more than 250 million people engage in code-switched communication, blending English with Hindi, Tamil, Kannada, Telugu, Bengali, and other languages. When a customer says “mera account mein last month ka statement nahi aaya hai, can you check please,” that’s textbook Hinglish, and it’s perfectly natural.

Why Standard ASR Fails

A model that has never been trained on Hinglish won’t handle it just because it has seen a lot of Hindi and a lot of English separately. The mixing patterns, syntactic borrowings, and phonological adaptations that emerge when languages blend require the model to learn from examples of the blend itself.

Research shows ASR models experience a relative increase in WER of 30-50% when exposed to code-switched speech compared to monolingual input. That gap alone can make the difference between a useful voice agent and an unusable one.

Beyond Hinglish

Hinglish is the most commercially prominent code-switching pair, but it’s far from the only one. Tanglish (Tamil-English) dominates Tamil Nadu contact centers. Kanglish (Kannada-English) is common in Bengaluru’s tech and financial services sector. Telglish (Telugu-English) matters across Andhra Pradesh and Telangana.

Practitioners on Reddit have noted that marketing pages often claim 90%+ accuracy on Hinglish, but the reality from teams working in production looks very different. This gap between benchmarked and production accuracy is one of the most persistent frustrations in Indian voice AI deployment.

For a deeper exploration of how code-switching affects voice AI systems, the code-switching voice AI guide covers strategies for handling language mixing in production.

ASR Applications in BFSI and Contact Centers

Contact centers are where ASR accuracy matters most, because every misrecognized word has a direct cost. About 46% of banks already use speech recognition for customer verification and fraud detection, and adoption is accelerating.

Collections and EMI Reminders

Voice-based collection calls are a massive use case in Indian BFSI. ASR converts the conversation into structured data that the AI agent uses to understand borrower responses, confirm payment commitments, and update records. When a borrower says “kal deposit kar dunga” (I’ll deposit tomorrow), the ASR needs to get that right so the system can log the promise-to-pay accurately.

KYC and Onboarding

Voice-first KYC calls require ASR to accurately capture names, addresses, PAN numbers, and Aadhaar details. Entity error rates matter far more than overall WER here, because a single wrong digit in an Aadhaar number renders the entire interaction useless.

Post-Call Analytics and Compliance

ASR transcripts feed into quality assurance systems that measure agent sentiment, check for script adherence, and flag compliance violations. In regulated BFSI workflows, these transcripts serve as audit trails. If the ASR introduces errors, compliance teams can’t trust the data.

Live Captioning and Accessibility

Real-time ASR enables live captioning for hearing-impaired customers and multilingual live translation on calls. This is an accessibility requirement that’s growing in regulatory importance.

The Indian ASR Ecosystem

India has developed its own ASR research infrastructure, driven by the reality that global models optimized for English simply aren’t good enough for Indian languages.

Open-Source Efforts

AI4Bharat’s IndicConformer is a multilingual ASR model suite designed for accurate transcription across all 22 official Indian languages. It represents India’s first open-source ASR system at this scale, and it has catalyzed significant research and development in the Indic speech community.

Benchmarks Built for Indian Speech

The community has developed several benchmarks specifically for evaluating ASR on Indian speech: IndicSUPERB, Vistaar, accent-focused datasets like Svarah and Lahaja, and large-scale demographically diverse datasets like IndicVoices. Indic DiarBench, covering all 22 scheduled languages with approximately 108 hours of multi-speaker audio, captures conversational nuances like English code-mixing, dialectal variation, and frequent speaker overlap.

New Purpose-Built Models

The pace of model releases is accelerating. Shunya Labs’ Zero Codeswitch, announced in 2026, is purpose-built for Indian code-switching scenarios. Their Zero STT Indic targets native-level accuracy on Hindi, Tamil, Telugu, Bengali, and other Indian languages.

These developments matter because they represent a shift from adapting Western ASR models to Indian speech toward building models designed for Indian speech from the ground up.

Key ASR Challenges

Noise and Audio Quality

Background noise distorts speech signals. ASR manufacturers often claim recognition accuracy as high as 99%, but in actual field conditions, reaching 90% can be difficult. Call center floors, mobile phone calls from busy streets, and rural telephony with poor network quality all degrade accuracy substantially.

Accents and Speaker Diversity

Most ASR training data comes from native English speakers, creating a bias against non-native accents. An ASR system trained primarily on American English will struggle with South Indian English accents, and even more with the prosodic patterns of someone speaking English with Hindi intonation. Building vernacular conversational experiences requires training data that reflects actual speaker diversity.

Low-Resource Languages

Training effective ASR models for low-resource or marginalized languages is difficult because there simply isn’t enough labeled data. This affects many Indian languages beyond the major ones. Odia, Assamese, Manipuri, and Dogri all face significant data scarcity.

The Benchmark vs. Production Gap

Tests on clean, read speech from academic datasets like LibriSpeech declare a winner. But production audio is noisy, accented, and full of people switching between languages mid-sentence. The gap between benchmark scores and real-world performance catches many teams off guard.

Latency

For ASR in real-time voice agents, the latency target is under 200 milliseconds. A highly accurate transcript delivered with significant delay still results in a poor conversational experience, with awkward pauses that make callers hang up. This is why the telephony stack underneath voice AI matters as much as the model itself.

Error Propagation

In a voice AI pipeline, WER errors compound downstream. Each component trusts the output of the previous one. If ASR misrecognizes a critical word, the intent classifier routes incorrectly, the response generator says the wrong thing, and the customer gets frustrated. One ASR error can cascade into a completely failed interaction.

How to Evaluate ASR for Your Use Case

Choosing an ASR system based on published benchmarks is like choosing a car based on its fuel economy on a dynamometer. The number is real, but it doesn’t reflect your driving conditions.

Test on your audio. Collect a representative sample of real calls from your production environment. Include noisy calls, code-switched conversations, accented speakers, and low-bandwidth telephony. Run every ASR vendor against this same dataset.

Measure what matters. Track WER, but also track entity error rate (are names, numbers, and dates correct?) and task completion rate (did the system accomplish what the customer needed?). A system with 18% WER that gets phone numbers right every time might outperform a system with 12% WER that garbles digits.

Test code-switching explicitly. If your customers switch between English and a regional language, test for that specifically. Ask vendors for their code-switching training data sources and WER on mixed-language audio.

Check latency under load. An ASR system that delivers 150ms latency on a demo call might hit 400ms when processing 10,000 concurrent streams. Test at your expected scale.

Evaluate domain vocabulary. Financial terminology, product names, and acronyms specific to your business need to be recognized correctly. “CIBIL” should not become “civil.” “SIP” should not become “sip.”

If you’re starting an evaluation process for Indian BFSI voice AI, this implementation guide walks through the practical steps.

Frequently Asked Questions

What is the difference between ASR and speech recognition?

They’re the same thing. ASR (automatic speech recognition), speech recognition, STT (speech-to-text), and computer speech recognition all refer to the technology that converts spoken audio into written text. ASR is the more technical term used in research and industry.

What is the difference between ASR and NLU?

ASR converts speech into text. NLU (natural language understanding) takes that text and interprets its meaning, identifying intent, entities, and sentiment. ASR is the ears; NLU is the brain. Both are required for a functioning voice AI agent.

What WER is acceptable for production use?

It depends on the use case. For general transcription, a WER under 10% is considered good. For contact center automation where entities like names and numbers matter, what counts is entity accuracy and task completion rate, not just overall WER. Many production teams aim for under 15% WER on their specific audio while tracking entity error rate separately.

Can ASR handle Hinglish and other Indian code-switching?

Standard ASR models trained on monolingual data struggle with code-switching. Purpose-built models trained on code-mixed data perform significantly better, though accuracy still lags behind monolingual performance by 30-50% in relative terms. The gap is narrowing as more Indian-focused ASR models enter the market, but it remains a real limitation. Read more about how customers switch between languages on calls.

How fast does ASR need to be for voice agents?

For real-time conversational agents, the latency target is under 200 milliseconds. Anything slower creates noticeable pauses that degrade the conversational experience and increase caller drop-off.

Is ASR accurate enough to replace human transcriptionists?

On clean English speech, the best ASR systems now match or exceed average human transcriptionist accuracy. On noisy, accented, code-switched Indian telephony audio, humans still outperform machines in most conditions. The practical approach is using ASR at scale with human review for high-stakes interactions.

How large is the ASR market?

The global ASR software market is valued at approximately $17.91 billion in 2026, projected to reach $29.42 billion by 2035. The broader speech and voice recognition market, which includes ASR alongside related technologies, is expected to grow from $23.7 billion to $104 billion over the same period.

What is the role of ASR in banking and financial services?

ASR powers voice-based customer verification, fraud detection, collections call automation, KYC voice workflows, post-call quality analytics, and compliance monitoring. About 46% of banks already use speech recognition for customer verification and fraud detection, and that number is growing rapidly.

To see how automatic speech recognition works inside a production voice AI system built for Indian BFSI, book a demo with Awaaz AI and test it against your own call recordings.