What is automatic speech recognition?
Automatic speech recognition (ASR) is the technology that turns speech into text. You use it whenever you dictate a message on your phone or see live captions on a video. In a contact centre, it is the foundation for almost every voice AI application, because most AI works with text: before a system can understand or analyse a call, it has to know what was said.
ASR is also called speech-to-text (STT). Its counterpart, which turns text back into speech, is text-to-speech.
How ASR is used in a contact centre
In real time
- AI voice assistants transcribe the caller as they speak so natural language understanding can work out what they need.
- Agent assist tools follow the live conversation to suggest knowledge articles, prompts or next steps.
- Live monitoring can flag keywords or sentiment changes during a call.
After the call
- Speech analytics transcribes recorded calls at scale so themes, trends and compliance issues can be found across every interaction.
- AI call summarisation uses the transcript to write up what happened, cutting after-call work.
- Search. Transcripts make a library of call recordings searchable by what was said.
How it works, briefly
Modern ASR uses neural network models trained on very large amounts of recorded speech paired with text. The model breaks audio into small slices, predicts the most likely sounds and words, and uses its knowledge of language to choose the most plausible sentence. For example, it uses context to decide between “their” and “there”, or between “four” and “for”.
Many systems also separate speakers (known as diarisation), so a transcript shows which words came from the customer and which from the agent. That matters for analytics, where you often want to know what customers said rather than what agents said.
What affects accuracy
ASR accuracy is often reported as word error rate, the proportion of words transcribed wrongly. Results vary a great deal with conditions:
- Audio quality. Phone audio is compressed, and mobile calls in poor coverage areas or with wind and traffic noise are harder.
- Accents and speech patterns. Systems perform best on the accents most represented in their training data.
- Vocabulary. Local place names, te reo Māori, product names and industry terms are frequent sources of error. Some systems let you add custom vocabulary.
- Overlapping speech. When the caller and agent talk at once, both are harder to transcribe.
- Numbers and spellings. Account numbers, addresses and email addresses are easy to get slightly wrong, and a single wrong digit matters.
Good practice
- Test on your own calls across a realistic mix of callers, lines and topics.
- Confirm critical details such as addresses and account numbers back to the caller when an AI is acting on them.
- Treat transcripts as evidence, not gospel. For quality or dispute purposes, check the recording.
- Consider privacy. Transcripts can contain personal and payment information, so decide where they are processed and stored, who can see them and how long they are kept.