Speech Recognition: How Computers Turn Your Voice into Text

When you speak, the air leaving your mouth vibrates slightly and creates sound waves. A microphone picks up those waves and measures their voltage tens of thousands of times per second, recording them as a dense digital signal. It's like a fast-moving video that gets saved as a huge stack of still photos.

Next, the computer reduces background noise, like wind, and makes the human voice stand out. Then it calculates the pitch, the loudness, and how the frequencies of the sound change over time, and turns all of that into a visual graph.

After this step, a voice that would have just faded into the air becomes precise numerical data that a computer can work with. Say "apple" and then "banana," and each makes a completely different frequency pattern. That's how the computer can tell the two sounds apart.

Once the sound is turned into numbers, it goes into an artificial neural network. First, an AI called an acoustic model looks at each tiny slice of sound and calculates the probability of which speech sound it is closest to, like "k," "a," or "n." It has learned from the voices of huge numbers of people ahead of time, so it can handle unusual accents and fast talkers to a fair degree. But the further a voice is from the data it learned on, the more likely it is to make mistakes.

Still, knowing the sounds isn't enough to build a perfect sentence. Think of "their" and "there," or "write" and "right." They sound identical, so a computer can't tell them apart from sound alone.

That's where a second AI, called a language model, steps in. It carefully weighs how the words before and after are connected, and picks the word that fits the context most naturally. Say you hear "I want to [write/right] a letter." The model looks at the whole sentence and settles on "write" as the final word. These days, a single big neural network is also widely used that learns everything at once, from sound all the way to text, without splitting it into these two stages.

A concept people often mix up with speech recognition is "speaker recognition." Speech recognition listens to sound and figures out what was said. Speaker recognition analyzes the unique resonance of each person's vocal cords to identify who is speaking. It's a kind of biometric authentication.

The two sometimes work together when you say "Hey Siri" or "Hey Google" to your phone. Depending on the device, speaker recognition first checks whether the voice belongs to the registered owner. Then speech recognition turns the words that follow into a sentence, and the device carries out the command.

Today's speech recognition is going beyond simple dictation by combining with large language models (LLMs). It transcribes more accurately than before, even in a noisy cafe or when someone mumbles, and it can even polish the transcript into smooth, clean sentences.