RustSTT
Real-time speech-to-text in Rust: streaming over WebSocket, several recognition architectures, one service that runs on CPU, CUDA or Metal.
- Ownership
- Owned by Red Sentra.
- Availability
- Offered commercially through its own site. Source is private.
- Stack
- Rust / WebSocket / CUDA / Metal
The problem
Transcription that arrives after the conversation has moved on is not real-time. Target hardware also varies: some deployments have a GPU, some have Apple silicon, some have neither, and the service has to be worth running on all of them.
The system
A streaming recognition service in Rust. Audio arrives over WebSocket and results come back while the speech is still happening. Several recognition architectures sit behind a single interface, and the same service targets CPU, CUDA or Metal depending on the machine.
My role
Systems and performance engineering: the streaming path, the abstraction across compute backends, and keeping latency predictable under load.
The hard part
One service, three very different compute targets, and a latency budget that does not move. What is fast on a CUDA GPU is not fast on a laptop CPU, and the streaming path has to stay predictable on all of them.
Architecture decisions
- One interface in front of several recognition architectures. Callers ask for transcription, not for a particular model, so an engine can be added or swapped without touching the transport.
- Streaming over WebSocket rather than request and response. Audio goes in continuously and partial results come back while the speaker is still talking, which is the only way the output is useful during a conversation.
- Compute targets are a deployment choice, not a fork. CPU, CUDA and Metal sit behind the same abstraction, so one codebase serves a laptop and a GPU server.
Where it stands
In production use. Commercial enquiries go through the project's own site.