Unspokn is a silent speech app. You mouth a short phrase at your camera, a lip-reading model turns it into text you check and edit, and then Unspokn shows it full-screen or speaks it in a natural voice. The microphone stays off. Typing and saved phrases work with no camera at all. It is built for people with speech disabilities, people with temporary voice loss, and anyone in a place where they can't speak aloud. It runs on iPhone and as a Mac menu-bar app that dictates into any app, both in private beta.
The problem
Speech-to-text needs your voice. People with speech disabilities, people recovering from surgery or laryngitis, and people in a quiet carriage or a shared office can’t always use it. The alternatives are slow: typing, picking from phrase boards, or carrying special hardware.
The product rules start from one fact. Many sounds look the same on the lips. So Unspokn must never claim to read any sentence reliably, must let you check every word before it is said, and must never swap in a guess for what you meant.
What we built
You mouth a short phrase at the front camera. Unspokn can start and stop on its own when your lips move and pause. The phrase comes back as text with “Or did you mean…” alternatives. You fix it or accept it, and a natural voice speaks it. Show mode puts the text full-screen and can flip it for the person across from you.
Typing and saved phrases work with the camera off. The app interface comes in eight languages, and results can be translated before they are spoken. Share clips make a short video of just your lips with the spoken audio, only when you tap Share.
On the Mac, Unspokn lives in the menu bar. Hold a shortcut and talk, or switch to Silent and mouth the words with the microphone off. The text lands in whatever app you are typing in.
How we built it
A Python service does the lip reading. It resamples the clip to 25 frames a second, finds face keypoints with MediaPipe, crops the mouth exactly the way the model was trained, and runs an open research lip-reading model with a language model. It returns three candidates and never a confidence score, because the beam scores are not probabilities.
A Next.js gateway sits in front of it. It checks uploads, applies rate limits, and calls Deepgram for voices and Anthropic’s API for translation. The phone and the Mac both talk to the same gateway. Shared TypeScript packages hold the API contracts, the state machine for the nine visible states and the design tokens.
The iPhone app is Expo and React Native, with a Swift keyboard extension. The Mac app is native Swift and SwiftUI, with optional on-device speech models. It started as a hackathon MVP. The commit that set up deployment to app.unspokn.xyz landed about two hours after the first commit.
What was hard
Being honest about lip reading. Every reading shows up as text on screen. You can edit it, stop playback the moment you tap, or turn off automatic speaking so nothing is said until you tap Speak. We chose not to pass the model’s output through a language model that “corrects” it, because that quietly replaces what you said with a guess. The public site has a section on what it can’t do yet.
Matching the model’s training. The model expects a fixed frame rate and a specific mouth crop. Browser and phone video arrive at variable frame rates, and the MediaPipe API the reference pipeline used no longer ships. We rebuilt the crop on the newer face detector using the same four points, so the crops still match what the model was trained on.
One Mac app, two editions. The Mac App Store sandbox forbids typing into other apps. So one source tree builds two editions. The Developer ID build inserts text directly. The App Store build copies it and tells you to press ⌘V. Features the sandbox forbids are compiled out, not just hidden.
Privacy you can see. The camera and microphone are separate permissions, and both states stay on screen. Clips are deleted as soon as they are read and never used for training. Text sent to an outside provider needs your consent first, and the gateway never logs it.
By the numbers
- 122 commits on main from November 9 to November 13, 2026, by 2 people
- 4 merged pull requests
- 1,215 test cases defined in 85 test files: 931 TypeScript, 98 Python and 186 Swift (counted in source, not from a CI run)
- 8 app languages; English lip reading, with Spanish, Portuguese, French and Mandarin as early previews
- On five 4-second studio clips of a person talking aloud, the model got 18% of words wrong. The team’s own notes say to expect worse for silent mouthing on a webcam
- About 2.4 to 3.3 seconds of server time to read a 4-second clip on an Apple M4 Max CPU
Where it is now
Unspokn is in private beta on iPhone and Mac. unspokn.xyz is live with a waitlist, and people are let in a few at a time. Everything is free during the beta. The browser app at app.unspokn.xyz has been retired, but the gateway behind it still runs for the native apps.
Next, according to the README and docs: an App Store release for iPhone, the Mac App Store edition through TestFlight, a two-pane layout for iPhone Duo, and Android later.



