Projects Hoshidub
Live AI dubbing for anime, running locally
A Windows application that gives every character in a show their own English voice as they speak, with the soundtrack left intact and an AI watch companion that cannot spoil the episode.
In short
Hoshidub watches a show alongside the viewer and speaks it in English in real time. It assigns a distinct synthetic voice to each character, keeps that voice consistent for the whole episode, lowers only the original Japanese dialogue, and leaves music and effects at full level. A side-panel companion character reacts to the episode and answers questions about it out loud.
It runs entirely on the user's own machine. There is no account, nothing is uploaded, and the application never records or decrypts video — it plays the service the user already subscribes to, in that service's own player.
Most of the world's anime is released in Japanese first and dubbed much later, or never. Subtitles work, but reading a screen is not the same as watching it, and viewers who want both the original performance and an understandable one have no good option.
The existing automated approaches have two failures. They lower the entire soundtrack to make the synthetic voice audible, which destroys the thing people actually watch anime for — the score and the sound design. And they lag, because they transcribe, translate and synthesise after the line has already been spoken.
Two insights carry the product.
First, latency is a scheduling problem, not a speed problem. When a show has an official subtitle track, the timing of every line is known in advance. Hoshidub reads ahead, synthesises each line before its cue, and plays it on time — so the delay is zero by construction rather than by optimisation. Shows without a subtitle track fall back to listening and translating, which does lag, and the interface says so instead of pretending otherwise.
Second, only the voices need to move. Rather than ducking the whole mix, the original Japanese dialogue is attenuated by about 15 dB while music and effects stay within roughly 2 dB of where they started. The result is a dub that does not cost you the soundtrack.
The companion character is a constrained language-model application: it is given only what has already played, so it is structurally unable to spoil what is coming. That is an architectural guarantee rather than an instruction in a prompt, which matters, because an instruction can be talked around and a missing context window cannot.
Press one button and every character in a show starts speaking English, in their own voice, exactly on cue, with the music untouched. All of it runs on the machine in front of you. Here is how that is possible.
When a show has an official subtitle track, every line and its exact timing are known in advance. The application reads ahead instead of listening along, which is the whole trick: it is not reacting to speech, it is working from a schedule.
Lines are assigned to distinct synthetic voices, and the assignment holds for the whole episode. A character who sounds different between scenes breaks the illusion faster than a wrong translation, so consistency is treated as a correctness property, not a nice-to-have.
Each line is generated before the moment it is needed and held ready. When the cue arrives the audio is already waiting, so it plays on time. The delay is zero by construction rather than by being fast.
Most dubbing tools lower the whole soundtrack to make room, which costs you the score and the sound design — the things people actually watch anime for. Here the original dialogue is attenuated by about 15 dB while music and effects stay within roughly 2 dB of where they started.
For a show with no subtitle track, the audio is transcribed and translated live. This path does lag, because nothing can transcribe a line before it is spoken — and the interface says so rather than pretending the two paths are the same.
A character in the side panel reacts to the episode and answers questions out loud. It is given only the portion of the episode that has already played, so when asked how it ends, it genuinely does not know.
Application
Speech
Language
Audio
Platform
Before building: never record video, never decrypt a stream, never require an account, never spoil an episode. Those four constraints eliminated most of the obvious architectures and left a defensible one.
The zero-delay path only exists where an official subtitle track does. Rather than treating that as a limitation, I made it the primary path and the listen-and-translate route the documented fallback.
The released version is labelled 0.1.0, with the GPU requirement stated up front rather than discovered after installation. Overstating the hardware story is how this class of application loses the users it has just acquired.
Pre-synthesising from a known subtitle track gives exact cueing. Listening live does not — speech recognition cannot transcribe a line before it is spoken. Rather than average the two into a vague claim, the product documents both paths separately: zero delay with official subtitles, translate-by-ear without them. Being specific about where a guarantee applies is more valuable than a stronger claim that fails on some shows.
Separating dialogue from music in a finished mix is the whole difficulty. Attenuating everything is trivial and ruins the show. The product's audio targets are explicit — Japanese dialogue down about 15 dB, music and effects within about 2 dB of original — which makes the behaviour testable rather than subjective.
A prompt instruction not to reveal the ending fails the first time a viewer asks cleverly. The design answer was to withhold the information entirely: the companion's context contains only what has already played. When asked how the episode ends it genuinely does not know. The distinction matters: a policy can be argued with, a missing context window cannot.
A tool in this category can very easily become a piracy tool. The posture is deliberate and stated publicly: it plays the service the user pays for, inside that service's player, and listens to the audio the way a person in the room would. Nothing is captured, stored or decrypted. The product's legal position is a design output, not a disclaimer.
Measured offset between a subtitle cue and the start of the synthesised line on the zero-delay path.
Level measurement confirming dialogue attenuation and that music and effects stay inside the stated tolerance.
A character keeps one voice for a whole episode; drift between scenes is a defect, not a variation.
Adversarial questioning of the companion about events later in the episode, verifying the context boundary holds.
This project is publicly inspectable. The links below are the demonstration — a live page you can open is stronger evidence than a screenshot I could have made in any image editor.
Source code is not currently published. Publishing this portfolio repository and making my work reviewable is step 2 on my roadmap, and it is a hiring problem before it is a technical one. I am happy to walk through any of this code in a screen share.