Tshivhidzo Mbedzi AI & Technology Specialist

Projects Project Luna

Built · not yet proven live AI Systems & Agents October 2026 – present

Project Luna

Autonomous AI presenter and live-stream host

A desktop application in which an animated AI character hosts a live stream unattended — reading chat, choosing topics, reacting to what is on screen and controlling her own performance, with no operator pressing buttons.

My role
Sole builder
Period
October 2026 – present
Category
AI Systems & Agents
  • 0 Operator actions required By design: no buttons during a stream
  • LLM + TTS + ASR Models running locally Language, speech synthesis and audio understanding
  • 2 Platforms integrated Live chat ingestion from two services
  • 0 Public broadcasts Stated plainly — it has not gone live

In short

  • An AI presenter that runs a session unattended, deciding for itself when to speak and when to stay silent.
  • Emergency stop, moderation and per-viewer data removal were built before any audience existed.
  • Built and tested, but it has never run a real public broadcast, and this page says so throughout.

Project overview

Project Luna is an attempt at a genuinely autonomous presenter. A rigged 3D character hosts a stream: she reads the live chat, answers it out loud, opens a new topic when chat goes quiet, reacts to video playing on the virtual set, and selects her own expressions and movements. There is no control surface to operate during a broadcast, which is the point — an AI host that needs a human driver is a puppet, not an agent.

Around it sits the operator tooling you need before you would ever point this at an audience: an emergency stop, moderation with a blocked-term list, the ability to make the system forget a specific viewer, and session recording that produces an event log and a report afterwards.

The problem being addressed

An autonomous presenter is an agent problem wearing a costume. It has to decide what to respond to, when to speak, when to stay quiet, what it is allowed to say, and how to behave when nothing is happening — all in real time, in public, with no undo. Most "AI streamer" projects solve the character and skip the agency, so a person ends up choosing what she reacts to.

It also raises a problem I had not expected to be the hardest one, which is honesty about the audience.

The solution

A director loop that owns the decisions, with the model as one input rather than the whole system. Chat arrives as events; the director decides whether an event deserves a response, what kind, and whether the moment calls for a topic change instead. Personality is a constraint on style, not a free-form instruction, and stiff assistant phrasing is filtered out before it is spoken, because the failure mode of a language model in this role is sounding like a help desk.

Everything runs locally — language, voice and audio understanding — so a broadcast does not depend on a metered API and a viewer's message does not leave the machine. Video awareness comes from transcribing the audio of what is on the set screen and detecting scene cuts, so her reactions are tied to something real rather than invented.

How it works

An animated character hosts a live session on her own. She reads the chat, answers it out loud, notices when the room goes quiet, reacts to what is playing on screen, and picks her own expressions. There is no control panel and nobody driving her.

  1. Everything arrives as an event

    A chat message, a stretch of silence, a scene change in the video on the set screen — all of it enters the system as the same kind of thing: an event with a timestamp. That uniformity is what lets one loop reason about all of them together.

  2. A director loop makes the decisions

    The loop decides whether an event deserves a response at all, what kind, and whether the better move is to change the subject instead. This is deliberately not the language model. The model is one input to the decision, the same as the silence timer, rather than the thing in charge.

  3. The model proposes, a filter disposes

    A language model running locally drafts what she might say. The default register of an instruction-tuned model is a polite assistant, which is precisely wrong for a live host, so a filter rejects assistant phrasing before it ever reaches speech. Personality is a constraint on style, not a long instruction asking the model to be fun.

  4. She can see what is on screen

    Audio from the video playing on the set is transcribed and scene cuts are detected, so her reactions correspond to something that actually happened rather than being invented. A host reacting to nothing is worse than a host saying nothing.

  5. Voice, face and body

    The line is spoken by a local synthesiser, drives the mouth, and is accompanied by an expression and a movement chosen for the moment. The character is rendered against a 3D set, in portrait or landscape.

  6. The stop button existed before the start button

    Emergency stop, moderation with a blocked-term list, and the ability to make the system forget a specific viewer were all built before there was any thought of an audience. A system that speaks in public unsupervised needs its controls first.

Technology stack

Application

  • Electron
  • Node.js

Language

  • Local LLM (Qwen3-4B) on GPU
  • Scripted fallback layer

Speech

  • Kokoro TTS
  • Whisper ASR
  • Loudness-driven lip sync

Presentation

  • VRM avatar
  • Unreal Engine set render
  • 16:9 and 9:16 layouts

Integration

  • Live chat ingestion
  • Moderation
  • DPAPI-encrypted credential storage

System architecture

Project Luna autonomy loop A director loop owns every decision and treats the language model as one input among several. Output passes a register filter before synthesis, and operator safety controls sit outside the loop. Live chat events Silence timer On-screen video ASR + scene cuts Director loop respond · stay silent · open a new topic The model is an input to the decision, not the decision-maker. Local LLM on GPU · no API key Scripted fallback Register filter rejects assistant phrasing Speech synthesis fixed character voice Avatar performance expression · movement · lip sync Rendered 3D set Operator safety, outside the loop emergency stop · moderation · forget a viewer Built before any thought of a real audience.
Project Luna autonomy loop — A director loop owns every decision and treats the language model as one input among several. Output passes a register filter before synthesis, and operator safety controls sit outside the loop.

My responsibilities

  • Designed the director loop that gives the character autonomy over her own performance.
  • Built the local model stack so no broadcast depends on a paid API.
  • Built the video-awareness path so reactions correspond to real on-screen events.
  • Built the operator safety tooling: emergency stop, moderation, per-viewer forget.
  • Built session recording, event logging and post-session reporting.
  • Made the integrity decision described below, and set the default accordingly.

Development process

  1. Autonomy first, polish second

    The director loop was built before the presentation layer, because a beautiful character with no agency is a different and much easier project than the one I wanted to attempt.

  2. Local by default

    Every model runs on the machine. A streamer that stops working when an API key hits its quota, mid-broadcast, is not viable, and viewer messages should not leave the host.

  3. Build the stop button before the start button

    Emergency stop, moderation and viewer-forget exist because this is a system that speaks in public without supervision. The controls came before any thought of a real audience.

  4. Test quietly and sparingly

    Test launches run silenced and with simulated chat behind an explicit environment flag, so a test can never be mistaken for or leak into a broadcast.

Challenges encountered

A simulated-audience feature that was dishonest the moment it was visible

Simulated chat existed so the director loop could be developed without an audience. During a real stream it displayed fabricated viewers. That is not a bug in the ordinary sense — the feature worked exactly as written — it is a feature that was fine as a test harness and became a lie the moment it was on screen in front of real people.

It is now off by default and only enabled by an explicit environment flag that no normal launch sets. The general lesson is the one I keep relearning: a development convenience that fabricates data must be structurally impossible to leave on, not merely switched off.

Making a language model not sound like a language model

The default register of an instruction-tuned model is a polite assistant, which is precisely wrong for a live host. The fix was not a better prompt but a filtering layer that rejects assistant phrasing before it reaches speech, and a personality defined as a constraint on style rather than as a long instruction asking the model to be fun.

An integration that is a licensing liability

Live chat from one platform relies on an unofficial, read-only community library under a strong copyleft licence. It works. It also means that if this application is ever sold, that module must be removed or the whole source published under the same licence. That is recorded as a condition on the project rather than discovered later by someone else.

Built is not proven

The honest status: this works in testing and has never hosted a real public broadcast. The live-chat path for one platform has never been exercised against a real stream at all. I have deliberately not written anything on this page that implies otherwise, because the gap between "works on my machine, unattended, for an hour" and "ran a public stream" is exactly where projects like this fail.

Testing and validation

Per-module unit tests

Each module is runnable and tested in isolation under a timeout, so a hang is a failure rather than a wait.

Simulated-chat harness

The director loop is exercised against synthetic chat behind an explicit flag, which is how autonomy was developed without an audience.

Screenshot verification

Rendered output is captured and inspected rather than assumed, since a layout fault is invisible to a log.

Session reports

Every recorded session produces an event log and a report, which is what makes a run reviewable after the fact.

Evidence and demonstrations

There is no public URL for this project, and there are no screenshots on this page. That is deliberate: a screenshot is trivial to fabricate and I would rather show you a description of the real architecture than an image you have no way to verify. The diagram above is authored from the actual system.

Source code is not currently published. Publishing this portfolio repository and making my work reviewable is step 2 on my roadmap, and it is a hiring problem before it is a technical one. I am happy to walk through any of this code in a screen share.

Current development status

Built, tested and in use

  • Autonomous director loop: chat response, topic initiation, expression and movement selection.
  • Local language, speech-synthesis and speech-recognition stack on the GPU.
  • Video awareness via audio transcription and scene-cut detection.
  • Portrait and landscape stream layouts with a 3D rendered set.
  • Operator tooling: emergency stop, moderation, blocked terms, per-viewer forget.
  • Session recording with event log, report and highlight clip extraction.
  • Encrypted local credential storage.

Not built, or not proven

  • It has never run a real public broadcast. Zero.
  • One platform's live-chat integration is untested against a real live stream.
  • Lip sync follows loudness only; it is not phoneme-accurate.
  • The 3D set is a blockout, not a finished environment.
  • The analytics dashboard and any learning-from-past-streams capability are not built.

Future improvements

  • Run an actual broadcast. Everything else on this list is secondary to that.
  • Replace loudness-driven lip sync with phoneme-aligned visemes.
  • Remove or relicense the copyleft chat module before any commercial use.
  • Score the director loop against a rubric — response appropriateness, silence handling, repetition — rather than judging it by watching.