Reachy Conv

git clone https://huggingface.co/spaces/ArtFix0/reachy_conv_app
cd reachy_conv_app
pip install -e .

# In another terminal:
reachy-mini-daemon --sim

# Back in the first:
python -m reachy_conv_app
→ dashboard at http://localhost:7860/

All runtime settings live in config.yaml at the project root. The dashboard edits this file in place and the conversation loop hot-reloads it on every save — no restart needed. A documented config.example.yaml is checked in for reference.

# llm
llm:
  base_url: http://localhost:11434
  model: llama3.1:8b-instruct-q4_K_M
  api_format: ollama              # or "openai" for OpenAI-compat endpoints
  system_prompt: "You are Reachy Mini, a small expressive desk robot..."
  temperature: 0.7
  max_tokens: 600                 # reasoning models need room for thought + answer
  think: false                    # set true to allow chain-of-thought on qwen3.5/gpt-oss
  num_ctx: 4096                   # context window; raise for longer conversations
  keep_alive: 5m                  # how long Ollama keeps the model in VRAM ("-1" = never)
  timeout_s: 300.0                # HTTP timeout per chat call (cold loads can take 60-120s)

# stt
stt:
  engine: parakeet                # or "whisper"
  parakeet:
    variant: v3                   # v2 (EN-only) / v3 (multilingual) / v3sq / canary
  whisper:
    model: small                  # tiny / base / small / medium / large-v3
    device: cpu                   # or "cuda"

# tts
tts:
  engine: edge                    # edge (default) / silero / qwen (opt-in)
  voice: en-US-AriaNeural         # ~300 edge voices; silero en_0…en_118
  rate: "+0%"                     # -50%…+100% (edge-tts only)
  volume: "+0%"                   # -50%…+50%
  pitch: "+0Hz"                   # -20Hz…+20Hz
  strip_emoji: true               # strip pictographs from LLM reply before TTS

# vad
vad:
  engine: smolvad                 # "smolvad" (silero-vad ONNX) or "silero"
  threshold: 0.5
  min_speech_ms: 250
  min_silence_ms: 300             # max silence window for short utterances
  min_silence_floor_ms: 300       # floor for long utterances (adaptive mode)
  silence_curve_knee_ms: 1500     # speech length (ms) at which we reach the floor
  silence_curve: adaptive         # "adaptive" (window shrinks) or "off" (fixed)

# conversation
conversation:
  mode: ptt                       # "ptt" (antenna) or "vad" (always-on)
  ptt_antenna: right              # "right" or "left"
  max_listen_seconds: 30.0
  interrupt_on_touch: true
  wake_word: ''                   # reserved for future use

# motion
motion:
  enabled: true                   # false = breathing + head-wobble only
  reply_overrides_enabled: true   # false = no post-TTS punctuation gestures
  emotion_map:                    # state → (move_name, loop?)
    wake:      laughing2
    listening: breathing
    thinking:  laughing2
    question:  inquiring1
    affirm:    yes1
    deny:      no1
    sad:       sad1
    cheerful:  cheerful1
    sleep:     tired1

# dashboard
dashboard:
  host: 0.0.0.0
  port: 7860
  autostart_loop: false

Single FastAPI server in a daemon thread on the robot. Static assets are version-stamped (/static/app.js?v=<version>) so the browser is forced to fetch the new bundle after every reinstall.

1. Mic pump — A thread in main.py reads 16 kHz PCM frames from the Reachy daemon's media manager and feeds them to LocalConversationHandler.receive().
2. Handler — LocalConversationHandler (local_conversation.py) implements the fastrtc.AsyncStreamHandler interface, but the local loop bypasses fastrtc.Stream — mic samples go straight in.
3. VAD — vad/backends.py consumes each frame, accumulates 512-sample chunks, and runs the silero-vad ONNX model. Its state machine implements an adaptive silence window: short utterances get the full min_silence_ms, long utterances get the min_silence_floor_ms. The transition is linear over silence_curve_knee_ms.
4. STT → LLM → TTS — When the VAD fires utterance_ended=True, the handler flushes the buffered audio to the STT backend, gets the transcript, sends it to the LLM, gets the reply, sends it to the TTS backend, and pushes the resulting audio to the Reachy daemon's speaker.
5. Self-feedback guard — While the robot is speaking, the mic is muted (_speaking_until in local_conversation.py) to prevent the robot from transcribing its own TTS output and replying to itself.
6. Motion loop — Runs MovementManager at 100 Hz. On each conversation state transition (start, arm, VAD start, LLM call, TTS start, TTS end), GestureDispatcher looks up the move name in motion.emotion_map and asks the MovementManager to play it from the pollen-robotics/reachy-mini-emotions-library dataset. State-driven mapping is hot-reloaded on every settings save.
7. Procedural fallback — LISTENING and IDLE use the procedural BreathingMove (5 mm z-sway at 0.1 Hz + ±15° antenna sway at 0.5 Hz) rather than a recorded move — it feels natural without committing to a heavy gesture. THINKING uses a short one-shot (default laughing2, loop forced off).

Click any item to expand the fix.