# Workstation voice for Relay

Luna's voice runs on your own machine. The SVEN cluster has no GPU, so the
speech models (Parakeet for recognition, Kokoro for synthesis) run locally. Luna's
reasoning still runs in the Relay backend, under your own account and model
connection. Only audio-to-text and text-to-audio happen here.

This is the development setup. Hosted realtime voice models will replace it later.
Until then, keep Relay's voice flows working on these small local models; they
will then also work on stronger hosted ones.

The guide is written so that a coding agent (Codex, Claude Code) can follow it on a
colleague's machine. Each step ends with a check.

## What you get

```
browser (relay.atomisk.ai or localhost:5180)
   └─ speech broker  127.0.0.1:5180/__relay/speech/*   scripts/speech-broker.mjs
        └─ voice worker  127.0.0.1:18041               voice-worker/serve.py
             Parakeet v3 (speech to text) · Kokoro 82M (text to speech, ONNX)
```

- Everything binds to loopback, and the broker rejects foreign origins.
- The worker's bearer token lives only in the broker's memory.
- No audio, transcript or prompt is written to disk or browser storage.

## Requirements

- Linux x86_64, or Windows with WSL2. macOS is untested: the pinned CPU wheels
  (`torch==2.11.0+cpu`) do not exist there.
- `git`, `python3` (any 3.x, used only to bootstrap `uv`), Node 24 and pnpm.
- About 10 GB of disk: the Python environments, plus the Parakeet and Kokoro weights
  in `~/.cache/huggingface` and `~/.cache/relay-conversation`.
- An NVIDIA GPU is optional:
  - Without one, synthesis runs on the CPU. That works, but replies start more slowly.
  - With one, you need a driver that supports CUDA 12.8. Synthesis then takes about
    1 GB of VRAM, and Kokoro answers in roughly 130 ms on an RTX 2080.
- A headset is recommended. The browser's echo cancellation helps, but speakers can
  still feed Luna her own voice.

## Setup

Run these from a clone of `atomisk-ai/relay-spatial`.

1. **Install the Node dependencies.**
   ```bash
   pnpm install
   ```
   *Check:* the command exits 0.

2. **Install the speech worker (CPU).**
   ```bash
   bash scripts/setup-speech.sh
   ```
   - It pins and clones Hugging Face `speech-to-speech` into `.tools/`.
   - It installs Python 3.12.15 through `uv`, and builds `voice-worker/.venv312` with
     CPU torch.

   *Check:* the script prints "Speech worker installed", and
   `voice-worker/.venv312/bin/python` exists.

3. **Optional: GPU synthesis.**
   ```bash
   bash scripts/setup-conversation-gpu.sh
   ```
   - It downloads the checksummed Kokoro ONNX files into `~/.cache/relay-conversation`.
   - It builds `voice-worker/.venv-rt` with `onnxruntime-gpu` and the CUDA 12.8
     libraries.
   - It also downloads an offline Qwen model and a llama.cpp runtime (about 3 GB).
     They stay unused unless you set `RELAY_CONVERSATION=1`; leave that unset.

   *Check:* `voice-worker/.venv-rt/bin/python -c "import onnxruntime as o; print(o.get_available_providers())"`
   lists `CUDAExecutionProvider`.

4. **Build a frontend to serve.** This step is needed only until the hosted app can
   use the broker (see below).
   ```bash
   bash scripts/bootstrap.sh && bash scripts/bootstrap-trunk.sh   # Rust toolchain and Trunk
   (cd next && CARGO_BUILD_JOBS=4 ../.local-tools/trunk build)    # Relay Next, ~4 min cold
   ```
   *Check:* `next/dist/index.html` exists.

5. **Start the adapter. It owns the broker and the worker.**
   ```bash
   RELAY_PORT=5180 RELAY_DIST="$PWD/next/dist" \
   RELAY_INTEGRATION_ORIGIN=https://relay-integration.atomisk.ai \
   node scripts/serve.mjs
   ```
   - Keep **port 5180**. Google sign-in is registered for `http://localhost:5180`,
     and the hosted app will look for the broker there.
   - It proxies `/api` to `RELAY_BACKEND_ORIGIN` (default `https://relay.atomisk.ai`).
   - `RELAY_INTEGRATION_ORIGIN` sends the organization API (`/api/v1/relay/*`, used
     by the Agent fleet) straight to the IntegrationService. Production's router only
     gains that route with RELAY-11572.
   - It starts the worker with `voice-worker/.venv-rt` when that exists, and with
     `.venv312` otherwise.

   *Check:* the worker logs to this terminal. The first start takes 10 to 30 seconds
   while the weights load.

6. **Try it.**
   - Open http://localhost:5180 and sign in with Google.
   - Hold **Space** and say "bring up engineering chief", or press **/** and type it
     to Luna.

   *Check:* Luna turns to the Agent fleet, opens the briefing and answers aloud. The
   browser console shows `luna streaming: speaking a sentence`.

## Using it from relay.atomisk.ai

Ticket RELAY-11572, stage 2, adds this:
- The hosted app probes `http://localhost:5180/__relay/speech/config` when you opt
  in to workstation voice. The opt-in is stored in your UiSettings.
- Chrome asks once to allow local network access; allow it.
- If no broker answers, the hosted app stays text-only and links to this guide.

Until that ships, use the local frontend at http://localhost:5180 (steps 4 to 6).
It talks to the same production backend.

## Tuning and troubleshooting

| Symptom | Fix |
| --- | --- |
| GPU memory is tight (a game, a build, another model) | `RELAY_SPEECH_DEVICE=cpu node scripts/serve.mjs` moves Kokoro to the CPU |
| Port 18041 is taken | `RELAY_SPEECH_PORT=<free port> node scripts/serve.mjs` |
| Luna answers in text but never speaks | Check the worker output in the adapter's terminal. A missing `.venv312` or `.venv-rt` means step 2 did not finish |
| Recognition is slow during heavy compiles | Expected: Parakeet runs on the CPU. Pause the build, or keep it bounded with `nice` and `CARGO_BUILD_JOBS=4` |
| The voice sounds metallic | Leave `RELAY_SPEECH_TONE` unset. `ship` is the experimental effect |
| Non-English replies | Kokoro v1 speaks English only. Recognition (Parakeet v3) is multilingual |

## Rules for agents doing this setup

- Do not commit `.tools/`, `voice-worker/.venv*` or anything under `~/.cache`; they
  are gitignored.
- Never expose the broker or the worker beyond loopback, and never put tokens in
  files or browser storage.
- Do not enable `RELAY_CONVERSATION=1` unless the person asks for the offline Qwen
  experiment.
- More background lives in `docs/VOICE-V1.md`: the protocol, measurements and
  provenance. Pins live in `voice-worker/provenance.json` and
  `voice-worker/conversation-provenance.json`.