HomeTeam FeaturesNova ContactAbout Download
VOICE AI

Meet Nova

A fully local voice assistant running entirely on the robot — no cloud, no internet required. Wake it, command it, hear it respond — all in Italian.

"Hey Nova…"

The custom wakeword model (trained on Google Colab via OpenWakeWord) listens 24/7 on the ReSpeaker 2-Mic HAT. Stereo GCC-PHAT estimates the direction of arrival and can orient the robot toward the speaker (±20° deadzone).

PIPELINE

How Nova Works

01

Wakeword Detection

OpenWakeWord ONNX model (hey_no_va.onnx) runs on CPU via ONNX Runtime. 32ms chunks, 16kHz mono. Threshold 0.7, 3s cooldown to prevent re-triggering. Buffer cleared immediately after detection.

hey_no_va.onnx
02

Audio Streaming

Raw PCM16 audio streamed via WebSocket to external server at 100.120.32.86:5000. Pure binary — no headers, no timestamps, no framing metadata. Float32 → int16 LE conversion.

WebSocket · PCM16
03

Whisper STT

faster-whisper (int8 quantized, CPU) transcribes the audio to text. Optimized for Italian. Runs on the external PC to offload the Pi.

faster-whisper · int8
04

LLM via Ollama

Transcribed text sent to Ollama running on the external PC. Model selection flexible — Llama, Mistral, Gemma etc. Response streamed back to Pi.

Ollama · local LLM
05

Piper TTS → Speaker

Response synthesized by Piper TTS with the "Miro" Italian voice (~/AIsburra/piper_voci_italiano/Miro/). Audio played via ReSpeaker output (plughw:CARD=seeed2micvoicec,DEV=0).

Piper TTS · Miro IT
LagmaBills assemblato
FACE ANIMATION

Emotional Expressions

Nova has a Pygame-based face display with 12+ emotional sequences. When the wakeword fires, it switches to SEQ_WAKEWORD (124): a two-phase animation with animated microphone, pulsing sound arcs, and blinking "In ascolto…" text.

Happy

Task completed successfully

Listening

SEQ_WAKEWORD — pulsing arcs

Thinking

Processing command or obstacle

Alert

Cliff / obstacle / emergency stop