local-voice-ai

Local Voice Agent

A private, low-latency voice assistant that runs on your hardware.

Powered by LiveKit Agents.

Local Voice Agent combines speech recognition, a language model, and speech generation in one supervised application. It selects a model stack that fits the available hardware and memory.

[!TIP] The Jetson profile supports real-time voice conversations on a Jetson Orin Nano.

The application includes:

• A browser voice interface.
• Local streaming speech recognition with Nemotron Q8.
• Local language models through llama.cpp.
• Local speech generation with Kokoro.
• Automatic setup for CPU, NVIDIA, Apple Silicon, and Jetson.
• A remote-client mode for devices that run without a local browser.

Requirements

Clone this repository before you start:

git clone https://github.com/ShayneP/local-voice-ai.git
cd local-voice-ai

The setup launcher needs Python 3.10 or later.

Install the additional tools for your platform:

On Apple Silicon, install the native server tools with Homebrew:

brew install livekit llama.cpp
uv sync --extra ml --extra dev

The first start needs an internet connection. Later starts reuse downloaded model files and native components. Docker also reuses its image layers.

Quick start

Start the setup launcher:

python3 run.py

The launcher shows the detected hardware, memory budget, and recommended models. Accept the recommendation or select a different profile.

When the application is ready, open . When the browser requests microphone access, permit it.

For a non-interactive start, run:

python3 run.py start --profile auto --yes

Model profiles

Automatic selection uses the device type to select a runtime. It then uses the memory budget to select a model profile.

The memory values are planning targets, not hard limits. The automatic mode keeps memory available for the operating system and active conversations.

All profiles use the native streaming Nemotron Q8 runtime. The launcher selects the CPU, CUDA, or Metal runtime for the device.

The default language is English. For English, the application uses the English-specific Nemotron model. For another supported language, it uses Nemotron 3.5.

Set the language in .env.local:

STT_LANGUAGE=fr-FR

If the speaker language can change, use STT_LANGUAGE=auto. This value selects the multilingual model. Whisper remains available as a manual fallback:

STT_PROVIDER=whisper

Whisper waits for a complete utterance before transcription. Nemotron sends partial transcripts while the user speaks, so Nemotron has lower voice latency.

To set a memory budget, use --memory-gb:

python3 run.py start --profile auto --memory-gb 5.5 --yes

Use a Jetson as the voice server

The recommended Jetson setup runs the voice stack on the Jetson and the browser interface on a laptop. This gives the browser a localhost address for microphone access.

The Jetson setup needs approximately 29 GB of free disk space. The first build compiles native components, so it takes longer than later builds.

1. Configure the Jetson address

On the Jetson, find its LAN address:

ip -4 -brief address

Create .env.local in the repository root. Replace the example address with the Jetson address:

LIVEKIT_URL=ws://192.168.1.40:7880
LIVEKIT_NODE_IP=192.168.1.40
MANAGE_LIVEKIT=1

2. Permit local network traffic

The laptop needs these ports on the Jetson:

If UFW is active, permit only the local subnet. Replace the example subnet with your local subnet:

sudo ufw status
sudo ufw allow proto tcp from 192.168.1.0/24 to any port 7880,7881,8080 comment 'local voice ai'
sudo ufw allow proto udp from 192.168.1.0/24 to any port 7882 comment 'local voice ai media'

CAUTION: Do not expose these ports to the public internet. The default service uses development credentials.

3. Start the Jetson

python3 run.py start --profile auto --memory-gb 5.5 --yes

Wait until the launcher reports that all services are ready.

4. Start the laptop client

Install Node.js 20 on the laptop. Then run:

git clone https://github.com/ShayneP/local-voice-ai.git
cd local-voice-ai
corepack enable
python3 run.py client --server 192.168.1.40

Open . The client installs its frontend packages on the first start.

Common commands

The launcher saves the selected profile in .local-voice-ai.toml. This file is local to the device and is not committed to Git.

Configuration

Put device-specific configuration in .env.local. This file overrides the selected profile and the defaults in .env.

Common values include:

See .env for the complete list.

Use an external service

Set a remote base URL to replace one local service. The supervisor does not start the matching local process.

Store API keys in .env.local. Do not commit this file.

Troubleshooting

A model shows several gigabytes during startup

The startup value is the model cache size on disk. It is not the memory used by the process.

The laptop cannot connect to the Jetson

From the laptop, request the Jetson status:

curl -fsS http://192.168.1.40:8080/api/status | python3 -m json.tool

If this command times out, make sure that the firewall permits the laptop subnet.

The interface connects without audio

Make sure that UDP port 7882 is open between the laptop and the Jetson.

A service does not become ready

Show the current status and logs:

python3 run.py status
python3 run.py logs

Local development

Local development needs Python 3.11–3.13, uv, Node.js 20, pnpm, livekit-server, and llama-server.

Install the Python environment:

uv sync --extra ml --extra dev
.venv/bin/python -m local_voice_ai.agent download-files

Start the application:

.venv/bin/python -m local_voice_ai serve

This reads .env.local, then the saved profile in .local-voice-ai.toml, then .env, so it starts with the same settings python3 run.py would use.

Serve it to other computers

serve binds the web port to every interface, but it tells browsers to connect to LiveKit on loopback, which no other machine can reach. Name the address they should use instead:

# .env.local
LIVEKIT_PUBLIC_URL=ws://192.168.1.40:7880

That is the only variable needed: the ICE address follows it, and LiveKit is still started here because LIVEKIT_URL remains on loopback. Set LIVEKIT_URL itself only to use a LiveKit you run elsewhere, such as LiveKit Cloud.

serve does not host the web interface, so connect from the other computer with python3 run.py client --server 192.168.1.40, which needs Node.js and pnpm there but not Docker.

If you change the frontend, start its development server in another terminal:

corepack enable
pnpm --dir frontend install --frozen-lockfile
pnpm --dir frontend dev

Run the automated tests:

.venv/bin/python -m pytest -q
pnpm --dir frontend build

Security

The default configuration is for local development and trusted private networks. It does not provide authentication for local model endpoints.

• Keep .env.local out of Git.
• Limit firewall rules to the local subnet.
• Do not publish the LiveKit or model ports directly to the internet.
• Use authentication and TLS before you expose the application through a public service.

Credits

• LiveKit
• LiveKit Agents
• NVIDIA Nemotron Speech
• NVIDIA Nemotron 3.5 ASR
• NVIDIA NeMo-Speech.cpp
• llama.cpp
• Gemma 4
• Kokoro
• Kokoro ONNX
• faster-whisper

Questions and feature requests are welcome through GitHub Issues.