Realtime

A voice agent on one box

karti-realtime-1 is a complete conversational voice pipeline — listen, think, speak — running end to end on a single NVIDIA DGX Spark. No cluster, no cloud round-trip. It hears you, reasons, and answers entirely offline, on hardware that fits in your hand.

128 GB

Unified memory

One GB10 pool, CPU + GPU

3 models

Co-resident

ASR · LLM · TTS, always warm

~25 tok/s

LLM decode

35B-A3B MoE @ NVFP4, single GPU

1 box

Whole pipeline

No cluster, no cloud call

The cascade · listen → think → speak

Three specialised models, chained and kept warm so there’s no cold-start tax between turns.

Listen

~2 GB resident

Nemotron streaming ASR

FastConformer-RNNT, multilingual across ~40 locales, kept warm for low-latency transcription with automatic language detection.

Think

~66 GB resident

Qwen3.6 35B-A3B · NVFP4

A mixture-of-experts LLM — 35B total, only ~3B active per token — quantized to NVFP4 so it decodes fast while leaving room for the voice models.

Speak

~38 GB resident

Audex 30B-A3B + causal decoder

NVIDIA's Nemotron-Labs Audex, a unified audio-text model, paired with a streaming causal speech decoder — audio starts playing before the full clip finishes generating.

The hard part: memory

a harness that can’t OOM-wedge

Unified memory is the whole trick — and the whole risk. Inference engines happily over-commit the shared pool, and with little swap, hitting 100% doesn’t fail gracefully; the machine thrashes and wedges. So one small orchestrator is the only thing allowed to start or stop a model, and it guarantees the box can never lock up:

Admission control

A model only launches if its declared footprint fits the memory budget given what’s already running, plus a safety margin.

A 1 Hz watchdog

Watches available memory every second and kills the newest model before the box thrashes — a backstop for when a footprint estimate is wrong.

How it compares

Natural conversation wants the round-trip under a second — ideally near the ~300 ms of a human pause. We stand on great tools to get there: LiveKit solves transport and orchestration; NVIDIA builds the models and the datacenter path. This project asks a narrower question: how much of that runs, well, on one local box?

LiveKit Agents

Transport + orchestration

WebRTC transport with a framework for building voice agents
Cascaded ASR → LLM → TTS, model-agnostic — you bring the backends
Custom end-of-utterance model for turn-taking; targets sub-1s round-trips

NVIDIA stack

Riva · ACE · Nemotron · Audex

Production speech microservices and reference agent frameworks
State-of-the-art ASR / TTS / audio models — the ones we build on
Designed to scale out across datacenter GPUs and the cloud

karti-realtime-1

The whole cascade, one boxOurs

ASR + LLM + TTS co-resident on a single GB10 — self-hosted, offline-capable
NVFP4 quantization to fit a full stack in unified memory with headroom
A memory harness that makes an over-committable box impossible to wedge

What’s next

A real benchmark suite

End-to-end voice-to-voice latency, TTS time-to-first-audio, and sustained tok/s under load — measured and published openly.

Tighter turn-taking

Streaming ASR into the LLM into streaming TTS, so the box starts answering while you're still finishing your sentence.

Speech-to-speech

Evaluating unified audio-language models against the cascade — fewer hops, one model, different tradeoffs.

On the numbers. Figures here are early and measured while the box serves all three models concurrently — a deliberately honest case, not an isolated single-model benchmark. A reproducible suite is on the roadmap. The hardware this runs on lives in Compute.