Runs on your siliconOwned
A whole inference cluster — ~14 language models plus image, video, music and speech — on one desktop-class box. No model provider ever sees a prompt.
← all projects
project deep-dive
GaudeauAI
Every model — language, vision, voice, music, and code — runs locally on a single NVIDIA DGX Spark. No prompt, document, or memory is ever sent to a third-party model provider. Reachable from an iPhone, a Mac, and CarPlay over a private mesh.
The one exception, stated plainly: web search calls an external API (Brave) — deliberately isolated behind a server-side proxy so the assistant's sandbox can never even see the key. Everything else is inference on the box.
121 GB unified memory~14 language models120B-class models on one box128K assistant context5 generative studios~250 news items/day enriched8 Xcode targets · iOS · Mac · CarPlay848 backend tests green0 cloud LLM calls
At a glance · The premise
Owned. Private. Complete.
Reachable only by youPrivate
Devices connect over a private WireGuard mesh. Raw model ports are loopback-only; every API call is bearer-authenticated behind one choke point.
A platform, not a chatbotComplete
Chat, a tool-using agent, agentic coding, generative studios, RAG project workspaces, a news-intelligence pipeline, voice, and CarPlay — one integrated system.
Live service status
| Service | Detail | Status |
|---|---|---|
| llama · qwen35 | Tier-1 · Qwen3 35B MoE · always-on | active |
| llama · fleet | 14 models · on-demand | idle |
| embedding | RAG vectors · always-on | active |
| kokoro-tts | neural speech | active |
| comfyui | FLUX.2 · SDXL · video | active |
| acestep | music generation | active |
| nemo · pod | openclaw agent · k3s | active |
| gaudeau-api | orchestrator · Node/SQLite | active |
Subsystem · The model fleet
Fourteen language models, plus a full generative stack — all local.
Each language model is its own llama.cpp service on a dedicated port. A memory-aware router loads them on demand, evicts idle ones, and keeps the assistant model permanently hot — treating 121 GB of shared memory as a live budget across LLMs, diffusion, video, music, and speech.
Language models
| Model | Role | Params · active | Quant · memory | Lifecycle |
|---|---|---|---|---|
| Qwen3.6-35B-A3B — Tier-1, the brain behind Nemo | Primary assistant & premium chat | 35B · 3B MoE | Q4_K_M · ~28 GB | always-on |
| Mistral-Small-3.2-24B (+ vision projector) | Workhorse — voice, translation, enrichment | 24B | Q4_K_M · ~21 GB | on-demand |
| Qwen3-Coder-Next | Coding mode & the Forge agent | 79B · 3B MoE | Q4_K_M · ~54 GB | on-demand |
| Qwen3-30B-A3B | Deep Research / reasoning | 30B · 3B MoE | Q4_K_M · ~26 GB | on-demand |
| Qwen3-VL-30B-A3B | Vision — OCR, receipts, describe-image | 30B · 3B MoE | Q4_K_XL · ~22 GB | on-demand |
| Qwen3.5-122B-A10B — the largest LLM | Premium — maximum quality | 122B · 10B MoE | Q4_K_M · ~86 GB | manual |
| gpt-oss-120B | Open 120B alternative | 120B | MXFP4 · ~70 GB | manual |
| Gemma-3-27B-it | Mid-size general | 27B | Q4_K_M · ~20 GB | manual |
| Cydonia-24B | Creative / general | 24B | Q4_K_M · ~18 GB | manual |
| LFM2-8B-A1B | Scientific & factual specialist | 8B · 1B MoE | Q4_K_M · ~8 GB | on-demand |
| Gemma-3n-E4B | Quick answers | E4B | Q4_K_M · ~12 GB | on-demand |
| Gemma-3n-E2B | Educational / lightest | E2B | Q4_K_M · ~8 GB | on-demand |
| EmbeddingGemma-300M | RAG vectors — 768-dim embeddings | 300M | Q8_0 · ~1 GB | always-on |
| Gemma-3n-E2B — on-device (Apple MLX) | Offline fallback — runs on the iPhone | E2B | 4-bit · ~1.5 GB | on phone |
Model identity is verified by readlink, never filename — the Tier-1 path is a stable symlink so ~15 downstream references never churn on a model swap.
Generative media & voice
| Model | Capability | Runtime | Performance |
|---|---|---|---|
| FLUX.2-dev | Flagship text-to-image (fp8, 35.5 GB DiT + Mistral text encoder) | ComfyUI v0.27 | ~75s / 1024px |
| SDXL 1.0 (+ refiner, ControlNet, IP-Adapter) | Fast image · img2img · inpaint · pose-match | ComfyUI | ~14s / image |
| SUPIR-v0Q | Gold-standard photo restore & upscale (2–4×) | ComfyUI + KJNodes | 1024→2048 ~316s |
| LTX-2.3-22B | Fast video — t2v / i2v / HQ 2× upscale (≤201 frames) | ComfyUI | ~46 GB cold-load |
| Wan 2.2 TI2V-5B | Balanced text/image-to-video | ComfyUI | ~96s / test clip |
| HunyuanVideo 1.5 | Highest-quality 720p video (t2v / i2v) | ComfyUI | 1280×720 |
| ACE-Step v1.5 Turbo | AI music — 320 kbps, reproducible seeds, vocals | uvicorn service | ~10.8s / 10s track |
| Kokoro-82M | Neural text-to-speech — 27 voices, streamed MP3 | FastAPI service | ~0.8s to first audio |
| Whisper.cpp (ggml-medium.en) | Speech-to-text — podcast/video transcription & ad-trim | DGX GPU | 45-min ep ~1–2 min |
ComfyUI runs pinned at v0.27 with restored DynamicVRAM (a one-line comfy-aimdo bump that ended a five-crash bring-up saga) and 14 SHA-pinned custom-node packs — FLUX.2 got ~3.4× faster in the upgrade. Voice-mode STT is separate: it runs on-device with Apple's Speech framework, never leaving the phone.
Subsystem · Inference & memory
Running a whole fleet on one box is a systems problem before it's an AI problem.
On unified memory the CPU and GPU draw from the same 121 GB pool — cgroup limits are cosmetic and torch never sees an OOM, so the kernel starves first. Everything that allocates has to coordinate through one shared budget. This is the layer that makes 120B-class models coexist without the box hard-resetting.
A cross-service memory ledgerAdmission control
One reservation surface both the LLM router and the image/video guard read and write, so a model cold-load and a FLUX.2 render can't each independently pass a free-memory check and then both allocate. Each side registers its in-flight loads and can ask the other to evict idle work. This is the fix that closed the box's first-ever hard crash.
An allocator-aware GPU reaperLive reclamation
After the v0.27 upgrade, model weights moved into GPU memory the container's cgroup can't see — docker stats read 2.7 GB while nvidia-smi showed the same process pinning 51 GB. The reaper now decides on max(RSS, per-PID GPU), so it's correct regardless of which allocator is in play. A 20-second headroom watchdog evicts the largest idle model before any allocation can fail.
Automatic model selectionRouting
Every chat message is scored for topic (coding, scientific, research, vision…) and complexity, then routed to the best-fit specialist — you just type. Coding always lands on the coder; a hard question can escalate to the 122B. Ported from the retired n8n 'Question Analyzer' into native JS.
MoE + quantization by designEconomics
The fleet is built on mixture-of-experts models so nominally huge models cost little active compute — a 122B model activates only 10B parameters per token. Aggressive quantization (Q4_K_M / UD-Q4 / MXFP4) plus statically pre-allocated KV cache means memory budgets are load-time and calibrated from real journal telemetry, not guesses.
Runtime footprint
- Single Node ESM server · ~130 modules
- core.db · 38 SQLite tables
- Separate vectors.db for embeddings
- 13 functional health canaries
- 11-step hash-verified deploys
- 44 days uptime before the first crash
Subsystem · Chat modes
Pick a mode. Press and hold to tune it.
The chat isn't one-size-fits-all. A row of mode pills sits above the message box — tap one and your message routes to the local model best suited to the job; press and hold it to open that mode's Options sheet and dial in length, depth, effort, and tone. No config file, no restart.
| Mode | Routes to | Press-and-hold tunes |
|---|---|---|
| Quick Response | A fast, lightweight model | Conciseness, response length, creativity |
| Coding | The code specialist (Qwen3-Coder-Next) + the Forge agent | Comments on/off, explanation depth, chain-of-thought, output size |
| Deep Research | The analytical reasoning model | Research depth (Surface → Exhaustive), reasoning effort (Low → Max) |
| Premium | The flagship 35B — maximum quality | Creativity, response length |
| Voice Chat | Hands-free, spoken back | Creativity, response length |
| Translate | Mistral-Small 24B + a purpose-built translation prompt | Source/target language, formality, literal-vs-creative |
Every setting persists per mode and per conversation, and syncs across devices — so each mode remembers how you like it. Prefer to steer manually? Pin a specific model to a conversation from the model picker; otherwise each mode auto-selects the best fit.
Translate mode
Multi-language translation is built right into the chat — no separate app, no cloud service, nothing leaving the rack. Tap the Translate pill, press and hold for options, then type or paste; GaudeauAI returns the translation only — no commentary, no echoed source — preserving tone, formatting, and technical terms.
Auto-detect in, anything out32 languages
Leave the source on auto-detect or set it explicitly, then translate into any of 32 languages (English by default) — from Spanish, French and German to Japanese, Arabic, Hindi, Ukrainian, Swahili and Tamil.
Formality + approachDialed to taste
Set the register (Informal, Neutral, Formal) and how faithful the rendering stays (Literal, Natural, Creative) right from the Options sheet.
Runs on your own hardwareLocal · private
Translation runs on a local open-weight model — Mistral-Small-3.2 (24B) on the DGX — driven by a dedicated translation prompt. Every language pair stays on the box.
Subsystem · Nemo
Nemo — a genuine tool-using agent, not a stateless chatbot.
The primary assistant runs the openclaw agent framework inside an isolated Kubernetes pod, driven by the always-hot 35B model. It has a persistent identity, a durable transcript, a long-term memory that survives context overflow, a least-privilege tool sandbox, and DNS egress filtering — engineered end-to-end for prompt-injection resistance.
128K context--ctx 131072
Holds roughly a full novel of conversation before anything is summarized away — GQA + q8 KV cache keep that buffer at just 1.36 GB.
Rotation-with-recall memory
When a session gets long it's distilled into a digest that seeds the fresh session, and chunked + embedded so past exchanges can be recalled later by meaning.
Editable personalitySOUL · AGENTS
Character, self-image, and behavior contracts live in Markdown files re-read every turn — editable from the app, effective on the next message with no restart.
Least-privilege sandbox
The web-search key was pulled out of the agent's sandbox entirely (injected text could have cat'd it) and put behind a rate-limited server proxy. Injection can't exfiltrate what it can't see.
DNS egress allowlist
A dnsmasq sidecar blocks all outbound domains except an explicit allowlist — a compromised agent can't phone home (verified: youtube.com → blocked, wikipedia.org → allowed).
Reminders & recaps via chat
Ask in plain language and Nemo emits a hidden JSON block the server turns into a real scheduled reminder, or a background job that renders a spoken topic recap.
Drafts new skills
Nemo can draft a whole automation from a conversation — but it lands as an inert pending item; every activation gate stays behind your manual tap. It can suggest, never self-install.
CTO Modetransient
A per-turn strategist persona for enterprise-AI-strategy questions — injected at exec time, never written to the identity files; flip it off and the next turn drops it with zero residue.
Self-healing recovery
On a reboot or pod recreation Nemo comes back on its own — container auto-restart, preserved pod, and wiped workspace files restored from an hourly backup. A live 8-minute cold boot recovered with zero manual steps.
Subsystem · Forge CLI
An agentic coder that runs on your box — and that you approve.
A natural-language command drives a real tool-use loop (read, write, edit, run, search, patch) against the local coding model — controllable from an iPhone or Mac, with no cloud coder anywhere. Its defining trick is safety by construction: every run executes in its own throwaway git worktree and nothing touches the real files until you review the diff and tap Apply.
Worktree sandbox
Each task is its own git repo; a run branches into an isolated worktree, works to completion autonomously, and never mutates the task folder until you approve.
Approve-before-apply diff
You get a per-file unified diff with include/exclude toggles — Apply All, apply only the files you picked, or discard the whole worktree.
8 built-in tools
read / write / edit / undo / applyPatch / listFiles / searchCode / runCommand — plus any registered skill exposed as a tool.
Survives disconnect
Background the app or lose the network mid-run and the work keeps executing server-side; reopening replays the real result and re-offers any pending review.
Sandboxed HTML preview
Generated artifacts — including JS canvas games — run live in the app inside a CSP-locked WebView that can't phone home or navigate away.
Admin-gated + audited
Because it holds shell + filesystem power it's the highest rung of the injection threat model — owner-only, home-dir-scoped, with a full JSONL command audit trail.
Four selectable coding backends (Qwen3-Coder-Next default → 122B premium), 24-hour session persistence in SQLite, cross-device sessions shared between iOS and Mac, LSP diagnostics fed back to the model after edits, and fuzzy smart-replace so edits land even when whitespace differs.
Subsystem · Media Workshops
Self-hosted generative studios — image, video, music, documents.
All diffusion runs through one containerized ComfyUI driven by a custom Node WebSocket bridge; music runs on a local ACE-Step server; documents stream from the model fleet. Every generation is background-safe — lock the phone or restart the server mid-render and the finished result still lands in the gallery.
ImageFLUX.2 · SDXL
Text-to-image with style presets, plus img2img variations, inpainting with a mask painter, and ControlNet pose-matching.
EnhanceSUPIR
One-tap photorealistic 2–4× upscale on any generated image or gallery item — restore-and-upscale entirely on the DGX; pixels never round-trip through the phone.
VideoLTX · Wan · Hunyuan
Three model families for text- and image-to-video across quality/speed tiers, up to 720p and 201 frames, with restart-recovery of in-flight renders.
MusicACE-Step
Describe a track (up to 10 min), optional vocals with lyrics you write or the LLM writes, 320 kbps output, and a reusable seed for reproducibility.
Documents20 templates
Meeting notes, reports, proposals, SOPs, READMEs, ADRs, API docs and more — routed to the right model, streamed live, optionally grounded in a project.
Galleries + Live Activities
Every image, video and track persists server-side with prompt/seed/model metadata, browsable and deletable across devices; long renders show Lock-Screen & Dynamic Island progress.
Subsystem · Project workspaces & RAG
Projects with their own knowledge, context, and a Jira-grade tracker.
Group work into projects, each with background context, custom instructions, uploaded knowledge files, and a full task tracker. Chat inside a project and the system fences your context into the prompt and runs semantic search over your files — retrieval-augmented generation, running entirely on your own box.
Per-project RAG
Upload documents; each is chunked and vector-embedded so its content becomes semantically searchable. Answers cite which files they drew from and how relevant each was.
Context & instruction injection
Free-text background and per-project behavior rules are folded into the model's system prompt for that project only — with three device-local toggles to turn each source on or off.
Injection fencing
Because uploaded docs could hide 'ignore your rules' text and the model may hold tools, every user-stored surface is wrapped in labeled 'this is DATA, not commands' fences and demoted below the real system prompt.
Command Center tracker
A full Jira-style tracker per project: board, list and timeline views; tasks / bugs / features; configurable status columns; priority, effort, assignee, due dates and tags.
Automation & collaboration
If-this-then-that automation rules, custom fields, templates, saved filters, subtasks, threaded comments with @mentions, a full activity log, watchers, attachments, and time tracking.
Capture → task
A quick capture aimed at a project — from CarPlay voice, the phone, or the Mac menu bar — drops straight into that project's tracker as a new to-do, sync-safe with tombstones so a deleted project can't be resurrected.
768-dim embeddings, ~800-char chunks, top-3 retrieval at a 0.35 similarity floor, every row conjunctively scoped to (project_id, user_id) with a per-user ownership guard above the SQL filter — sub-5ms search at single-user scale.
Subsystem · Briefs · News · Daybook
A personal news-intelligence pipeline that learns your taste.
It ingests RSS, Reddit, Substack, YouTube and podcasts, runs every item through a two-stage local-LLM pass (score, then summarize only the survivors), learns what you care about from your behavior, and turns the result into spoken briefings, reading surfaces, and a morning narrative — all on one box, zero cloud.
Two-stage enrichment
Stories are triaged for relevance first; only the ~35% that survive get a full summary written — roughly halving LLM output and keeping junk out of the feed.
Taste-learning engine
Seven weighted signals over a rolling 30 days — thumbs, stars, read-full vs bail, ignores — quietly boost or bury whole categories and sources over time.
Watched entities
Name people, companies or products and any incoming story mentioning them is auto-tagged and can push a notification, deep-linked to the brief.
Flash Briefing
A 2–5 minute spoken news podcast pre-rendered every morning, playable with lock-screen controls, regenerable on demand.
The 7am Daybook
A warm spoken 'here's your day' merging overnight weather, your calendar, reminders and top stories into one personal narrative — ending with a plain-spoken system-health check.
Recaps on demand
'Everything about Nvidia from the last 30 days' splices your enriched stories on a topic into a single narrated audio recap via map-reduce synthesis.
Podcasts + ad-trim
Add any podcast by name or URL; episodes get their ad reads transcribed (Whisper) and spliced out automatically, and YouTube items skip sponsors via SponsorBlock.
Drop-a-link ingest
Paste any article URL and it's fetched, cleaned to reader view, summarized, tagged, and saved — behind an SSRF guard and rate limit.
HEARD / READ / SEEN
Every brief is stamped for whether you heard, read, or just saw it, and audio playback position resumes across iPhone, iPad, CarPlay and Mac — the server is the single source of truth.
Subsystem · Clients
One Swift codebase — iPhone, iPad, Mac, and CarPlay.
A single SwiftUI codebase across eight Xcode targets ships as three first-class clients of the DGX backend, plus widgets, Live Activities, Siri intents, and system extensions — sharing view and service code behind #if os() shims.
iPhone & iPad
Five locked tabs
Today, Chats (model-routed with mode toggles), Nemo, Playlists, and Tools — plus a universal Settings gear. Deep links survive lazy tab mounting via a stash-drain router.
Voice Mode
Tap the orb and talk: on-device speech recognition → the 35B model streams a reply → gapless Kokoro speech back, first audio in ~0.5s. Interrupt it just by talking.
On-device offline LLM
When the DGX is unreachable the phone answers from a 4-bit Gemma model running on-device via Apple MLX — with an offline workspace and queued mutations.
macOS · native app
11-section sidebar
A sandboxed, hardened-runtime Mac app sharing ~55 source files with iOS: Today, Chats, Projects, Notes, Nemo, Briefs, Playlists, Workshops, Skills, Health, Settings.
Command panel + ⌘K search
A Spotlight-style global-hotkey panel for quick capture, clipboard AI and drag-in image OCR, plus a ⌘K universal search across briefs, notes, conversations and files.
Full parity
Streaming chat, briefs reader, media workshops with cancel/abort, the Forge coding client, and the voice orb — all sharing server state with the phone.
CarPlay · Widgets · Voice engineering
Full CarPlay app
A five-tab Driving-Task interface — Today, Playlists, Nemo (voice-first), Ideas (capture), More — with a full-screen player, motion-aware read-only-while-driving, and graceful offline degradation.
Audio-exclusivity arbiter
One coordinator guarantees only one thing ever makes sound at a time across 4+ engines — a Nemo reply, a brief, a flash briefing and voice mode can never overlap, in the car or on the phone.
Barge-in detection
While the assistant speaks, a per-turn auto-calibrated VAD listens for you to start talking and stops playback instantly — no hardware echo cancellation needed.
8 widgets + Live Activities
Home- and Lock-Screen widgets (tasks, projects, briefs, quick-capture, next-skill countdown, Nemo, smart home, quick actions), Dynamic Island generation progress, and 6 Siri App Intents.
Share & Notification extensions
Share a URL/text/image into the app from anywhere (images route to OCR or receipt extraction); pushes are enriched in-flight with the real sender and decrypted preview.
Offline-first sync
Pair once with a code, then conversations, projects, captures, notes and settings sync across devices with field-level last-write-wins merges and conflict handling.
Subsystem · Security · Skills · Privacy
A small attack surface, on purpose — and a threat model, not an afterthought.
Every /api/* route sits behind one bearer-auth choke point, the box is reachable only over a private mesh, and the raw model servers are loopback-only. On top runs a permission-gated Skills system where the AI can draft automations but nothing runs until the owner reviews the full source, types a confirmation, and the content hash matches byte-for-byte.
One gate, mesh-only, loopback raw portsPerimeter
A single choke point authenticates every call before any handler runs — a bogus path returns the same 401 as a real one, so the API isn't even mappable unauthenticated. Raw llama.cpp / Kokoro / ComfyUI / embedding ports bind to localhost; access is Tailscale-mesh-only, with a deploy-time verifier that fails the release on any new non-loopback binding.
A privilege ladder for untrusted textInjection doctrine
Before trusting any stored or fetched text, the surface is placed on a ladder of how much power the receiving model holds — Forge (shell+fs, admin-gated) at the top, then tool-enabled chat, then tool-less workshops, then read-aloud. Secrets never sit where a tool-holding model could read them.
Four gates before AI-drafted code executesSkills — approve-before-run
Drafted skills land inert. Approving requires all four: you review the complete handler source on the phone; the approve call must present the exact SHA-256 hash; dangerous permissions demand a typed 'install <id>' phrase; and a static analyzer must find no undeclared capabilities. Rollback re-verifies the on-disk hash so a tampered skill can't be trusted.
Restore-verified backups + disaster runbookResilience
Two nightly schedules capture the whole system with off-site iCloud sync and a multi-stage secrets scrubber that aborts the backup if any known key prefix survives. Nine recovery scripts cover eight failure scenarios; backups have restored deleted media byte-identical.
256-bit device tokens in the iOS Keychain · 8-char ambiguity-free pairing codes with a 30-min expiry · scrypt password hashing with anti-enumeration · per-user data + vector isolation · layered rate limits a patch-wrapper can't bypass · ~37 permission-gated skills with sick-machine health, a real-cron scheduler, an autonomy dial, and semantic search.
By the numbers
Telemetry.
121 GB
Unified memory
14
Language models
122B
Largest model
128K
Assistant context
~250
News items/day
8
Xcode targets
38
SQLite tables
13
Health canaries
848
Backend tests
~2 yr
In development
1
External API · sandboxed
0
Cloud LLM calls
How it's built
Built like a product. Run for one.
A single-tenant deployment architected to multi-user-correct standards, shipped with the rigor of a system that has to stay up — because it does.
Zero-assumption
Docs and comments are leads, not truth. Every claim is verified against the live system before it ships.
Verified deploys
Hash-checked releases, journal boot-scans, and a live probe on every change. Green tests are the floor, not the finish line.
Adversarial review
Changes are reviewed to refute, not to rubber-stamp — the way a real bug gets found before a user does.
Self-hosted personal AI · iOS ⇄ DGX Spark ⇄ Nemo · every token generated on owned hardware.