Case study
An ambient AI assistant for the home
A wall-mounted display and Persian-speaking voice assistant that lives in the room instead of on a screen: weather as art, music as color, and a voice that knows my day.
Product designer and owner · Sep–Oct 2026 · Product brief, interaction, motion and voice UX · Raspberry Pi 5, React, realtime voice
Hero photo: the wall display in its frame, weather state
At a glance
Role
Product designer and owner: product brief, decisions, interaction and motion design, voice UX. Designed and built with AI coding tools, with me directing and reviewing every pull request.
Team
Me and AI coding agents.
Timeline
First commit on 22 September 2026, active through October 2026. Hardware bring-up is in progress.
Status
Software works end to end in a desktop preview and an ARM64 virtual machine. Validation on the Raspberry Pi is underway.
Platform
Raspberry Pi 5 kiosk, React and TypeScript UI, a Node backend over WebSocket, a WebGL weather engine, an on-device wake word (openWakeWord), a full-duplex realtime voice model, and a self-hosted agent server (OpenClaw) for memory and actions.
One assistant, many surfaces
Phones demand attention, smart speakers are blind, and “smart displays” are dashboards: ten tiles, all equally loud. I wanted something on the wall that is beautiful when you ignore it, useful at a glance, and conversational when you talk to it, in Persian as people actually speak it.
It had a brain to plug into: a personal assistant I’d built on a self-hosted agent server, with memory, reminders, a morning brief, home control and Telegram. So the real problem was never the screen.
“How does one assistant show up on many surfaces (chat, voice, a wall) and stay the same assistant?”
On the wall · one mode at a time
9 modes
57 decisions
916 tests
19 days
The numbers, written the way the display would show them: one quiet line. Sources in By the numbers.
PRD snapshot
What V1 had to be
Goal
An ambient home computer that never looks like a dashboard, with a voice that shares one memory with the text assistant.
Users
The owner, hands busy, across the room. Guests, who should see nothing personal.
Success criteria
Glanceable in under a second from across the room: one state on screen at a time.
Voice answers to “what’s my day” or “what’s playing” are immediate, with no backend round trip.
The voice and the chat assistant never disagree about what’s true today.
Nothing personal is shown or said when guests are over.
Runs on a 2 GB Raspberry Pi, a budget kept even after a 4 GB board arrived.
Non-goals for V1
Camera, presence and ambient-light sensing. Touch UI on the wall. A chat transcript on screen. Multi-user profiles.
Constraints
Low-power ARM hardware. A model quota shared with my own daily AI use. Free or non-commercial APIs. No public ports, private network only. Persian and English mixed in one sentence.
Challenge 1 of 5
Do not look like a dashboard
Situation
Weather, music, lyrics, voice, morning brief, reminders, trip nudges, cards and night mode all want permanent real estate. That is how smart displays become tile grids.
Options
Tiles. A carousel. Or one exclusive full-screen state with a priority order.
Decision
One state at a time (D4), resolved by a pure state machine with a fixed priority. Interrupts win; everything falls back to weather.
I wrote the principles down: ambient first, utility second; voice is a mode, not a chat window.
Priority order, highest first · D4
01 Brief
02 Voice
03 Card
04 Night
05 Camera
06 Lyrics
07 Music
08 Weather
On screen now
Nothing else wants the wall, so it shows the sky.
1 of 9 full-screen states · everything falls back here
A stack, not a grid. Anything higher in the deck interrupts anything lower, and only the front screen is ever drawn.
Outcome
Nine full-screen states, and the state logic is a pure, unit-tested function.
State-machine diagram: the 9 states, interrupts and the priority order
Every state, and what is allowed to interrupt it.
Challenge 2 of 5
Voice that is fast, cheap and actually remembers
Situation
A full-duplex realtime voice model handles the conversation, but memory, reminders and tools live in the agent server. Its model quota is shared with my own use. On 24 September it hit its limit, and replies failed.
Why it was hard
Routing everything to the agent is slow, and silence feels broken in voice. It also burns quota. Routing nothing makes the voice a stranger.
Function tools were not an option: the voice model’s delegation channel carries no task text.
Decision
Four layers (D48), ordered by how close the answer already is. Weather and music come from the display’s own state (D43, D44), and filler lines cover the waits.
The question
Spoken to the wall.
L1 Identity and tone
No model call
A file in the agent’s workspace, re-read only when its timestamp changes.
L2 Today
No model call
The “today” snapshot the chat assistant writes every 5 minutes.
L3 Memory search
Search only
Only for a strictly defined pure recall question.
L4 Agent
Agent call
Actions, and everything else.
The line gets longer the further an answer has to travel. Three of four layers answer without the agent.
Outcome
Day, weather and music answers are instant; the agent runs only when something has to happen. The bilingual recall classifier is tested on edge cases: “who directed…” must not hit personal memory.
Challenge 3 of 5
Persian voice: wake word, turn-taking and colloquial speech
Situation
Off-the-shelf wake-word engines had no Persian, and Persian voice etiquette doesn’t map onto English phrase lists.
Decision
I trained a custom “Hey Ari” wake word (D27) on synthetic clips in 13 voices, with near-miss negatives such as “hey Siri” and “hey Harry”.
Turn-taking
A conversation ends on 15 seconds of silence, a goodbye in either language, or 5 minutes (D30).
A goodbye said while the agent is working is held, and «صبر کن، یه چیز دیگه» (“wait, one more thing”) takes it back (D37). An evening “good night” also puts the display to sleep (D41).
One conversation, start to end
Illustration, not a recording
Hey Ari
15 s of silence
Ends on
15 seconds of silence, a goodbye in either language, or 5 minutes (D30).
Held
A goodbye said while the agent is still working waits for the answer.
Taken back
Saying «صبر کن، یه چیز دیگه» (“wait, one more thing”) cancels the goodbye (D37).
Wake word (D27)
Trained on synthetic clips in 13 voices. Ignores near misses like “hey Siri” and “hey Harry”.
Outcome
An on-device wake word, and endings that behave the way people talk. These rules came from living with it, not from a spec.
Wake-word training pipeline: synthetic voices, near-miss negatives, on-device model
Trained on synthetic speech, tested against the names it must ignore.
The voice screen
While the assistant listens, thinks and speaks, the screen shows one orb and no transcript. The orb changes color and motion with each phase, so you can tell what it is doing from across the room.
Listening: the orb swells with the level of your voice. Demo data.
Speaking: the same orb, with a different color and motion. Demo data.
Challenge 4 of 5
Privacy in a shared room
Situation
The display lives in a shared space, and it knows a lot.
Guest mode
Saying «مهمون داریم» (“we have guests”) or flipping a switch on the phone hides everything personal and stops memory search (D40). It turns itself off at 03:00.
Guardrails
The mic sleeps from 01:00 to 08:00 (D36). The display’s agent credential is never admin (D29). Phone control is private-network only, with no open ports (D17). Personal context is held in memory and never logged.
Twenty-four hours in the room
01:00 → 08:00 mic asleep (D36)
03:00 guest mode switches itself off (D40)
00:00
06:00
12:00
18:00
Privacy runs on the room’s clock, not on a settings page.
Guest mode off: the everyday screen. Demo data.
Guest mode on: the same moment, nothing personal. Demo data.
Outcome
Privacy is a mode you switch on by voice in one sentence, not a settings page.
Challenge 5 of 5
Motion and color that read from across the room
Situation
Music and weather had to feel alive without becoming noise.
Music
Music is only a circular cover, rotating every 30 seconds with a 24 px, 440 ms blur crossfade (D20), over a wash of the album’s main colors, darkened to keep 7:1 text contrast (D42).
Lyrics
One line at a time, crossfading 300 ms before the cue (D21). The music API is polled every 15 seconds, so playback is projected locally for up to 20 seconds to stay in sync (D22).
Weather
WebGL with the real sun, moon phase and stars for the city, with a CSS fallback (D38). Reduced motion is respected throughout.
Now playing · lyrics
One line at a time.
The next line is already fading in.
30 s
The round cover turns (D20)
24 px · 440 ms
Blur crossfade between covers (D20)
300 ms
Lyric fades in before its cue (D21)
15 s → 20 s
API poll, then local projection to stay in sync (D22)
7:1
Text contrast floor on the color wash (D42)
The music screen, drawn as its own spec sheet.
Music and lyrics on the wall
While a song plays, the screen shows only the cover. If synced lyrics exist, the current line takes over and sits centered over colors taken from the album.
Now playing: only the cover, nothing else on screen. Demo data.
Lyrics: the current line, a public-domain verse. Demo data.
Recording: weather, day to night
Recording: cover rotation and color wash
Recording: lyric crossfade, non-copyrighted text
Recording: voice orb, listening, thinking, speaking
Screen recordings with placeholder data.
Design decisions and system
A decision log as the system’s spine
Decision log
57 numbered decisions, each Locked, Provisional or Rejected, with a date and a rationale, and a master brief that went through 36 versions. Each change to a locked decision names what it amends.
Six of 57, pinned
D4
One state at a time
A pure state machine with a fixed priority decides what is on the wall.
D20
The cover turns
Every 30 seconds, with a 24 px, 440 ms blur crossfade.
D27
«Hey Ari»
A custom wake word, trained on synthetic clips in 13 voices.
D29
Never admin
The display’s agent credential is never an admin one.
D40
Guest mode
One sentence hides everything personal and stops memory search.
D48
Four voice layers
Files first, the agent last.
Each entry in the log carries an ID, a status (locked, provisional or rejected), a date and the reason. A change to a locked decision names what it amends.
Information architecture
The wall is ambient output. Voice is the primary input. A phone control page is the settings: city, quiet hours, the guest switch, music pairing. No touch on the wall.
Typography
Vazirmatn for Persian, Inter for English, chosen per line so mixed-language lyrics and cards render correctly right to left and left to right.
Voice screen
No transcript. Just an orb with listening, thinking and speaking looks.
One memory, many surfaces
One source of truth for reminders and automations (D5). The voice reads the same “today” snapshot the chat assistant writes every 5 minutes, and finished voice conversations are written back as text, so the chat side knows what was said on the wall.
Edge cases
A 5-second pause grace, lyric intros and outros, failed reads that keep the last good copy, a brief that interrupts a conversation (D26), and “couldn’t check” only when every layer fails. Never guess.
Evidence
By the numbers
smart-display · main · counted 11 Oct 2026
$ git log --oneline | wc -l
420
commits on main
$ gh pr list --state all
51
pull requests, 50 merged
$ git log --format=%ad
19
days, 22 Sep → 10 Oct 2026, with commits on 12
→ docs/decisions.md
57
numbered product decisions, D1–D57
→ docs/
36
versions of the brief, v0.3 → v0.38 (2,093 lines)
→ app/src/states/
9
display states in stateMachine.ts
$ grep test( / it( / def test_
916
test cases: 907 JavaScript, 9 Python
$ wc -l app/ server/ wakeword/
~16.9k
lines of source, excluding tests
→ D42
7:1
contrast floor for lyrics and music text
Counted from the project’s repository on 11 October 2026. Still to measure on the Pi: wake-word false accepts and rejects, time to first word per layer, the share of answers with no agent call, and WebGL frame rate.
What I learned
A decision log beats a spec for a living product. Fifty-seven dated decisions let me and the AI agents change things without losing the why.
Latency is the voice UX. The biggest win was moving answers closer: files, local state, search, then the agent.
Privacy has to be a gesture, not a setting.
Next: finish validation on the Pi, measure the numbers above, keep or drop WebGL weather by its real frame rate, then presence sensing in V2.