This is a submission for Weekend Challenge: Dog Days Edition

What I Built

I went looking for a dog in the GPU family tree. I found fish, dead physicists, and a bridge in Florence.

NVIDIA names its architectures after scientists — Tesla, Kepler, Hopper, Blackwell. AMD names its GPUs after fish: Sienna Cichlid, Navy Flounder, Hotpink Bonefish. Intel uses Italian bridges. There is exactly one dog in the entire lineage: Husky, the CPU cores in AMD's 2011 Llano APU. And because Llano was an APU, those Husky cores shared a die with an integrated Radeon — codenamed Sumo. The closest a dog has ever gotten to a GPU is sitting next to one, and even then the graphics half got named after a wrestler.

So I put one there myself.

Dog or Not is a live-video security scanner with one job. You hold something up to the camera, say "scan", and it tells you whether it is a dog. If it is, it barks.

It is deliberately not charming about it. The scanner is a cold threat-assessment system that happens to have been pointed at dogs, and the entire joke is that it does not know it is making one. Hold up a golden retriever and it says, in the flattest voice available:

"Woof."

Hold up a wolf and it says "Negative. Grey wolf." Hold up a cat and it suffers a fatal system error. Get three dogs in frame and it declares a containment breach — in whichever of nine languages you picked.

The classification line is where it gets interesting. A wolf is not a dog. Neither is a coyote, a fox, a plush toy, a bronze statue, a cartoon, or a person in a costume. That is a choice rather than a fact, and it is the choice that makes the thing measurable — "is this a dog" is otherwise solved zero-shot and there is nothing to find out.

Demo

Live: https://dog-or-not-289270257791.us-central1.run.app

Grant camera access, press INITIATE, then hold something up and say "scan".
Chrome or Edge for the voice command — everywhere else, use the SCAN button,
which does exactly the same thing.

Code

Running it yourself is four commands, and only one of them costs anything:

./scripts/install_deps.sh      # NOT a bare pip install -r; see below
make frontend                  # npm ci && vite build — the backend serves dist/
make mock                      # offline fake server on :8080, free, no API key
make run                       # the real Live API — this one bills

The dependency script exists because pip install -r requirements.txt hard-fails here with ResolutionImpossible. requirements.txt pins websockets==17.0.1, deliberately above the caps google-adk (<16) and google-genai (<17) declare — those bounds are "last version we tested", not a real incompatibility, and overrides.txt overrides them. uv applies it with --override; pip has no equivalent, so the script installs the tree without websockets and then forces the pin with --no-deps. Do not "fix" this with a bare --no-deps on the whole file: it skips every transitive dependency too and leaves an app that cannot import.

Forked from way-back-home, which was the same scanner counting fingers for a biometric handshake. The multimodal plumbing came from there — bidirectional WebSocket, 1 FPS video, local wake-word detection, the accuracy harness, the Cloud Run chain — and it is the reason this exists at all in a weekend.

How I Built It

So where does the model actually sit?

Nothing here is a request/response call. The browser holds one WebSocket open for the life of a scan, and the same socket carries frames up and speech down.

Browser (Chrome / Edge)                        Cloud Run  --min-instances=1
┌─────────────────────────────┐                ┌──────────────────────────────┐
│ getUserMedia                │                │ FastAPI  main.py             │
│  ├ video → canvas → JPEG    │  binary  2  →  │  ├ origin allowlist          │
│  │    640×480 q60 @ 1 FPS   │                │  │   (CORS does not apply    │
│  └ audio → Web Speech API   │  {"type":      │  │    to WebSockets)         │
│      never leaves the tab   │   "text"}  →   │  ├ LiveRequestQueue          │
│                             │                │  └ ADK Runner  run_live()    │
│ audioStreamer.js            │  ←  binary  3  │           │                  │
│  └ mu-law → PCM → worklet   │     (mu-law)   │           ▼                  │
│                             │                │  biometric_agent             │
│ DogScanner.jsx              │  ←  {"type":   │   report_verdict             │
│  └ DOG / NOT A DOG          │      "match"}  │   trigger_system_error       │
└─────────────────────────────┘                │   trigger_heavy_metal_mode   │
                                               └──────────────┬───────────────┘
                                                              │ bidiGenerateContent
                                                              ▼
                                                     Gemini Live API

Two decisions in that picture cost the most to arrive at.

The microphone does not stream. Its entire job is to catch the word "scan", and doing that over the wire costs 256 kbit/s of raw PCM — about two thirds of the uplink. Worse, continuous audio is what stops the model taking turns: measured 0/5 with speech in the room against 5/5 for the identical prompts sent as text. So the audio stays in the browser and the wake word sends the same text frame the offline harness sends. Uplink drops from ~385 to ~128 kbit/s, all of it video.

Model audio comes back as mu-law on a binary frame, not base64 in the JSON. Base64 inflates binary by a third, and over a 14-second session that was measured at 220 KB → 95 KB, downlink 128 → 55 kbit/s. The encoder is 40 lines of pure Python; the decoder is a 256-entry table in the browser. Change one and you must change the other — a mismatch is static, not an error.

I did not write this. A self-paced loop did, and I want to talk about what that was actually like — because the interesting part is not that it worked.

Loop-driven development, concretely: instead of a conversation, the agent schedules its own next wake-up. It reads a build log, picks the highest-priority unblocked task, does it, and appends an honest entry about what moved and what broke. Then it decides when to come back — fifteen minutes while there was work, thirty once it ran out — and arms a file watcher so it wakes immediately if the thing it is waiting on arrives.

That build log is in the repo, tick by tick, written as it happened rather than reconstructed afterwards. This section is drawn from it.

Every green checkmark was, at some point, green over something broken

This is the thing I did not expect, and it is the whole reason the post is worth reading.

35 passing tests over an app that could not work. The backend started sending {is_dog, subject, confidence}. The tests went green. The frontend was still reading msg.count || msg.digit off that same frame, so the verdict never reached the UI. Nothing covers that seam — the Python suite stops at the socket and there were no frontend tests at all. The tests measured exactly what they cover, which was not the broken part.

There was a nastier bug hiding in the same line. is_dog is a boolean, so the old msg.count || ... idiom would have silently discarded every NOT-A-DOG verdict — half the answers, and the more interesting half.

Clean lint over a failed build. Deleting a dead function took a live one out with it, four lines below. ESLint passed — it does not resolve cross-module imports by default — and the failure only appeared in vite build, buried under twelve lines of rollup stack trace.

A green test asserting the exact thing it was supposed to prevent. The project keeps a non-public model id out of the repo, enforced by a gate wired into make test and make deploy. One test asserted that id as a string literal. A leak with a checkmark on it.

A gate that caught itself. That same gate failed on its first run — on its own list of patterns, which necessarily contains every string it searches for. Funny once, an infinite loop thereafter.

The same bug, three times, in three places

A route that exists in the real backend and not in the place people actually develop:

1. /api/config 404ing under the Vite dev proxy, which forwarded only /ws. Symptom: the header read AWAITING LINK — the exact bug that endpoint was written to fix.
2. /api/config and /api/fixtures missing from the mock server — the documented way to work on the UI without billing a session. The fixture portal could not load a single image there.
3. Waiting to happen on the next endpoint.

Every one failed soft. Nothing crashed, nothing logged an error, and the feature just looked unfinished. The graceful fallback hid its own cause. There is now a test that diffs the real app's /api/* routes against the mock's and fails naming the difference — and I checked it is not vacuous, because two empty sets compare equal and a parity test over nothing passes forever.

The model that narrates calling a tool and then doesn't

The whole architecture hangs off one tool call: the model sees a subject, calls report_verdict(is_dog, confidence, subject), and everything downstream — UI, bark, scoring — follows from that.

Exactly one non-preview model offers the Live API. On it, every single trial came back SILENT. Meanwhile its own thinking said:

"A visual identification scan reveals a clear image of a Golden Retriever. The subject is verified as a real dog with 95% confidence, as is_dog is true... I'm executing the report_verdict tool with the dog's details, then I will say 'Woof.'"

It sees the dog. It narrates the call. Zero tool calls, ever. Meanwhile the fallback model inherited from the earlier build could not open a Live session at all, under a comment claiming it could — because until the first real session, nothing had ever opened one.

One fixture in twenty was poisoned

The eval set came from Wikimedia Commons, sourced by search term. One of them — "Dog with Goofy plush toy" — is a real dog chewing a toy, filed as not-a-dog. It would have scored every correct answer as a failure.

There is no test for that. The only way to catch it is to look at it. The project's own older documentation already says so: generate the input, never the expectation.

And the harness was scoring obedience as failure

One dog fixture came back SILENT twice, consistently. It is a Halloween dog park — beagle, cavalier, retriever in the background — and the model had called trigger_heavy_metal_mode(). The containment breach fired unprompted, on an unposed real-world photograph.

The harness only scored report_verdict. But the instruction gives the easter eggs absolute priority over report_verdict — so the harness was marking the model doing exactly as it was told as a failure.

Is it actually right, though?

There is exactly one thing in this repo that measures the model rather than the plumbing, and it needs no human in front of the camera:

python scripts/scan_accuracy.py --blur-prob 0.3 --jitter 2 --min-rate 0.8

It drives the real deployed endpoint with fixture images at 640×480 JPEG q60 — the exact format the browser sends — puts a {"type":"text"} frame in where you would say "scan", and scores every report_verdict against a count verified by eye before it was committed. One billed session per run. --blur-prob and --jitter approximate a real webcam; --min-rate turns it into a gate that can fail a build.

What it cannot see: it holds one static fixture per trial, so it is structurally incapable of catching a sampling miss — a pose held for less than a second at 1 FPS that lands in no frame at all. That blind spot is how "1 FPS and 2 FPS score identically" got measured, believed, and written down.

The numbers

Twenty fixtures, one session each:

Latency 0.68–1.56s, mostly around 0.7. It correctly said "bronze statue" for a sculpture of a man with a bronze dog beside him.

What the loop got wrong

It reported the wrong blocker for four hours. It kept saying it was waiting on dog photos, while (a) there was no API key, which it had never checked, and (b) twenty public-domain fixtures with generated attribution turned out to be about ten minutes of work.

It optimised what it could reach instead of what mattered, and reported a blocker it had not verified. That is a very human failure mode and it is worth naming, because the honest version of "I let an AI build this over a weekend" includes it.

Prize Categories

Best Use of Google AI — Gemini Live via the Agent Development Kit. Bidirectional WebSocket streaming video at 1 FPS, structured verdicts through tool calls rather than parsed prose, and a nine-language session config where the model translates its own lines rather than reading a shipped phrasebook. That last part is the cheapest possible proof the model is really being called: a recording cannot answer in Japanese.

Best Use of ElevenLabs — the bark pack, generated with the Sound Effects API at build time, never at runtime. The clips are fetched and decoded once and held in memory, so the bark adds zero latency to the response path and cannot fail during a session. This project had already measured what a second audio stream does to a Live session — 0/5 against 5/5 — and the way to use a sound API here was to make the app touch the network less, not more.

Sound effects generated with ElevenLabs. Fixture images from Wikimedia Commons, attributed in tests/fixtures/ATTRIBUTION.md.

Measured on: gemini-3.1-flash-live-preview via google-adk==2.6.3 / google-genai==2.17.0, Python 3.13, deployed to Cloud Run in us-central1 with --min-instances=1 and --timeout=3600 (a WebSocket is one long request; the default 300s cap would end every session at five minutes). Video 640×480 JPEG q60 at 1 FPS, audio 16 kHz PCM up / 24 kHz down. Client tested in Chrome — the wake word is SpeechRecognition, which is Chrome and Edge only; everywhere else the SCAN button does the identical thing.