Four Ways My Unattended Video Pipeline Died Overnight — and How I Made It Heal Itself

The morning after I lost my job, my Mac finished and filed an ASMR video. Nobody asked it to. It just ran.

In the first post, I walked through the structure of the pipeline itself — ComfyUI × FFmpeg × the Freesound API, generating long-form ASMR videos with nothing but free tools. This second post covers the other half: putting that pipeline on macOS launchd so it fires at a fixed time every day, and the self-healing logic that gets the script past the "cold start" problem, where you boot the Mac and ComfyUI simply isn't running. Two numbers do most of the work here: the ComfyUI startup wait went from 180 seconds to 600, and the Freesound download timeout went from 90 seconds to 240. Before those changes, mornings failed 2–3 days a week.

Why this setup works

The ceiling on manual work

Making a single 30-minute ambient ASMR video carefully takes 2–3 hours of hands-on time. Tuning image-generation prompts, layering the BGM, checking the loop points, building the thumbnail, filling in YouTube metadata — each step is small, but they stack up.

Trying to hold 30 videos a month means 60–90 hours of pure labor. I attempted it while holding a side job, and it collapsed in two weeks. That was the first time I understood that "scaling output" isn't about moving your hands faster — it's about building a state where output accumulates without your hands at all. When I was laid off and my income went to zero, the first thing I rebuilt was this environment.

Own an environment, not a workflow

The essence of automation is constructing, exactly once, a mechanism where output keeps growing while you do nothing. That's precisely what daily.sh delivers: when the script finishes, ~/Desktop/ASMR/<date>_<theme>/ lands atomically with the video, thumbnail, youtube.md, and still image all in place. I just check it the next morning. Whether I step away mid-generation or I'm asleep, the files keep piling up.

One line in the code embodies the whole philosophy:

# 冪等性は「その日に1本でもあればskip」(1日1本・テーマ違いでも二重生成しない)
EXIST=$(find "$DEST" -maxdepth 1 -type d -name "${DATE}_*" 2>/dev/null | head -1)
if [ -n "$EXIST" ]; then log "already stocked for $DATE ($EXIST); skip (idempotent)"; exit 0; fi

Even if launchd tries to fire in both the 07:00 and 14:00 slots, the script returns immediately with exit 0 when today's directory already exists under ~/Desktop/ASMR/. No double generation. If the Mac sleeps partway through, the 14:00 slot handles recovery. That's the core of the idempotent-slot design.

Why it costs ¥0/month

This pipeline is deliberately designed to cost nothing monthly.

• Image generation: ComfyUI (~/dev/comfyui / RealVisXL / MPS) = free
• Motion synthesis: FFmpeg (displace / perlin flicker) = free
• Ambient sound: Freesound API, CC0-licensed = free (FREESOUND_TOKEN is issued on free signup)
• Upload target: YouTube Data API v3 = the free quota is plenty

The only things I actually need are electricity for the Mac itself and a Google Cloud OAuth client (free registration, just issuing an API key). No paid SaaS image generation, no cloud GPU, no monthly subscription.

This design eliminates the risk of "an external service changes its pricing and my machinery breaks." Since ComfyUI and FFmpeg both run locally, a sudden price hike or the death of a free tier can't stop the automation. Even right after my income dropped to zero, this one thing kept running.

coverage.csv kills the anxiety

To sustain monthly revenue, you have to break the loop where you get anxious about whether the machinery is still working and go check on it. Appending to coverage.csv is the design that does that:

# ---- 11. カバレッジログ(CSV) ----
COV="$ROOT/coverage.csv"
[ -f "$COV" ] || echo "date,theme,duration_s,sounds,size,status" > "$COV"
echo "$DATE,$THEME_ID,$DUR,$idx,$SZ,OK" >> "$COV"

One glance at a single line of this CSV each morning tells me whether yesterday's 07:00 slot succeeded. When the YouTube upload succeeds, the trailing status field changes to OK+uploaded (overwritten via sed -i '' "s|,OK\$|,OK+uploaded|"). Less anxiety means fewer unnecessary edits to the machinery, which means longer stretches of stable operation.

The other benefit of idempotency

Idempotent design also gives you backfill as a byproduct. Point the --date option at a past date and the same script fills in the gap after the fact:

./daily.sh --date 2026-06-28   # 欠けていた6/28分を手動で補完
./daily.sh --still scene.png --theme-id rainy_cafe --force  # 既存画像で音だけ再mix

--still skips launching ComfyUI and reuses an existing still image. Because ComfyUI's MPS inference takes several minutes, I've actually used this a few times when I liked the image but wanted different BGM. Without that flag, ensure_comfyui() runs as usual — and that function is the star of this post.

The whole flow

Bird's-eye view of the pipeline

Here is everything daily.sh executes, in order:

launchd (7:00 / 14:00, catch-up)
    │
    ▼
daily.sh
    │
    ├─[lock]──────── mkdir LOCKDIR atomic / ゾンビPID自動回収
    │
    ├─[冪等確認]──── ~/Desktop/ASMR/<DATE>_* 存在? → exit 0
    │
    ├─[ComfyUI]───── down? → 自動起動(.venv mps) → 最大600秒待機
    │
    ├─[テーマ]─────── 日付UNIXtime÷86400 % 7種 → 決定的ローテ
    │
    ├─[1] 画像生成  comfy_gen.py (seed=日付ベース / 最大3リトライ)
    │
    ├─[2] 自動マスク auto_masks.py (rain窓 / fire炎 自動検出)
    │
    ├─[3] 雨変位マップ gen_rain_glass_map.py (16秒ループ / キャッシュ流用)
    │
    ├─[4] ループ動画  render_loop.sh (FFmpeg displace + flicker)
    │
    ├─[5] 音取得+mix  freesound_fetch.py (timeout 240s) + mix_audio.sh
    │
    ├─[6] 30分化      make_full.sh (ループタイル)
    │
    ├─[7] サムネ       make_thumb.py (1280×720)
    │
    ├─[8] メタデータ  make_meta.py → youtube.md
    │
    ├─[9] 検証────── 尺±3秒 / video+audioストリーム / サムネサイズ
    │
    ├─[10] atomic移動 work/_deliver → ~/Desktop/ASMR/<DATE>_<THEME>/
    │
    ├─[11] CSV追記    coverage.csv (date,theme,duration_s,sounds,size,status)
    │
    └─[12] YouTube    非公開アップ (token有時のみ / 失敗してもstock保持)

Every step uses die() and hits exit 1 the instant something fails. It never proceeds to the next step in a half-finished state. Nothing arrives in ~/Desktop/ASMR/ unless all 12 steps succeeded.

Small details at the top of the script

set -uo pipefail
export LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8   # launchdの最小環境でも日本語UTF-8を安定処理
cd "$(dirname "$0")"
ROOT="$(pwd)"

set -uo pipefail guarantees that failures inside a pipe propagate to exit 1. The shell launchd spawns has a minimal PATH and locale. So that filenames containing Japanese theme names and CSV writes don't get mangled, the locale is forced at the very top. cd "$(dirname "$0")" moves into the script's own directory so relative paths to themes.json and lib/ always resolve.

Details of the lock mechanism

LOCKDIR="$ROOT/.daily.lock.d"
acquire_lock(){
  if mkdir "$LOCKDIR" 2>/dev/null; then echo $$ > "$LOCKDIR/pid"; return 0; fi
  local opid; opid=$(cat "$LOCKDIR/pid" 2>/dev/null || echo "")
  if [ -n "$opid" ] && ! kill -0 "$opid" 2>/dev/null; then
    log "stale lock (pid $opid dead); reclaiming"; rm -rf "$LOCKDIR"
    if mkdir "$LOCKDIR" 2>/dev/null; then echo $$ > "$LOCKDIR/pid"; return 0; fi
  fi
  return 1
}
if ! acquire_lock; then log "another run holds the lock; exiting"; exit 0; fi
trap 'rm -rf "$LOCKDIR"' EXIT

mkdir is atomic under POSIX. Even if several processes try at once, exactly one succeeds. That gives reliable mutual exclusion even in a macOS launchd environment where flock isn't available.

Automatic reclamation of zombie locks is the key part. If a previous run crashed and left LOCKDIR behind while that PID's process is already dead, the failure of kill -0 $opid is detected and the lock is re-acquired. No human has to manually delete .daily.lock.d; the next run self-heals.

Theme rotation

NTHEME=$(python3 -c "import json;print(len(json.load(open('themes.json'))['themes']))")
DAYIDX=$(( $(date -j -f "%Y-%m-%d" "$DATE" +%s 2>/dev/null || date -d "$DATE" +%s) / 86400 ))
PICK=$(( DAYIDX % NTHEME ))
THEME_ID=$(python3 -c "import json;print(json.load(open('themes.json'))['themes'][$PICK]['id'])")

themes.json currently defines 7 themes. The date is converted to a day count via UNIX time ÷ 86400, and the theme is chosen by the remainder modulo the theme count of 7. Specify the same date and you always get the same theme — a deterministic rotation (overridable with --theme-id).

date -j -f "%Y-%m-%d" is macOS BSD date syntax and is incompatible with Linux's date -d, so 2>/dev/null || date -d "$DATE" +%s falls back to the Linux version. The script assumes launchd + macOS, but this is a concession to occasionally running it in Docker on Linux during development.

Timeout design for sound fetching

for lic in cc0 any; do
  if timeout 240 python3 "$LIB/freesound_fetch.py" \
      --query "$Q" --minlen 25 --license "$lic" --out "$SRC" \
      >"$WORK/fs_${i}.json" 2>>"$LOG"; then
    fetched=1; break
  fi
done
if [ "$fetched" -eq 1 ]; then MIXARGS+=("$SRC" "$G"); else log "WARN: sound fetch failed: $Q"; fi

Freesound's preview download endpoint has no officially configured timeout. Even after the connection is established, the download can stall and wait forever. timeout 240 (4 minutes) kills the whole process and moves on to the next loop iteration, treating it as a failure.

If nothing can be fetched under CC0, it falls back to the any license (for lic in cc0 any); sources that still fail are skipped with a WARN in the log. Only when all sources come back empty does the whole pipeline stop with die "no sound sources fetched (won't ship silent)" — the judgment being that I don't ship silent videos.

Quality assurance via atomic move and verification

# ---- 9. 検証 ----
DUR=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$VIDEO" 2>/dev/null | cut -d. -f1)
WANT=$((MIN*60))
[ -n "$DUR" ] && [ "$DUR" -ge $((WANT-3)) ] && [ "$DUR" -le $((WANT+3)) ] || die "duration check failed ($DUR != $WANT)"
STREAMS=$(ffprobe -v error -show_entries stream=codec_type -of csv=p=0 "$VIDEO" 2>/dev/null | sort | tr '\n' ',')
echo "$STREAMS" | grep -q "audio" && echo "$STREAMS" | grep -q "video" || die "missing stream ($STREAMS)"
TW=$(python3 -c "from PIL import Image;print('x'.join(map(str,Image.open('$THUMB').size)))")
[ "$TW" = "1280x720" ] || die "thumb size $TW != 1280x720"
[ -s "$META" ] || die "meta empty"

Before moving anything, ffprobe confirms the duration falls within MIN*60 ± 3 seconds, checks that both video and audio streams exist, and verifies the thumbnail is 1280×720. This exists to prevent silent failures like "ffmpeg quietly produced an empty video" or "a video missing its audio track landed on the Desktop."

And the move itself is atomic:

STAGE="$WORK/_deliver"; mkdir -p "$STAGE"
cp "$VIDEO" "$STAGE/video.mp4"
cp "$THUMB" "$STAGE/thumbnail.png"
cp "$META"  "$STAGE/youtube.md"
cp "$STILL" "$STAGE/scene.png"
mkdir -p "$DEST"
rm -rf "$FINAL"
mv "$STAGE" "$FINAL"

Everything is assembled inside the work directory, then landed with a single mv. Even if the machine sleeps and wakes while the CPU or GPU is mid-process, files never end up half-written in ~/Desktop/ASMR/. The invariant holds: "any folder you can see on the Desktop is a complete one."

The next chapter gets concrete about the biggest obstacle in this whole flow — the ComfyUI cold-start problem — and about registering the job with launchd.

Implementation details

ensure_comfyui(): a 150 × 4-second = 600-second wait, by design

I promised in the previous chapter that ensure_comfyui() is the star of this post. Here's the whole function:

ensure_comfyui(){
  curl -s -m 3 http://127.0.0.1:8188/system_stats >/dev/null 2>&1 && return 0
  log "ComfyUI down; starting (.venv mps)..."
  ( cd "$HOME/dev/comfyui" && PYTORCH_ENABLE_MPS_FALLBACK=1 nohup .venv/bin/python main.py --port 8188 \
      >/tmp/comfyui_asmr.log 2>&1 & )
  for i in $(seq 1 150); do
    sleep 4
    curl -s -m 3 http://127.0.0.1:8188/system_stats >/dev/null 2>&1 && { log "ComfyUI up"; return 0; }
  done
  return 1
}
[ -n "$REUSE_STILL" ] || ensure_comfyui || die "ComfyUI unavailable (could not start)"

First it hits http://127.0.0.1:8188/system_stats with curl -s -m 3. If a response comes back within 3 seconds, ComfyUI is considered already up and it returns 0 immediately. Terminating the polling right there means that on most days — when yesterday's generation ran and ComfyUI is still up — this passes in milliseconds.

If ComfyUI is down, it starts in the background in a subshell. The crucial part is explicitly naming .venv/bin/python. Pointing directly at the virtualenv's Python instead of python3 is necessary because launchd's PATH is roughly the minimal /usr/bin:/bin:/usr/sbin:/sbin. Calling python3 gets you system Python, which lacks ComfyUI's dependencies (torch, safetensors, kornia, etc.), so it dies instantly on an import error. PYTORCH_ENABLE_MPS_FALLBACK=1 is the environment variable that makes Apple Silicon's MPS backend fall back to CPU for unsupported operators; forget it and you get RuntimeError: Not implemented for MPS on certain layers and model loading stalls.

nohup blocks signals and redirects stdout to /tmp/comfyui_asmr.log. By not daemonizing, daily.sh's EXIT trap can't accidentally drag ComfyUI down with it (trap 'rm -rf "$LOCKDIR"' EXIT only deletes the lock directory and has no effect on background processes).

Post-launch polling runs seq 1 150 for 150 iterations with sleep 4 — a 4-second interval, for a maximum wait of 600 seconds (10 minutes). RealVisXL loads a 13GB+ model into MPS memory, so on a morning after a cold start, the model isn't even in the disk cache. A sequential SSD read runs, PyTorch's MPS graph optimization runs, ComfyUI's workflow registration runs — and I measured this whole chain taking 5–8 minutes on some days. 150 × 4 = 600 seconds was chosen as a number that covers that worst case. If HTTP still doesn't respond past 600 seconds, return 1 hands control to the caller's die "ComfyUI unavailable" and the entire pipeline stops.

When the --still flag is passed (reusing an existing still image), ensure_comfyui is skipped:

[ -n "$REUSE_STILL" ] || ensure_comfyui || die "ComfyUI unavailable (could not start)"

That's the flag for when I want the same image but different BGM, and it skips the entire 10-minute ComfyUI startup wait.

Registering with launchd: 07:00, 14:00, and delayed catch-up

The launchd label is com.lily.asmr-daily (as documented in the README); you drop a plist into ~/Library/LaunchAgents/ and run launchctl load. Setting two StartCalendarInterval entries secures the 07:00 and 14:00 slots.

Catch-up is the behavior where, if the scheduled time passes while the Mac is asleep, launchd runs the job late the next time the machine wakes. StartCalendarInterval has this enabled by default, so even if the Mac was asleep at 07:00, launchd invokes daily.sh the moment I open the lid. At that point the idempotency check runs:

EXIST=$(find "$DEST" -maxdepth 1 -type d -name "${DATE}_*" 2>/dev/null | head -1)
if [ -n "$EXIST" ]; then log "already stocked for $DATE ($EXIST); skip (idempotent)"; exit 0; fi

If today's output isn't on the Desktop, it just runs; if it is, it returns immediately with exit 0. The 14:00 slot exists so that recovery on a day when the 07:00 slot failed for some reason happens without human involvement. Because the design gives two chances per day, coming home and opening the Mac (say, at 15:00) triggers the catch-up for the 14:00 slot.

As an extra piece of launchd configuration, I recommend explicitly injecting HOME and PATH via the EnvironmentVariables key. As noted at the top of daily.sh, the shell launchd starts is a separate process from your login shell. The nvm path set in ~/.zshrc, brew-related PATH entries, environment variables like FREESOUND_TOKEN — none of it carries over. Since FREESOUND_TOKEN is designed to be read from a .env file ([ -f "$ROOT/.env" ] && { set -a; . "$ROOT/.env"; set +a; }), writing just PATH and HOME into the plist's EnvironmentVariables is enough.

Logs land in two places: daily.log (appended via tee inside the script) and launchd.log (the plist's StandardOutPath/StandardErrorPath). launchd.log records the time launchd started daily.sh and its exit code, so you can immediately distinguish "daily.sh was never even invoked" from "it was invoked but died internally."

Private YouTube uploads: unattended OAuth refresh

The heart of the upload piece is get_creds() and upload() in youtube_upload.py.

def get_creds(interactive=False):
    creds = None
    if os.path.exists(TOKEN):
        creds = Credentials.from_authorized_user_file(TOKEN, SCOPES)
    if creds and creds.valid:
        return creds
    if creds and creds.expired and creds.refresh_token:
        creds.refresh(Request())
        _save(creds); return creds
    if not interactive:
        sys.exit("ERROR: no valid token. Run once: python3 lib/youtube_upload.py --auth")
    ...

If token.json exists and is valid, it's used as-is; if expired, creds.refresh(Request()) renews the access token and _save(creds) overwrites the file. Google OAuth access tokens last one hour, but the refresh token persists until it's revoked. Because of those three lines, you consent in the browser once (--auth) and from then on daily.sh refreshes unattended. _save() restricts permissions with os.chmod(TOKEN, 0o600), so there's no risk of token.json being readable by group/other.

The upload itself:

media = MediaFileUpload(spec["video"], chunksize=8 * 1024 * 1024, resumable=True,
                        mimetype="video/mp4")
req = yt.videos().insert(part="snippet,status", body=body, media_body=media)
resp = None
while resp is None:
    status, resp = req.next_chunk()
    if status:
        print(f"  upload {int(status.progress()*100)}%", file=sys.stderr)

It sends in 8MB chunks with resumable=True. A 30-minute H.264 video runs 1–2GB, so a single-shot upload means starting over from zero on any momentary network drop. Resumable mode can pick up mid-stream, so it holds up fine on flaky home WiFi late at night. On the daily.sh side there's a timeout 1200 (20 minutes) wrapped around it:

if timeout 1200 python3 "$LIB/youtube_upload.py" --upload "$FINAL/upload.json" >>"$LOG" 2>&1; then

On success, the Studio URL is recorded in daily.log and the status in coverage.csv is rewritten to OK+uploaded:

sed -i '' "s|,OK\$|,OK+uploaded|" "$ROOT/coverage.csv" 2>/dev/null || true

Publishing is done by a human. The privacyStatus the script sets is hardcoded to "private" (spec.get("privacy", "private")). Title, description, tags, and thumbnail are already configured, so it's ready to publish with one click in Studio. Because the thumbnail is set with a separate request from the video upload (thumbnails().set()), there are rare cases where the upload succeeds but only the thumbnail step fails. In that case you can re-set it afterward with --set-thumb VIDEO_ID thumb.png:

def set_thumb(video_id, thumb):
    """既存動画にサムネだけ設定(再アップせず・要確認済みアカウント)"""

This repair command ships inside youtube_upload.py.

Where I got stuck

1. The ComfyUI startup wait was too short, and every morning died with an error

When I first wrote ensure_comfyui(), I set the polling count to 45. 45 × 4 seconds = 180 seconds. The intuition was "three minutes should be plenty."

When ComfyUI starts up in the background while I'm actively using the Mac, three minutes really is usually enough. The problem is the cold start — powering down at night and having launchd fire at 07:00 the next morning. At that point RealVisXL's safetensors file (located under ~/dev/comfyui) isn't in the disk cache at all. A sequential SSD read runs, Python startup overhead runs, PyTorch initializes the MPS device, workflow nodes load — and I measured days where that whole sequence took 5–6 minutes. Cutting it off at 180 seconds and falling into die "ComfyUI unavailable" happened 2–3 mornings a week.

Checking daily.log:

[2026-06-14 07:04:22] ComfyUI down; starting (.venv mps)...
[2026-06-14 07:07:04] FAIL: ComfyUI unavailable (could not start)

Startup attempted at 07:04, timed out at 07:07 (about 3 minutes). It would retry in the 14:00 catch-up slot, but on days I'd taken the Mac out with me, that didn't run either — so there were days with zero output.

After changing the polling count to seq 1 150 (600 seconds), cold-start failures disappeared entirely. A 10-minute wait makes the script look frozen, but it's only waiting for ComfyUI to come up; the generation, mixing, and uploading afterward don't run in parallel with it (it's serial). By the time I've made coffee and come back, coverage.csv has an OK in it.

2. A Freesound download stalled silently, and daily.sh waited forever

The Freesound API's preview download endpoint has no official timeout. After Content-Length comes back in the header and the download starts, the server side can silently hold the connection open and stop mid-transfer. Download naively with wget or requests.get(stream=True) and the socket timeout never fires — the connection is "alive" — so the process waits forever.

My initial implementation didn't account for this and simply called freesound_fetch.py directly. README.md still says "fetch is held at timeout 90, missing sources are gracefully skipped," but the actual daily.sh uses timeout 240 (4 minutes):

if timeout 240 python3 "$LIB/freesound_fetch.py" \
    --query "$Q" --minlen 25 --license "$lic" --out "$SRC" \
    >"$WORK/fs_${i}.json" 2>>"$LOG"; then

I originally set 90 seconds on the assumption that "sound sources are short preview files, so that's plenty of headroom." The reality is that Freesound's preview servers are overseas, and from my home line late at night the throughput can sit around 400kbps. A preview MP3 (128kbps) of a 25-second-plus source is about 400KB, but if the transfer stalls mid-way, 90 seconds isn't enough.

The instant the 90-second timeout fired, freesound_fetch.py was killed with SIGTERM, leaving $SRC empty or incomplete. The downstream mix_audio.sh then chewed on that incomplete file and FFmpeg died with Invalid data found when processing input — a nice little chain reaction. After bumping to 240 seconds and adding the cc0 → any fallback, this class of failure went away. Only when every source comes back empty does it stop with die "no sound sources fetched (won't ship silent)", preventing the silent failure of a silent video landing on the Desktop.

3. launchd's PATH was too narrow to resolve .venv

The PATH of a shell started by launchd is /usr/bin:/bin:/usr/sbin:/sbin. None of the brew, nvm, or Conda paths from ~/.zshrc are there. My first implementation had this inside ensure_comfyui():

nohup python3 main.py --port 8188 &

Invoked via launchd, python3 resolves to system Python (/usr/bin/python3). System Python doesn't have torch etc., so ComfyUI dies right after startup with:

ModuleNotFoundError: No module named 'torch'

Worse, because it's thrown into the background with nohup, that error only gets written to /tmp/comfyui_asmr.log and nothing appears in daily.sh's log. Never noticing that ComfyUI failed to start, the 600-second polling grinds on and eventually stops at die "ComfyUI unavailable".

The fix is simple — specify .venv/bin/python by full path:

( cd "$HOME/dev/comfyui" && PYTORCH_ENABLE_MPS_FALLBACK=1 nohup .venv/bin/python main.py --port 8188 \
    >/tmp/comfyui_asmr.log 2>&1 & )

$HOME does expand under launchd (because HOME is set in EnvironmentVariables). That effectively uses ~/dev/comfyui/.venv/bin/python, and ComfyUI starts with all its dependencies present.

For the same reason, I explicitly put PATH into the launchd plist. If ffmpeg, ffprobe, and python3 are installed via brew, those commands won't resolve either unless the plist's EnvironmentVariables includes /opt/homebrew/bin. I hit that failure too in the first few days: daily.sh dying with command not found at the step that calls ffprobe (the verification stage).

4. Without LC_ALL, Japanese theme names got mangled and coverage.csv broke

launchd's shell also has a minimal locale. Write a string containing Japanese into a filename or CSV with LC_ALL unset, and macOS's BSD environment mangles it. Concretely, taking jp_title from themes.json (e.g. 「雨カフェ」), putting it in a variable, and writing:

echo "$DATE,$THEME_ID,$DUR,$idx,$SZ,OK" >> "$COV"

THEME_ID itself is ASCII (rainy_cafe, etc.) so it's fine, but the JP_TITLE written to the log output and to youtube.md gets garbled. Looking at daily.log:

[2026-06-10 07:12:34] === daily start date=2026-06-10 theme=rainy_cafe rain=True -> ...
title: ??? ??? ???

The title had become a row of question marks. The status column of coverage.csv was fine, but since the contents of youtube.md were mangled, I had to fix them by hand when copy-pasting.

What I added as line 2 of daily.sh:

export LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8   # launchdの最小環境でも日本語UTF-8を安定処理

As the comment records, without this, Japanese breaks only on launchd-invoked runs. Running ./daily.sh manually from a terminal picks up the locale via ~/.zshrc, so it doesn't reproduce locally — which is why "why does this only break under launchd?" took a while to isolate.

The current daily.sh is what came out the other side of all four of these failures. Every single line in the comments that says "even in launchd's minimal environment…", "macOS-compatible mkdir atomic", "prevent DL hangs" is a scar from something that actually broke and got fixed.

Next time I'll dig into the design of themes.json, the file defining the 7 themes, and how to write ComfyUI prompts so that automatic masking (auto_masks.py) works reliably.

Gotchas

Here are the failures I hit building all this, ordered by how easily they reproduce. The four already covered above — ComfyUI 180 → 600 seconds, the Freesound DL hang 90 → 240 seconds, launchd PATH and .venv resolution, and mangled text from unset LC_ALL — are omitted; this section focuses on everything else I actually got stuck on.

Cold start and model loading

• Forget PYTORCH_ENABLE_MPS_FALLBACK=1 and the error never reaches daily.sh's log. The line that starts ComfyUI in the background:

( cd "$HOME/dev/comfyui" && PYTORCH_ENABLE_MPS_FALLBACK=1 nohup .venv/bin/python main.py --port 8188 \
    >/tmp/comfyui_asmr.log 2>&1 & )

Without PYTORCH_ENABLE_MPS_FALLBACK=1, an operator Apple Silicon's MPS doesn't support raises RuntimeError: Not implemented for MPS and model loading stalls partway. Because it's backgrounded with nohup, that error only lands in /tmp/comfyui_asmr.log and nothing shows up in daily.sh's log. ComfyUI up never returns, 600 seconds burns away, and it finally stops at die "ComfyUI unavailable" — that pattern failed two days in a row.

• Hitting daily.sh's EXIT trap doesn't stop ComfyUI. trap 'rm -rf "$LOCKDIR"' EXIT only deletes the lock directory. Since nohup detached it from daily.sh's process group, ComfyUI stays running until the next morning even if daily.sh exits 1. That's intentional (it skips the startup wait in the next day's 07:00 slot), but at first I was confused about "why isn't ComfyUI dead?" The fact that curl -s -m 3 http://127.0.0.1:8188/system_stats returns 0 immediately is a byproduct of the same thing.

• comfy_gen.py's --timeout 300 and the retries are independent. On the daily.sh side, the 3 retries shift the seed by +777 each time:
  

for att in 0 1 2; do
  SEED=$(( SEEDBASE + att*777 ))
  if python3 "$LIB/comfy_gen.py" --prompt "$PROMPT" --seed "$SEED" --out "$STILL" --timeout 300 ...; then
    ok=1; break
  fi
  sleep $(( (att+1)*10 ))
done

Three failures in a row gives die "image generation failed after retries". Because the seed changes, there have been real cases where the model hangs on a particular seed and the next attempt escapes it.

Files and state

• rm -rf "$WORK" only runs on success. At the end of daily.sh, just before the log line:

rm -rf "$WORK"
log "=== daily done"

On failure, ~/dev/asmr-factory/out/work_YYYY-MM-DD_THEME_ID/ survives. That's intentional, but left alone it eats disk. Since still.png, loop.mp4, and bed.wav all remain, you can debug on the spot with ffprobe out/work_.../video.mp4. I delete old work directories manually on a weekly basis.

• sed -i '' is macOS BSD sed-specific syntax. The step that rewrites the tail of coverage.csv:

sed -i '' "s|,OK\$|,OK+uploaded|" "$ROOT/coverage.csv" 2>/dev/null || true

Test on Linux and sed -i is the correct form; bring it to macOS and it dies with illegal option -- i (or vice versa). || true keeps a failure on this line from stopping everything, but there were cases where I only later noticed that the rewrite to OK+uploaded had been silently failing.

• A leftover zombie lock means it skips forever. Without the kill -0-based reclamation in acquire_lock(), a crashed previous run leaving .daily.lock.d/ behind means the next run exits immediately with another run holds the lock; exiting. In daily.sh the reclamation logic is explicit:

if [ -n "$opid" ] && ! kill -0 "$opid" 2>/dev/null; then
  log "stale lock (pid $opid dead); reclaiming"; rm -rf "$LOCKDIR"
  if mkdir "$LOCKDIR" 2>/dev/null; then echo $$ > "$LOCKDIR/pid"; return 0; fi
fi

Leave it out and "why is it skipping every morning?" is invisible unless you read daily.log.

• The video/thumbnail paths in upload.json are overwritten with absolute paths after the atomic move. At the time make_meta.py generates them, the paths are close to relative paths under work/. daily.sh re-injects them with Python after the move to stock:

s['video']='$FINAL/video.mp4'; s['thumbnail']='$FINAL/thumbnail.png'; s['privacy']='private'

Forget that injection and youtube_upload.py dies with video not found. When inspecting upload.json by hand, check whether the video field is an absolute path.

YouTube API

• The thumbnail is set in a separate request from the video upload. In youtube_upload.py this is caught with try/except and kept to a WARN:

try:
    yt.thumbnails().set(videoId=vid, media_body=MediaFileUpload(thumb)).execute()
    print("  thumbnail set")
except Exception as e:
    print(f"  WARN: thumbnail set failed: {e}", file=sys.stderr)

Even when the video upload succeeds and coverage.csv records OK+uploaded, a black thumbnail in YouTube Studio means this path failed. You can re-set it later with --set-thumb VIDEO_ID thumb.png. Setting thumbnails requires a verified YouTube account (phone-verified) — on an unverified account this API always fails with 403.

• Forget cache_discovery=False and the discovery cache can contend. The cache_discovery=False in build("youtube", "v3", credentials=get_creds(), cache_discovery=False) is quietly important. In scenarios where multiple processes start simultaneously via launchd (a lock crash followed by a restart racing), I got lock-contention errors on the cache file.

• Delete token.json by mistake and you're back to --auth. _save() calls os.chmod(TOKEN, 0o600):
  

def _save(creds):
    os.makedirs(CRED_DIR, exist_ok=True)
    with open(TOKEN, "w") as f:
        f.write(creds.to_json())
    os.chmod(TOKEN, 0o600)

Since a refresh token is only issued once per session, include ~/.youtube/ in your Time Machine or encrypted-storage backups.

• YouTube Data API v3's daily quota is 10,000 units. A video upload consumes 1,600 units and setting a thumbnail consumes 50. That's no problem for normal one-video-a-day operation, but backfilling past dates with the --date flag hits the quota ceiling at 6 videos. Spread backfills across multiple days.

launchd and shell environment

• StartCalendarInterval catch-up may not fire after a reboot. If 07:00 passes while the Mac is asleep and you then open it, catch-up runs; but with a full shutdown followed by powering on the next morning, there are cases where catch-up doesn't fire. Running the Mac on sleep rather than powering it off is more stable. If you must power off, run ./daily.sh manually or leave it to the 14:00 slot.

• If the parent directory of the plist's StandardOutPath doesn't exist, launchd may ignore the job. If launchd.log stays empty with no trace of daily.sh being invoked at all, check the parent directory of the log path in the plist.

• Use set -a; . "$ROOT/.env"; set +a to load .env. Plain source .env can hit unbound variable on empty-string variables when combined with set -u. Loading it after set -a — which auto-exports everything that follows — reliably makes os.environ["FREESOUND_TOKEN"] visible to Python scripts called in subshells.

Best practices

Based on all the failures and fixes above, here are the rules I actually follow while running an unattended launchd × local-ML × external-API pipeline stably for six months.

1. Nail down set -uo pipefail + LC_ALL in the first two lines

set -uo pipefail
export LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8

pipefail propagates errors inside pipes, and LC_ALL keeps Japanese strings stable even in launchd's minimal environment. -u turns references to undefined variables into immediate errors, preventing the "it kind of works but it's actually passing an empty string" class of bug. Making it a habit to start every launchd-only script with these two lines saves enormous amounts of time later, otherwise spent stuck on mangled text and silent errors.

2. Funnel everything through one die() so failure locations are instantly identifiable

die(){ log "FAIL: $*"; exit 1; }

Looking at daily.log, the text after FAIL: tells you immediately which step stopped. die "duration check failed ($DUR != $WANT)" shows the actual and expected durations as numbers. Not hardcoding exit 1 all over the place and always going through die means a single FAIL: grep works on the logs.

3. Always wrap external dependencies in timeout, at 1.5–2× the measured value

The principle is: anything that could hang on external I/O gets a kill switch. APIs where the vendor set no timeout deserve special caution.

4. Use atomic moves so "whatever's on the Desktop is complete"

Collect every file in work/_deliver/, then land it in stock with a single mv. Within the same filesystem, mv is effectively atomic — it just swaps the inode. Whatever happens — sleep/wake, crash, external API failure — a half-finished folder never appears in ~/Desktop/ASMR/.

5. For launchd, specify .venv/bin/python by full path

launchd's PATH is only /usr/bin:/bin:/usr/sbin:/sbin. python3 and ffmpeg aren't on that PATH. Even source-ing .venv/bin/activate only affects resolution within that subshell. Writing the full path like $HOME/dev/comfyui/.venv/bin/python and including /opt/homebrew/bin in the plist's EnvironmentVariables is the most reliable approach.

6. Lock with atomic mkdir + automatic zombie reclamation via kill -0

Since flock isn't available on macOS, exploit the atomicity of mkdir. Without automatic reclamation of zombie locks, the pipeline skips forever after a crash unless a human manually deletes .daily.lock.d.

7. Put the idempotency check before ComfyUI startup

EXIST=$(find "$DEST" -maxdepth 1 -type d -name "${DATE}_*" 2>/dev/null | head -1)
if [ -n "$EXIST" ]; then log "already stocked for $DATE ($EXIST); skip (idempotent)"; exit 0; fi

If today's output exists, exit 0 immediately — not even the 10-minute ComfyUI startup wait happens. No matter how many times launchd's catch-up invokes it, every call after the first returns in milliseconds. The invariant "no regeneration happens without a --force flag" is what keeps the Desktop's state stable.

8. A two-slot design (07:00 / 14:00) puts recovery on the machine, not a human

If the 07:00 slot fails, the 14:00 slot handles catch-up recovery. I've repeatedly experienced coming home to find today's output already generated, thanks to this design. Spacing the slots 7 hours apart means no contention even when ComfyUI is in the middle of heavy processing.

9. Let coverage.csv end the "is it still working?" anxiety in one line each morning

date,theme,duration_s,sounds,size,status
2026-07-03,rainy_cafe,1800,3,412M,OK+uploaded
2026-07-02,forest_night,1801,4,398M,OK+uploaded

As long as OK+uploaded keeps stacking up, there's nothing to touch. The main goal is breaking the loop where anxiety drives you to make unnecessary changes that create bugs. If status is stuck at OK (no upload), check the expiry of ~/.youtube/token.json or the YouTube API quota.

10. Use the verification step to catch "the empty video FFmpeg quietly produced"

DUR=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$VIDEO" 2>/dev/null | cut -d. -f1)
WANT=$((MIN*60))
[ -n "$DUR" ] && [ "$DUR" -ge $((WANT-3)) ] && [ "$DUR" -le $((WANT+3)) ] || die "duration check failed"

FFmpeg can produce a 0-byte or 10-second video without returning an error code. Confirming thumbnail size (1280x720), the presence of both video and audio streams, and a non-empty metadata file before the atomic move prevents the silent failure of an incomplete video landing on the Desktop.

11. Keep work on failure for debugging, and sweep it manually on a schedule

Deleting work/ on failure destroys reproducibility. Because work survives with still.png, loop.mp4, and bed.wav intact, you can inspect it immediately with ffprobe or by playing it. In exchange, check the growth of ~/dev/asmr-factory/out/ weekly and delete old work directories by hand.

12. Protect token.json with 0o600 and back it up

os.chmod(TOKEN, 0o600)

Lose the two files ~/.youtube/client_secret.json and token.json and you're starting over from --auth. Include them in Time Machine or encrypted-storage backups to guard against accidental deletion.

13. The real code is canonical, not the docs — leave the "why" in comments

README.md says fetch is timeout 90, but the actual daily.sh uses timeout 240. Values changed during operation get reflected in code, but the README usually can't keep up. Don't create a second source of truth elsewhere; leave the reasoning in the code itself:

# timeoutでDLハング(freesoundのプレビューDLにtimeout無し)を防ぐ→失敗扱いで次へ
if timeout 240 python3 "$LIB/freesound_fetch.py" ...

One comment line explaining "why 240 seconds" is the best documentation you can leave for your future self.

Wrap-up

The week after I was laid off and my income went to zero, this pipeline was the first thing I put in order.

Every morning at 07:00, launchd starts daily.sh, ComfyUI comes up on its own, CC0 sources are fetched from Freesound, FFmpeg finishes an ASMR video, and it gets uploaded to YouTube as private — the entire chain runs completely unattended. Asleep or out of the house, the line count in coverage.csv keeps growing.

I've spent two posts on the technical points, but the core is simple: don't make the hands-on work faster — build, once, a state where output accumulates without your hands. ComfyUI's 600-second wait problem, the Freesound DL hang, launchd's PATH problem — all of them reached their current form through repeatedly breaking and fixing. Every "why" line I left in the comments is a trace of that. ¥1.2M in monthly revenue is the result of keeping this thing running 24 hours a day for six months.

Next time I'll dig into the 7-theme design of themes.json, and how to write ComfyUI prompts so auto_masks.py reliably detects fire and rain.

The full picture of the setup, the breakdown of the ¥1.2M/month, and the 30-day procedure are collected in a paid note.
📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*