I Broke My Own GitHub Trending Scraper 5 Ways on Purpose. All 5 Builds Stayed Green.

I run a zero-API GitHub Trending aggregator. In the 52 days since I set it up, it has fired 201 times and 199 of those runs succeeded. The only two failures happened on day one and had nothing to do with scraping — GitHub Pages wasn't enabled yet, so the deploy step returned a 404.

A wall of green builds reads like a health signal. So this week I stopped trusting it and tried to break my own parser on purpose.

I fed it five kinds of damaged HTML — the kind of damage GitHub actually introduces during a routine CSS refactor. Every single test returned normally. No exception, no non-zero exit, no failed build. The site would have published anyway, with empty or corrupted data, and CI would have filed it as a success.

Here is the full teardown, the exact numbers, and the completeness canary I now run before anything ships.

What the pipeline actually is

No token. No REST API. No rate limits. That was the design constraint, so the whole thing is:

1. GitHub Actions cron, 0 */6 * * * — four runs a day.
2. urllib.request GETs six pages: github.com/trending?since=weekly plus the python, javascript, typescript, rust, and go variants.
3. A regex parser extracts repo name, description, language, total stars, forks, and weekly star gain.
4. It writes trending.json + summary.json + a static index.html and uploads the folder as a Pages artifact.

The latest run collected 116 rows across those six pages (22 on the all-languages page, 20 / 18 / 18 / 19 / 19 on the language pages). Across 201 runs that's roughly 23,000 repo rows, all from HTML, all for $0.

What that weekly snapshot actually says

Before the failure analysis, the payload, from the run at 2026-09-17 04:39 UTC — all numbers below come straight out of the published trending.json:

The 22 repos on that page represent 1,709,069 stars in total.

I classified them with an explicit, reproducible rule: a repo counts as agent infrastructure if its name or description contains one of skill, plugin, agent, harness, context, memory, mcp, prompt, spec-kit, superpowers, humanizer, no-ai-slop.

Result: 19 of 22 repos (86%), holding 1,397,405 of those stars (82%).

The three that didn't match are bilawalsidhu/gods-eye-view (browser spy-satellite simulator), microsoft/markitdown (document → markdown), and home-assistant/core. The rule is keyword-based and I'm not going to pretend it's a semantic classifier — but when the three exceptions are a satellite simulator, a document converter, and Home Assistant, the trend isn't ambiguous. What developers are starring hardest in September 2026 is the layer that wraps the model: skills, harnesses, context stores, plugin registries, and anti-slop filters. blader/humanizer (49,268★) and petergyang/no-ai-slop (10,178★) are both in the list — tools whose entire job is cleaning up output that another model produced.

The five ways I broke it

Here's the test harness. I monkeypatched urllib.request.urlopen to return saved GitHub HTML from disk, so every run is offline and deterministic, then scored the parser on four dimensions instead of just "did it crash":

• rows — article blocks found
• name — non-empty repo names
• desc — non-empty descriptions
• lang / stars — populated fields

The results:

Test 4 and 5 are not hypothetical. GitHub is in the middle of prefixing its utility classes with tmp-, and today's trending markup reads class="col-9 color-fg-muted my-1 tmp-pr-4". A parser pinned to the full class string doesn't crash on that — it silently returns zero descriptions while still reporting 21 healthy rows. Your row count looks perfect and every card on the site is blank.

And the one that genuinely frightened me:

--- TEST 6: selector requiring <a href="/ to be the FIRST attribute ---
    articles: 21
    extracted: 21  of which wrong (end with /stargazers): 21
    sample wrong value: omacom/omarchy/stargazers

--- TEST 7: same markup, attributes allowed before href ---
    extracted: 21 | sample: omacom/omarchy | wrong: 0

GitHub reordered the attributes on the anchor inside each <h2>. The old-style selector <h2[^>]*>.*?<a href="/([^"]+)" requires the tag to literally begin with <a href=". It no longer does, so the regex skipped the repo link entirely and latched onto the next <a href="/..."> in the block — which is the stargazers link.

21 out of 21 repo names came out as owner/repo/stargazers. Not an error. Not a warning. A perfectly-formed string with the wrong content, in every row, in every build.

Why the green build is the wrong instrument

The aggregator had exactly one health channel: the build status. The build status measures whether the code ran. It says nothing about whether the output was shaped like the output.

That means a green run has two indistinguishable meanings:

• fresh data, or
• no data, published confidently

There's a second blind spot in the same archive. Of the 201 runs, 17 pairs of consecutive runs were more than 8 hours apart, the longest being 13.2 hours, against a nominal 4-per-day schedule of ~208 slots. GitHub Actions cron drifts and queues. From the outside, a 13-hour hole and a 6-hour heartbeat look identical — both are green.

This is the same class of failure I keep running into with agents: success is a statement about the executor, not about the artifact. (I wrote about a nastier version of it — twelve days of success: true with zero articles actually published — here.)

The fix: a completeness canary, run before publish

Four assertions, none of which is "did it crash":

1. Row floor — fewer than 15 rows means the selector rotted.
2. Description coverage ratio — if under 80% of rows have a description, the field selector broke even though the row count looks fine. This is the assertion that catches Tests 4 and 5.
3. Name shape — every repo name must contain exactly one /. This catches the owner/repo/stargazers corruption. A shape check, not a length check, because the failure was well-formed.
4. Snapshot age — an unparsable or stale updated_at fails. This is the only thing that catches the 13-hour cron hole.

Run against the live artifact and against deliberately damaged copies:

LIVE  (2026-09-17 04:39 UTC): PASS
EMPTY scrape               : ['row count 0 < 15']
DESCRIPTIONS wiped         : ['description coverage 0/5 < 80%']
STALE snapshot (9.0h old)  : ['snapshot age 9.0h > 8h']

(The 0/5 is because the trimmed summary.json only keeps the top five rows for the Actions log — the canary itself runs against the full trending.json.)

def verdict(doc, lang_counts, min_rows=15, min_desc_ratio=0.8, max_age_hours=8):
    problems = []
    repos = doc.get('repos') or []
    total = doc.get('total_repos') or 0

    if total < min_rows:
        problems.append(f'row count {total} < {min_rows}')

    desc = [r for r in repos if (r.get('description') or '').strip()]
    if repos and len(desc) / len(repos) < min_desc_ratio:
        problems.append(f'description coverage {len(desc)}/{len(repos)} < {min_desc_ratio:.0%}')

    bad = [r.get('name') for r in repos
           if not r.get('name') or r['name'].count('/') != 1]
    if bad:
        problems.append(f'{len(bad)} malformed repo names e.g. {bad[:2]}')

    for k, v in (lang_counts or {}).items():
        if v < min_rows:
            problems.append(f'language page "{k}" only {v} rows')

    t = datetime.datetime.strptime(doc['updated_at'], '%Y-%m-%d %H:%M UTC')
    age = (datetime.datetime.utcnow() - t).total_seconds() / 3600
    if age > max_age_hours:
        problems.append(f'snapshot age {age:.1f}h > {max_age_hours}h')

    return problems

Wire it into the workflow so a non-empty problems list exits 1 and the last-good artifact stays published. A failed build that keeps good data beats a green build that serves an empty page — and it is the only way the alert channel ever fires.

Two selector rules that would have prevented Tests 4–7 outright, both learned the hard way:

• Never require attribute order. <a[^>]*href="/([^/]+)/([^"/]+)", not <a href="/.
• Never pin a full class string. class="col-9[^"]*", not class="col-9 color-fg-muted my-1 pr-4".

Both are one-character-worse regexes that survive a refactor that already happened once this month.

The takeaway

Scrapers don't fail loudly. They degrade into well-formed nonsense, and the more CI you have, the more confident you get about it, because a green checkmark is a much louder signal than a missing description.

If you're running anything that scrapes, check the shape of your output on every run: field coverage ratios, value-format assertions, and freshness — not just exit codes. Count what should be there, not what happened to come back.

The aggregator is open source and updating every 6 hours: github.com/Felixwang007/github-daily-trending — a live snapshot site plus the parser and workflow, MIT, no API keys needed. If you build something on top of it, the canary above is the part worth stealing.

I also publish the smaller tools I use daily — an A-share three-pillar screener, a technical-indicator library ported to match TongDaXin's formulas, and the de-slop writing checkers — on 虾评 and at github.com/Felixwang007.

Next week: I'm wiring the same canary pattern into a stock-data pipeline, where the failure mode is worse — a silently dropped field doesn't blank a web page, it corrupts a signal.