License Referee — a compatibility ruling agent backed by Sanity Context

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

License Referee — paste a package.json, a GitHub repo URL, or an npm package name, and it rules on whether every dependency's license can legally combine with your project's, in which direction, under what conditions, and who said so.

"Can I use this?" is usually answered by glancing at a license badge, which is wrong in ways that matter. Compatibility is directional (Apache-2.0 code may go into a GPLv3 work; GPLv3 code may not go into an Apache-2.0 work — same two licenses, opposite answer). It's version-exact (Apache-2.0 is compatible with GPLv3 and incompatible with GPLv2, per both the Apache Software Foundation and the FSF). And authorities disagree with each other — the OSI approves licenses the FSF doesn't call free software; CC0 is FSF-recommended for software but was withdrawn from OSI review.

None of that survives a keyword search. It needs a graph: license → directional ruling → authority → obligation, with the disagreements kept visible instead of averaged away.

Demo

Live app: https://license-referee.vercel.app

Video: https://youtu.be/-cBBpuPs8IU

Try it with a real project: paste
{"license":"GPL-2.0-only","dependencies":{"@grpc/grpc-js":"^1"}}
and watch it explain, correctly, why that combination isn't allowed — and what would fix it.

Code

https://github.com/harsha-iiiv/license-referee

How I Used Sanity

Sanity project: 9qqt8rm4, dataset production. Schema: license (SPDX id, category, OSI/FSF flags, permissions/conditions/limitations), compatibilityRuling (directional from → into, verdict, combination kind, verbatim quote, source URL, optional dissent when authorities disagree), obligation (what to do, per license per trigger), source (the authorities, ranked by what they're authoritative for).

Knowledge Base: indexed from that dataset — every ruling carries a quote and source URL checked live against the authority itself (FSF, GNU, Apache, Mozilla, Eclipse, OSI, choosealicense.com) during verification.

Sanity Context tools used: initial_context and knowledge_base_read against the Knowledge Base endpoint for explanations, obligations, and cross-source reconciliation; groq_query and schema_explorer against a GROQ endpoint over the same dataset for exact, deterministic verdict lookups.

What the agent does with what it retrieves: resolves every dependency's declared license from the npm registry, normalizes it to an SPDX expression, looks up an exact ruling for each (dependency license → project license) pair, falls back to the Knowledge Base for conditions and edge cases when no direct ruling exists, and never guesses — an unmapped pair comes back "unknown," not a fabricated verdict.

Did structure matter? I ran a 22-question golden set through three configurations: this agent, a keyword-search baseline over the identical source pages, and a no-retrieval baseline.

The telling case: an obscure license (BlueOak-1.0.0) into a GPL-3.0-only project. All three modes reached the correct verdict, but in one no-retrieval run the model backed it with a citation that doesn't exist — a fabricated FSF license-list entry, checked live against the real page. Both retrieval-backed modes correctly said no FSF ruling exists for it instead. A right verdict with an invented authority is worse than a right verdict with an honest "no ruling exists."

Sanity Project Details

Project ID: 9qqt8rm4, dataset production
Studio: https://license-referee.sanity.studio/

Agent Session

The build started with a design pass: what would make this trustworthy rather than just plausible? The answer was a typed graph instead of prose — license, compatibilityRuling, obligation, and source as separate Sanity document types, with from/into as distinct fields so the direction of a ruling could never get silently inverted by a model reading too casually.

The dataset itself went through an adversarial verification pass: every ruling's quote and source URL was checked against the live page it cited. That pass caught 16 quotes that were paraphrased rather than verbatim, one wrong verdict (CC-BY-4.0 into GPL was marked incompatible; the FSF's actual position is the opposite), and a structural gap — zero rulings for a copyleft dependency going into a permissive or closed-source project, the single most consequential direction for a real npm tree. All of that got fixed and the ruling set grew from 70 to 138 entries.

The last mile was proving the thesis instead of asserting it: a golden set of 22 real questions and project audits, run through the production agent, a keyword-search baseline over the identical source pages, and a no-retrieval baseline. The gap that mattered wasn't the score — it was a no-retrieval run inventing an FSF citation that doesn't exist on the real page, while both retrieval-backed modes correctly said no such ruling exists. That's the whole argument for structured content in one example: a right answer with a fabricated source is worse than a right answer with an honest gap.