Every government funding program answers the same five questions: who can apply, how much money, what it's for, what the deadline is, and how to apply. That consistency is a trap. The moment you try to extract those five answers programmatically across thousands of sources, you discover that no two agencies agree on how to say any of them — and the field you'd least expect to cause trouble, the deadline, turns out to be the one that breaks the most things.
Here's what that actually looks like once you try to parse it at scale.
The Same Program, Five Different Shapes
A funding opportunity can arrive as:
A structured JSON record from an agency API (rare, and even then inconsistently populated)
An HTML table on a state economic development site
A scanned PDF Notice of Funding Opportunity (NOFO) with eligibility buried in paragraph four
A press release announcing a program that links out to a different page for actual details
A one-page flyer with the real rules referenced as "see 2 CFR 200" and nothing else
The same underlying program — say, an SBIR Phase I solicitation — might be machine-readable on SBIR.gov and simultaneously exist as a 40-page PDF on the funding agency's own site with additional, non-redundant eligibility detail that isn't on SBIR.gov at all. There's no single "canonical" version to rely on. You need both, reconciled.
This is the core data engineering problem: you're not parsing one format, you're maintaining parsers for an open-ended, growing set of formats that don't share a spec.
Eligibility: The Field That Refuses to Be a Field
Eligibility is the highest-value field in the entire dataset — it's the difference between a relevant result and wasted applicant time — and it's also the least likely to exist as structured data.
When it is structured, you might get something like:
json
{
"eligible_applicant_types": ["small_business", "nonprofit"],
"naics_codes": ["541511", "541512"]
}
When it's not, which is most of the time, you get prose like:
"Applications will be accepted from small business concerns as defined in 13 CFR 121.201 that are majority-owned by U.S. citizens or permanent residents, provided the applicant has not received more than one prior Phase II award under this topic in the preceding three fiscal years."
That sentence contains at least four distinct, separately-checkable constraints, cross-references an external regulation by number, and uses phrasing ("small business concerns as defined in...") that only resolves to a concrete rule if you already have 13 CFR 121.201 parsed too. Extracting this reliably means combining rule-based pattern matching for well-known regulatory phrases with LLM-assisted extraction for everything else, then flagging low-confidence extractions for review rather than silently guessing. Treating this as a plain NLP entity-extraction task undersells it — half the work is knowing which external regulations a sentence is implicitly pointing to.
Deadlines: Small Field, Huge Failure Cost
Of every field in a funding record, deadlines cause the most damage when wrong, and they're surprisingly hard to get right. A few reasons:
They come in incompatible formats. "Rolling," "Q3 2026," "the 15th of each month," "45 days after LOI acceptance," "no later than 5:00 PM ET on the due date," and plain ISO dates all show up across different sources for the same type of program.
They're relative, not absolute, more often than you'd expect. "60 days from opportunity posting" only resolves to a real date if you correctly captured the posting date — which itself might have been silently updated. Programs get amended, and the amendment sometimes shifts every downstream deadline without a clear changelog.
Time zone and cutoff time are usually missing. "Due August 15" with no time zone is ambiguous by up to three hours across the US, and federal deadlines are frequently interpreted strictly — missing a submission portal's cutoff by minutes, because you inferred Eastern instead of the portal's own server time, is the kind of failure users don't forgive.
Stale re-posts are common. A program page gets re-crawled, the text is 99% identical to last year's cycle, but the deadline field is the one line that changed. If your diffing logic isn't specifically deadline-aware, this is exactly the kind of change a generic "has the page changed" check can miss or mis-flag.
The fix that worked for us: treat deadline extraction as its own subsystem, not a side effect of general page parsing. Parse to a structured (date, time, timezone, confidence, relative_basis) tuple, always preserve the source phrasing alongside the parsed value, and never surface a normalized date to a user without also showing what the original text said. When parsing confidence is low, show the raw text instead of a guessed date — a wrong-looking guess is worse than an honest "couldn't parse."
Building the Normalization Layer
A pattern that holds up well in practice:
Format-specific extraction, not one universal parser. HTML tables, PDFs, and API responses need different extraction strategies feeding into the same target schema — don't try to force one parser to handle all input shapes.
A canonical schema everything maps to. Applicant type, funding amount range, sector/NAICS, deadline (structured), and geography, regardless of source format.
Confidence scoring per field, not per record. A record can have a rock-solid deadline and a garbage eligibility extraction. Blending that into one record-level confidence score throws away exactly the information you need to know what to double check.
Preserve source text next to every normalized value. Every structured field carries its original phrasing. This is your debugging tool, your audit trail, and your fallback UI when confidence is low, all at once.
Re-crawl with diffing tuned to high-value fields. Treat deadline and eligibility changes as higher-priority diffs than cosmetic page changes, since those are the fields most likely to silently shift and most costly when missed.
The Actual Lesson
It's tempting to treat a deadline as a solved problem — dates are dates, right? In practice, a deadline field is only as trustworthy as the messiest source that feeds it, and government funding data is nothing but messy sources. Getting "deadline: August 15" to reliably mean what an applicant needs it to mean takes a dedicated parsing subsystem, not a regex and a prayer. Treat it like the high-stakes field it is, because getting it wrong doesn't just corrupt a record — it costs someone a real opportunity.
