How Fillable finds the blanks in a flat PDF form

Originally published at getnobody.org.

A lot of forms still arrive as flat PDFs: an intake form, a volunteer application, an order sheet. Nothing on them can be typed into, so people print them, fill them in by pen and scan them back. The tools that add form fields mostly want a monthly plan for something you need once. Fillable is the one-off version: you drop in the PDF, it finds the fields, you fix what it got wrong and pay $4 once for that document.

Finding the fields is the interesting part, and it all runs in the browser. This post covers how it works.

(Chart: see the original post.)

What a PDF page actually contains

A PDF page doesn't hold "a text field here" or even "a line here". It holds a content stream: a list of drawing operators like "move to", "line to", "rectangle", "stroke", "fill" and "show this text". pdf.js exposes that list through page.getOperatorList(), so the first job is to replay it and write down every straight line and rectangle it draws.

Replaying it means keeping the graphics state the way a renderer does. Every cm operator changes the current transform matrix, q and Q save and restore it, and a form XObject carries its own matrix on top. A line drawn at (0, 0) inside three nested transforms could be anywhere on the page, so every point goes through the full matrix before I keep it. The line width matters too, and it can come from a graphics state dictionary rather than the w operator.

Form generators disagree about how to draw a line. Some stroke a path. Some fill a rectangle 0.5 points tall. Some draw a box as four separate strokes. So a thin filled rectangle counts as a line, and rectangles are also rebuilt from separate lines that meet at their corners. That second step is also what finds every cell of a ruled table.

Text comes from page.getTextContent(), one run per item with its position measured from the baseline. Two extra rules come from real forms: a run of three or more underscores is a writing line typed as text, and characters like ☐ are checkboxes typed as text.

From lines and boxes to fields

With geometry and text in PDF points, classification is a set of plain rules:

1. Checkboxes are near-squares between 6 and 20 points. Each one takes the text to its right as its name. Short options also take the question on their row, so "Smoker? Yes / No" becomes smoker_yes and smoker_no.
2. Boxes that are empty become text fields, or multiline fields when they are taller than 36 points. If a label is printed inside the box, the field goes in the space under it or beside it. A box that contains other boxes is a panel, not a field.
3. Table cells that are empty are named from their column header and row label.
4. Writing lines are split wherever text sits on them, so "Signature ____ Date ____" drawn as one long rule gives two fields. The label is the nearest text to the left, then below, then above.

The kind comes from the label. "Date of birth" gets a date field and "Signature" a digital signature field (signing one needs a reader that supports digital signatures, such as Acrobat Reader; switch it to text for a typed name). Number fields are only for amounts and quantities. My first version also made phone and zip fields numbers, until I remembered what a number field does to a zip code: "02134" is shown as "2,134.00".

Last, overlapping candidates are merged, everything is clamped to the page box, and names are made unique. The server clamps again when it writes the PDF, because it never trusts coordinates from the browser.

Scanned pages have no operators

A scan is one big image and no text. When a page is mostly image and has almost no text runs, Fillable switches paths: it renders the page at 2 pixels per point, converts it to grayscale and picks a black-and-white threshold with Otsu's method (the gray level that best separates two groups of pixels, ink and paper).

Then it looks for long dark runs along each row and each column, and stacks runs on neighbouring rows into lines with a thickness. Anything thicker than 4 points is a filled bar or a photo, not a rule. Rectangles come from the lines the same way as before, and a rectangle that is mostly ink inside is thrown away. The rest goes through the same classifier as a drawn form.

There's no text recognition, so fields on a scan start unnamed. In the editor you select one, type a name and press Enter to jump to the next.

A quarter of a degree

My first version worked on a perfectly straight test scan and fell apart on the next one, which was tilted by 0.25 degrees. That tilt is invisible to the eye. But at 2 pixels per point, a line that rises by tan(0.25°) climbs one pixel every 229 pixels. A 468 point rule is 936 pixels wide, so it steps up four times, and each step splits it into a separate run. The big boxes lost their edges and the table lost a column.

The fix is to straighten the page before looking for lines. To measure the tilt, I take the dark pixels and, for each candidate angle, count how many land on each row after rotating by that angle. At the right angle, all the pixels of each rule land on the same row, so the counts are very uneven. Summing the squared counts turns that into a score. Fillable tries every 0.05 degrees between -3 and 3, then every 0.01 degrees around the best angle. It rotates the bitmap, finds the lines, and maps each field's centre back to where it sits on the original page.

On a letter page, the whole line and box search, straightening included, takes about 45 ms in my tests. The chart shows the difference on my sample intake form, scanned at different tilts:

(Chart: see the original post.)

How well it does

These numbers are from one intake form I made for testing. It has 25 fields: writing lines, two boxes, five checkboxes, a comments box, a ruled table and a signature line. Real forms are messier, which is why every field can be checked and fixed before you pay.

On the scan, the three extra fields are the table's header cells. Without text recognition they look like empty boxes. The three wrong kinds are the date and signature fields, which come out as plain text because there is no label to read.

What you get

When you download, the PDF goes to the server once, together with the field list. The server writes real AcroForm fields with pdf-lib and sends the file straight back. The PDF is never stored: the server keeps its SHA-256 hash, page count and field count, so a payment unlocks exactly that document. Date fields carry their format, checkboxes are real checkboxes, and every field keeps its name. Text fields and checkboxes work in Acrobat Reader, Preview and browsers; date formatting needs a reader that runs form scripts, and signature fields a reader that supports digital signatures.

Finding fields, editing and previewing are free. The fillable PDF is $4, once, for that document, with free re-downloads for 24 hours if you spot something to fix. There's no account and no monthly plan.

Try Fillable