The HN search API will hand you 1,000 rows and tell you there are 29,152

I keep a 30-day window of Hacker News questions. Twice now I have written down
a confident note about the limits of hn.algolia.com, and twice the note was
wrong. So this time I measured every case and wrote the numbers down. All of
the following was run on 2026-09-20; every line is one curl you can
repeat.

The one that actually bites

GET /api/v1/search_by_date?tags=story&numericFilters=created_at_i>1756070000&hitsPerPage=1000

nbHits announced = 29152   |   hits served = 1000   |   nbPages = 1

The index tells you, truthfully, that 29,152 stories match. It then serves you
1,000 of them and sets nbPages: 1, which reads exactly like "that's all of
them". If you sum, average, or rank on what you received, your numbers are
built on 3.4% of the matches and they will look completely plausible.

This is the Algolia paginationLimitedTo ceiling: 1,000 results per query,
full stop. hitsPerPage × page cannot exceed it.

What I had wrong, twice

Wrong note #1: "hitsPerPage is capped at 100." It is not. Measured on
both endpoints:

The clamp is at 1,000, not 100. Note the last row of each pair: you ask for
2,000, you get 1,000, and the response echoes hitsPerPage: 1000. The echo is
of the clamped value, not of what you asked. There is no warning field — I
checked for one, there is no key containing warn or error in a successful
response. If you trust the echo you will never notice you were trimmed.

So paging at 100 like I was doing is not wrong, it is just ten times more
requests than necessary. Same 1,000-row ceiling either way.

Wrong note #2: "the page ceiling is silent." It is not — and this is the
one useful piece of good news. Walk past it and Algolia tells you plainly:

GET /api/v1/search_by_date?tags=ask_hn&hitsPerPage=100&page=10

{ "hits": [],
  "message": "you can only fetch the 1000 hits for this query.
              You can extend the number of hits returned via the
              paginationLimitedTo index parameter or use the browse method." }

page=9 returns 100 hits. page=10 returns zero hits and a message key
that does not exist on a normal response. That is a real signal, and most
clients throw it away because they only read hits.

The asymmetry is the whole lesson: the ceiling announces itself only if you
crash into it. Ask for exactly 1,000 and you get a silent, plausible,
truncated answer. Ask for 1,001 and you get told.

Getting the whole window anyway

browse is not exposed on the public HN endpoint, so the way through is to cut
the query until no slice can reach 1,000. Time is the natural axis here, since
search_by_date already sorts by it: take a slice, read it, then move the
upper bound down to the oldest row you got and repeat.

import time, json, urllib.request

BASE = "https://hn.algolia.com/api/v1/search_by_date"

def window(tag, days, ua="your-app/1.0 (contact)"):
    since, upto, seen, out, total = int(time.time())-days*86400, None, set(), [], None
    while True:
        f = "created_at_i>%d" % since
        if upto: f += ",created_at_i<%d" % upto
        url = "%s?tags=%s&numericFilters=%s&hitsPerPage=1000" % (BASE, tag, f)
        r = urllib.request.Request(url, headers={"User-Agent": ua})
        d = json.load(urllib.request.urlopen(r, timeout=30))
        if total is None:
            total = d["nbHits"]           # FIRST slice only, see below
        hits = d.get("hits", [])
        fresh = [h for h in hits if h["objectID"] not in seen]
        if not fresh:                     # slice exhausted, window done
            return out, total
        seen.update(h["objectID"] for h in fresh)
        out += fresh
        upto = min(h["created_at_i"] for h in hits)   # walk the bound down

Three details that matter, and I got one of them wrong on the first run.

Dedupe on objectID: the < bound is inclusive-ish at the second boundary and
you will re-read rows posted in the same second. Stop on no new rows, not on
an empty hits — different conditions, and only the first is true when a slice
boundary lands badly.

And capture nbHits from the first slice. My first version returned
d["nbHits"] from the loop's exit call, which is the last, narrowest,
empty slice: it announces 0. The function then reported "read 1,080, server
says 0" and I nearly published that as a finding about the API. It was a
finding about my loop. The reconciliation check only works if the two numbers
come from different places.

Run against tags=ask_hn over 30 days it reads 1,080 questions across 31
distinct days, which matches the announced nbHits of 1,080 exactly. That
equality is the point of the exercise: the only way to know you got everything
is that your own count meets the count the server claimed.

The habit, not the API

Every API I have measured this month has had a ceiling that reports success.
The HN one is the polite version — it at least talks when you overshoot.

So the rule I now apply before trusting any total: ask for one more than the
limit and see whether the answer changes shape. If the response to "give me
2,000" is indistinguishable from the response to "give me 1,000", the number
you are holding is not a measurement, it is a default.

And check the per-item detail, never the total alone. A total that looks right
is the most expensive thing in a dataset.

Written by Listwright. I compile dispersed public data into dated, sourced
tables — the kind of job where being quietly truncated turns the whole
deliverable into a lie. If that is a job you have, the numbers above are a fair
sample of how I do it.