n8n Loop HTTP Request Until Done

n8n Loop HTTP Request Until Done

If you’re googling how to loop an n8n workflow until the API says done, the common trap is making n8n hold the entire batch and timing out. The real trick: make your API resumable, then let n8n be a dumb loop that re‑calls it until done is true. We built exactly that for a newsletter send pipeline that mails 40 recipients per call—without double‑sending or rate‑limit meltdowns.

Why a Single HTTP Request Won’t Cut It for Batch Jobs

n8n’s HTTP Request node times out after 300 seconds by default. Push a full bulk email, import, or backfill through one call and you’re betting the job finishes before the timeout. If it doesn’t, n8n retries the same call—and if the endpoint isn’t resumable, you re‑process everything you already handled. Double‑sends, duplicate imports, corruption.

Takeaway: you need an endpoint that can be called repeatedly, makes progress each time, and never repeats work on a retry. Then the loop in n8n becomes trivial.

The Resumable API Pattern (The Part You Build)

Instead of one “send‑everything” call, we expose a POST /send-issue endpoint that processes a fixed‑size batch and reports back the job’s status. The endpoint stamps each subscriber with the issue they last received, so a re‑call after a crash picks up where it stopped and cannot double‑send.

app.post('/send-issue', async (req, res) => {
  const { issueId } = req.body;
  const issue = await getIssue(issueId);

  // Human approval gate — deliberate stop, never retry
  if (issue.status === 'draft') {
    return res.status(409).json({ error: 'Issue is still a draft' });
  }

  // Grab 40 recipients who haven’t been stamped for this issue
  const batch = await getUnsentSubscribers(issueId, 40);

  // Send with controlled concurrency (3 workers) and pacing (120ms pause)
  const results = await sendBatchWithPacing(batch, {
    concurrency: 3,
    delayMs: 120,
  });

  // Stamp every successful send so it won’t repeat
  await stampSent(results.sent, issueId);
  const remaining = await countUnsent(issueId);

  res.json({
    sent: results.sent.length,
    failed: results.failed.length,
    remaining,
    done: remaining === 0,
  });
});

The response always carries done: true/false and a count of remaining work. n8n never needs to know how many batches are left—it just asks “are we done?”.

Why the API owns pacing: Our SMTP provider rate‑limits bursts, so we tuned the worker pool to 3 parallel sends with a 120 ms pause. Those numbers live in the app, not in n8n. When you push pacing logic into the workflow, every retry, timeout, or parallel execution can explode your rate‑limit budget. Keep the API responsible for its own downstream limits.

Wiring the n8n Loop: POST, Check done, and Wait

In n8n, the Loop node runs a set of steps repeatedly while a condition holds. Here’s the exact flow we use:

1. Loop node – condition {{ $json.done !== true }}. Initial data: { "issueId": "abc-123" }. Set “Wait Between Iterations” to 1000 ms (or longer if your endpoint is heavy).
2. HTTP Request node – POST to your /send-issue endpoint with the current issueId. Set the node to “Never Error” so you can inspect the status code instead of letting n8n abort the workflow.
3. Switch node – route by statusCode:

• 409: deliberate stop. Send a Slack alert and terminate the workflow (a “Stop” node or a “No Operation, do nothing” exit). Never feed 409 back into the loop.
• 200, 201, 5xx: continue. For 5xx you may want an extra Wait of 5 seconds before the loop re‑calls, but in practice our API rarely returns a transient error mid‑batch, so the plain loop is enough.

1. Loop input – the filtered output (everything except 409) feeds back into the Loop node. The condition re‑evaluates on the done flag.

That’s it. n8n acts as a patient polling agent—no batch state, no resumption logic, no time‑bomb retries.

Status Codes Are Your Stop Signs — Classify Before You Loop

A Loop node treats every non‑200 as a failure worth retrying. If you wire a 409 “draft” response into the loop without a bypass, you turn a safety gate into an infinite retry storm. We saw this during design and deliberately built the 409 classification into the workflow.

Classify deliberately:

Route 409 to a Slack/email node and a Stop node before the loop input. That single branch costs nothing but saves you from a 3 am incident.

Rate Limits, Concurrency, and Pacing: Keep Them in the API, Not n8n

We learned this the hard way when our SMTP provider throttled bursts. Initially we tried to control concurrency inside n8n, but any workflow change or duplicate execution risked blowing the limit. Moving the worker pool (3 concurrency, 120 ms delay) into the endpoint eliminated that risk. n8n calls once, receives a batch, waits a second, calls again. The API enforces its own pacing.

When you’re consuming rate‑limited services, define limits on the server that calls them. Let n8n be a thin client.

When You’d Rather Not Build the Endpoint Yourself

Designing a resumable batch API, stamping idempotency keys, and handshaking with n8n loops is a few hours of work—until you hit the edge cases around partial failures, stamp race conditions, and idempotent retries. If you’re not a backend engineer, or you’re moving dozens of pipelines into n8n, our n8n automation agency builds the server‑side logic alongside the workflows so you ship production‑grade batch processing without the gotchas. We’ve packaged patterns like this one into reusable business process automation templates, and you can see real workflow examples in our lab.

FAQ

Can I loop an HTTP request in n8n without building a custom API?

Yes. If the endpoint doesn’t maintain progress (e.g. it sends everything in one shot), a retry after a timeout will re‑process all work. You can mitigate that by tracking state in a database, but the moment you need retry‑safe batch work, the resumable pattern pays for itself.

What if my existing API doesn’t return a done flag?

Add a done flag yourself on the server side. Count the remaining items and set done to true when the count hits zero. In n8n you only need to check that flag—the loop becomes trivial.

How do I stop the loop when something goes wrong permanently?

Classify status codes before you wire the loop. Route deliberate stops (like 409 “still a draft”) to a separate branch that halts the workflow and alerts a human. Let transient 5xx errors flow back into the loop so they retry naturally, but add a Wait node between iterations to give the downstream a chance to recover.