Daily Cleanup Job Operations: Delete Old Node.js Express Uploads and Logs Safely

Short answer: run one externally triggered daily cleanup job, keep retention rules in the application database, delete in bounded and restartable batches, and treat the scheduler as a wake-up signal rather than the owner of correctness. For a fintech service that also retries outbound webhooks, use the same operational pattern for both workflows: durable state, an idempotency key, a lease that prevents overlapping workers, and a recovery cursor that an operator can inspect.

The least complex deployment is usually a scheduler outside the Node.js Express web process calling a dedicated worker command once per day. The familiar cron expression 0 2 * * * means 02:00 according to the scheduler's configured time zone, but the expression is the easy part. The hard part is proving what was eligible, what finished, and what happens after a process stops between selecting a row and changing it.

That distinction matters in payments. A duplicated webhook can trigger repeated downstream work; an over-broad retention sweep can erase evidence needed to reconcile it. Scheduling is therefore an operational recovery problem — not a timer problem.

How should a Node.js Express service schedule daily cleanup of old uploads and logs?

Start by separating policy, orchestration, and destructive work. Policy answers which records may be removed. Orchestration decides when a run may begin. The worker performs small, observable state transitions. Don't hide all three behind an Express route whose success is inferred from one request status.

For example, define retention from an immutable cutoff captured at the beginning of a run. If the run starts at 2026-08-12T02:00:00Z and the policy is 30 complete days, every batch uses the same computed cutoff. Recomputing now for each batch creates a moving boundary; under a long backlog, items can become eligible halfway through the same run, which makes an audit harder to reproduce.

The database should record a run identifier, policy version, cutoff, current phase, cursor, counts, and timestamps. The scheduler can retry its trigger because acquiring the run lease is idempotent. A second trigger for the same logical date either observes the existing run or resumes it; it doesn't create an independent sweep.

Keep it boring.

The scheduler also needs an explicit time zone. If the business rule is based on UTC days, configure UTC and write the rule that way. If the rule follows a local regulatory day, daylight-saving transitions need a policy decision rather than an implicit server default. I'm not sure there is one correct choice for every jurisdiction; legal retention language and the organization's reconciliation calendar resolve that question, not cron syntax.

Make deletion a state machine, not a query

A single statement such as DELETE ... WHERE created_at < cutoff looks clean until it competes with production traffic, reaches a statement timeout, or removes database metadata before an object-store deletion is confirmed. Upload metadata and object bytes live in different failure domains, so pretending the operation is atomic doesn't make it atomic.

Use phases instead. First mark eligible upload rows with a deletion run ID while holding a short database transaction. Then delete the corresponding objects with an operation that tolerates an already-absent key. Finally, remove or tombstone the metadata according to the audit policy. Logs and ordinary relational records can use their own bounded phases. A run may advance only after its current batch has a durable outcome.

This is the same shape as outbound webhook delivery. Persist an event and a stable delivery ID before attempting the call. Each attempt records a result, while the receiving system gets the stable ID as its idempotency key. Consumer acknowledgements in messaging systems exist for the same fundamental reason: delivery and processing completion are separate events, and acknowledgement controls when the broker may consider work handled. Redelivery can happen, so the handler still needs idempotent behavior.

Here is deliberately generic Python that shows the contract. The surrounding service may be Node.js Express; the algorithm belongs in a worker boundary and doesn't depend on an HTTP framework.

from dataclasses import dataclass
from datetime import datetime
from typing import Protocol, Sequence


@dataclass(frozen=True)
class Candidate:
    record_id: str
    object_key: str | None


class CleanupStore(Protocol):
    def acquire_run(self, logical_day: str, cutoff: datetime) -> str | None: ...
    def next_batch(self, run_id: str, limit: int) -> Sequence[Candidate]: ...
    def mark_object_removed(self, run_id: str, record_id: str) -> None: ...
    def finalize_record(self, run_id: str, record_id: str) -> None: ...
    def finish_run(self, run_id: str) -> None: ...


class ObjectStore(Protocol):
    def delete_if_present(self, key: str) -> None: ...


def execute_cleanup(
    store: CleanupStore,
    objects: ObjectStore,
    logical_day: str,
    cutoff: datetime,
    batch_size: int = 200,
) -> None:
    run_id = store.acquire_run(logical_day, cutoff)
    if run_id is None:
        return

    while batch := store.next_batch(run_id, batch_size):
        for item in batch:
            if item.object_key is not None:
                objects.delete_if_present(item.object_key)
                store.mark_object_removed(run_id, item.record_id)
            store.finalize_record(run_id, item.record_id)

    store.finish_run(run_id)

There is a subtle ordering choice here. Removing object bytes before metadata can leave a row pointing to an absent object during recovery; removing metadata first can orphan bytes that continue to incur storage and evade ordinary discovery. In a system where financial audit evidence matters, I prefer an explicit deleting state that remains visible to operators until both sides converge. Your mileage may vary when objects are derived, replaceable artifacts and the database is the sole authority.

Do not let the loop run without a bound merely because the sample is compact. Production workers need an execution deadline, per-batch metrics, cancellation between batches, and a cursor whose update is committed with the batch's state transition. The cursor should be based on a stable ordered key, commonly a timestamp plus a unique ID, rather than an offset that shifts as rows disappear.

Choose the scheduler by its recovery contract

The simplest service is the one whose failure model your team can operate, not the one with the shortest setup page. Compare scheduler classes only after the worker is restartable; otherwise a more capable scheduler merely invokes a fragile program more reliably.

Host cron is not suitable when the web tier is ephemeral and there is no stable worker host. A platform scheduler is a weak fit when its maximum execution window is shorter than the worst plausible backlog and it cannot hand work to a durable worker. Stick with an orchestrator-native schedule when the team already has cluster ownership, workload identity, logs, and runbooks there; adding a second scheduling control plane buys little.

A queue is justified for webhook retry when delivery attempts need independent backoff, acknowledgement, and redelivery. It need not own the daily retention policy. One scheduled message can create or resume the durable cleanup run, while the database remains authoritative about progress. Google Cloud Pub/Sub is one documented example of a messaging service, and RabbitMQ documents the distinction between consumer acknowledgements and publisher confirms; those references clarify messaging semantics, not a universal product recommendation.

The catch is operational surface area. If the workload is one daily bounded sweep, introducing a broker solely to express 0 2 * * * means another persistence system, another backlog to monitor, and another recovery procedure. Use it only when its delivery model solves a problem the database lease and existing scheduler do not.

Test the ugly transitions before rollout

Happy-path tests prove almost nothing about cleanup. The useful tests stop the worker after each durable transition: after claiming candidates, after deleting an object, after recording that deletion, and before finalizing metadata. Restart with the same logical day and verify that no eligible item is skipped, no protected item crosses the cutoff, and completed work is harmless when observed again.

Consider one interruption drill with three upload records, ordered (created_at, id) as A, B, and C. The worker claims all three under run cleanup-2026-08-12, removes A's object, records A's object phase, removes B's object, and then loses its process before recording B's phase. On restart, the ledger proves A can advance directly to metadata finalization; B's object removal is repeated through the delete_if_present contract and then recorded; C proceeds for the first time. The batch cursor advances only after all three durable row states are settled, so a crash cannot move the cursor past B. Now repeat the drill after metadata finalization but before the cursor commit: A, B, and C are selected again, their terminal row states make each operation a no-op, and only the cursor changes. This small exercise exposes more than a day of happy-path runs because it forces the team to name the authority for every transition, including the ambiguous gap between a remote object operation and its local record.

Stop there.

For webhooks, test duplicate delivery IDs, receiver timeouts, rate limiting such as HTTP 429, and a response arriving after the sender has decided the attempt timed out. A 409 from a receiver that explicitly uses it to signal an already-processed idempotency key may be a successful business outcome, but only if that contract is documented; status codes alone can't tell the worker whether the effect occurred.

Observability should answer operator questions rather than emit a celebratory “job ran” log. Record the logical day, cutoff, policy version, lease owner, oldest remaining candidate, attempted and completed counts, per-phase latency, retry count, and terminal reason. Alert on age of unfinished work and lease staleness. A zero-deletion run may be correct, so count alone is not a failure signal.

Deployment should begin in report-only mode. Compute candidates and reason codes without deleting them, compare the result with policy owners, then enable one data class with a small batch and an execution deadline. Keep the old path disabled but available until several full retention windows have passed — the exact number depends on the retention period and audit requirements, so it cannot be responsibly prescribed from scheduler mechanics alone.

A compact operational recovery rule

Use external scheduling for availability, a database lease for exclusivity, stable keys for idempotency, and checkpoints for recovery. The daily expression may remain 0 2 * * *; correctness lives in the worker's state transitions.

Roll out in this order: inventory retention rules, add a report-only run ledger, implement bounded phases, rehearse interruption and replay, enable one class of records, and only then attach the production schedule. Apply the same ledger discipline to outbound webhook attempts, but keep webhook delivery state separate from retention state so a cleanup policy change cannot rewrite delivery history.

No scheduler eliminates duplicate triggers or ambiguous remote outcomes. The design passes review when an operator can identify the last durable transition, replay from it, and explain why every deleted upload, log, or record was eligible.

References

• RabbitMQ, “Consumer Acknowledgements and Publisher Confirms”: https://www.rabbitmq.com/docs/confirms
• Google Cloud, “Pub/Sub overview”: https://cloud.google.com/pubsub/docs/overview

Further reading

• https://www.rabbitmq.com/docs/confirms
• https://cloud.google.com/pubsub/docs/overview