Cross-Region S3 Replication Without the Gotchas (2026)

Cross-region S3 replication copies objects from a source bucket to a destination in another region, continuously. AWS S3, MinIO, and RustFS all do it, but the failures almost never come from the copy itself. They come from versioning, IAM, and delete-marker defaults you forgot to set.

Key facts before you start

What is cross-region S3 replication, in plain terms?

CRR is a server-side rule that copies every new object from a source bucket to a destination in another region, asynchronously. 'Asynchronously' matters here: the source write returns success before the copy lands, so the two buckets are eventually consistent, not locked in step. Anything that speaks the PUT Bucket Replication API does the same: AWS, MinIO, RustFS. The point is disaster recovery and reads closer to the user. But 'just copy my bucket to another region' hides a dozen small defaults. Versioning state, IAM trust, delete-marker handling, KMS keys: each one fails silently instead of throwing an error you'd notice.

Versioning is the prerequisite nobody can skip

This is the one I see missed most in postmortems. Versioning has to be on, at both ends, or replication doesn't work. I've spent multiple sessions debugging broken replication from skipping the 15-minute propagation wait after enabling versioning, so now I turn it on and walk away before writing anything. AWS tracks objects by version ID, so the destination needs it too. Enabling it is one CLI call (verbatim below). AWS flags a gotcha here that's worth taking seriously:

# Enable versioning on BOTH the source and destination bucket
aws s3api put-bucket-versioning \
    --bucket amzn-s3-demo-bucket \
    --versioning-configuration Status=Enabled

AWS note: "When you enable versioning on a bucket for the first time, it might take a short amount of time for the change to be fully propagated... We recommend that you wait for 15 minutes after enabling versioning before issuing write operations."

If you attach the rule to a bucket without versioning, AWS rejects the config outright. If only the destination is missing it, the source accepts writes but every replica write fails silently, and nothing shows up remotely. That's the failure I get pinged about more than any other: replication looks healthy, the destination is just empty.

How do you set up the AWS IAM role without a silent AccessDenied?

S3 assumes an IAM role to do the copy. The role has two halves: a trust policy letting s3.amazonaws.com assume it, and a permissions policy granting the exact replication actions. Both blocks below are verbatim from the AWS replication permissions docs. The trust policy:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "Service":"s3.amazonaws.com"
         },
         "Action":"sts:AssumeRole"
      }
   ]
}

The permissions policy (source bucket = amzn-s3-demo-source-bucket, destination = amzn-s3-demo-destination-bucket):

{
   "Version":"2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetReplicationConfiguration",
            "s3:ListBucket"
         ],
         "Resource": [
            "arn:aws:s3:::amzn-s3-demo-source-bucket"
         ]
      },
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetObjectVersionForReplication",
            "s3:GetObjectVersionAcl",
            "s3:GetObjectVersionTagging"
         ],
         "Resource": [
            "arn:aws:s3:::amzn-s3-demo-source-bucket/*"
         ]
      },
      {
         "Effect": "Allow",
         "Action": [
            "s3:ReplicateObject",
            "s3:ReplicateDelete",
            "s3:ReplicateTags"
         ],
         "Resource": "arn:aws:s3:::amzn-s3-demo-destination-bucket/*"
      }
   ]
}

Where this goes wrong is quiet: drop the s3.amazonaws.com principal from the trust policy, or leave s3:ReplicateDelete out of the permissions, and replication just doesn't happen. No CloudTrail error, no SNS alert; the destination stays empty. Check the role ARN in the replication config matches what you attached, then test with one object.

The replication configuration file, field by field

With versioning and the role set, you attach a replication config to the source bucket. The command is verbatim from the AWS CLI reference:

aws s3api put-bucket-replication \
    --bucket amzn-s3-demo-bucket1 \
    --replication-configuration file://replication.json

The minimal replication.json (verbatim from AWS docs) already bakes in two defaults that bite people:

{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [
        {
            "Status": "Enabled",
            "Priority": 1,
            "DeleteMarkerReplication": { "Status": "Disabled" },
            "Filter" : { "Prefix": ""},
            "Destination": {
                "Bucket": "arn:aws:s3:::amzn-s3-demo-bucket2"
            }
        }
    ]
}

An empty Filter prefix means 'all objects'; Priority decides which rule wins when two rules match the same object. The two fields worth adding for production are Metrics / ReplicationTime, which switch on lag tracking. Note they live inside the Destination block, per the AWS schema:

{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [
        {
            "ID": "replicate-all-with-metrics",
            "Status": "Enabled",
            "Priority": 1,
            "Filter": { "Prefix": "" },
            "DeleteMarkerReplication": { "Status": "Disabled" },
            "Destination": {
                "Bucket": "arn:aws:s3:::amzn-s3-demo-bucket2",
                "Metrics": { "Status": "Enabled", "EventThreshold": { "Minutes": 15 } },
                "ReplicationTime": { "Status": "Enabled", "Time": { "Minutes": 15 } }
            }
        }
    ]
}

Metrics and ReplicationTime are real fields in the AWS replication schema; enabling them is what makes replication lag visible instead of invisible.

Why are your old objects missing from the replica?

This trips up almost everyone, and it's burned me on migrations that looked green until someone noticed six months of history missing from the replica. AWS says it plainly: 'Objects that existed before you set up replication aren't replicated automatically. In other words, Amazon S3 doesn't replicate objects retroactively.' A rule you add today only covers objects written or changed after it goes live. That 2 TB of history already in the source? It sits there until you backfill it explicitly.

On AWS, the supported way to handle that is S3 Batch Replication, a one-time job that copies existing objects on demand (the older per-rule ExistingObjectReplication option is no longer supported — AWS removed it and points to Batch Replication). Note that backfilling is a large, billable operation for cross-region transfers. MinIO does the same through the --replicate flag instead of a JSON field:

mc replicate add myaistor/mybucket \
   --remote-bucket https://user:secret@minio.mysite.tld/remotebucket \
   --replicate "delete,delete-marker,existing-objects"

The existing-objects value tells MinIO to replicate objects already there. MinIO needs versioning on first: objects written before versioning has no version ID and get excluded. Check the current MinIO docs for the exact mc version subcommand before running that, since the flag name drifts between versions.

Delete markers and KMS keys: the two settings that bite later

Delete markers and KMS encryption both default to the safe-but-surprising option, and neither one errors when it's wrong. First, delete markers. With versioning on, deleting an object writes a delete marker instead of removing data. DeleteMarkerReplication is Disabled by default, so a delete in the source does not delete in the destination, so your DR replica keeps serving the 'deleted' object. Set "DeleteMarkerReplication": { "Status": "Enabled" } if you want deletes to follow, but in a bidirectional setup that can trigger delete storms, so decide on purpose. From production experience I rarely run bidirectional replication with delete-marker sync on; a delete storm across sites is painful and slow to unwind, so I keep delete propagation off and clean up the destination by hand when something is actually gone.

Second, KMS-encrypted objects. If the source uses SSE-KMS, replication needs SourceSelectionCriteria.SseKmsEncryptedObjects.Status = "Enabled" plus a Destination.EncryptionConfiguration.ReplicaKmsKeyID, and the IAM role has to be allowed kms:Decrypt on the source key and kms:Encrypt on the destination key (kms:GenerateDataKey is only needed when you replicate plaintext objects into a destination bucket that has default SSE-KMS/DSSE-KMS encryption). Miss any of that and encrypted objects never show up at the destination while unencrypted ones do. That's a partial, silent failure that's a pain to diagnose. Safest move until the KMS grants are confirmed: keep the same key, or SSE-S3, on both ends.

How do self-hosted engines replicate across regions?

If you run your own S3-compatible storage, the model holds: a rule or a site link copies objects between two endpoints, but the tooling is different. On MinIO, the mc replicate add above is the whole flow (after versioning). On RustFS, cross-region is site replication, which links two or more independent deployments and syncs buckets, object versions, and IAM across them. The commands are from the RustFS rc client:

# Configure one alias per site (repeat for each deployment)
rc alias set site1 https://site1.example.com:9000 \
    <your-access-key> <your-secret-key> \
    --region us-east-1 --bucket-lookup path

# Confirm both sites are reachable
rc ready site1

# Link the sites (first alias receives the admin request)
rc admin replicate add site1 site2

# Inspect replication status from either site
rc admin replicate status site1

RustFS is explicit about what this needs: two or more independent deployments, a stable S3 API endpoint per site, bidirectional connectivity on the S3 API port (normally 9000), trusted TLS certs, the rc client on a secured host, root admin creds at every site, and bucket-versioning support at every site. Bring each deployment up with the verified Docker command:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default console credentials are rustfsadmin / rustfsadmin, and the console listens on port 9001.

The RustFS docs are clear about the limits: site replication is asynchronous (a successful write at one site doesn't mean the other already has it) and provides no DNS failover, no traffic routing, no application recovery orchestration. Plan your own recovery. Distributed Mode is still 'Under Testing' in the Feature & Status table, so treat each linked site as single-node-ready today, not a multi-node cluster, and test the workflow on empty sites before you link production data. RustFS gives you cross-region sync, but not a turnkey active-active setup that resolves conflicts. The docs don't pretend otherwise. If replication is the only reason you'd stand up a second cluster, do the egress math first: the bandwidth you save is usually smaller than a second deployment's run cost, and self-hosting only wins when you already own the hardware or you have a data-residency requirement.

Monitoring replication lag and the cost of cross-region traffic

You can't see replication lag until you turn on metrics. On AWS that means setting Metrics and ReplicationTime in the rule (above) and alarming on the CloudWatch metrics they produce, and without those blocks AWS emits no replication-latency metric at all. ReplicationTime lets you request a target replication window in minutes; AWS also offers S3 Replication Time Control (S3 RTC) for workloads that need a contracted SLA (99.9% of objects replicate within 15 minutes). Alarm on sustained backlog, not a single sample. Async replication falls behind temporarily under bursty writes.

On cost, cloud and self-hosted aren't a tweak apart, they're structurally different. AWS bills inter-region transfer for every replicated byte (rates vary by region pair), so a high-churn bucket racks up real egress. A self-hosted pair, RustFS via rc admin replicate add or MinIO via mc replicate add, moves bytes over your own network, so there's no per-GB cloud egress for the replication itself (you still pay your bandwidth provider). I'm not quoting a per-GB number here because AWS inter-region rates are region-pair-specific and move around. The point that holds: the engine choice decides who sends the egress bill, not whether bytes move.

FAQ

Does S3 cross-region replication copy existing objects?

No. AWS states that objects created before the replication configuration is attached are not replicated automatically. To move historical data, run an S3 Batch Replication job (the older per-rule ExistingObjectReplication option is no longer supported). MinIO covers the same case with the existing-objects value in mc replicate add --replicate.

Does cross-region replication replicate delete markers?

Not by default, and I'd leave it off unless you have a reason. In the AWS replication schema, DeleteMarkerReplication defaults to Disabled, so deleting an object in the source does not delete its replica. Flip it to "Status": "Enabled" only when the destination is a true mirror and you accept that a bad delete now spreads to both sides.

Does the destination bucket need versioning for S3 replication?

Yes, on both ends, no exceptions. Replication keys off version IDs, so the destination has to track them too. One gotcha: the change isn't instant. AWS says wait about 15 minutes after you flip versioning on before you start writing, or you'll replicate into a half-propagated bucket and wonder why objects are missing.

Can MinIO or RustFS do cross-region S3 replication?

Yes. MinIO uses mc replicate add with a --replicate flag that controls delete, delete-marker, and existing-object replication. RustFS uses site replication via the rc client: rc admin replicate add site1 site2 links two deployments and syncs buckets, object versions, and IAM. Both require versioning on the source and (for RustFS) bidirectional network access on the S3 API port.

How do I monitor S3 replication lag?

On AWS, enable the Metrics and ReplicationTime blocks inside the replication rule; only then does AWS emit replication-latency CloudWatch metrics you can alarm on. Without those blocks, lag is not exposed by default. RustFS exposes per-site replication status through rc admin replicate status site1 --metrics, which reports backlog and metrics from each deployment.

Originally published at RustFS blog. If you want S3-compatible replication without the per-GB cloud egress, RustFS is open source under Apache 2.0 and links regions through site replication. read the docs or download to try it.