Devices publish MQTT to a broker in the cloud. It works, until the link to the site goes down for forty minutes.
What happened to the readings in between is usually discovered weeks later, by someone looking at a gap in the historian while investigating something unrelated.
There are two standard answers, and they're both reasonable.
QoS 1 with a persistent session. The client retries unacknowledged messages on reconnect. Works up to the client's queue depth, which on constrained hardware is small. When it fills, messages drop silently. Reboot mid-outage and the in-memory queue goes with it.
A local broker bridged to the cloud. Mosquitto at the site, queueing while the link is down, draining when it returns. This is the correct shape and where most mature deployments land.
The cost of the second one isn't technical. It's that you now run a broker per site — its own config, its own credentials, its own topic tree, its own upgrade window. One site is fine. Two hundred is a job.
And the namespaces drift. site-14/line1/temp in one place, plant14/Line_1/temperature in another, because they were configured eighteen months apart by different people. So the cloud side grows a translation layer, and adding a site means touching it.
The part that actually hurts
Buffering is a solved problem. Every bridge does it.
The unsolved problem is that each site is its own island with its own identity and its own naming, and the integration burden grows with every site you add. The buffering question gets all the attention because it's the one that shows up in an incident review. The namespace question is the one that quietly sets how fast you can roll out.
A different split
Keep MQTT at the device — nothing on the plant floor changes. Put something at the site that speaks MQTT to those devices, persists locally, and is part of one network with the hub rather than bridged into it.
With NATS that's a leaf node with JetStream enabled at the site, and the MQTT interface turned on:
server_name: edge-site-1
jetstream {
domain: edge
store_dir: /data
}
mqtt {
port: 1883
}
leafnodes {
remotes: [ { url: "nats-leaf://hub:7422" } ]
}
Devices publish to port 1883 exactly as before. The site holds its own storage. The hub sources from it, and catches up when the link returns.
Two details worth knowing. MQTT support requires JetStream — sessions and QoS 1 state live in streams, so it isn't optional. And MQTT topic levels map onto NATS subject tokens, so factory/line1/temp becomes factory.line1.temp. The / becomes a ., which matters before you name anything sensor.1.
The difference from bridging is that factory.line1.temp means the same thing at the site and at the hub, because it's one network rather than two with a mapping table between them. Site 201 doesn't require editing anything.
What it doesn't solve
You still run a process at every site. Retention at the edge is a real sizing decision — how long an outage you survive is a number you choose and pay for in disk. And catching up is asynchronous, so if something downstream needs strict ordering across sites, that's a design conversation rather than a config flag.
If this is your problem
I work at Synadia, where we maintain NATS. We're running a free 90-minute session on this on October 14, going deeper on security, fleet configuration, and what changes at a few hundred sites: synadia.com/lp/enterprise-nats-use-cases
