A temperature sensor reports 71.6, and it means Fahrenheit. Meanwhile, the canonical unit for temperature in the system is Celsius. It is a simple conversion: a subtraction and a multiplication, and the driver bringing the sensor's reading into the system has everything it needs to perform it.
But it must not convert. What it publishes is 71.6, tagged with the unit the device actually used: the conversion happens on the far side of a wire boundary.
Two things break if the conversion happens at the driver. The number the sensor actually produced stops existing anywhere, so the reading cannot be reproduced once the unit tag or the scale factor turns out to have been wrong - and on a long enough timeline one of them is bound to be. The other is that the driver becomes a second place where the meaning of a point gets decided, when the registry is supposed to be the only one. A protocol adapter running on a Raspberry Pi in a cupboard is a poor place to keep half a semantic model.
Converting a unit is the first item on a list of eight things a driver is forbidden to do, and that list is the spine of this document. Everything else here - the process model, the assignment export, the subject layout, the buffering rules, the deliberately narrow command contract - is what remains once protocol ugliness has been confined to one side of that line and meaning to the other.
What this document owns: the driver host and instance process model, the assignment export and why it is pushed rather than queried, the instance lifecycle, the bus subject layout and the limits on the observation stream, the driver-supplied half of the observation envelope, local buffering and the one place in the path where data is dropped by design, availability declarations, driver-side command handling, discovery and quarantine, the prohibitions list, and the conformance harness every module is tested against. What it deliberately does not own is anything about what a value means, which belongs to The Semantic Model; command verification and the epoch whose token the driver checks, which belong to Core Runtime; and the permissions that turn the subject layout into an authorisation surface, which belong to Security Model. The boundary drawn here is what lets all three assume a device looks the same whatever protocol it speaks.
The driver contract has one job: make every device look identical to the core while keeping all protocol ugliness on the far side of a wire boundary. Everything below follows from drivers being dumb by design.
Process model
Driver host is the deployment unit — one container or systemd unit per network segment. It loads driver modules and supervises driver instances, one per configured device or device group.
driver host "rpi-c1"
├── modbus-tcp → instance "boiler-relay"
├── mqtt → instance "esp32-fleet"
└── ble → instance "ble-local"
Supervision is per instance. A wedged Modbus poller must not take down MQTT ingest in the same host.
One northbound transport, for every protocol without exception. Every driver host reaches the core the same way: over the bus, in the observation envelope below. A second northbound path for any one protocol buys two failure models, two buffering implementations, and two places to fix backpressure, in exchange for convenience in a single module.
The temptation is strongest for BLE, because the radio work genuinely is bespoke — supervision, retry-on-connect, per-device connection state — and it is easy to let that bespokeness leak upward into how the host talks to the core. It stays below the line. Whatever a BLE host has to do to hold a connection is its own business; what it emits is an observation like any other.
Assignment, not configuration
Drivers must not read the registry, but they need to know what to poll. The registry exports assignments rather than having drivers query it.
On startup a driver host requests its assignment from the core and caches it to local disk. If the core is unreachable, it starts from cache. An assignment carries addressing only:
driver_instance: boiler-relay
module: modbus-tcp
transport: { host: 10.0.4.12, port: 502, unit_id: 3 }
points:
- point_id: boiler.ch1.flow-temp
read: { fn: input_reg, addr: 30001, type: int16, scale: 0.1 }
poll: { interval: 30s }
- point_id: boiler.ch1.pump
read: { fn: coil, addr: 1 }
write: { fn: coil, addr: 1 }
idempotent: true
No quantity, no unit, no subject, no freshness policy. The driver sees a point ID, an address, and a scale factor. scale is a raw-register-to-native adjustment, not a unit conversion — a Modbus register holding tenths of a degree is a wire encoding, not a unit.
Credentials are referenced, never carried. Where a transport needs one — an MQTT broker password, a vendor API key — the assignment names it:
transport: { host: mqtt.lan, port: 8883, credential: mqtt/fleet }
The driver host resolves that reference against its own local store, provisioned out of band. The bus never carries a secret and neither does the on-disk assignment cache, which matters because the cache sits on every host and the assignment subject is readable by anything that can subscribe to it. See Security Model.
Assignment changes arrive as a push on the bus with a version number. The driver applies them without restarting: new points start polling, removed points stop, changed addresses rebind. This is also the load-shedding lever — raising poll intervals across a fleet is an assignment change that takes effect in seconds, which is the manual response to sustained ingest lag described in Core Runtime.
Lifecycle
init → connecting → connected → ready → degraded → connecting
↓
stopping → stopped
-
connecting— transport not established, exponential backoff with jitter, capped around 60s -
connected— transport up, initial reads in flight -
ready— initial read pass complete, or explicitly skipped for points with no readable state -
degraded— transport up, some points failing; instance stays up and keeps serving the rest
The connected → ready transition is what makes cold start work. Until an instance is ready, the core keeps its points in restored or unknown rather than promoting them, rule warmup gates on it, and the command plane treats every claim on its points as undeliverable rather than sending into the dark.
Instances publish their own lifecycle as points under subject: self — driver.boiler-relay.state, .reconnects, .poll-latency, and the queue and rejection counters below. Driver health is not a separate observability channel; it is the same pipeline, with the same freshness and alerting. Driver monitoring comes free, and the ingest path is exercised constantly as a side effect. The full catalogue of what a driver must publish is in Observability.
Bus subjects
obs.{host}.{instance}.{point_id} observations
avail.{host}.{instance} device/component availability
quarantine.{host}.{instance} unregistered points
cmd.{host}.{instance}.{point_id} commands, core → driver
cmdres.{host}.{instance}.{command_id} command outcome, driver → core
assign.{host} assignment push, core → host
Every driver-plane subject leads with the host, and that is what makes bus permissions workable: one credential grant per host covers every instance it will ever run, so credentials stop needing to change whenever an assignment does. Two places to keep in step is one too many, and the one that gets forgotten is the credential.
cmdres carries host and instance for the same reason. A subject of cmdres.{command_id} alone would force every driver to hold cmdres.>, and any driver could then forge an outcome for another driver's command. See Security Model.
Commands route by instance rather than by point, because the core knows the binding and the driver should not have to filter a firehose. Each instance subscribes to cmd.{its host}.{its own id}.>.
Stream limits
obs is a single JetStream stream, and it is the system's outage buffer. Three properties, all load-bearing:
- Reject new publishes when full; never discard old. A full stream pushes back onto the driver's local queue, which is bounded and reports its drops. Discarding old messages would delete observations that had already been accepted, between two components that both believe the handoff succeeded.
-
Bounded by bytes and messages, with no
max_age. An age limit deletes unconsumed messages silently, which is the same failure by a different route. Age-based deletion belongs only where dropping old data is the actual intent, which here is Postgres retention. - Sized from tolerable outage and utilisation, not from a duration that feels sufficient for a restart. The arithmetic is in Core Runtime, and the size cannot be computed until capacity targets exist.
Durability lives in Postgres. The stream exists so that a core outage costs nothing, not as an archive.
Northbound: the observation
Exactly the envelope from the semantic model:
point_id, value_native, unit_native?,
quality: live | bad | unavailable,
reason?, detail?,
device_time?, observed_at, source_seq,
driver_id, driver_instance, assignment_version
unit_native is populated only when the source self-describes — Shelly and UniFi JSON do, Modbus does not. When present it drives the mismatch check; when absent the registry is trusted.
source_seq is monotonic per instance and resets on restart, with the restart marked by an epoch counter. (epoch, seq) is what the core dedups on.
A driver may publish only for points its own assignment names. The core enforces this rather than trusting it — see the publisher check in Ingest Path & Log Schema — because subject permissions confine a host to its own subjects but say nothing about which point IDs it puts inside them.
Push and poll are the same thing
The driver module implements one method: emit observations. Whether that is an MQTT callback or a poll loop is internal.
Poll scheduling lives in the driver, never the core, for two reasons: the scheduler must survive a core outage, and it needs to be adjacent to the transport to do jitter, coalescing, and batching. A Modbus driver reading twenty registers should issue one multi-register read, not twenty round trips the core scheduled independently.
Every poll produces an observation. A poll that returns the same value still publishes, because the deadband decision is core-side and freshness needs the refresh. A poll that fails produces quality bad with the reason — not silence.
Buffering
When the bus is unreachable or full, the driver buffers to a local disk queue and replays with original observed_at values intact.
This is exactly why ingested_at exists as a separate field. A driver host that buffers through a twenty-minute partition and then floods must not have those readings evaluated as fresh. The core orders by ingested_at, computes freshness from observed_at, and a rules engine seeing a burst of old readings treats them correctly.
The queue is bounded by size and age. On overflow it drops oldest and publishes a self point recording the drop count — silent data loss is the failure mode nobody notices, and this is the single place in the whole path where data is dropped by design.
Availability
The driver declares unavailability; the core fans it out.
avail.{host}.{instance}: {
scope: instance | device | component,
target: "boiler" | "boiler.ch1",
available: false,
reason: transport_timeout,
observed_at
}
The driver names the scope and the core marks every affected point unavailable. This is the difference between "the cause of the silence is unknown" (stale) and "the device is off the network" (unavailable), and only the driver can tell which.
A driver that dies without saying anything is caught by its own heartbeat going stale — the core marks the instance unavailable after a grace period. Both paths converge on the same state.
Southbound: commands
command_id, point_id, value, issued_at, deadline, idempotent, core_epoch
The driver's contract is narrow. It attempts the write and replies once:
command_id,
outcome: accepted | rejected | failed | timeout,
device_ack?: { value, observed_at },
reason?, detail?
accepted means the transport accepted it. device_ack is populated only when the protocol returns confirmation inline. Everything else the core owns.
core_epoch is a fencing token. The driver tracks the highest epoch it has seen and rejects any command carrying a lower one, with outcome rejected and reason stale_epoch. It needs to understand nothing else about the value, only that it never goes backwards. The reasoning is in Core Runtime; the effect is that a core which has already been replaced cannot still be driving relays through a partition it has not noticed yet.
Verification is core-side, without exception. poll_verify is the core issuing a read command after a delay. async_event is the core correlating a later observation. optimistic is the core marking the point assumed. The driver has no concept of confirmation modes, no timers, and no memory of what it was asked to do.
That boundary matters because verification needs durable timers and registry knowledge, and because a driver that retries on its own behalf will eventually double-fire a non-idempotent command. Drivers retry transport establishment, never command delivery — unless idempotent: true, which permits one retry within the deadline.
Commands are deduplicated on command_id within a short window, so a bus redelivery does not pulse a relay twice.
Discovery and quarantine
An observation for an unknown point_id goes to quarantine.{host}.{instance} with its full native payload, never to obs.*. The core records it, deduplicates by point ID, and surfaces it for triage. Nothing auto-registers.
This gives discovery as a side effect of normal operation. Connect a new device, let the driver run, and watch what accumulates in quarantine. Promoting a point is a registry commit supplying quantity, subject, unit, and freshness — the four things no driver can guess correctly.
For protocols with real discovery (mDNS, vendor announce messages, device lists), the driver publishes candidates to the same quarantine subject in the same shape. One triage path regardless of how the point was found.
What drivers must never do
Worth writing down and enforcing in review, because each of these is a plausible-looking shortcut that dissolves the boundary:
- Convert units, or emit anything but
value_native - Assert quality other than
live,bad, orunavailable - Invent, rename, or auto-register points
- Publish for a point its assignment does not name
- Implement debounce, deadband, moving averages, or hysteresis
- Retry a non-idempotent command
- Hold verification state or timers
- Read the registry directly
Conformance
The module interface is small enough to test properly, and with six or more modules to write, a shared harness pays for itself immediately.
connect() → transport
read(points) → observations
write(point, value) → outcome
close()
A conformance suite is built alongside the first driver: a mock transport that produces malformed payloads, unmapped enum values, out-of-range readings, mid-poll disconnects, slow responses that exceed a deadline, and clock skew. Every module runs the same suite. The failure modes are protocol-independent even though the protocols are not.
mqtt and modbus-tcp come first — push and poll, self-describing and not, verifiable and not. Between them they exercise every branch of this contract. Other protocols are then variations rather than new problems.
Exit criteria
The conformance suite above is this step's exit criterion, not a nice-to-have. A driver module is done when it passes the shared harness, and mqtt and modbus-tcp both pass it before anything else is written.
Three additions to the harness that come from later documents and belong in the same suite:
-
Epoch rejection. A command carrying a
core_epochbelow the highest seen is rejected withstale_epoch, and the driver does not attempt the write. Tested by replaying an old command after a higher epoch has been observed. - Publish scope. The module publishes only under its own host and instance prefix, and only for point IDs its assignment names. This is checked here as well as core-side, because a driver that gets it wrong should fail its own tests rather than fill quarantine.
- Buffer bounds. Fill the local queue past its limit and assert that the oldest entries are dropped, the drop count is published, and nothing is lost silently.
The step as a whole is done when both modules pass, and when an instance taken from connecting to ready to degraded and back publishes the lifecycle transitions that cold start and deliverability depend on.
On Monday - the Dev Diary. The third entry of what the build has had to send back to the specification.
Then the Wednesday after - Ingest Path and Log Schema. Two logs rather than one, and the column that makes the unit conversion a driver is forbidden to do reversible once it happens in the right place.
Start of the series: An Introduction. The map, the two decisions every later document is downstream of, and why a specification at this scale is being published in public while the system it describes gets built.
