Documentation

Sample quality

A failed, overridden or out-of-service sensor keeps sending values. Before #183, those values reached fault rules, Cx checks and his queries looking exactly like good data. Now each sample in point_history can carry a quality flag. A flagged value is kept exactly as received: Baler flags it and never rewrites it.

The flags

Flag	Meaning	Set by
(none)	ok. Stored as NULL, so healthy history costs nothing extra.
`fault`	The device says the value is unreliable: BACnet status-flags `fault`, or a `reliability` other than `no-fault-detected`.	the reading
`overridden`	The device says someone overrode it at the panel.	the reading
`out_of_service`	The device says the object is out of service.	the reading
`out_of_range`	Outside the plausible range for the point's Brick class and unit (below).	Baler's sweep
`spike`	An isolated jump: more than 6 scaled MADs from the median of the 5 samples on each side.	Baler's sweep
`unit_suspect`	The values fit the other temperature unit, not the one the point claims (21 "°F" in a zone).	Baler's sweep

Stale and stuck stay as they were: derived when a rule runs (his_checks), not stored. They describe a series, not one sample.

Where flags come from

The reading. An MQTT reading or a POST /ingest body may carry quality: either a flag name, or BACnet's own status:


Code
{"value": 140.0, "ts": 1786454667, "quality": {"status_flags": ["fault"], "reliability": "open-loop"}}

in-alarm alone isn't bad data: the value is real, and the alarm is about it. A reading without quality is ok, as before, and so is a quality Baler doesn't recognise. An unknown field never makes good data look bad. Windrow will send this once Windrow#20 lands. Until then, only Baler's own checks set flags.

Baler's sweep. fault_detector runs quality.sweep every 5 minutes (--quality-interval, 0 turns it off). It covers the last hour of every point that has both a Brick class and a unit tag:

Range. A wide plausible range per class and unit, meant to catch a failed sensor, not a comfort complaint. Examples: zone air temperature 32–120 °F (0–50 °C); outside air -60–140 °F; water temperature 32–260 °F; humidity 0–100 %RH; damper, valve, position and speed 0–100 %; CO2 0–10000 ppm. The table is quality.RANGES. A class with no row isn't range-checked; power is one, since it can go negative with PV.
Spikes. Each sample is compared with the median of the 5 samples before it and the 5 after. The window is centred, so a real step change (a setpoint change) isn't flagged. The newest 5 samples wait for the next sweep.
Unit suspicion. The median and 90% of the values fit the claimed unit's range, or they fit the other unit's and not their own. In the second case Baler writes an open suggestion to quality_suggestions, for example "this zone reads 21–23 °F; °C?". The samples are flagged unit_suspect. The unit is never changed; a person decides.

The sweep only fills a sample that has no flag yet, so a device's own flag always wins. It's idempotent.

Who skips, who shows

Consumer	What it does with a flagged sample
Fault rules, Cur mode	A reading the device flags neither opens nor resolves a fault. The fault stays as it was until a good reading arrives.
Fault rules, `stuck`	Counts only good samples: a spike doesn't count as "the value moved".
Fault rules, `stale`	Unchanged. A flagged sample still arrived.
Cx checks (`commissioning.history`)	Latest-value checks and windows read good samples only.
The tagging model step	The value stats it shows a model come from good samples only.
`GET /history`	Returns every sample. A flagged one has a `quality` key; an ok one looks exactly as before.
`GET /his` (his queries, raw reads)	Same: `quality` on flagged samples. In `compact=1`, a parallel `quality` array appears only when something in the series is flagged.

Not covered yet

Rollups. /his reads with a rollup, and the 5-minute and 1-hour continuous aggregates behind them, still fold every sample, flagged or not. Excluding flags means recreating those aggregates on a live hypertable. That needs its own migration.
Derivations (derivation_engine) and the Haystack API's hisRead read every sample and don't show the flag.
Cx availability counts every sample that arrived, flagged or not.
There's no UI or API for quality_suggestions yet. The review queue (#184) is where it belongs.

Last modified on November 8, 2026

Fault Detection Derivations

The flags

Flag

Meaning

Set by

(none)

ok. Stored as NULL, so healthy history costs nothing extra.

fault

The device says the value is unreliable: BACnet status-flags fault, or a reliability other than no-fault-detected.

the reading

overridden

The device says someone overrode it at the panel.

the reading

out_of_service

The device says the object is out of service.

the reading

out_of_range

Outside the plausible range for the point's Brick class and unit (below).

Baler's sweep

spike

An isolated jump: more than 6 scaled MADs from the median of the 5 samples on each side.

Baler's sweep

unit_suspect

The values fit the other temperature unit, not the one the point claims (21 "°F" in a zone).

Baler's sweep

Stale and stuck stay as they were: derived when a rule runs (his_checks), not stored. They describe a series, not one sample.

Where flags come from

The reading. An MQTT reading or a POST /ingest body may carry quality: either a flag name, or BACnet's own status:

Code

{"value": 140.0, "ts": 1786454667, "quality": {"status_flags": ["fault"], "reliability": "open-loop"}}

Baler's sweep. fault_detector runs quality.sweep every 5 minutes (--quality-interval, 0 turns it off). It covers the last hour of every point that has both a Brick class and a unit tag:

Range. A wide plausible range per class and unit, meant to catch a failed sensor, not a comfort complaint. Examples: zone air temperature 32–120 °F (0–50 °C); outside air -60–140 °F; water temperature 32–260 °F; humidity 0–100 %RH; damper, valve, position and speed 0–100 %; CO2 0–10000 ppm. The table is quality.RANGES. A class with no row isn't range-checked; power is one, since it can go negative with PV.

Spikes. Each sample is compared with the median of the 5 samples before it and the 5 after. The window is centred, so a real step change (a setpoint change) isn't flagged. The newest 5 samples wait for the next sweep.

Unit suspicion. The median and 90% of the values fit the claimed unit's range, or they fit the other unit's and not their own. In the second case Baler writes an open suggestion to quality_suggestions, for example "this zone reads 21–23 °F; °C?". The samples are flagged unit_suspect. The unit is never changed; a person decides.

The sweep only fills a sample that has no flag yet, so a device's own flag always wins. It's idempotent.

Who skips, who shows

Consumer

What it does with a flagged sample

Fault rules, Cur mode

A reading the device flags neither opens nor resolves a fault. The fault stays as it was until a good reading arrives.

Fault rules, stuck

Counts only good samples: a spike doesn't count as "the value moved".

Fault rules, stale

Unchanged. A flagged sample still arrived.

Cx checks (commissioning.history)

Latest-value checks and windows read good samples only.

The tagging model step

The value stats it shows a model come from good samples only.

GET /history

Returns every sample. A flagged one has a quality key; an ok one looks exactly as before.

GET /his (his queries, raw reads)

Same: quality on flagged samples. In compact=1, a parallel quality array appears only when something in the series is flagged.

Not covered yet

Rollups. /his reads with a rollup, and the 5-minute and 1-hour continuous aggregates behind them, still fold every sample, flagged or not. Excluding flags means recreating those aggregates on a live hypertable. That needs its own migration.

Derivations (derivation_engine) and the Haystack API's hisRead read every sample and don't show the flag.

Cx availability counts every sample that arrived, flagged or not.

There's no UI or API for quality_suggestions yet. The review queue (#184) is where it belongs.

Last modified on November 8, 2026