# Query (hisRead) "What is this air handler doing right now?" One line — `readAll(ahu and air).hisRead(thisMonth)` — and every matching point's history is on screen: `GET /ingest/his?expr=...` takes that expression and returns every matched point's history as one JSON body, and `/ui/his.html` (through the gateway: `localhost:8080/ui/his.html`) is the shell-plus-chart surface over it — type a query, get charts, flip to a grid, download CSV. The syntax is Axon-shaped (familiar if you've used SkySpark) but this is *not* Axon. It is the two calls people actually reach for in a hurry — a Folio-style tag filter and `hisRead`/`hisRollup` — implemented over Baler's own two stores (the RDF entity graph in Oxigraph for the filter, `point_history` in Postgres/TimescaleDB for the history). Everything else Axon can do in that shell is deliberately out of scope; see "What is and isn't supported" below. ## The query language ``` query := source ("." call)* source := ("read" | "readAll") "(" filter ")" | filter # a bare filter means readAll call := "hisRead" "(" span ")" | "hisRollup" "(" fold "," interval ")" filter := term (("and" | "or") term)* term := "(" filter ")" | "not" term | path cmpOp value | path | ^symbol path := name ("->" name)* value := "string" | number[unit] | true | false | @ref | YYYY-MM-DD | hh:mm[:ss] ``` Some real ones: ``` readAll(ahu).hisRead(thisMonth) every point on every air handler, this month readAll(zone and air and temperature).hisRead(today) every zone air temperature point, by the words read(vav and zone).hisRead(yesterday) the first VAV zone point only ahu and temp bare filter — readAll(...).hisRead(today) readAll(temperature_sensor).hisRead(2026-09-01..2026-09-07) any-case Brick class, subclasses included readAll(point and unit == "kW").hisRead(pastWeek).hisRollup(avg, 1hr) readAll(id == @site1).hisRead(2026-08) readAll(temp and not air).hisRead(lastWeek).hisRollup(max, 1day) ``` ### The filter The filter is Haystack's filter grammar — `and`/`or`/`not`, parentheses, names, and `tag value` comparisons with `==`, `!=`, `<`, `<=`, `>`, `>=`. It compiles to a single SPARQL `SELECT DISTINCT ?s` over the entity graph, which is what makes it fast enough to type against interactively; `hisquery.compile_filter()` is the whole compiler if you want to see the translation. #### Querying Brick without knowing Brick Baler's entities are Brick-typed, and Brick is hard to query by hand: the class names are long (`Zone_Air_Temperature_Sensor`), the hierarchy matters (a `Temperature_Sensor` query that misses every subclass is useless), there are aliases (`AHU` is `Air_Handling_Unit`, `Discharge_Air_Temperature_Sensor` is `Supply_Air_Temperature_Sensor`), and a plain word like `temperature` isn't a class at all. The filter absorbs all of that. A bare name is matched **three ways at once**, and an entity that satisfies any of them counts: - **As a Brick tag word.** Brick attaches a tag vocabulary to every class — the words the class name is made of (`Zone_Air_Temperature_Sensor` carries `Zone`, `Air`, `Temperature`, `Sensor`, `Point`; `Air_Handling_Unit` carries `AHU`, `Air`, `Handling`, `Unit`, `Equipment`). So `zone and air and temperature and sensor` finds every zone air temperature sensor, whatever class it was typed with, and `ahu` finds air handlers *and* rooftop units, with no Haystack tag anywhere. Haystack's spellings where they differ from Brick's are mapped: `temp`, `sp`, `cmd`, `equip`, `elec`, `occ` mean `Temperature`, `Setpoint`, `Command`, `Equipment`, `Electrical`, `Occupancy`. Aliases and deprecated classes inherit the tags of what they point at, so nothing is left out for lack of its own tag list. And the project's own tag→class mapping rules (`rules/haystack_*_to_brick.yaml`) run backwards too — `run` has no Brick word, but `[fan, run, sensor] → Fan_Status` means `fan and run` still finds every `Fan_Status` point. - **As a Brick class, with its whole subtree.** Any name with an uppercase letter or an underscore is a class name, matched **case-insensitively** and including every subclass and every alias: `Temperature_Sensor` (or `temperature_sensor`) finds the discharge, return, mixed, outside and zone air temperature sensors; `Air_Handling_Unit`, `AHU`, and `air_handling_unit` all find air handlers, rooftop units, DOAS units, and anything typed with Brick's alias classes. Classes the live graph adds beneath Brick count as their parents and resolve by their own name the same way — a [Brick extension](#your-own-brick-extension) of yours, or the PROJ fallback classes `mapping.py` mints (`proj:Fan_Status_Variant`, a direct `rdfs:subClassOf` of a Brick class). - **As a Haystack marker.** `ahu`, `air`, `temp`, `sensor`, `zone`... anything ingest wrote as a `haystack:hasTag` triple (every marker a source sent, verbatim). This is what a Haystack-fed point has; the two ways above are what a Brick-only entity (a loaded model, a derivation output) has. Mixed data works because the same word means the same thing through all three. - **Structural aliases** — `point`, `equip`, and `site` also match on the graph's shape (has a `brick:hasPoint`, is a `brick:isPointOf` something, and so on), so a BACnet point that arrived with no `point` marker and no class still counts. *Brick's own* hierarchy, aliases and tag words come from the vendored `ontology/Brick-only.ttl` (1.4.4), read once per process by `brick_vocab.py` — not from the live graph. That's deliberate: nothing in `docker-compose.yml` loads Brick into Oxigraph (see [Architecture](/architecture)), and even when it is loaded, Oxigraph takes tens of seconds on an `rdfs:subClassOf*` walk inside a filter over the full ontology. Every hop is resolved in Python and handed to SPARQL as a bound class list instead, so a filter is ~0.07s with the full ontology loaded, whether it names 5 classes or 1,400. What the live graph declares *on top of* Brick is read from the live graph, in one cheap query per request (`hisquery.graph_vocab`): every `rdfs:subClassOf`, `owl:equivalentClass` and `brick:hasAssociatedTag` whose subject isn't in Brick's namespace, plus every subject typed as a class, tag or shape. So an extension class is found under its Brick parent, by its own name, through its own aliases, and by the tag words it declares — see below. Ontology-level subjects are never entities. With Brick loaded into Oxigraph its ~2,600 class/tag/shape/quantity subjects (and the REC ontology Brick 1.4 embeds) are all in the same union graph as your equipment, and a `readAll(not point)` would otherwise list them. They're dropped after the query by what they are — anything in a vocabulary namespace (`brickschema.org`, `w3id.org/rec`, `w3.org`, `qudt.org`), and anything the live graph itself declares as a class, tag, or shape (so an extension's declarations are excluded too, whatever namespace they use). This happens in Python, not as a `FILTER NOT EXISTS` per candidate: that clause alone was ~0.4s of every query, and it only knew about `owl:Class`-typed subjects, so `readAll(Quantity)` used to hand back Brick's quantity individuals. When a filter matches nothing and a name is *almost* a Brick class or tag word, the response carries `hints` — `Air_Handeling_Unit` → `Air_Handling_Unit`, `temperture` → `temperature` — and the UI shows them as "did you mean" links. It's never an error: the name may be a perfectly good Haystack marker the vendored Brick can't know about. A class or tag word your extension declares is never "corrected" to a Brick near-miss. ### Your own Brick extension The reason to pick Brick is that you can extend it, and the filter treats an extension exactly like Brick. Write it the way [Brick's extension guidance](https://docs.brickschema.org/extra/extending.html) says — your own namespace, `rdfs:subClassOf` the nearest Brick class, a `skos:definition`, optionally `owl:equivalentClass` aliases and `brick:hasAssociatedTag` words: ```turtle @prefix acme: . @prefix brick: . @prefix tag: . acme:Heat_Recovery_AHU a owl:Class ; rdfs:subClassOf brick:Air_Handling_Unit ; skos:definition "An air handler with an integral heat recovery wheel." ; brick:hasAssociatedTag tag:Heat, tag:Recovery . acme:HRU a owl:Class ; owl:equivalentClass acme:Heat_Recovery_AHU . acme:Rotor a brick:Tag . acme:Wheel_Speed_Sensor a owl:Class ; rdfs:subClassOf brick:Speed_Sensor ; brick:hasAssociatedTag acme:Rotor, tag:Speed, tag:Sensor . ``` Then an equipment typed `acme:Heat_Recovery_AHU` (or `acme:HRU`, or a further subclass of either) is found by `AHU`, `Air_Handling_Unit`, `Equipment`, `ahu`, `equip`, and also by `Heat_Recovery_AHU`, `heat_recovery_ahu`, `HRU`, `hru`, and `heat and recovery`. Its `acme:Wheel_Speed_Sensor` point is found by `Speed_Sensor`, `speed and sensor`, `Sensor`, `Point`, `Wheel_Speed_Sensor`, and `rotor`; the response reports `Wheel_Speed_Sensor` as its `brickClass`. A class declared by `rdfs:subClassOf` alone, with no `a owl:Class`, works the same way. Two things to know: - **Load it into a graph of its own.** `RemoteStore.load_ontology` `PUT`s into a single named graph (`urn:baler:ontology`) and replaces it, so it can't hold Brick and your extension at once. Oxigraph's graph-store endpoint takes any graph name: ```sh curl -X PUT -H 'Content-Type: text/turtle' --data-binary @acme-ext.ttl \ 'http://localhost:7878/store?graph=urn:acme:ontology' ``` The query reads through `--union-default-graph`, so the graph name is yours to pick. Brick itself does not need to be loaded for any of this to work — the filter needs your `rdfs:subClassOf brick:...` edges, and Brick's side of the hierarchy comes from the vendored file. Loading Brick as well (`python ontology_demo.py`) is fine; it costs ~0.05s per query. - **Stay out of Brick's namespace.** Triples whose subject is a `brick:` IRI are skipped when the live graph is read (Brick's own are already known, faster, from the pinned file), so a class you mint *as* `brick:Something` isn't picked up as an extension. That also means a *newer* Brick loaded live doesn't extend the vendored one: a class added in Brick 1.5 matches only by exact `rdf:type` until `ontology/Brick-only.ttl` is bumped (see `ontology/README.md`). Comparisons: `dis == "AHU-1"`, `unit == "°F"`, `kind != "Bool"` compare `haystack:*` value tags as-is; `id == @some-id` matches the entity's own id (and it's the only thing `id` may be compared to — `id == 5` is a parse error). Ref-valued tags compare by id too: `siteRef == @site1`, and `equipRef == @ahu-1` also matches through the graph's own `brick:isPointOf` edge, so it works for points whose source never sent an `equipRef` tag. A comparison on a tag the entity doesn't have is false, as in Haystack (`x != 1` doesn't match recs with no `x`), and so is a comparison between different kinds (`foo < "c"` never matches a number). **Numbers compare only with the same unit**, as in Haystack: `temp > 72°F` matches `75°F` but not a unitless `75`, and `temp > 72` matches only unitless values. There's no unit conversion (`2min` is not `120s`). A bare name matches a tag with any value, not only markers: `dis` matches every record with a `dis`. Dates and times compare as dates and times (`installed < 2020-01-01`, `ts->time >= 08:00`), and `^hot-water` is shorthand for `hot and water`. DateTimes compare by instant (`ts > 2020-09-01T05:00:00Z UTC`); `==` also needs the same Haystack timezone, and `ts->tz == "New_York"` tests the zone itself. Uri literals (`` link == `http://x/a` ``) compare only with Uri values. Paths step into nested Dicts as well as follow Refs (`ref->thru == "e"`), and `//` starts a comment. These rules are checked against haxall's own filter test cases on every build (`tests/test_haxall_filter_vectors.py`, run on both rdflib and Oxigraph). **After the read** (Baler#18), in the order written: - `.sort("tag")` / `.sortr("tag")` order the series by a tag (default `dis`, case-insensitive; series missing the tag go last). `.first()` keeps only the first series. - `.hisClip()` trims to the `hisRead` span (already the case); `.hisClip(span)` trims further, e.g. `.hisRead(thisWeek).hisClip(today)`. - `.hisInterpolate()` puts every series on the same timestamps (the union of them all), each within its own first..last sample: linear between numbers, the previous value for bools and text. Capped at 500,000 values; `hisRollup` first for long spans. Lambdas (`map(p => ...)`) and general Axon aren't supported - a filter path such as `equipRef->dis == "AHU-1"` covers most `map`/`findAll` uses. **`read` vs `readAll`.** `readAll(filter)` returns every match. `read(filter)` takes the first match only, and if nothing matches it's a **404**, the same as Axon's `UnknownRecErr` — a `readAll` that matches nothing is an empty, successful 200. A bare filter with no `read`/`readAll` and no `.hisRead(...)` is `readAll(filter).hisRead(today)`. ### Equipment expands to its points This is the one deliberate departure from Axon, and the reason to have the feature. In Axon, `read(ahu).hisRead(today)` is an error: an equipment record has no history. Here, any matched entity that isn't itself a point is expanded to every point it `brick:hasPoint` (following `owl:sameAs` merges, the same expansion `ingest.points_of_equip` does), so "show me this AHU" is `read(ahu and dis == "AHU-1")`, not a hunt for the right `equipRef` syntax. A filter that matches points directly (`temp and zone`) returns those points unexpanded. The expansion is one hop — `brick:hasPoint` — so a matched *site* contributes only points attached directly to it, not everything under its equipment; query the equipment (`readAll(equip and siteRef == @site1)`) for that. ### Spans `hisRead(span)` takes Axon's `DateSpan` vocabulary. Every span is a half-open `[start, end)` range of *local* days in the request's timezone (see `tz` below), never a rolling clock window — `today` is local midnight to local midnight tomorrow, exactly as SkySpark treats it. | Span | Meaning | | --- | --- | | `today`, `yesterday` | one local day | | `thisWeek`, `lastWeek`, `pastWeek` | Mon–Sun weeks; `past*` is the trailing 7 days ending today | | `thisMonth`, `lastMonth`, `pastMonth` | calendar months; `pastMonth` is the trailing month ending today | | `thisQuarter`, `lastQuarter` | calendar quarters | | `thisYear`, `lastYear`, `pastYear` | calendar years; `pastYear` is the trailing year ending today | | `2026-09-03` | one explicit day | | `2026-09` | one explicit month | | `2026` | one explicit year | | `2026-09-01..2026-09-07` | inclusive date range (both ends are whole days) | Omit `.hisRead(...)` entirely and the span is `today`. ### Rollups `.hisRollup(fold, interval)` aggregates on the server in one SQL statement instead of shipping raw rows and folding in the browser — a month of 1-minute data comes back as 720 hourly buckets, not 43,000 rows. `fold` is one of `avg`, `min`, `max`, `sum`, `count`. `interval` is an Axon duration: `15min`, `1hr`, `6h`, `1day`, `1wk`, `1mo`, `1yr` (units: `s`/`sec`, `min`, `h`/`hr`, `day`, `wk`, `mo`, `yr`). Fixed-length intervals use TimescaleDB's `date_bin`, anchored at the span start; `mo`/`yr` have no fixed length and use `date_trunc` in the request timezone, so a `1mo` bucket really is a calendar month and a `1day` bucket really starts at local midnight. Boolean points fold as 0/1, so `avg` of a `Bool` point is its duty cycle for the bucket — the most useful single number for a fan-status or occupancy point over a day. ## `GET /ingest/his` | Param | Meaning | | --- | --- | | `expr` | the query, required | | `tz` | IANA timezone for span resolution and calendar bucketing; defaults to `BALER_TZ`, then `UTC` | | `limit` | per-point raw-row cap; defaults to 10,000, max 200,000 (rollups aren't capped — they're already bucketed) | The response is one object with the resolved `span` (`start`/`end` epoch seconds, the `label`, the `tz` used), the `rollup` in effect (or `null`), `matched` (the entity ids the filter hit, previewed) and `matchedCount`, and `series` — one entry per point with `id`, `dis`, `unit`, `kind` (`Number`/`Bool`/`Str`), its `brickClass`, its parent `equip`/`equipDis`, the `history` as `[{ts, value}, ...]` oldest-first, and a `truncated` flag that's `true` when `limit` clipped that point. Full schema: [the Ingest API reference](/api/ingest). `limit` matters. Axon's `hisRead` applies its own default row limit for the same reason: a month of 5-second data across forty points is millions of rows, and one browser request shouldn't be able to ask for that by accident. When you see `truncated: true`, add a `.hisRollup(...)` — that's what it's for — rather than raising `limit`. Errors: a malformed or unsupported expression is a **400** whose body names the problem and the position in the text (`unsupported function 'foo' at position 12; only hisRead(span) and hisRollup(fold, interval) are supported`, `unknown span 'thisFortnight'`, `id must be compared to a @ref`); `read(...)` with no match is a **404**. Nothing is ever silently reinterpreted — an expression that uses something this grammar doesn't cover fails loudly, by design. Through the gateway the route is `GET /ingest/his`, and it requires the `viewer` role (or higher) like every other read — see [Gateway auth](/gateway-auth). ```bash curl -s -H "Authorization: Bearer $TOKEN" \ --get 'http://localhost:8080/ingest/his' \ --data-urlencode 'expr=readAll(ahu and air).hisRead(pastWeek).hisRollup(avg, 1hr)' \ --data-urlencode 'tz=America/Chicago' ``` ## `/ui/his.html` The page is the query bar and three views of whatever it returns, with no build step and no charting library — the charts are hand-drawn ``, same as the rest of `ui/`. - **Query bar** — type any expression above and press Enter (or Run). Below it, span chips (`today`, `yesterday`, `thisWeek`, `thisMonth`, ...) and a rollup dropdown *rewrite the expression in the bar* rather than being hidden state: clicking `lastWeek` turns `readAll(ahu).hisRead(today)` into `readAll(ahu).hisRead(lastWeek)` in front of you, so the text is always the whole truth about what's on screen, and copying it into `curl` gives you the same result. - **By unit** — the default. Points are grouped onto one chart per unit (`°F`, `cfm`, `%`, `kW`...), so every temperature on the AHU shares an axis and a supply/return/mixed-air comparison is one glance. `Bool` points draw as a step chart; `Str` points list as a timeline of value changes. Hovering gives a crosshair and a tooltip with every series' value at that instant; clicking a legend row hides/shows that series; the small `↗` next to a name opens that one point alone in a new query. - **Per point** — one small chart per point with min/max/avg/last stats, for when the by-unit chart is too crowded. - **Grid** — the same data as a table (one column per point, one row per timestamp), plus a **Download CSV** button. - **Live** — on by default; re-runs the query every 10 seconds while the span's end is still in the future (`today`, `thisWeek`...), for watching a piece of equipment respond. A span that's entirely in the past (`yesterday`, `2026-08`) can't change, so it isn't polled. The URL updates as you go (`his.html?expr=...`) so a query is a link you can send someone. The rest of the UI links here too: every equipment row in the tree on `index.html` has a `📈` that opens `readAll(id==@).hisRead(today)`, and the point detail panel has "Open in Query →" for a single point. ## Timezones `BALER_TZ` (in `.env`; see `.env.example`) sets the default `tz` for `GET /his`, so `today` means the building's today, not the server's UTC today. Any request can override it with `?tz=...`. The UI sends the browser's own IANA zone, so it always sees local days; the env var is for API clients and scripts that don't say. Set it to the zone the *building* is in — a multi-site deployment spanning zones should pass `tz` per request instead (see [Multi-site](/multi-site)). ## Tag paths `a->b` follows the ref tag `a` and tests `b` on the entity it points at, the same as Haystack: ``` readAll(point and equipRef->ahu).hisRead(today) readAll(temp and equipRef->dis == "AHU-1").hisRead(yesterday) readAll(point and not equipRef->vav).hisRead(today) ``` - **equipRef** follows the point's `brick:isPointOf` edge, so it works for every point in the graph, not only Haystack-sourced ones. A point whose equip hasn't been linked yet is followed through its raw `equipRef` tag instead. - **siteRef** also follows Brick structure: the `brick:Site` that reaches the entity through `hasPart` (Site > Building > Floor > Zone, or equipment's own parts), backwards along `feeds` (an AHU feeds a Floor, a VAV a Zone) and on to `hasPoint`. That's the `siteRef` `/api` shows on every record, and `nav` walks site → equip → point the same way. A raw `siteRef` tag (Haystack imports) still counts when no Brick Site reaches the entity. - **Any other ref tag** (`spaceRef`, a custom `fooRef`) is followed through its raw Haystack value: a `urn:...` ref names the entity directly, a bare id resolves to the Haystack-sourced record with that id. - The far end is tested with the full filter vocabulary, so `equipRef->ahu` also matches points on equipment typed `brick:Rooftop_Unit` that carry no `ahu` marker. - A path that can't be followed (no such ref) is false, and `not` of it is true, as in Haystack. - Paths can chain: `equipRef->siteRef->dis == "HQ"`. ## What is and isn't supported Supported: the grammar above, exactly. Not supported, and each one is an explicit 400 naming what was rejected, never a silent approximation: - **Arithmetic and every other Axon function** — no `hisFoldAll`, `hisInterpolate`, `hisClip`, `.map(...)`, no `toDateSpan`, no lambdas. There are two calls, `hisRead` and `hisRollup`, and that's the whole library. Anything computed from history belongs in a [Derivation](/derivation), which writes a real point you can then `hisRead` like any other. - **Unit conversion.** `temp > 22°C` doesn't match `75°F`; units must be identical. - **Paths into nested dicts** (`ref->thru` where `ref` is a Dict) and `ts->tz`. - **Time-of-day spans** (`hisRead(2026-09-03T08:00..2026-09-03T17:00)`) — spans are whole local days. - **Writes.** There is no `hisWrite` here; that's [`POST /ingest/history`](/api/ingest) or the [Haxall drop-in](/haxall-drop-in). For the wider question of what a SkySpark user gives up moving here — and what's on the backlog to close — see `todo/skyspark-haxall-migration-gaps.md` in the repo. ## Where the code is - `src/baler/hisquery.py` — tokenizer, parser, filter→SPARQL compiler, span resolution, and `run_query()`, which strings them together. Each stage is a pure function so `tests/test_hisquery.py` runs the parser, compiler (against an in-memory `rdflib` graph), and span math with no services. - `src/baler/brick_vocab.py` — Brick's class hierarchy, aliases, and per-class tag words, parsed from the vendored `ontology/Brick-only.ttl` once per process; `tests/test_brick_vocab.py` pins the facts the compiler relies on against the real file, so a Brick version bump that changes the file's shape fails loudly. - `src/baler/timeseries.py` — `read_range_many()` (raw, multi-point, per-point limit), `read_rollup()` (`date_bin`/`date_trunc` aggregation), `read_point_labels()` (unit/label fallback from history rows, for derivation-engine points that carry no graph tags). - `src/baler/ingest_api.py` — the `GET /his` route. `tests/test_his_route.py` covers it end-to-end against live Oxigraph and Postgres (`pytest -m integration`). - `gateway/njs/policy.js` — the `GET /ingest/his` → `viewer` row. - `ui/his.html`, `ui/his.js`, `ui/his.css` — the page; `ui/app.js` for the tree/detail deep links. ## How rollups stay fast `point_history` has two TimescaleDB continuous aggregates, created on first start: `point_history_5m` and `point_history_1h`. They hold sum, count, min and max per point per bucket, and are refreshed every 5 and 30 minutes over the last 35 days. A rollup can use a view when the view's bucket divides the requested interval and the span starts on one of its boundaries. It then covers the span with the coarsest pieces that give exactly the raw answer: whole 1-hour buckets that are up to date, 5-minute buckets for what the hourly view hasn't caught up on, and raw rows for the rest. "Up to date" comes from TimescaleDB's own bookkeeping (its invalidation logs and each view's watermark), so the unrefreshed tail, a source backfilling history since the last refresh, or a span that was never materialized are all read from raw rows. Results are the raw fold's, never stale. Calendar (month/year) rollups and odd intervals fold raw samples as before. A day of 614 points at 5-second samples (6.2M rows) folds from about 100k pre-summed rows; `readAll(equip).hisRead(today)` takes about 0.4 s end to end on a dev machine. Big responses are encoded and decoded in small pieces (`timeseries.iter_rows`, `http_handler_base.json_pieces`), so one large `/his` read never holds Python's GIL long enough to stall other requests, such as Ops' health probe.