Copy page

Notebooks

Notebooks: decisions

Status: recommended, not yet confirmed. These are the recommendations from the Code epic (issues 90-95). The notebook image and baler_fn are built around them, but the final call is the project owner's. If he picks differently, the last section lists what changes.

1. Feature name: "Baler Notebooks"

SkySpark calls the equivalent feature Code, with Shell for the REPL and Eggbox for packages. Reusing "Code" would be confusing in a product that people will put next to SkySpark. It would also clash with "code" in its everyday sense: source code, Axon code, fault-rule code.

user-facing name	what it is	SkySpark counterpart
Baler Notebooks	the opt-in JupyterLab service and the workflow around it	Code, Shell
function package	a pip-installable repo with a baler.toml	pod / Eggbox package
function index	the static docs page listing published packages	Eggbox
baler_fn	the Python package (connect, @baler_function, list)	the Code app's function list

"Notebooks" says what the user actually opens, and it doesn't promise server-side execution, which Baler deliberately doesn't do. Internal names (the compose profile notebook, the service jupyter, the docs folder pages/code/) follow the issues and don't need to change if the name does.

2. Notebook client library: phable (recommended)

The question was which Haystack client the notebook ships for anything beyond baler_fn's wrapper: phable, or openhay. openhay is ours, and haystack_api.py already uses it on the server side.

Either way, baler_fn.connect() returns a client that wraps Baler's own baler.haystack_client, as issue 91 asks. That client gained API-key auth, Hayson decoding and hisWrite for this. So the choice decides what's preinstalled in the image, and what client.phable() (or an equivalent) hands back. It doesn't decide what baler_fn itself is built on.

The trial

Both libraries were run against a live Baler stack (this branch, docker compose, the gateway at /api/demo) on 2026-11-07. Each one authenticated with an operator API key from POST /auth/api-keys and did the four things the issue asks for.

• phable 0.1.29, from PyPI.
• openhay at main, commit eb89b9e, built from git with maturin.

step	phable 0.1.29	openhay eb89b9e
install	pip install "phable[pandas,pyarrow]": pure Python, wheels on PyPI	not on PyPI. pip install git+... builds Rust with maturin: needs cargo, about 1.5 min cold
auth with an API key	✅ phable.HaystackClient(url, api_key) (sends BEARER authToken=)	✅ oh.Client(url, bearer=api_key)
about	✅	✅
read	✅ read_all("point")	✅ read("point")
hisRead to a DataFrame	✅ through the batch form: his_read_by_ids([...]).to_pandas() gives a typed long-format DataFrame. ⚠️ The single-id his_read_by_id(...).to_pandas() fails, see below.	⚠️ no DataFrame support: you build one from Grid.rows by hand
hisWrite	✅ his_write_by_id(ref, [{"ts": ..., "val": Number(...)}])	✅ his_write(ref, [(ts, Number(...))])
ops coverage	no nav, watches, defs/libs or close-and-reuse	everything Baler serves, including watches, nav, defs, the pointWrite array and close
who maintains it	third party (an independent check on Baler's Haystack conformance)	us

Both clients also pass every Baler cell of the CI conformance matrix (Haystack conformance), with the gaps marked there.

The single-id phable gap. Baler answers a single-point hisRead with the point's id in the grid meta, as the Haystack spec describes. phable's Grid.to_pandas() insists on an id in the val column meta, and raises ValueError: Column 'val' must have metadata with a valid 'id' of type Ref. The batch form carries column ids, so it works. baler_fn's own his_read_df() doesn't depend on either. A later change could add the column meta in haystack_api.py. That's harmless to other clients, but it touches the conformance matrix, so it's left as a follow-up rather than done here.

Why phable

• It's made for this. phable targets Jupyter, with pandas and Polars conversion built in. openhay is a protocol library: complete and fast, but with no DataFrame layer.
• It's cheap to ship. A notebook image, or a user's own environment, gets phable with one pip line. openhay needs a Rust toolchain or a separately built wheel until it publishes wheels to PyPI. The main Dockerfile already pays that cost with a Rust build stage. Repeating it in every user's notebook and function-project image would make the user-facing path the slow one.
• It's independent. It's a second implementation that Baler is already tested against, so examples written with it show that Baler works with ordinary Haystack tooling.
• Its gaps don't matter much here. Watches and nav are rarely what a report or statistics notebook needs, and baler_fn covers read, history and writes on its own.

openhay stays the server-side library, and remains the better choice for anyone who wants full op coverage. Once it ships wheels to PyPI, adding it to the image is one line in docker/notebook.Dockerfile, and it could then be revisited as the default.

If the decision goes the other way

Choosing openhay for the notebook would mean:

• docker/notebook.Dockerfile: replace the phable[...] pip line with a Rust stage that builds the openhay wheel, the same as the main Dockerfile. Pin the same ref as pyproject.toml and OPENHAY_REF.
• src/baler_fn/client.py: replace BalerClient.phable() with an openhay() method returning oh.Client(base_url, bearer=api_key).
• Docs: the notebook pages, and the starter notebook's last cell.

Nothing else depends on the choice: baler_fn, the manifest, the template and the index are library-neutral.

3. Later phase (not decided here): running functions inside Baler

Promoting a published function to a server-side derivation or fault rule (through derivation_api.py) would mean Baler executing user code. sandbox.py states that it's not a hard security boundary. That phase needs its own design and security review before any work starts. Until then the rule stands: Baler never executes notebook or package code.