Core Utils Runner
The Core Utils runner (core-utils-runner) provides domain-agnostic transforms: normalize tabular data, extract protobuf capture payloads, reshape JSON, and build $heat-dataservice for Next dashboards. Use it in integrator session templates when tasks are routed to the core-utils deployment.
For platform-only multi-upload tabular → dataservice on system-utils, see system-tabular-to-dataservice.
Node template selection
| Template | Purpose | Limitations | Details |
|---|---|---|---|
tabular-remap | Rename, derive, or filter CSV/TSV columns with Python expressions before downstream analytics. | Single tabular parent; expressions run per row in memory; not for publishing dashboards. | tabular-remap |
hydrate-protobuf | Extract legacy protobuf types (Gaze, FolderUpload, OpenDIS PDU, SimulationFrame) to JSON, CSV, or binary. | Prefer hydrate-protobuf-v2 for capture-aware SimulationFrame slices; v1 SimulationFrame uses component flattening, not envelope metadata. | hydrate-protobuf |
hydrate-protobuf-v2 | Merge all parents and emit SimulationFrame, EntityState, or EntityDataEvent records with capture envelope fields. | JSON or CSV only (no raw bin); dedupe upload flag is log-only today. | hydrate-protobuf-v2 |
tabular-query | Run SQL SELECTs over one CSV/TSV in in-memory SQLite; output named tables as JSON. | One input table; all columns are TEXT in SQLite; not for very large files. | tabular-query |
tabular-to-dataservice | Build $heat-dataservice from one tabular parent for dashboard-v2. | Single parent only; for all uploads on one input-node use system-utils system-tabular-to-dataservice. | tabular-to-dataservice |
json-template | Build output JSON from a template tree with JMESPath leaves. | Single parent; strict fails on JMESPath errors. | json-template |
json-flatten | Flatten nested JSON (or NDJSON lines) to CSV/TSV. | Single parent; wide sparse tables possible on deep nesting. | json-flatten |
json-merge | Shallow or deep merge of multiple JSON object inputs. | Not a dashboard node; arrays off unless allowArrays is set. | json-merge |
simulation-frame-merge | Pivot long-format SimulationFrame JSON to wide CSV by tick and entity. | Expects hydrate-style simulation_frame arrays; include origin_uuid in groupBy for multi-capture merges. | simulation-frame-merge |
converge-outputs | Re-expose every upstream output on this node without copying blobs (aggregation). | Maps references only; configure whenNoOutputs for empty upstream behaviour. | converge-outputs |
Example workflow
Supported tabular formats
| Input | Output | Notes |
|---|---|---|
| CSV | CSV / TSV | Auto-detect delimiter in auto mode |
| TSV | CSV / TSV | Delimiter preserved or converted per config |