`system-tabular-to-dataservice` (platform, system-utils)

The system-tabular-to-dataservice node is a platform-only transform on system-utils. It reads every node output from a single parent input-node (CSV/TSV uploads or tabular JSON), builds canonical $heat-dataservice, and adds tabularUploadSummary for a Next CustomRenderer panel.

Configuration matches tabular-to-dataservice on core-utils, plus multi-upload behaviour.

Typical pipeline


input-node (tabular-upload) → system-tabular-to-dataservice → heat-system-next-dimension

Shipped reference session templates:

system-tabular-dashboard-next (explicit timeColumn): shipped template
system-tabular-dashboard-autodetect-next (autodetect + dynamic layout): shipped template

How realm and time are chosen

Question	Configuration
Time axis	`timeColumn` (required). Values become each series point’s `timeMs`.
Realm split	Optional `realmColumn`: distinct cell values → `realms[].name`. If unset, rows use `defaultRealm` (default `default`).
Primary key / dedupe	`timeColumn` plus optional `primaryKeyColumns`, with `dedupePolicy`: `last` or `first`.
Milliseconds vs ISO	`timeColumnIsMs` (default `true`) with `baseCapturedAt`, or `timeColumnIsIso8601` (`timeMs` = ms since minimum ISO timestamp per realm).

Set time and realm explicitly in defaultConfiguration, or enable autoDetectTimeColumn (platform extension) to infer the time column from headers and sample rows. Pair with emitSuggestedLayout and heat-system-next-dimension layoutFromParentOutput for an upload-any-CSV dashboard (template system-tabular-dashboard-autodetect-next).

Multi-upload behaviour (platform extensions)

Property	Default	Description
`realmPerFile`	`false`	When `true` and `realmColumn` is unset, each parent output (each upload) becomes its own realm (name = slug of filename).
`whenNoOutputs`	`complete`	Parent has no outputs yet: `complete` (empty payload + summary), `suspend`, or `fail`.
`maxFiles`	`50`	Maximum parent outputs processed per run.
`maxRowsPerFile`	`100000`	Row cap per file (memory guardrail).

When multiple files are uploaded and realmColumn is set, realm names are prefixed with the file slug to avoid collisions (for example metrics_pilot_a).

When multiple files share defaultRealm without realmColumn, channel id values are prefixed with the file slug.

Shared configuration (parity with tabular-to-dataservice)

Property	Required	Default	Description
`inputFormat`	✖	`auto`	`auto` \| `csv` \| `tsv` \| `json`
`sourceTable`	✖*	,	`tables` key for JSON tabular-query shape
`timeColumn`	✔*	,	Header for `timeMs` (*optional when `autoDetectTimeColumn` is true)
`autoDetectTimeColumn`	✖	`false`	Infer `timeColumn` and ISO vs ms mode from CSV headers and sample rows
`emitSuggestedLayout`	✖	`false`	Add `suggestedLayoutConfiguration` (v2 Next layout) to node output
`maxAutodetectSampleRows`	✖	`200`	Rows sampled per file for autodetect
`timeColumnIsMs`	✖	`true`	Ms offset from `baseCapturedAt`
`timeColumnIsIso8601`	✖	`false`	ISO timestamps per realm
`baseCapturedAt`	✖**	,	ISO8601 base when using ms mode
`sortByTime`	✖	`true`	Sort series by `timeMs`
`defaultRealm`	✖	`default`	Fallback realm name
`realmColumn`	✖	,	Column whose distinct values become realms
`discardRowsWithEmptyRealm`	✖	`false`	Drop rows with empty realm cell
`groups` / `defaultGroupId`	✖	`metrics`	Channel groups
`excludeColumns` / `includeColumns`	✖	,	Column filters
`channels`	✖	`[]`	Explicit channel definitions
`treatEmptyAsNone` / `discardEmptyValues` / `discardEmptyRows`	✖	see core-utils	Empty handling
`primaryKeyColumns`	✖	`[]`	Extra dedupe keys
`dedupePolicy`	✖	`last`	`last` \| `first`
`dataserviceVersion`	✖	`1.0`	Payload version
`dashboardUsers`	✖	`[]`	Optional ACL GUIDs

* Required when JSON has multiple tables.
** Required when timeColumnIsMs=true and timeColumnIsIso8601=false.

Auto-discovered channels

Numeric columns (excluding time, realm, primary keys, excluded) become shape: "series" channels. Channel id = slugified header (Speed (m/s) → speed_m_s). Layout channels must use these ids.

Output


{
  "$heat-dataservice": { "version": "1.0", "groups": [], "realms": [] },
  "dashboard_users": [],
  "tabularUploadSummary": {
    "schemaVersion": 1,
    "processedAt": "2026-05-20T12:00:00.000Z",
    "files": [],
    "totals": { "fileCount": 0, "rowCount": 0, "errorCount": 0 }
  }
}

CustomRenderer reads $session.payload.tabularUploadSummary on the session page (sandbox mode). See CustomRenderer.

Example configuration


{
  "timeColumn": "time_ms",
  "timeColumnIsMs": true,
  "baseCapturedAt": "2025-01-01T00:00:00Z",
  "realmColumn": "pilot_id",
  "realmPerFile": false,
  "whenNoOutputs": "complete"
}

system-tabular-to-dataservice (platform, system-utils)