Node Templates

Node Templates define the type of operation a node can perform within HEAT’s Directed Acyclic Graph (DAG) workflows. They act as reusable blueprints for different kinds of data-processing tasks, such as input, transform, processing, or aggregation. The actual configuration, connections, and scheduling of these operations are determined when you place them in a Session Template, where the Session Template Node references a Node Template.

What Is a Node Template?

A Node Template describes:

• Operation Type: For instance, whether it’s an input node that receives data from an Ingest, a transform node that reshapes data, a processing node running algorithms, or an aggregation node combining multiple data streams.
• Expected Inputs/Outputs: Defines what kind of data (e.g., JSON, CSV, or custom object) the node expects and what it will produce.
• Configuration Schema: Declares any parameters or settings required by the node. This could include a destination Data Source, runtime options, or file format preferences.
• Multi-Input Support: Indicates if the node can handle multiple upstream inputs at once (e.g., merging two CSV outputs) or requires a single input only.
• Feature/Capability: References the underlying feature or runner (e.g., a custom transformation runner or machine-learning algorithm) that will execute this operation.

You can think of Node Templates as “kinds of operations.” Each node in a Session Template points to a Node Template, along with specifics like how it connects to other nodes and any parameter overrides.

Relationship to Runners

While a Node Template defines what the node is supposed to do conceptually, a runner is what actually executes the logic:

• Runners are containerized processes deployed by HEAT to handle one or more Node Template types.
• A Node Template typically references a runner name or ID, but it does not itself dictate how the task is performed internally.
• This separation of concerns allows you to update or replace the underlying runner without changing every Session Template node.

Defining Node Templates

Like other core HEAT entities (e.g., Projects, Session Templates), Node Templates can be defined or updated:

• Via the HEAT API: Programmatically create or modify Node Templates for integration into your CI/CD pipelines.

Using Node Templates in Session Templates

When building a Session Template:

1. Choose a Node Template: For each step in your data pipeline, select the Node Template that aligns with the operation needed (e.g., an “XML-to-JSON Transform Node”).
2. Set Configuration: Override or fill in any parameters required by that node (e.g., specifying the Data Source, file formats, or API endpoints).
3. Link to Dependencies: Indicate which nodes feed data into this node, and which node or nodes it outputs to.

Example Use Cases

• Input Node
  An input node is typically processed by our Ingests and the payload gets stored in a Data Source for downstream processing.
• Transform Node
  A Node Template called “transform-csv-biometrics” could read raw CSV input, extract particular columns, and convert to a consistent format for another downstream node.
• Processing Node
  A Node Template called “process-biometrics” might involve a runner that takes JSON input and applies a machine-learning model to generate predictions or additional metadata.
• Aggregation Node
  “Combine Metrics” could accept outputs from multiple transform or processing nodes, merging them into a single dataset or summary.

Runner hubs

Template selection tables (purpose and limitations) live on each runner index:

• Core Utils
• System Utils
• Dashboard Utils (docs path dashboard-tools, runner image dashboard-utils)
• Audio Utils

Platform templates (system-utils)

Summaries below point to System Utils detail pages. For integrator Next dashboards, use dashboard-v2 on dashboard-utils and the dashboard-v2 upstream contract.

Shipped examples: system-tabular-dashboard-next, system-tabular-dashboard-autodetect-next, sample-arbex-composable-dashboard, heat-system-resource-monitor-next, sample-arbex-legacy-dashboard (Legacy v1 via Arbex). Legacy payload authoring: v1 dashboard RAG  in the monorepo.

heat-workspace-group-count (platform reserved, system-utils only)

See heat-workspace-group-count. The template reads one parent JSON object, clones the root, and appends heatWorkspaceGroupCountV1 for Next BarChart value channels.

Node configuration (instance DefaultConfiguration)

Output: heatWorkspaceGroupCountV1

• schemaVersion: integer, currently 1.
• aggregation: literal countByKey in v1.
• itemsPath, groupKeyPath, filters, sort, maxGroups: echo effective settings for auditing.
• groups: { "key", "label", "count" }[] sorted per sort.
• barSeries: Apex payload. Single-series (default): [{ "name": string, "data": [{ "x", "y", "cpuRequestsMilli"?, "memoryRequestsMi"? }] }] where y is the item count per bucket. Grouped metrics when resourceTotalsAsGroupedBarSeries is true: two entries [{ "name": "<cpu>", "data": [{ "x", "y" }] }, { "name": "<memory>", "data": [...] }] with the same x order; y is millicores or MiB respectively. The Next BarChart uses dual colour and dual y-axes when multiple series are present.

Bind in heat-system-dataservice-envelope with valueFromBindings and copyPath: for example copy heatWorkspaceGroupCountV1.barSeries into a value channel key (e.g. barSeries) so data.value.barSeries is available to BarChart in ui/dashboard.

system-arbex-js (platform reserved, system-utils only)

Sandboxed JavaScript on the platform runner. API reference: Arbex API reference. Platform guide: system-arbex-js. Arbex RAG corpus: tools/arbex/rag/.

Best Practices

• Keep Node Templates Small & Focused
  Each template should handle a single logical step. This approach reduces complexity and makes troubleshooting easier.
• Document Expected Inputs/Outputs
  Make sure the Node Template’s schema is well-described, so other team members or external integrators know how to pass data between nodes.
• Ensure Idempotence
  Node Templates that rely on external state or have side effects can lead to unpredictable results. Strive for operations that produce the same output given the same inputs.

Retention (session template nodes)

In a Session Template, each node’s defaultConfiguration may include retention.skipArchive: true so session retention does not trim that node’s output history during ArchiveByAge. Input-type templates are always exempt from archival.

Summary

• Node Templates define what an operation is supposed to do (type, expected inputs/outputs, config schema).
• Runners handle how that operation is executed at runtime.
• By combining Node Templates in a Session Template, you build out an end-to-end data workflow.
• They can be created or updated programmatically via the HEAT API and stored for reuse across multiple workflows.

Next Steps

• See how Node Templates fit into complete workflows with Session Templates.
• Explore how data enters the system through Ingests.
• Learn how runners execute node templates in the cluster.