Building Node Templates: Developing Runners

This chapter shows how to turn business logic into an executable Runner. We will build a minimal example called generic-sum-csv which receives a CSV file, sums every numeric column, and returns a JSON object of the form:


{
  "speed": 120.7,
  "altitude": 3412,
  "temperature": null
}

Columns that cannot be coerced into numbers are reported as null.

Runtime contract

Every runner is a container that:

Installs the heat-runtime Python package.
Subclasses BaseProcessor and implements two async hooks: validate() and process().
Registers itself with HeatRuntime.register() and starts the event loop with HeatRuntime().init().
Remains idempotent, process a single task using only the provided inputs + config.

The typical event loop for the HEAT runtime polls the platform for tasks. When a task is available it is claimed so no other runner can process it. Your logic runs, then results are posted back through the runtime. You do not call HEAT HTTP APIs directly from runner code.

Implementation

src/sum_processor.py


from io import BytesIO
from typing import Dict, Any
 
import pandas as pd
from heat_runtime import BaseProcessor, HeatRuntime
 
class SumProcessor(BaseProcessor):
    """Aggregates all numeric columns of a CSV (supplied in‑memory) into a single JSON row."""
 
    TEMPLATE_NAME = "generic-sum-csv"  # must match the node template
 
    async def validate(self, inputs: Dict[str, Any], config: Dict[str, Any]):
        await super().validate(inputs, config)
        if "file" not in inputs:
            raise ValueError("expected 'file' artefact in inputs")
 
    async def process(self, inputs: Dict[str, Any], config: Dict[str, Any]):
        """Sum every numeric column; non‑numeric become null."""
        raw = inputs["file"]  # bytes or file‑like object provided by HEAT
 
        # normalise to a file‑like buffer that pandas can read
        if isinstance(raw, (bytes, bytearray)):
            buffer = BytesIO(raw)
        else:
            buffer = raw  # assume already file‑like (e.g. StreamingBody)
 
        df = pd.read_csv(buffer)
 
        result = {}
        for col in df.columns:
            if pd.api.types.is_numeric_dtype(df[col]):
                result[col] = float(df[col].sum())
            else:
                result[col] = None
 
        return result
 
if __name__ == "__main__":
    HeatRuntime.register(SumProcessor)
    HeatRuntime().init()

Why async? heat-runtime handles polling and heartbeat in the event loop; your CPU‑bound work can stay synchronous inside process().

Dockerfile

runner/Dockerfile


FROM python:3.12-slim
 
WORKDIR /app
 
# Install runtime and dependencies
RUN pip install heat-runtime pandas==2.*
 
# Copy source
COPY src/ ./src/
 
# Entrypoint
CMD ["python", "src/sum_processor.py"]

Push this image to your registry and reference it in runnerTypes.containerImage.

Updating the manifest

Add a new runner type and node template:

my-feature.heat


{
  "runnerTypes": [
    {
      "name": "sum-csv",
      "containerImage": "registry.example.com/sum-csv:1.0.0",
      "cpuLimit": "1",
      "memoryLimit": "1Gi",
      "description": "Sums numeric columns of a CSV.",
      "enabled": true,
      "featureInternalName": "myFeature"
    }
  ],
  "nodeTemplates": [
    {
      "name": "generic-sum-csv",
      "type": "Processing",
      "acceptsMultipleInputs": false,
      "requiresConfiguration": false,
      "configurationSchema": {},
      "featureInternalName": "myFeature",
      "supportedRunners": ["sum-csv"],
      "hasNonJsonArtifacts": false
    }
  ]
}

Upload the updated manifest via Cluster Manager (Features screen). You can then reference it in session templates; the platform provisions runner workloads for generic-sum-csv tasks when sessions need them.

Test locally (optional)

You can exercise the processor outside HEAT for quick feedback:


python src/sum_processor.py

When no environment flags indicate the HEAT cluster, the Python runtime runs in local dev mode for quick iteration.

Next steps

Building a Manifest - field‑level deep‑dive and JSON‑Schema validation.
Deploying Changes - CI/CD tips for pushing images and manifests.
Using in a Session Template - wiring generic‑sum‑csv into a DAG.