Model Runner and Client

The Model Runner and Model Runner Client are the two halves of remote model execution. The Runner sits next to each model and serves it over gRPC. The Client sits in your Crunch Node and calls many models concurrently.

Model Runner

The Model Runner is a gRPC server that runs alongside a submitted model and makes it callable over the network. It handles three things:

Dynamic loading — discovers and instantiates the participant’s model class at startup
Remote execution — exposes model methods over gRPC so the Coordinator can call them
Access control — enforces the Secure Model Protocol to verify that only authorized Coordinators can reach the model

Dynamic subclass pattern

When the Coordinator initializes a model through the Client, it specifies a base class (e.g., starter_challenge.tracker.TrackerBase) that the model must implement. The Model Runner:

Searches the submitted code for a class that inherits from this base class
Instantiates it
Exposes its methods over gRPC

The Runner handles all input/output serialization, so both sides exchange structured data reliably.

Health checks

The Runner exposes a health check service so the Coordinator can verify availability remotely. If a model becomes unresponsive, the Client detects it and disconnects.

Model Runner Client

The Model Runner Client is the Coordinator-side Python library. It connects to the Model Orchestrator, maintains a live list of available models, and fans out calls to all of them concurrently over gRPC. It is designed for reliability at scale — even when some models are slow, buggy, or offline.

Initialization

runner = DynamicSubclassModelConcurrentRunner(
    timeout=50,
    crunch_id="your-crunch-id",
    host="localhost",
    port=9091,
    base_classname="starter_challenge.tracker.TrackerBase",
    max_consecutive_failures=10,
    max_consecutive_timeouts=10,
)

Parameter	Description
`timeout`	Maximum seconds to wait for all models during a single call
`crunch_id`	On-chain identity of the Crunch
`host` / `port`	Location of the Model Orchestrator
`base_classname`	Base class that all participant models must implement
`max_consecutive_failures`	Disconnect a model after this many consecutive errors
`max_consecutive_timeouts`	Disconnect a model after this many consecutive timeouts

Then start the connection:

await runner.init()   # connect to the orchestrator and all models
await runner.sync()   # keep the model list updated in the background

Calling models

Every call fans out to all connected models concurrently. You specify the method name and arguments:

await runner.call(
    method="tick",
    arguments=[
        Argument(
            position=1,
            data=Variant(type=VariantType.JSON, value=encode_data(VariantType.JSON, prices)),
        )
    ],
)

To collect predictions:

results = await runner.call(
    method="predict",
    arguments=[
        Argument(position=1, data=Variant(type=VariantType.STRING, value=encode_data(VariantType.STRING, asset_code))),
        Argument(position=2, data=Variant(type=VariantType.INT, value=encode_data(VariantType.INT, horizon))),
        Argument(position=3, data=Variant(type=VariantType.INT, value=encode_data(VariantType.INT, step))),
    ],
)

Each result contains:

Field	Description
model identifier	Which model produced this result
status	`SUCCESS`, `FAILURE`, or `TIMEOUT`
output	The prediction payload
latency	Time spent predicting (microseconds)
`model_name`	User-defined model name
`cruncher_name`	Cruncher display name
`cruncher_id`	Unique Cruncher identifier on-chain
`deployment_id`	Deployed version identifier

Use deployment_id to detect when a Cruncher deploys a new version of their model. You can reset metrics or apply version-specific behavior based on this value.

You can also target a subset of models — for example, to allocate more inference budget to top performers.

Timeout and failure handling

All calls execute with the configured timeout. The Client applies safeguards so a single slow or broken model cannot block the entire system:

A call is marked TIMEOUT if the model takes too long to respond
A call is marked FAILURE if the model raises an exception or returns invalid data

The library tracks consecutive failures and consecutive timeouts per model. When either limit is reached, the model is automatically disconnected. This protects your pipeline from buggy models, very slow models, and models that don’t respect the interface.

Async architecture

The Client is fully asynchronous because it needs to:

Maintain a persistent connection to the Orchestrator
Call many models concurrently on every tick

It uses an event loop to keep the model list up to date, detect when models join or leave, and reconnect automatically when needed.

Avoid blocking the event loop with heavy computations in the predict worker. Delegate CPU-intensive work (scoring, aggregation) to the score worker instead.

Next: Model Orchestrator

How model containers are deployed, monitored, and kept reachable.

Overview

Core Concepts

Getting Started

CLI Reference

Appendix

Model Runner

Dynamic subclass pattern

Health checks

Model Runner Client

Initialization

Calling models

Timeout and failure handling

Async architecture

Next: Model Orchestrator

​Model Runner

​Dynamic subclass pattern

​Health checks

​Model Runner Client

​Initialization

​Calling models

​Timeout and failure handling

​Async architecture

Next: Model Orchestrator

Model Runner

Dynamic subclass pattern

Health checks

Model Runner Client

Initialization

Calling models

Timeout and failure handling

Async architecture