Define your own Crunch

Now that you understand the Crunch Node and the challenge package through the default scaffold, this guide walks you through customizing them to build your own Crunch.

Build your custom Crunch Node (8:08)

You will:

Update the model interface for your prediction task
Build your scoring function
Configure the Crunch Node
Customize the challenge package for participants
Test the full loop locally

Your workspace

When you ran crunch-node init my-challenge, you got this structure:

my-challenge/
├── node/                          ← Crunch Node (Docker Compose, config, scripts)
│   ├── docker-compose.yml
│   ├── .local.env                 # Environment variables
│   ├── config/
│   │   ├── crunch_config.py       # CrunchConfig — type shapes and behavior
│   │   └── callables.env          # Callable overrides
│   ├── api/                       # Custom API endpoints
│   ├── extensions/                # Custom callables
│   └── Makefile
├── challenge/                     ← Participant-facing package
│   ├── starter_challenge/
│   │   ├── tracker.py             # Model base class
│   │   ├── scoring.py             # Scoring function
│   │   ├── backtest.py            # Backtest harness
│   │   └── examples/              # Quickstarter models
│   └── pyproject.toml
├── webapp/                        ← Local Coordinator Platform clone (starter + platform apps)
└── Makefile

The node/docker-compose.yml setup uses this local webapp/ clone as the build context for report-ui, so you can evolve the dashboard UI alongside your node and challenge package.

Step 1: Define your prediction task

Your prediction task defines what participants need to predict and how their models interact with your Crunch Node.

Design the model interface

Edit challenge/starter_challenge/tracker.py to define the contract between your node and every Cruncher submission. Ask yourself:

What data will models receive? (e.g., price ticks, tabular features, images)
What should models return? (e.g., a class label, a probability distribution, a numeric value)
What methods do models need? (e.g., tick + predict, or train + infer)

The default TrackerBase uses a tick() → predict() pattern for real-time streaming:

class TrackerBase:
    def tick(self, data: dict[str, Any]) -> None:
        """Receive latest market data. Override to maintain state."""
        ...

    def predict(self, subject: str, resolve_horizon_seconds: int, step_seconds: int) -> dict[str, Any]:
        """Return a prediction for the given scope."""
        raise NotImplementedError

For a batch classification problem, you might use a completely different pattern:

class MyModelBase(abc.ABC):
    @abc.abstractmethod
    def train(self, train_data: pd.DataFrame) -> None:
        """Train on labeled data."""
        pass

    @abc.abstractmethod
    def predict(self, features: pd.DataFrame) -> pd.DataFrame:
        """Return predictions with a 'prediction' column."""
        pass

Define output types

Update your CrunchConfig in node/config/crunch_config.py to match the types your models will produce:

from coordinator_node.crunch_config import CrunchConfig

contract = CrunchConfig(
    raw_input_type=RawInput,        # Shape of feed data
    output_type=InferenceOutput,    # Shape of model predictions
    score_type=ScoreResult,         # Shape of scoring results
)

The MODEL_BASE_CLASSNAME in node/.local.env must match your tracker class path. For example, if your package is my_challenge and the class is MyModelBase, set MODEL_BASE_CLASSNAME=my_challenge.model_base.MyModelBase.

Step 2: Build your scoring function

Edit challenge/starter_challenge/scoring.py to implement your evaluation logic:

def score_prediction(prediction, ground_truth):
    """Score a single prediction against ground truth.
    
    Return a dict matching your ScoreResult shape.
    """
    error = abs(prediction["value"] - ground_truth["value"])
    return {
        "value": 1.0 / (1.0 + error),
        "success": True,
        "failed_reason": None,
    }

Then point the score worker at your function in node/.local.env:

SCORING_FUNCTION=starter_challenge.scoring:score_prediction

Choose your evaluation approach

Approach	When to use	Example
Accuracy / F1	Classification tasks	Iris classification
Log-likelihood	Probabilistic predictions	Density estimation
MSE / MAE	Regression tasks	Price forecasting
Custom metric	Domain-specific evaluation	Risk-adjusted returns

Configure multi-metric scoring

Beyond your per-prediction scoring function, the engine computes portfolio-level metrics. Configure which ones in your CrunchConfig:

contract = CrunchConfig(
    metrics=["ic", "ic_sharpe", "hit_rate", "max_drawdown"],
    aggregation=Aggregation(ranking_key="ic_sharpe"),  # Rank by IC Sharpe
)

Choose your payout schedule

Set CHECKPOINT_INTERVAL_SECONDS in node/.local.env:

Continuous payouts — e.g., 604800 (1 week) for ongoing competitions with live data
One-time payout — set a long interval and trigger manually at competition end

See Crunch lifecycle for the full checkpoint and payout flow.

Step 3: Configure the Crunch Node

Edit node/.local.env with your competition settings:

# Competition identity
CRUNCH_ID=my-challenge

# Data source
FEED_SOURCE=pyth
FEED_SUBJECTS=BTC,ETH
FEED_KIND=tick
FEED_GRANULARITY=1s

# Model interface
MODEL_BASE_CLASSNAME=starter_challenge.tracker.TrackerBase

# Scoring
SCORING_FUNCTION=starter_challenge.scoring:score_prediction
CHECKPOINT_INTERVAL_SECONDS=604800

For advanced customization, you can override additional callables via environment variables:

Env var	Purpose
`INFERENCE_INPUT_BUILDER`	Transform raw feed data into model input
`INFERENCE_OUTPUT_VALIDATOR`	Validate model output shape and values
`MODEL_SCORE_AGGREGATOR`	Aggregate per-model scores across predictions
`LEADERBOARD_RANKER`	Custom leaderboard ranking strategy

Step 4: Customize the challenge package

Update the participant-facing package:

Update the model interface

Edit challenge/starter_challenge/tracker.py with your base class (done in Step 1).

Write quickstarter examples

Replace the default examples in challenge/starter_challenge/examples/ with working models for your competition. Provide at least one that participants can submit immediately.

Update the scoring function

Edit challenge/starter_challenge/scoring.py with your evaluation logic (done in Step 2). This lets participants score locally.

Configure backtest data

Update challenge/starter_challenge/config.py so the backtest harness knows where to fetch historical data from your Crunch Node.

Update package metadata

Edit challenge/pyproject.toml — change the package name, version, and description.

Publish to PyPI

cd challenge
pip install build twine
python -m build
twine upload dist/*

Step 5: Test the full loop locally

Finishing a Custom Crunch Node Implementation (4:38)

With your customizations in place, run the complete stack:

cd node
make deploy

Then verify:

Models connect — Open the Coordinator Platform at http://localhost:3000 and check that test models appear
Predictions flow — Submit a test model and confirm predictions are being collected
Scoring works — Wait for the scoring interval to pass, then check the leaderboard
API returns data — Hit http://localhost:8000/reports/leaderboard to verify results

You can also run the full preflight check:

make preflight

This validates config, deploys, checks model connectivity, and runs end-to-end verification.

Scoring requires ground truth. If your predictions have a 1-minute horizon, you need to wait 1 minute before scores appear on the leaderboard.

Checklist

Before moving to deployment, confirm:

Model interface defined in challenge/starter_challenge/tracker.py
Scoring function implemented in challenge/starter_challenge/scoring.py
CrunchConfig updated in node/config/crunch_config.py
Environment variables set in node/.local.env
Quickstarter examples work and are easy to modify
Backtest harness fetches data and produces scores
Full loop works end-to-end with make preflight
Challenge package published to PyPI

Next: Wallet & CLI setup

Set up your Solana wallet and register as a Coordinator on the protocol.

Overview

Core Concepts

Getting Started

CLI Reference

Appendix

Your workspace

Step 1: Define your prediction task

Design the model interface

Define output types

Step 2: Build your scoring function

Choose your evaluation approach

Configure multi-metric scoring

Choose your payout schedule

Step 3: Configure the Crunch Node

Step 4: Customize the challenge package

Step 5: Test the full loop locally

Checklist

Next: Wallet & CLI setup

​Your workspace

​Step 1: Define your prediction task

​Design the model interface

​Define output types

​Step 2: Build your scoring function

​Choose your evaluation approach

​Configure multi-metric scoring

​Choose your payout schedule

​Step 3: Configure the Crunch Node

​Step 4: Customize the challenge package

​Step 5: Test the full loop locally

​Checklist

Next: Wallet & CLI setup

Your workspace

Step 1: Define your prediction task

Design the model interface

Define output types

Step 2: Build your scoring function

Choose your evaluation approach

Configure multi-metric scoring

Choose your payout schedule

Step 3: Configure the Crunch Node

Step 4: Customize the challenge package

Step 5: Test the full loop locally

Checklist