Runs

Execute and monitor your Daft workflows.

What are Runs?

A run is a single execution of your code on Daft Cloud. Runs allow you to execute Python scripts, modules, or functions with automatic scaling, logging, and monitoring.

You can create runs through the web UI (described below) or programmatically using the Python SDK.

Creating a Run

1. Navigate to Runs in your project sidebar
2. Click Create run
3. Configure the entrypoint (file, module, or function)
4. Set environment options (Python version, dependencies)
5. Click Create

Run Entrypoints

Daft Cloud supports three types of entrypoints:

File

Execute a Python script file. Specify the path to your script relative to your repository root.

Example: scripts/process_data.py

You can also pass command-line arguments that will be available via sys.argv.

Module

Execute a Python module as a script. Specify the module path using dot notation.

Example: my_package.data_processor

Function

Call a specific function within a file. Specify the file path and function name separated by a colon.

Example: scripts/pipeline.py:process_data

You can pass positional and keyword arguments to the function. Use the Arguments section to configure:

• Positional arguments (in order)
• Keyword arguments (by name)
• Data source and catalog mappings

Run Lifecycle

Runs progress through the following states:

State	Description
Pending	Run created, waiting to start
Running	Code is actively executing
Succeeded	Execution completed successfully
Failed	Execution encountered an error
Cancelled	User cancelled the run

Monitoring Runs

Run Details

Each run has multiple tabs for monitoring:

• Details: Entrypoint, source, environment configuration
• Logs: Real-time and historical execution logs
• Problems: UDF exceptions with stack traces
• Results: Return value from the run
• Metrics: CPU usage, token consumption, request counts

Logs

Logs stream in real-time while the run is active:

2024-01-15 10:30:01 | INFO | Starting data processing...
2024-01-15 10:30:05 | INFO | Loaded 10,000 rows
2024-01-15 10:30:15 | INFO | Processing complete

Metrics

Track resource usage and model inference:

Metric	Description
CPU Seconds	Allocated compute time
Total Requests	Number of inference calls
Tokens In	Input tokens for LLM calls
Tokens Out	Output tokens from LLM calls

Run Configuration

When creating a run, you can configure:

Python Version

Runs use Python 3.12. Support for other versions coming soon.

Dependencies

Add Python packages to install before execution. You can specify:

• Package names: pandas, numpy
• Pinned versions: pandas==2.0.0
• Version ranges: numpy>=1.24.0

Environment Variables

Set environment variables for the run. These are available in addition to your project secrets.

Re-running

To re-run a completed run with the same configuration:

1. Open the run details
2. Click the Run again button
3. Optionally modify the configuration
4. Click Create

Saving Runs as Tasks (Beta)

Tasks allow you to save a run configuration for repeated execution. Once saved, tasks can be triggered manually. Support for triggering tasks on a schedule (CRON), or via webhooks is coming soon.

To save a run as a task:

1. Open a completed run’s details
2. Click Save as Task
3. Give the task a name
4. The task will be available in the Tasks section

Tasks are currently in beta. The interface and functionality may change.

Cancelling Runs

To cancel a pending or running execution:

1. Open the run details
2. Click the Cancel button
3. The run will transition to the Cancelled state

Source Code

Runs execute code from your connected Git repository:

• Commit hash: Specific version of your code
• Branch/tag: Reference to a branch or tag

Commit and push your code before creating runs to ensure reproducibility.