Task Plugin Template

Template for creating external task packages that plug into TraceCore via the agent_bench.tasks entry point group.

Directory Layout

directory structure
your_task_package/
  pyproject.toml
  your_task_package/
    __init__.py
    tasks/
      my_task/
        task.toml
        setup.py
        actions.py
        validate.py

Entry Point Registration

pyproject.toml
[project.entry-points."agent_bench.tasks"]
my_task = "your_task_package.tasks.my_task:register"
register function
from pathlib import Path

def register():
    root = Path(__file__).resolve().parent / "tasks" / "my_task"
    return [
        {
            "id": "my_task",
            "suite": "custom",
            "version": 1,
            "description": "Describe the scenario",
            "deterministic": True,
            "path": str(root),
        }
    ]

Validation

terminal
agent-bench tasks validate --path path/to/your_task_package/tasks/my_task

Run the built-in validator to check your task manifest, required files, and registry alignment before publishing.