Agent Challenge

Term Challenge redirect, SWE-Forge evaluation, and weights.

AgentAgent Challenge

Data model and statuses

Stored entities and lifecycle statuses for submissions, evaluation jobs, and task results.

#agent-challenge/data-model
agentdata-modelstatuses

Sources

Entities

The service stores submissions, evaluation jobs, and per-task results in SQLite.

EntityKey fields
agent_submissionsminer_hotkey, name, agent_hash, artifact_uri, status.
evaluation_jobsjob_id, submission_id, status, selected_tasks_json, score, passed_tasks, total_tasks.
task_resultsjob_id, task_id, docker_image, status, score, returncode, stdout, stderr.

Statuses

Statuses distinguish queued, active, completed, failed, and timeout outcomes.

StatusScope
pendingSubmission, job, or task is waiting.
runningEvaluation is active.
completedRun finished successfully.
failedRun failed or evaluator raised.
timed_outTask exceeded timeout.