Competition Instructions

The STLabs Compe-ticket-ion challenges you to build a container-based agent that classifies support tickets into ordered (integration, method) pairs. Your agent runs inside a sandboxed environment with a fixed 15-minute timeout, and is scored on correctness, speed, and cost efficiency.

1. Sign in with your @mit.edu email using a one-time magic link on the login page.
2. View your registry credentials on the Submissions page under “Registry Access”. You’ll receive a private ECR namespace and push-only credentials.
3. Build your agent as a Docker container targeting linux/amd64. Your container’s CMD / ENTRYPOINT is executed as-is inside the sandbox.

Your agent receives ticket data on stdin and must emit results on stdout, one JSON object per line.

{ "ticket_id": "t001", "text": "Customer reports Slack integration not sending notifications..." }

{ "ticket_id": "t001", "pairs": [{ "integration": "slack", "method": "send_message" }] }

Your container runs in an E2B sandbox with a 15-minute timeout and restricted network egress (platform domain only).

The proxy is fully compatible with the OpenAI Python SDK. Point base_url at the environment variable and authenticate with the provided proxy token:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["OPENROUTER_BASE_URL"],
    api_key=os.environ["OPENROUTER_API_KEY"],
)

response = client.chat.completions.create(
    model="openai/gpt-4o-mini",  # must be an allowlisted model
    messages=[
        {"role": "system", "content": "You are a ticket routing assistant."},
        {"role": "user", "content": ticket_text},
    ],
)

print(response.choices[0].message.content)

• Only allowlisted models may be used. Requests for other models will be rejected by the proxy.(All models are currently allowed.)
• A per-submission cost cap is enforced — exceeding it will terminate your run immediately.

1. Build and tag your container for linux/amd64.
2. Push the image to your assigned ECR namespace using the credentials shown on the Submissions page.
3. The platform automatically detects pushes and enqueues an evaluation run. You can track status on the Submissions page.
4. Up to 3 queued submissions are allowed at a time. You can cancel queued submissions before they start running.

Each submission is scored using the formula:

final_score = (w1 * completion_rate)
            + (w2 * accuracy)
            + (w3 * speed_score)
            + (w4 * cost_score)

• Accuracy is LCS-based on ordered (integration, method) pairs per ticket.
• Speed score = min(1, baseline_ms / mean_time_ms)
• Cost score = min(1, baseline_usd / cost_per_ticket)
• Fixed baselines are used — no peer normalization.

Scoring weights (w1–w4) are set before the competition begins and we do not plan to change them during the event. In the unlikely event that a weight adjustment is necessary, all leaderboard submissions will be automatically re-scored to ensure fairness.

• Your best score under the active scoring version is shown.
• Failed or timed-out runs are excluded.
• Tie-breaker: earliest completed_at timestamp.
• During the event, only aggregate metrics are shown. Per-ticket breakdowns are not revealed.

• Your container must target linux/amd64 and use digest-pinned image references.
• Network access is restricted to the platform domain only — no external API calls.
• Submissions are evaluated against a fixed 200-ticket set that is disjoint from any samples or templates.
• Ground truth is never revealed to participants.
• Credential rotation has a cooldown period. Plan key rotations accordingly.