Comet Opik

Comet Opik is an open-source platform for evaluating, testing, and monitoring LLM applications.

Step 1: Get your Opik credentials

In Comet, set up your Opik workspace and project:

1. Log in to your Comet account
2. Create or select a workspace for your LLM traces
3. Create a project within the workspace
4. Go to Settings > API Keys to create or copy your API key

Step 2: Enable Broadcast in OpenRouter

Go to Settings > Observability and toggle Enable Broadcast.

Step 3: Configure Comet Opik

Click the edit icon next to Comet Opik and enter:

• Api Key: Your Comet API key (starts with opik_...)
• Workspace: Your Comet workspace name
• Project Name: The project name where traces will be logged

Step 4: Test and save

Click Test Connection to verify the setup. The configuration only saves if the test passes.

Step 5: Send a test trace

Make an API request through OpenRouter and view the trace in your Opik project dashboard.

Custom Metadata

Comet Opik supports custom metadata on both traces and spans for organizing and filtering your LLM evaluations.

Supported Metadata Keys

Example

1
{
2
  "model": "openai/gpt-4o",
3
  "messages": [{ "role": "user", "content": "Evaluate this response..." }],
4
  "user": "user_12345",
5
  "session_id": "session_abc",
6
  "trace": {
7
    "trace_name": "Response Quality Eval",
8
    "generation_name": "Quality Assessment",
9
    "eval_suite": "quality_v2",
10
    "test_case_id": "tc_001"
11
  }
12
}

Additional Context

• Custom metadata keys from trace are included in both the trace and span metadata objects
• Cost information (input, output, total) is automatically added to span metadata
• Model parameters and finish reasons are included in span metadata when available
• The user field maps to user identification in trace metadata
• Opik uses UUIDv7 format for trace and span IDs internally; original OpenRouter IDs are stored in metadata as openrouter_trace_id and openrouter_observation_id

Privacy Mode

When Privacy Mode is enabled for this destination, prompt and completion content is excluded from traces. All other trace data — token usage, costs, timing, model information, and custom metadata — is still sent normally. See Privacy Mode for details.