Smoke OTEL - 26000186618

[github-actions]

Executive Summary

Overall: FAIL

The local OTEL mirror is healthy and Grafana (Tempo) received and indexed current-run spans. However, Sentry's OTLP ingest endpoint returned 401 Unauthorized, preventing any spans from reaching Sentry for this run.

Run URL: https://github.com/github/gh-aw/actions/runs/26000186618

Status Table

Backend	Write Config Present	Write Export Succeeded	Read Config Present	Read Query Succeeded	Overall
Local OTLP	✅	✅	—	—	✅
Sentry	✅	❌	✅	❌	❌
Grafana	✅	✅	✅	✅	✅

Evidence

Local OTLP

• OTEL env vars: OTEL_EXPORTER_OTLP_ENDPOINT=set, OTEL_EXPORTER_OTLP_HEADERS=set, GH_AW_OTLP_ENDPOINTS=set, OTEL_SERVICE_NAME=gh-aw.smoke-otel-backends
• Local mirror: /tmp/gh-aw/otel.jsonl — 1 span batch present
• Span for this run:

  {"name":"gh-aw.agent.setup","traceId":"a3927fc7a68ca5c3173434cfc6b82afd","run_id":"26000186618"}

• OTLP export errors: 1 error logged in /tmp/gh-aw/otlp-export-errors.jsonl:

  {"host":"o205451.ingest.us.sentry.io","status":401,"reason":"Unauthorized"}

Sentry

• MCP connection: ✅ (organization github discovered, project gh-aw found)
• OTLP write path: ❌ — 401 Unauthorized on o205451.ingest.us.sentry.io
• Current-run spans (run 26000186618): Not found — 0 results
• Recent gh-aw spans (last 24 h): Not found — 0 results (consistent with broken ingest)
• Representative Sentry link: https://github.sentry.io/explore/discover/homepage/?dataset=errors&queryDataset=error-events&query=&project=4511347087179777

Grafana

• MCP connection: ✅
• Tracing datasource: grafanacloud-ghaw-traces (Tempo, uid=grafanacloud-traces)
• Current-run spans: ✅ — trace a3927fc7a68ca5c3173434cfc6b82afd confirmed with github.run_id=26000186618
• Span attributes verified: github.run_id, github.sha, github.workflow_ref, runner.os, service.version
• Recent gh-aw spans: ✅ — 19 traces visible from multiple workflows (gh-aw.smoke-otel-backends, gh-aw.pr-sous-chef, gh-aw.smoke-ci)
• Representative trace: a3927fc7a68ca5c3173434cfc6b82afd — root span gh-aw.pre_activation.setup, service gh-aw.smoke-otel-backends

Failure Analysis

Sentry — Write Export Failed (401 Unauthorized)

Failing step: OTLP span export to o205451.ingest.us.sentry.io

Evidence observed:

• /tmp/gh-aw/otlp-export-errors.jsonl records: {"host":"o205451.ingest.us.sentry.io","status":401,"reason":"Unauthorized"}
• Sentry list_events returned 0 results for project gh-aw, both with and without the github.run_id:26000186618 filter

Most likely root cause: The GH_AW_OTEL_SENTRY_AUTHORIZATION secret injected into the OTLP headers is invalid, expired, or mismatched for the Sentry DSN endpoint o205451.ingest.us.sentry.io. The 401 is a hard auth rejection, not a rate limit or network issue.

Path: Write path — OTLP auth / configuration

Next debug step:

1. Verify GH_AW_OTEL_SENTRY_AUTHORIZATION secret value matches the active DSN for project gh-aw in the github org on https://us.sentry.io.
2. Run sentry find_dsns --organization_slug github --project_slug gh-aw to retrieve the current DSN and compare the auth token.
3. Rotate or update the secret in the repository/environment settings.

Sentry — Read Query Returned No Results

Failing step: list_events returned 0 results for project gh-aw

Evidence: Direct consequence of the broken write path above — no spans were ingested, so no spans can be read.

Most likely root cause: Same as above (401 on ingest means nothing was stored).

Path: Read path blocked by write path failure

Next debug step: Once the write-path auth is fixed and spans are re-ingested, re-run the same query: sentry list_events --organization_slug github --project_slug gh-aw --query "github.run_id:<run_id>".

Generated by 🧪 Smoke OTEL · ● 4.9M · ◷

• expires on May 17, 2026, 9:20 PM UTC