--- name: github-deploy-error-conversion-chain description: "Each morning and after a deploy: every production release lined up with the Sentry issues it introduced and the key event rate before and after it. Use when the user asks for this, or when it runs on a schedule." --- # Deploy to error to conversion chain Each morning and after a deploy: every production release lined up with the Sentry issues it introduced and the key event rate before and after it. For developers and PMs who have watched a sign-up rate sag and spent an afternoon working out which release did it. The deploy time is in GitHub or Vercel, the new error is in Sentry, and the drop is in PostHog or GA4. Each tool is right on its own; none of them puts the three on one clock. The agent takes every production deploy of the last 24 hours, finds the PR behind it, the Sentry issues that first appeared after it, and the hourly key event count around it, set against the same hours a week earlier. When a deploy lines up with a new error and a drop, the card lands in Needs you with the PR number on it. ## Two ways to get the data Use the API or MCP route when you have an MCP server, a CLI or an API token for the app. Otherwise, if you can drive a browser the user is signed in to (Claude in Chrome, a Playwright or Chrome DevTools MCP server), use the browser route. Say which one you used. ## Route A: API or MCP server ### What you need - Read access to deployments in GitHub (`gh` or `GITHUB_TOKEN` with Deployments: Read) or to the Vercel project (Vercel MCP or `VERCEL_TOKEN`), whichever records your production deploys. - A Sentry token with `event:read` and `project:read`, or the Sentry MCP server; releases set to the commit SHA make the match exact. - PostHog (`query:read`) or GA4 (Viewer) access, and the name of the one key event to watch, such as `signed_up` or `sign_up`. If something is missing, say what and stop; do not invent numbers. ### Steps 1. **List production deploys and their PRs** (GitHub): From whichever you use: `GET /repos/{owner}/{repo}/deployments?environment=production` with each deployment's `success` status time from `/deployments/{id}/statuses`, or Vercel `GET /v7/deployments?projectId=&target=production&state=READY&since=` using `ready` and `meta.githubCommitSha`. Map each SHA to its PR with `GET /repos/{owner}/{repo}/commits/{sha}/pulls`. Keep the last 24 hours (or the one deploy that triggered this run). 2. **Get the new errors** (Sentry): Call `GET /api/0/organizations/{org}/issues/?query=is:unresolved firstSeen:-24h&environment=production&statsPeriod=24h&sort=freq`. Keep `shortId`, `title`, `firstSeen`, `count`, `userCount` and the first release (from the issue detail's `firstRelease.version`). 3. **Pull the key event by hour** (PostHog): In PostHog, POST a HogQL query to `/api/projects/:id/query/`: `SELECT toStartOfHour(timestamp) AS hour, count() FROM events WHERE event = 'signed_up' AND timestamp > now() - INTERVAL 8 DAY GROUP BY hour ORDER BY hour`. In GA4 instead, `runReport` with dimension `dateHour`, metric `keyEvents` and a filter on `eventName`. Whichever you use, cover 8 days so last week's same hours are there. 4. **Line them up on one clock**: Convert everything to UTC (GA4 `dateHour` is in the property's time zone). Tie a Sentry issue to a deploy when its first release matches the deploy SHA or version, or, with no release set, when it was first seen within 60 minutes after that deploy and before the next one. For each deploy, compare key events from the deploy hour until the next deploy or now against the same hours 7 days earlier; a drop of 10% or more on at least 30 baseline events counts. Deploys with no new issue and no drop are listed as clear. 5. **Post the chain**: One row per deploy: PR, live time, new issues, key event change and a verdict (suspect, watch, clear). When a deploy has both a new issue and a drop, set `work_state: failed` and put the PR, the issue and the drop in the title. ## Route B: the browser ### What you need - Signed in to GitHub (or Vercel) with access to the project's deployments, to Sentry as an organization member, and to PostHog or GA4 with view access. If something is missing, say what and stop; do not invent numbers. ### Steps Start at https://github.com/. The user is signed in already; never type passwords or codes, and ask if a sign-in page appears. 1. **Find the production deploys**: Open the repository, then **Deployments** in the right sidebar (or `https://vercel.com/` > the project > **Deployments**, filtered to Production). Note each deploy of the last 24 hours, its time and commit, and open the commit to find its PR number. 2. **Read the new Sentry issues**: In `https://sentry.io/`, open **Issues**, set the environment to production and the date to the last 24 hours, search `is:unresolved firstSeen:-24h` and sort by Events. For each issue, open it and note the first seen time and the first release shown in the sidebar. 3. **Read the key event by hour**: In PostHog, create a Trends insight on your key event, interval **Hour**, last 8 days. In GA4, use Explore with dimensions Date + hour (YYYYMMDDHH) and Event name, filtered to the key event. Note counts from each deploy hour onward and the same hours 7 days earlier. 4. **Line them up**: Put all times in one time zone. Tie an issue to a deploy when its first release is that deploy's commit, or it was first seen within 60 minutes after it. Call a deploy suspect when it has a new issue and the key event fell 10% or more on at least 30 baseline events. Note: The UI shows Sentry and PostHog times in your profile's time zone and GA4 in the property's, so check both before lining them up. GA4 Explore can lag several hours. This route is fine for one suspect deploy; for a day with many deploys the API route is faster and less error-prone. ## The report - **The verdict**: the deploy that lines up with a new error and a drop, or "all clear". - **Per deploy**: PR, live time, new Sentry issues (events, users), key event change against the same hours last week. - **The timeline** for the suspect deploy: deploy, first error, and the hourly key event count. - One line on what to do first, for example "Roll back PR 123 or fix WEB-4F1 on the sign-up form". ## Post it to Briefkin When `BRIEFKIN_TOKEN` is set, or `~/.config/briefkin/env` exists (the installer from https://briefkin.dev/docs writes it), also send the result to the user's Briefkin board as one card. Keep the `item_key` the same on every run so the card updates in place, use a new `event_id` per run, and put today's real numbers in place of the example ones. If the request fails, say so once and carry on. ```bash [ -n "${BRIEFKIN_TOKEN:-}" ] || . ~/.config/briefkin/env curl -sS -X POST "${BRIEFKIN_URL:-https://briefkin.dev}/v1/events" \ -H "authorization: Bearer $BRIEFKIN_TOKEN" \ -H "content-type: application/json" \ -d @- <<'JSON' { "v": 1, "event_id": "github-deploy-error-conversion-chain/", "item_key": "github-deploy-error-conversion-chain", "project": "acme-web", "source": "Deploy to error to conversion chain skill", "kind": "update", "title": "Sign-ups down 18% since PR 123 shipped", "summary": "PR 123 went live at **14:02**. Sentry issue **WEB-4F1** was first seen at **14:09** in that release. Sign-ups since: **141**, against 172 in the same hours last week (**-18%**).", "work_state": "failed", "widget": [ { "type": "Steps", "props": { "items": [ { "label": "14:02 PR 123 \"New sign-up form validation\" live in production", "state": "done", "note": "Commit 9f2c41e, release 9f2c41e" }, { "label": "14:09 WEB-4F1 TypeError: email.trim is not a function", "state": "failed", "note": "First seen in release 9f2c41e; 212 events, 168 users" }, { "label": "14:00 to 20:00 sign-ups 141 vs 172 last week", "state": "failed", "note": "-18% on the same hours 7 days earlier" } ] } }, { "type": "Table", "props": { "columns": [ "Deploy", "Live", "New issues", "Sign-ups", "Verdict" ], "align": [ "left", "left", "left", "right", "left" ], "rows": [ [ "PR 123", "Today 14:02", "WEB-4F1 (212 events)", "-18%", "Suspect" ], [ "PR 121", "Today 09:40", "None", "+2%", "Clear" ], [ "PR 119", "Yesterday 17:15", "API-2C0 (9 events)", "-3%", "Watch" ] ] } }, { "type": "Callout", "props": { "text": "Roll back PR 123 or ship a fix for WEB-4F1 first: the error fires on the sign-up form submit, which matches the drop.", "tone": "bad" } } ] } JSON ``` Widgets come from the Briefkin catalog (https://briefkin.dev/v1/catalog). Without a token, just give the user the report. ## Run it on a schedule Suggested: Daily at 8:00, and after each deploy (`0 8 * * *`). From cron, with the agent signed in on that machine: ```bash 0 8 * * * claude -p "Run the github-deploy-error-conversion-chain skill" # Codex: codex exec, Gemini CLI: gemini -p, Copilot CLI: copilot -p, OpenCode: opencode run ``` Or use the agent's own scheduler (Claude Code routines, Codex automations, Claude or ChatGPT scheduled tasks): https://briefkin.dev/skills/github-deploy-error-conversion-chain#schedule Source: https://briefkin.dev/skills/github-deploy-error-conversion-chain