--- name: release-health-check description: "An hour after each production deploy: errors after the release against the hour before, new Sentry issues, key page checks, and the PRs that shipped. Use when the user asks for this, or when it runs on a schedule." --- # GitHub, Vercel and Sentry release health check An hour after each production deploy: errors after the release against the hour before, new Sentry issues, key page checks, and the PRs that shipped. Looking at the error chart after a deploy is the check everyone agrees on and everyone skips when the deploy goes out at 17:50. It also takes three tabs: the deploy in Vercel, the errors in Sentry and the diff in GitHub. Every hour the agent looks for a production deployment that has had a full hour of traffic, compares that hour with the hour before in Sentry, requests a few key URLs on the live site, and lists the pull requests in the release. A regression arrives with its suspects already named, and a clean release gets a one-line all clear. ## Two ways to get the data Use the API or MCP route when you have an MCP server, a CLI or an API token for the app. Otherwise, if you can drive a browser the user is signed in to (Claude in Chrome, a Playwright or Chrome DevTools MCP server), use the browser route. Say which one you used. ## Route A: API or MCP server ### What you need - Vercel deployments through a token in `VERCEL_TOKEN` or the Vercel MCP server, and Sentry events through a token with `event:read` and `org:read` or the Sentry MCP server. - Read access to the GitHub repo through `gh` or a GitHub MCP server, and Sentry release names you can map to a deploy (the commit SHA works well). If something is missing, say what and stop; do not invent numbers. ### Steps 1. **Find the new release** (Vercel): `GET /v7/deployments?projectId=&target=production&state=READY&limit=2` gives the newest production deployment and the one before. Continue only if the newest became ready 60 to 120 minutes ago (its `ready` time), so there is a full hour after it; otherwise post nothing. 2. **List what shipped** (GitHub): `gh api repos/acme/acme-web/compare/...`, with the SHAs from each deployment's `meta.githubCommitSha`, then pull the PR numbers and titles out of the commit messages. 3. **Compare errors before and after** (Sentry): `GET /api/0/organizations/{org}/events/?field=count()&field=count_unique(user)&query=event.type:error&project=` with `start` and `end` for the hour before and the hour after the ready time. Count new issues with `query=is:unresolved firstRelease:`, where the release is what your SDK reports (often the commit SHA). 4. **Run the key checks**: Request each URL on a short list kept in the prompt (home, login, checkout, `/api/health`) and record the status code and time to first byte. A non-2xx status or a time to first byte over 1 second fails the check. 5. **Judge and post**: Healthy when errors rose less than 25% and no new issue reached 10 users; watch when either line is crossed; failed when errors doubled or a key check failed. Post `work_state: failed` for a failed release so it lands in Needs you. ## Route B: the browser ### What you need - Signed in to Vercel (a Viewer seat is enough), to GitHub with read access to the repo, and to Sentry as a member of the organization, all in the same browser profile. If something is missing, say what and stop; do not invent numbers. ### Steps Start at https://vercel.com/dashboard. The user is signed in already; never type passwords or codes, and ask if a sign-in page appears. 1. **Find the release in Vercel**: Select the team, open the project (acme-web) and click **Deployments**. Set **All Environments** to `Production` and **Status** to `Ready`. The row marked **Current** is the release; note its time and commit SHA, and the SHA of the Ready row below it. Stop here if the release went live less than an hour ago. 2. **List what shipped on GitHub**: Open `https://github.com/acme/acme-web/compare/...`. The **Commits** tab lists the merged pull requests by number and title. 3. **Compare errors in Sentry**: Open https://sentry.io/issues/ for the project, set the time range picker to an **Absolute** range covering the hour before the deploy, and read the total events in the chart header with the search `event.type:error`. Repeat for the hour after. Then search `is:unresolved firstRelease:` for issues new in this release and read their Users column. 4. **Run the key checks**: Open `/`, `/login`, `/checkout` and `/api/health` on the live domain in new tabs. Record whether each loads with a normal page or JSON rather than an error page, and whether it took noticeably longer than a second. Open DevTools **Network** for the exact status and time to first byte when it matters. 5. **Judge**: Healthy when errors rose less than 25% and no new issue reached 10 users; watch when either line is crossed; failed when errors doubled or a key check failed. List the PRs so a problem has suspects. Note: This route takes three signed-in apps and an absolute time range in Sentry, so it suits a check after a specific deploy rather than an hourly schedule. Sentry's release name must match what the SDK sends; if that is not the commit SHA, use the release shown on the issue page instead. ## The report - **Verdict**: healthy, watch or failed, with the release and its deploy time. - **Errors**: events in the hour after against the hour before, and the change. - **New issues** first seen in this release, with users affected. - **Key checks**: status and time to first byte per URL. - **What shipped**: the PRs in the release, so a problem has suspects. ## Post it to Briefkin When `BRIEFKIN_TOKEN` is set, or `~/.config/briefkin/env` exists (the installer from https://briefkin.dev/docs writes it), also send the result to the user's Briefkin board as one card. Keep the `item_key` the same on every run so the card updates in place, use a new `event_id` per run, and put today's real numbers in place of the example ones. If the request fails, say so once and carry on. ```bash [ -n "${BRIEFKIN_TOKEN:-}" ] || . ~/.config/briefkin/env curl -sS -X POST "${BRIEFKIN_URL:-https://briefkin.dev}/v1/events" \ -H "authorization: Bearer $BRIEFKIN_TOKEN" \ -H "content-type: application/json" \ -d @- <<'JSON' { "v": 1, "event_id": "release-health-check/", "item_key": "release-health-check", "project": "acme-web", "source": "GitHub, Vercel and Sentry release health check skill", "kind": "update", "title": "Release 9c3e1b7 looks healthy, one issue to watch", "summary": "**312 errors** in the hour after the 15:20 deploy, against 298 the hour before. One new issue so far, affecting 4 users.", "widget": [ { "type": "Grid", "props": { "columns": 3 }, "children": [ { "type": "Stat", "props": { "label": "Errors in the hour after", "value": 312, "delta": 5, "better": "lower" } }, { "type": "Stat", "props": { "label": "New issues", "value": 1 } }, { "type": "Stat", "props": { "label": "Checks passing", "value": "4 of 4", "tone": "ok" } } ] }, { "type": "Table", "props": { "columns": [ "Check", "Status", "TTFB" ], "align": [ "left", "right", "right" ], "rows": [ [ "/", 200, "180 ms" ], [ "/login", 200, "210 ms" ], [ "/checkout", 200, "340 ms" ], [ "/api/health", 200, "45 ms" ] ] } }, { "type": "KeyValue", "props": { "items": [ { "key": "New issue", "value": "ACME-WEB-4FB TypeError in PlanBadge, 4 users" }, { "key": "Shipped", "value": "#1310 plan badges, #1311 audit log filters, #1312 stripe-node bump" } ] } } ] } JSON ``` Widgets come from the Briefkin catalog (https://briefkin.dev/v1/catalog). Without a token, just give the user the report. ## Run it on a schedule Suggested: Every hour (`0 * * * *`). From cron, with the agent signed in on that machine: ```bash 0 * * * * claude -p "Run the release-health-check skill" # Codex: codex exec, Gemini CLI: gemini -p, Copilot CLI: copilot -p, OpenCode: opencode run ``` Or use the agent's own scheduler (Claude Code routines, Codex automations, Claude or ChatGPT scheduled tasks): https://briefkin.dev/skills/release-health-check#schedule Source: https://briefkin.dev/skills/release-health-check