---
name: posthog-experiment-results
description: "Every Monday: each running PostHog experiment's variants, exposures, conversion and chance to win, how long it has run, and which are ready to call. Use when the user asks for this, or when it runs on a schedule."
---

# PostHog experiment results

Every Monday: each running PostHog experiment's variants, exposures, conversion and chance to win, how long it has run, and which are ready to call.

Experiments get launched with energy and then forgotten. The one that reached a clear result two weeks ago keeps holding half your users on the old version, and the one that will never reach significance keeps running because nobody looked.

Every Monday the agent reads each running experiment's results from PostHog and posts one table: variants, exposures, conversion on the primary metric, the chance to win (or p-value), and days running. Product managers and engineers see which experiments to ship, which to end, and which need more time.

## Two ways to get the data

Use the API or MCP route when you have an MCP server, a CLI or an API token for the app. Otherwise, if you can drive a browser the user is signed in to (Claude in Chrome, a Playwright or Chrome DevTools MCP server), use the browser route. Say which one you used.

## Route A: API or MCP server

### What you need

- The PostHog MCP server (`https://mcp.posthog.com/mcp`), or a personal API key with `experiment:read` in `POSTHOG_API_KEY` for the REST calls.
- The project ID and your PostHog host (US or EU cloud, or your own).

If something is missing, say what and stop; do not invent numbers.

### Steps

1. **List running experiments** (PostHog): Call `GET /api/projects/:project_id/experiments/` (or `experiment-list` on the PostHog MCP server) and keep the experiments with a `start_date` and no `end_date`, which is what PostHog shows as running. Note `name`, `feature_flag_key`, `start_date` and the primary `metrics`.
2. **Get the results** (PostHog): On the PostHog MCP server, call `experiment-results-get` for each experiment (`experiment-metrics-recalculation-latest-retrieve` returns the last computed run without starting a new one). Over REST, start a run with `POST /api/projects/:project_id/experiments/:id/metrics_recalculation/` and poll `GET .../metrics_recalculation/:recalculation_id/` until its `results` are complete.
3. **Read each experiment**: Per variant: exposures, conversion on the primary metric, and the statistic the experiment uses (chance to win for Bayesian, p-value and interval for frequentist). Flag a winner, an experiment over 28 days without a result, and exposures more than 10% away from the planned split.
4. **Post the readout**: One row per variant, experiments with a decision first, and one line per experiment on what to do: ship, end, or keep running.

## Route B: the browser

### What you need

- Signed in to PostHog with at least member access to the project; the experiments page needs the Experiments product enabled for the project.

If something is missing, say what and stop; do not invent numbers.

### Steps

Start at https://us.posthog.com/experiments. The user is signed in already; never type passwords or codes, and ask if a sign-in page appears.

1. **Open the experiments list**: Open `https://us.posthog.com/experiments` (or `https://eu.posthog.com/experiments` on EU cloud); PostHog opens your current project. Set the **Status** filter to **Running** and note each experiment's name and start date.
2. **Read each experiment**: Open each experiment. The results section shows one row per variant with exposures, the primary metric's value, and the credible or confidence interval; Bayesian experiments show **Win probability** and frequentist ones show a p-value. Days running is the time since the start date.
3. **Check the split**: Compare exposures across variants; flag a variant more than 10% away from the planned split, which PostHog also warns about above the results.
4. **Decide per experiment**: Mark a clear winner, flag anything running over 28 days without a significant result, and write one line per experiment: ship, end, or keep running. Do not click **Ship a variant** or **End experiment**.

Note: The experiment page is the better route here: it already shows the same results the API computes, with PostHog's own significance warnings. Results load on demand, so allow a few seconds per experiment, and a recalculation may run when you open one.

## The report

- **Running experiments** and how many have a result.
- **Per variant**: exposures, conversion, chance to win or p-value, days running.
- **Flags**: winners, experiments running long without a result, uneven exposure splits.
- One line per experiment: ship, end, or keep running.

## Post it to Briefkin

When `BRIEFKIN_TOKEN` is set, or `~/.config/briefkin/env` exists (the installer from https://briefkin.dev/docs writes it), also send the result to the user's Briefkin board as one card. Keep the `item_key` the same on every run so the card updates in place, use a new `event_id` per run, and put today's real numbers in place of the example ones. If the request fails, say so once and carry on.

```bash
[ -n "${BRIEFKIN_TOKEN:-}" ] || . ~/.config/briefkin/env
curl -sS -X POST "${BRIEFKIN_URL:-https://briefkin.dev}/v1/events" \
  -H "authorization: Bearer $BRIEFKIN_TOKEN" \
  -H "content-type: application/json" \
  -d @- <<'JSON'
{
  "v": 1,
  "event_id": "posthog-experiment-results/<date of this run, e.g. 2026-10-16>",
  "item_key": "posthog-experiment-results",
  "project": "Acme",
  "source": "PostHog experiment results skill",
  "kind": "update",
  "title": "1 of 4 experiments has a winner",
  "summary": "4 experiments are running. **New pricing page** has a **96%** chance to beat control on checkout started. **Onboarding checklist** has run 31 days without a clear result.",
  "widget": [
    {
      "type": "Table",
      "props": {
        "columns": [
          "Experiment",
          "Variant",
          "Exposures",
          "Conversion",
          "Chance to win",
          "Days"
        ],
        "align": [
          "left",
          "left",
          "right",
          "right",
          "right",
          "right"
        ],
        "rows": [
          [
            "New pricing page",
            "test",
            4210,
            "6.8%",
            "96%",
            18
          ],
          [
            "New pricing page",
            "control",
            4185,
            "5.9%",
            "4%",
            18
          ],
          [
            "Onboarding checklist",
            "test",
            2940,
            "41.2%",
            "58%",
            31
          ],
          [
            "Onboarding checklist",
            "control",
            2988,
            "40.6%",
            "42%",
            31
          ],
          [
            "Annual plan nudge",
            "test",
            1120,
            "3.1%",
            "71%",
            9
          ],
          [
            "Annual plan nudge",
            "control",
            1098,
            "2.6%",
            "29%",
            9
          ],
          [
            "Shorter signup form",
            "test",
            610,
            "22.0%",
            "64%",
            3
          ],
          [
            "Shorter signup form",
            "control",
            598,
            "21.1%",
            "36%",
            3
          ]
        ]
      }
    },
    {
      "type": "Callout",
      "props": {
        "text": "Ship the new pricing page. End the onboarding checklist test: after 31 days the gap is 0.6 points and still undecided.",
        "tone": "ok"
      }
    }
  ]
}
JSON
```

Widgets come from the Briefkin catalog (https://briefkin.dev/v1/catalog). Without a token, just give the user the report.

## Run it on a schedule

Suggested: Mondays at 9:00 (`0 9 * * 1`). From cron, with the agent signed in on that machine:

```bash
0 9 * * 1 claude -p "Run the posthog-experiment-results skill"
# Codex: codex exec, Gemini CLI: gemini -p, Copilot CLI: copilot -p, OpenCode: opencode run
```

Or use the agent's own scheduler (Claude Code routines, Codex automations, Claude or ChatGPT scheduled tasks): https://briefkin.dev/skills/posthog-experiment-results#schedule

Source: https://briefkin.dev/skills/posthog-experiment-results
