--- name: dataforseo-new-serp-competitors description: "Every Monday: the domains that entered or left the top 10 on your 25 highest-click keywords, and how many of your clicks those keywords carry. Use when the user asks for this, or when it runs on a schedule." --- # New competitors on your money keywords: DataForSEO SERP API and Search Console Every Monday: the domains that entered or left the top 10 on your 25 highest-click keywords, and how many of your clicks those keywords carry. A new competitor rarely announces itself. It shows up at position 6 on a keyword that pays your bills, climbs for a few weeks, and you notice when clicks are already down. Rank trackers follow your own position; they don't tell you who just arrived beside you. Every Monday the agent takes the 25 queries that bring you the most clicks in Search Console, fetches their live Google results through DataForSEO, and compares each top 10 with last week's. SEO leads, marketers and founders see which domains entered or left, on which keywords, and how many of your clicks ride on those keywords. ## Two ways to get the data Use the API or MCP route when you have an MCP server, a CLI or an API token for the app. Otherwise, if you can drive a browser the user is signed in to (Claude in Chrome, a Playwright or Chrome DevTools MCP server), use the browser route. Say which one you used. ## Route A: API or MCP server ### What you need - Read access to the Search Console property and the market you track (country, plus DataForSEO `location_code` and `language_code`, for example `2840` and `en`). - DataForSEO API login and password, or the DataForSEO MCP server. A run is 25 Standard-queue SERPs a week, about $0.015 at list price; try the task flow on `https://sandbox.dataforseo.com/v3/` first. - A folder the agent can keep `serp-watch.json` in between runs; the first run only records a baseline. If something is missing, say what and stop; do not invent numbers. ### Steps 1. **Fix the 25 money keywords** (Google Search Console): Call `searchanalytics.query` for the last 28 full days with `dimensions: ["query"]` and your country filter, sort by clicks and take the top 25, or use the fixed list from the settings. Save the list in `serp-watch.json` and refresh it only on the first Monday of the month, so the week-to-week comparison covers the same keywords. 2. **Queue 25 SERPs on the Standard queue** (DataForSEO): Send one `POST /v3/serp/google/organic/task_post` with 25 tasks (up to 100 fit in one call), each with the keyword in lowercase, `location_code` and `language_code` for your market, `depth: 10` and `tag` set to the keyword. Poll `GET /v3/serp/google/organic/tasks_ready` and fetch each with `GET /v3/serp/google/organic/task_get/advanced/`; fetching is free for 30 days. 3. **Compare each top 10 with last week**: From each result keep items with `type: "organic"` and `rank_group` 1 to 10, and normalize `domain` (lowercase, strip `www.`). Compare the set of domains per keyword with last week's in `serp-watch.json`: domains in this week's set only have entered, last week's only have left. Ignore your own domain, and note your `rank_group` (or "not in top 10"). 4. **Weigh the change by your clicks** (Google Search Console): Join the changed keywords back to the Search Console rows on the query string and add up the clicks they brought in the last 28 days. A domain new on several keywords is a competitor; one new on a single keyword is often a one-off article. Also read the previous card with `GET https://briefkin.dev/v1/cards/insight-dataforseo-new-serp-competitors` to say whether a new domain was already flagged last week. 5. **Post and save**: Post the keywords that changed with the domains in and out, their positions, your position and the clicks at stake. Write this week's top 10s to `serp-watch.json` for next Monday. ## Route B: the browser ### What you need - Signed in to Google with access to the Search Console property, and to the DataForSEO dashboard with a funded balance. The agent needs a file it can read and write for last week's lists. If something is missing, say what and stop; do not invent numbers. ### Steps Start at https://search.google.com/search-console/performance/search-analytics. The user is signed in already; never type passwords or codes, and ask if a sign-in page appears. 1. **Pick the money keywords**: Pick the property, set Date to `Last 28 days` and your country, and sort the Queries table by Clicks. Note the top 25 queries with their clicks, or use your fixed list. 2. **Fetch the top 10s in DataForSEO**: Open `https://app.dataforseo.com/`, go to API Playground, choose SERP API > Google > Organic > Live Advanced, set your location and language and depth `10`, and run each of the 25 keywords once. Note the organic results ranked 1 to 10 by domain. 3. **Compare with last week**: Compare each keyword's domains with last week's list in the saved file: new domains have entered, missing ones have left. Leave out your own domain and note your position. 4. **Report and save**: List the changed keywords with domains in and out, your position and your clicks from Search Console, and save this week's lists for next Monday. Note: The Playground runs Live requests (about three times the Standard queue price) one at a time, so 25 keywords take a while and cost more than the API route. Searching Google by hand is free but shows results personalized to your browser and location, so it is not comparable week to week. ## The report - **Keywords that changed**: how many of the 25 have a new domain in the top 10. - **New domains**, with the keywords they entered and their positions. - **Domains that left** the top 10. - **Clicks at stake**: last 28 days of your clicks on the changed keywords. - The competitor to look at first. ## Post it to Briefkin When `BRIEFKIN_TOKEN` is set, or `~/.config/briefkin/env` exists (the installer from https://briefkin.dev/docs writes it), also send the result to the user's Briefkin board as one card. Keep the `item_key` the same on every run so the card updates in place, use a new `event_id` per run, and put today's real numbers in place of the example ones. If the request fails, say so once and carry on. ```bash [ -n "${BRIEFKIN_TOKEN:-}" ] || . ~/.config/briefkin/env curl -sS -X POST "${BRIEFKIN_URL:-https://briefkin.dev}/v1/events" \ -H "authorization: Bearer $BRIEFKIN_TOKEN" \ -H "content-type: application/json" \ -d @- <<'JSON' { "v": 1, "event_id": "dataforseo-new-serp-competitors/", "item_key": "dataforseo-new-serp-competitors", "project": "example.com", "source": "New competitors on your money keywords: DataForSEO SERP API and Search Console skill", "kind": "update", "title": "2 new domains entered the top 10 on 6 money keywords", "summary": "**2 new domains** entered the top 10 for **6 of your 25** money keywords this week; **rival.com** took position 2 on \"acme pricing\". Those 6 keywords brought you **1,860 clicks** in the last 28 days.", "widget": [ { "type": "Table", "props": { "columns": [ "Keyword", "Entered (pos)", "Left", "You", "Your clicks 28d" ], "align": [ "left", "left", "left", "right", "right" ], "rows": [ [ "acme pricing", "rival.com (2)", "oldtool.com", 1, 610 ], [ "invoice software", "rival.com (7)", "", 9, 240 ], [ "invoice app", "quotepilot.io (6)", "", 4, 380 ], [ "recurring invoices", "rival.com (9), quotepilot.io (10)", "oldtool.com", 5, 290 ], [ "invoice template pdf", "quotepilot.io (8)", "", 6, 210 ], [ "billing software small business", "rival.com (5)", "", 7, 130 ] ] } }, { "type": "Badges", "props": { "items": [ { "label": "rival.com: 4 keywords", "tone": "bad" }, { "label": "quotepilot.io: 3 keywords", "tone": "warn" }, { "label": "oldtool.com left 2", "tone": "neutral" }, { "label": "19 keywords unchanged", "tone": "ok" } ] } }, { "type": "Callout", "props": { "text": "Look at rival.com first: new on 4 keywords at once, including your own brand query. Its /vs/acme page is the one ranking on \"acme pricing\".", "tone": "warn" } } ] } JSON ``` Widgets come from the Briefkin catalog (https://briefkin.dev/v1/catalog). Without a token, just give the user the report. ## Run it on a schedule Suggested: Mondays at 8:00 (`0 8 * * 1`). From cron, with the agent signed in on that machine: ```bash 0 8 * * 1 claude -p "Run the dataforseo-new-serp-competitors skill" # Codex: codex exec, Gemini CLI: gemini -p, Copilot CLI: copilot -p, OpenCode: opencode run ``` Or use the agent's own scheduler (Claude Code routines, Codex automations, Claude or ChatGPT scheduled tasks): https://briefkin.dev/skills/dataforseo-new-serp-competitors#schedule Source: https://briefkin.dev/skills/dataforseo-new-serp-competitors