Keyword Coverage — WordPress Plugin Build Specification
1. Purpose
Build a WordPress plugin called Keyword Coverage that helps bulk-content bloggers/affiliate sites decide *what to write next* and *where to publish it*, using semantic embeddings (not just exact-string matching) to compare a bulk list of keywords against a site’s existing published content and category taxonomy.
The plugin has three features, all built on the same core primitive: embed text → compare via cosine similarity.
1. Keyword Coverage Search — classify each input keyword as `Covered`, `Weak`, or `Gap` against existing published posts.
2. Semantic Grouper — cluster a bulk keyword list into groups of meaningfully-identical keywords so duplicates can be eliminated with one click.
3. Keyword Grouper (Category Mapper) — assign each surviving keyword to the best-matching existing WordPress category, or flag it as needing a new category.
2. Core Architecture
2.1 Embedding provider
- Use an embeddings API (OpenAI `text-embedding-3-small`, or Voyage AI, or Google `text-embedding-004`). Store the API key in a plugin settings page (`Settings > Keyword Coverage`).
- Batch requests (provider limits, e.g. 100–2000 inputs per call) with retry/backoff.
- Admin UI polls a REST endpoint for job progress and shows a progress bar.
- Tab 1: Coverage Search
- Tab 3: Category Mapper
- Large `
- Results in a sortable/filterable table (use `WP_List_Table` or a simple JS table).
- Adjustable similarity thresholds exposed as sliders/inputs (with sane defaults) since “covered” vs “weak” is inherently fuzzy and site owners will want to tune it.
- Allow importing a CSV with `keyword, search_volume` so volume can travel through the pipeline and be used for sorting Gap keywords by opportunity size.
- Sort keywords arbitrarily; for each unclustered keyword, compare to all existing cluster “centroids” (or just the first keyword added to that cluster); if similarity ≥ `threshold_group` (default 0.90, tighter than coverage threshold since these are meant to be near-duplicates), add to that cluster; else start a new cluster.
- Grouped/collapsible list: cluster header shows the suggested primary keyword; expand to see all keywords in the group with checkboxes.
– Abstract the provider behind a single PHP class `KC_Embedding_Client` with method `embed( array $texts ): array` so the provider can be swapped later.
2.2 Storage
Custom table `wp_kc_embeddings`:
| column | type | notes |
|
|
|
|
| id | bigint PK | |
| object_type | varchar(20) | `post`, `category`, `keyword_cache` |
| object_id | bigint | post ID or term ID (nullable for ad-hoc keyword cache) |
| source_hash | varchar(64) | md5 of the text that was embedded, to detect staleness |
| embedding | longtext (JSON array) or use a vector-capable table if MySQL/MariaDB version supports it | |
| updated_at | datetime | |
– Post embeddings: generated from `post_title + focus_keyword (Rank Math/Yoast meta if present) + first ~300 words of content`. Regenerate on `save_post` hook (debounced) or via a manual “Rebuild Index” button, since re-embedding all posts on every request is too slow/expensive.
– Category embeddings: generated from `category name + category description + titles of up to N posts in that category`. Rebuild on `create_category`/`edited_category`/manual refresh.
– Keyword embeddings: computed on-demand per request (bulk keyword pastes aren’t reused often enough to justify permanent caching, but cache for the duration of a session/job in a transient or the same table with `object_type = keyword_cache`).
2.3 Background processing
Bulk jobs (thousands of keywords × embedding calls + similarity math) must NOT run synchronously in a single HTTP request.
– Use Action Scheduler (bundled with WooCommerce/many plugins, or pull in as a library) or a simple custom queue table + WP-Cron, to process keyword batches (e.g. 200 at a time) in the background.
– Store job results in a custom table `wp_kc_jobs` (job_id, type, status, input, result JSON, created_at).
2.4 Admin UI
Three tabs under one top-level admin menu “Keyword Coverage”:
– Tab 2: Semantic Grouper
Common UI pattern per tab:
– “Run” button → creates a background job → progress bar.
– Bulk checkbox actions + CSV export on every results table.
3. Feature 1 — Keyword Coverage Search
Input: paste up to several thousand keywords (one per line).
Process (per keyword, batched):
1. Embed the keyword.
2. Compare against all post embeddings (post_type=post/page/product as configurable, status=publish).
3. Take the single highest cosine similarity score and its matching post.
4. Classify:
– `similarity >= threshold_covered` (default 0.86) → Covered
– `threshold_weak <= similarity < threshold_covered` (default 0.70–0.86) → Weak
– `similarity < threshold_weak` → Gap
Output table columns:
`Keyword | Status | Best-Match Post (title, link) | Similarity Score | Search Volume (optional manual column if user pastes it alongside keyword)`
Extras:
– Filter results by status; default sort Gap keywords by similarity ascending (least covered first).
4. Feature 2 — Semantic Grouper
Input: a keyword list (typically the “Weak” + “Gap” export from Feature 1).
Process:
1. Embed all keywords.
2. Cluster via a simple greedy/agglomerative approach (no need for full ML clustering libraries):
– This is O(n²) worst case — fine up to a few thousand keywords in a background job; if it needs to scale further, note that as a v2 optimization (e.g. approximate nearest neighbor / FAISS-style index) but don’t over-engineer for v1.
3. Within each cluster, pick a suggested “primary” keyword (e.g. shortest, or highest search volume if provided).
Output UI:
– User checks which keyword(s) to keep per group; “Export Selected” produces the deduplicated final list.
5. Feature 3 — Keyword Grouper (Category Mapper)
Input: the deduplicated keyword list from Feature 2.
Process:
1. Ensure category embeddings exist/are fresh (rebuild if stale).
2. Embed each keyword.
3. Compare each keyword against all category embeddings; take the best match.
4. If `similarity >= threshold_category` (default 0.75) → assign to that category.
5. If below threshold → flag as “New Category Suggested”; group these leftover keywords using the same clustering logic as Feature 2, then suggest a category name per cluster (either take the most frequent significant word/phrase across the cluster, or — better — make one LLM call per cluster asking for a short category name given the keyword list in that cluster).
Output table columns:
`Keyword | Suggested Category (existing) | Confidence | OR: New Category Suggestion + cluster members`
Bulk action: “Assign” writes the mapping into a results export (CSV: keyword, category) that the user takes into their content/publishing workflow — the plugin doesn’t need to auto-create posts, just produce the routing decision.
6. Non-Functional Requirements
– API cost/rate limits: batch aggressively, cache embeddings for posts/categories, never re-embed unchanged content (hash check).
– Large sites: index build for 2,500+ posts should run as a background job with a progress indicator, triggerable manually from settings (“Rebuild Content Index”).
– Extensibility: keep the embedding client and similarity/clustering logic as separate, swappable PHP classes so the SaaS-like features (this looks like the seed of a paid plugin) can later add other embedding providers or plug into external content APIs (e.g. multisite / agency use across several WP installs).
– Security: nonce-protected AJAX/REST endpoints, capability check `manage_options` (or a custom capability) for all admin actions, sanitize all pasted input.
– i18n: wrap all UI strings for translation (your audience includes Bangla-speaking users — consider bundling a `.po`/`.mo` for Bangla from day one).
7. Suggested Build Order (MVP → v1)
1. Plugin scaffold + settings page (API key, thresholds).
2. Post/category embedding indexer + storage table + manual rebuild button.
3. Feature 1 (Coverage Search) end-to-end, since it’s the simplest (single comparison pass).
4. Feature 2 (Semantic Grouper) — reuse embedding + similarity code from Feature 1.
5. Feature 3 (Category Mapper) — reuse clustering code from Feature 2, add category embeddings.
6. Background job queue + progress UI (retrofit into all three once the core logic is proven on smaller batches).
7. CSV import/export polish, bulk actions, thresholds as user-adjustable settings.
8. One-line prompt to hand to a coding AI
> “Build a WordPress plugin called Keyword Coverage per the attached spec. Start with the plugin scaffold, settings page, and the embedding indexer for posts (custom table `wp_kc_embeddings`, OpenAI `text-embedding-3-small` via a `KC_Embedding_Client` class), then implement Feature 1 (Coverage Search) fully before moving to Features 2 and 3. Use WordPress coding standards, nonce/capability checks on all AJAX/REST endpoints, and keep the similarity/clustering logic in a separate reusable class.”