[Feature]: Issue 45 | Contextual Tagging

[TylerB24890]

What?

Closes #45

This PR introduces Contextual Tagging to the AI Experiments plugin. It allows for one-click AI-powered suggestions for post tags and categories based on a comprehensive analysis of the post's content (as well as title and excerpt). It helps authors apply a relevant and consistent taxonomy, improving content organization, discoverability, and on-site navigation.

Why?

Properly categorizing and tagging content is fundamental to an effective content strategy. It helps readers find related articles and signals relevance to search engines. However, this process is often inconsistent and subjective. This feature removes the guesswork by providing intelligent, context-aware suggestions, leading to a better-organized and more interconnected website.

How?

• Registered a new ContextualTagging Experiment.
• Registered new settings to enable Contextual Tagging + determine the strategy (Suggest New, Use Existing) and max suggestions.
• Hook into the block editor editor.PostTaxonomyType and inject the Experiment UI (buttons, suggestions)
  • Opt'd for UI built-in to the existing term selectors rather than providing a modal experience as mocked-up in the issue description. See screenshots below for further details.
• Apply a custom system prompt to ensure the experiment settings are respected (new terms, existing terms, max terms, etc...)
• Various filters are available to customize the output;
  • ai_contextual_tagging_content - Filter content before AI processing
  • ai_contextual_tagging_suggestions - Filter suggestions after AI processing
  • ai_contextual_tagging_strategy - Filter the strategy setting
  • ai_contextual_tagging_max_suggestions - Filter the max suggestions setting

Use of AI Tools

• AI Assistance: Yes
• Tool(s): Claude Code (Opus 4.6)
• Used for:
  • Documentation review and workflow summary
  • Experiment scaffolding
  • Unit test boilerplates
  • Debugging support & code suggestions
  • Inline documentation and experiment documentation generation
  • Assistance with the system prompt generation

Testing Instructions

1. Navigate to the Experiments page and enable the Contextual Tagging experiment. Choose the strategy and maximum suggestions.
2. Navigate to the block editor the core post post type and confirm the Suggest Categories and Suggest Tags buttons are available in the term selector panels.
3. Click one of the buttons and confirm terms are returned from the AI according to your selected preferences. If "Existing terms only" is selected, your response should only consist of existing terms with no new terms suggested.
   • If no terms are available in the taxonomy an error message will appear at the top of the block editor.
4. Click on the returned terms and confirm they are assigned (or created + assigned) as expected.
5. Navigate back to the Experiments settings page and change the settings for the Contextual Tagging experiment.
   • If you previously suggested existing terms, change it to the "New Terms" option.
   • Increase/decrease the maximum suggestions.
6. Navigate back to the block editor and repeat steps 3 & 4, confirming the results have changed according to your selected preferences.

Screenshots or screencast

Settings UI:

Suggestion UI (block editor):

New Term Suggestions:

Existing Term Suggestions:

Additional Notes

• Various features from the issue discussion were put on hold for the initial PR. These features can be added in a new PR or after the initial PR review.
  • Empty taxonomy callout - Proactive "Need some tags? Get AI suggestions" prompt when post has 150+ words but no tags or categories.
  • Per-post strategy override - Can be achieved through the ai_contextual_tagging_strategy filter, but an inline UI to do so is possible.
  • Taxonomy toggle settings - Allow users to choose which taxonomies can be auto-generated/suggested (default is just category and post_tag)
  • Confidence Threshold Slider - Allow users to choose the confidence threshold for suggestions

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: TylerB24890 <[email protected]>
Co-authored-by: jeffpaul <[email protected]>
Co-authored-by: estelaris <[email protected]>
Co-authored-by: karmatosed <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 59.48276% with 141 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.07%. Comparing base (32c5e2d) to head (e0e8f48).

Files with missing lines	Patch %	Lines
...bilities/Contextual_Tagging/Contextual_Tagging.php	70.83%	70 Missing ⚠️
...eriments/Contextual_Tagging/Contextual_Tagging.php	35.57%	67 Missing ⚠️
...bilities/Contextual_Tagging/system-instruction.php	0.00%	4 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #313      +/-   ##
=============================================
+ Coverage      57.85%   58.07%   +0.21%     
- Complexity       615      679      +64     
=============================================
  Files             46       49       +3     
  Lines           3165     3513     +348     
=============================================
+ Hits            1831     2040     +209     
- Misses          1334     1473     +139     

Flag	Coverage Δ
unit	58.07% <59.48%> (+0.21%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jeffpaul]

1. Should probably center align the button text for Categories and Tags to align to other buttons in the sidebar

2. Can probably remove the loading spinner as the suggest buttons alternate to a loading state already

3. Clicking the suggested term (specifically NOT the X close) appears to also close/remove the suggested term where I thought it might allow me to edit the suggested term (e.g. to pluralize or capitalize it). There doesn't seem to be an obvious way to save suggested terms as the post was already autosaved so there wasn't a Save option in the Gutenberg top toolbar and nothing else obvious in the sidebar panel. What's the expectation to save/apply suggestions?

4. It might be helpful to add a subheading of "Suggested Categories" and "Suggested Tags" just below the separator and just above the suggested terms to make it obvious what that sub-section sidebar panel is?

5. (Continuing from screenshot above) Instead of "Regenerate" perhaps "Suggest again"?

[TylerB24890]

@jeffpaul -- Thanks for the feedback! I've resolved 1, 2 & 4 -- see below regarding 3 & 5;

Clicking the suggested term (specifically NOT the X close) appears to also close/remove the suggested term where I thought it might allow me to edit the suggested term (e.g. to pluralize or capitalize it).

That might be a little challenging with this UI being in the sidebar. We might be better off going with a dedicated modal if we're to support that. Is that something we would want to do as opposed to the sidebar UI?

There doesn't seem to be an obvious way to save suggested terms as the post was already autosaved so there wasn't a Save option in the Gutenberg top toolbar and nothing else obvious in the sidebar panel. What's the expectation to save/apply suggestions?

Clicking the suggestion (outside of the 'X') will automatically create (if new) & assign the post to the term -- but I would have expected the Save button for the post to re-appear, since we changed the post state 🤔 I'll have to see what's going on there.

(Continuing from screenshot above) Instead of "Regenerate" perhaps "Suggest again"?

Personally I feel "Regenerate" makes more sense because the initial button says "Generate", but happy to update it if you prefer.

[dkotter]

Handful of suggestions / comments / concerns, happy to talk through any of those as needed.

Would also be great to get some E2E tests here prior to this getting merged.

[dkotter]

So not to derail things here but wondering if we can come up with a better name for this than Contextual Tagging. Just my opinion so feel free to ignore but I'm worried too many people will associate that Tagging part of this with WordPress tags, and just assume this only works for tags even though it also works for categories (and presumably other taxonomies in the future).

That said, I don't necessarily have a better option though here's some ideas:

• Content Classification
• Contextual Classification
• Term Suggestions
• Suggested Terms
• Smart Classification

[dkotter]

cc/ @jeffpaul if you have thoughts or feel strongly on Contextual Tagging

[jeffpaul]

I feel that "terms" isn't something that non-technical site owners will understand means "categories and tags", so let's go with Content Classification or Contextual Classification?

[dkotter]

It's not unusual for sites to have thousands or tens of thousands of terms (tags especially). In that scenario, seems we still send all of those to the LLM which won't scale well.

Wondering if we need to take an approach where we don't send any existing terms and instead we check if the returned terms already exist? So basically instead of asking the LLM to only return terms in our defined list, we allow the LLM to return anything and then we remove any that aren't existing terms. May impact the results we end up with but seems better than sending tens of thousands of tags for the LLM to process

[jeffpaul]

That might be a little challenging with this UI being in the sidebar. We might be better off going with a dedicated modal if we're to support that. Is that something we would want to do as opposed to the sidebar UI?

I'm fine holding off for that now as the current implementation is good.

Clicking the suggestion (outside of the 'X') will automatically create (if new) & assign the post to the term -- but I would have expected the Save button for the post to re-appear, since we changed the post state 🤔 I'll have to see what's going on there.

For me clicking the term and not the X also closes/removes the term, but applying/saving it would be better so perhaps something's not working there? I'm running WordPress 7.0-beta5-62069 and AI Provider for Google Version 1.0.3.

(Continuing from screenshot above) Instead of "Regenerate" perhaps "Suggest again"?

Personally I feel "Regenerate" makes more sense because the initial button says "Generate", but happy to update it if you prefer.

Except the initial buttons say Suggest Categories and Suggest Tags?

[TylerB24890]

What's left:

It's not unusual for sites to have thousands or tens of thousands of terms (tags especially). In that scenario, seems we still send all of those to the LLM which won't scale well.
Wondering if we need to take an approach where we don't send any existing terms and instead we check if the returned terms already exist? So basically instead of asking the LLM to only return terms in our defined list, we allow the LLM to return anything and then we remove any that aren't existing terms. May impact the results we end up with but seems better than sending tens of thousands of tags for the LLM to process

Need to circle around to this to avoid sending all terms to the agent.

Took care of the existing terms being sent to the LLM. We no longer restrict the number of suggestions it will return and instead filter suggestions in post-processing as needed. We do still pass the currently assigned terms, though. I feel this provides it additional context and prevents redundant suggestions.

@dkotter I rebased the develop branch into my branch and it looks like it introduced additional changes to my PR. Can you confirm if those are okay or do I need to re-open this on a new branch or something?

[dkotter]

I rebased the develop branch into my branch and it looks like it introduced additional changes to my PR. Can you confirm if those are okay or do I need to re-open this on a new branch or something?

I am seeing image generation/edit code in this PR that shouldn't be here. Wondering if you just need to merge the latest changes from develop here?

[jeffpaul]

Latest round of updates works well for me, will await @dkotter return for final review/merge