Skip to content

[#281] Add generic metadata to function declarations - #282

Open
superdav42 wants to merge 5 commits into
trunkfrom
feature/gh281-deferred-tool-loading
Open

[#281] Add generic metadata to function declarations#282
superdav42 wants to merge 5 commits into
trunkfrom
feature/gh281-deferred-tool-loading

Conversation

@superdav42

@superdav42 superdav42 commented Aug 26, 2026

Copy link
Copy Markdown
Member

Summary

Revised following stakeholder feedback to add only generic function metadata, while tool-search behavior is explored in provider plugins.

$function = new FunctionDeclaration(
    'get_weather',
    'Gets the weather',
    null,
    ['deferredLoading' => true]
);

$metadata = $function->getMetadata();

The SDK stores annotations but does not interpret their names or values. The motivating provider experiment can combine function metadata with existing ModelConfig::setCustomOptions(['deferredLoading' => true]); no feature-specific core option is introduced.

Scope

One production file: src/Tools/DTO/FunctionDeclaration.php.

  • Optional fourth array $metadata = [] constructor argument.
  • getMetadata() accessor and optional metadata array/JSON representation.
  • Empty metadata is omitted, preserving the serialized shape of existing declarations; missing metadata restores to an empty array.
  • Existing constructor calls and PHP 7.4 compatibility are retained.

The previous proposal's ProviderData class, message type, serialization changes, and tests have been removed. There are no changes to messages, PromptBuilder, ModelConfig, model requirements, or enums. There is no usingToolSearch() method or dedicated deferred-loading property.

Provider experiment and limitations

Companion implementation: WordPress/ai-provider-for-openai #50.

The provider owns annotation semantics, automatic activation above 10 functions, model eligibility, and request-level overrides. It translates recognized annotations into native wire fields rather than blindly forwarding metadata. Automatic/request-level behavior also works with released SDK 1.3.1; only per-function annotations need this extension.

The experiment uses OpenAI-managed state via the existing previous_response_id custom option, with only new input/function results on subsequent turns. Stateless replay of search output is explicitly outside this proposal. This avoids adding any new message representation before the feature has demonstrated value.

For #281. This is an extension point, not a claim to complete provider adoption or establish tool-search utility.

Reference and review map

  • src/Tools/DTO/FunctionDeclaration.php: follows its existing constructor/accessor/schema/array transformation pattern.
  • tests/unit/Tools/DTO/FunctionDeclarationTest.php: legacy serialization shape, arbitrary nested annotations, boolean false/null preservation, array/JSON round trips, model-config round trips, and clone/getter independence for array metadata.
  • docs/FUNCTION_METADATA.md: generic contract, naming guidance, provider interpretation, and experiment boundaries.
  • Vercel AI SDK OpenAI provider uses per-tool providerOptions for comparable provider-owned settings. Its documented deferred-loading API is explicit rather than count-triggered.

Verification

  • composer test:unit: 1,202 tests, 4,366 assertions passed.
  • composer lint: PHPCS and PHPStan passed.
  • Verified no diff against trunk remains under src/Messages/ or tests/unit/Messages/.
  • Changes appended to the existing PR branch without rewriting history.
  • No live quality/cost benchmark or paid API call was performed. Feature usefulness and the automatic threshold remain questions for the provider experiment.

aidevops.sh v3.32.317 plugin for OpenCode v1.18.30 with gpt-6-astra spent 1h 10m and 556,600 tokens on this with the user in an interactive session.

@superdav42 superdav42 added the origin:interactive Created by interactive user session label Aug 26, 2026
@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.58%. Comparing base (20a1a6d) to head (dec7664).

Additional details and impacted files
@@             Coverage Diff              @@
##              trunk     #282      +/-   ##
============================================
+ Coverage     86.54%   86.58%   +0.03%     
- Complexity     1381     1383       +2     
============================================
  Files            69       69              
  Lines          4438     4449      +11     
============================================
+ Hits           3841     3852      +11     
  Misses          597      597              
Flag Coverage Δ
unit 86.58% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@superdav42
superdav42 marked this pull request as ready for review September 2, 2026 23:05
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: superdav42 <superdav42@git.wordpress.org>
Co-authored-by: JasonTheAdams <jason_the_adams@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@superdav42 superdav42 changed the title [#281] Add provider-agnostic deferred tool loading [#281] Preserve provider-native conversation data Sep 9, 2026
@superdav42 superdav42 changed the title [#281] Preserve provider-native conversation data [#281] Add generic metadata to function declarations Sep 9, 2026
@superdav42

Copy link
Copy Markdown
Member Author

Updated this PR to follow the function-annotations direction: the only production change is now generic metadata on FunctionDeclaration (optional fourth argument, getMetadata(), and serialization). Empty metadata preserves the old serialized shape.

All ProviderData/message-type changes are removed, along with the earlier tool-search builder method, dedicated deferred-loading property, and model requirements. Tests follow the existing DTO patterns in tests/unit/Tools/DTO/FunctionDeclarationTest.php; rationale is in docs/FUNCTION_METADATA.md.

The OpenAI provider experiment (#50) now consumes deferredLoading annotations and existing custom options, retains automatic threshold behavior, and uses previous_response_id for server-managed continuation. Stateless discovery replay is deliberately deferred rather than expanding this core proposal.

Verification: client 1,202 tests / 4,366 assertions and lint passed; provider 93 tests / 604 assertions (one existing skip) and lint passed. No live usefulness/cost benchmark has been claimed.


aidevops.sh v3.32.317 plugin for OpenCode v1.18.30 with gpt-6-astra spent 1h 10m and 559,833 tokens on this with the user in an interactive session.

@JasonTheAdams JasonTheAdams left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @superdav42 ! A few small suggstions.

Comment thread docs/FUNCTION_METADATA.md

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update ARCHITECTURE.md instead of adding this.

* description: string,
* parameters?: array<string, mixed>
* parameters?: array<string, mixed>,
* metadata?: array<string, mixed>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the word annotations and like that it's consistent with the AI SDK. Let's go with that as our word.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The descriptions throughout this are inconsistent. Let's clean that up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

origin:interactive Created by interactive user session

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants