Skip to content

test(sdk): assert the request in generated service tests - #2114

Merged
abnegate merged 1 commit into
mainfrom
test/sdk-service-request-assertions
Oct 20, 2026
Merged

abnegate merged 1 commit into
mainfrom
test/sdk-service-request-assertions

Conversation

@abnegate

Copy link
Copy Markdown
Member

What

Generated PHP, Python, Dart and Flutter service unit tests mocked the client with wildcards and only checked the returned model. A method that sent the wrong verb or path, or dropped its body, still passed. Hansi raised this on the createQuery tests in appwrite/sdk-for-php#97, appwrite/sdk-for-python#166 and appwrite/sdk-for-dart#136, but the pattern covers every generated method.

Each generated method test now checks:

  • PHP: ->expects()->call('<VERB>', '<path>', Mockery::any(), Mockery::on(...)). The closure checks array_keys($params) with assertSame. MockeryPHPUnitIntegration makes expects() verify that the call happened exactly once.
  • Python: requests_mock matches only the method's verb. The test checks m.last_request against the unquoted URL path and the names of the parameters sent: the JSON body keys, the query keys for GET, or the multipart field names for uploads.
  • Dart/Flutter: MockClient.call and chunkedUpload now include path, params and the other named arguments in the Invocation. The test then runs verify(client.call(HttpMethod.x, path: '<path>', params: captureAnyNamed('params'), ...)) and checks params.keys. For webAuth methods, the test checks the decoded URL path.

The expected path is the spec path with the example arguments substituted. Config-filled path parameters, such as oauth2/{project_id}, take the test client's config value, which is the SDK's real behaviour.

Evidence

New Generation test fails before and passes after. GenerationTest::testServiceTestsAssertTheRequest has 9 cases: path and params for PHP, Python, Dart and Flutter, plus Python body.

  • Original templates (origin/main): Tests: 9, Assertions: 9, Failures: 9.
  • This PR: OK (9 tests, 9 assertions)

Mutation check. I mutated Foo::put() in each generated fixture SDK and ran only that method's generated test:

SDK mutation of Foo.put() original template this PR
php PUT sent as POST passes (missed) fails (caught)
php path /mock/tests/bar passes (missed) fails (caught)
php body params dropped passes (missed) fails (caught)
python PUT sent as POST passes (missed) fails (caught)
python path /mock/tests/bar passes (missed) fails (caught)
python body params dropped passes (missed) fails (caught)
dart PUT sent as POST fails (caught) fails (caught)
dart path /mock/tests/bar passes (missed) fails (caught)
dart body params dropped passes (missed) fails (caught)

Dart already caught a wrong verb because its stub was verb-specific. It missed wrong paths and dropped bodies.

Generated tests pass on the real Appwrite 2.4.x spec (server, generated locally):

  • PHP: OK (945 tests, 2765 assertions)
  • Python: Ran 884 tests ... OK, and black --check passes
  • Dart: +804: All tests passed!

On the fixture, PHP, Python and Dart each pass 56 tests. The fixture includes the grant/special&id path argument and the multipart uploads.

Known limits

Python's query string and multipart encoding drop the empty []/{} arguments the tests pass. For GET and upload methods, the expected names therefore leave out required array and object parameters. JSON bodies still list them.

🤖 Generated with Claude Code

Generated PHP, Python, Dart and Flutter service tests mocked the client
with wildcards and only checked the returned model, so a method calling
the wrong verb or path, or dropping its body, still passed. Each test now
pins the verb, the path with the example arguments substituted, and the
parameter names the method sends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hansi-codes

hansi-codes Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

🟢 Tier S · Ready to merge

Generated PHP, Python, Dart, and Flutter service tests now assert the request verb, substituted path, and parameter names in addition to the response. Generation tests check representative request assertions for each SDK family, including Python body parameters.

Verdict New comments Fixed Still open
✅ Approved 0 0 0
📂 Walkthrough · 5
File Change
templates/dart/test/services/service_test.dart.twig Captures client request arguments in the mock and asserts path and parameter keys.
templates/flutter/test/services/service_test.dart.twig Applies Dart request-assertion coverage to Flutter service tests.
templates/php/tests/Services/ServiceTest.php.twig Uses exact client call expectations and asserts generated parameter keys.
templates/python/test/services/test_service.py.twig Matches mocked requests by verb and asserts URL paths and sent parameter names.
tests/generation/GenerationTest.php Adds generated-output checks for representative request assertions across the SDKs.

Reviewed e522a2d · Details · Comment @hansi-codes review to re-run, or mention @hansi-codes with a question.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Tier S · Looks good to merge. Summary

@abnegate
abnegate merged commit 5639cfe into main Oct 20, 2026
60 checks passed
@abnegate
abnegate deleted the test/sdk-service-request-assertions branch October 20, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant