Skip to content

feat: add authztest package for authorization test helpers with failure context - #1758

Closed
tlyyxjz wants to merge 1 commit into
apache:masterfrom
tlyyxjz:authztest
Closed

feat: add authztest package for authorization test helpers with failure context#1758
tlyyxjz wants to merge 1 commit into
apache:masterfrom
tlyyxjz:authztest

Conversation

@tlyyxjz

@tlyyxjz tlyyxjz commented Sep 5, 2026

Copy link
Copy Markdown

Closes #1624 — implements the authorization test helpers proposed there.

Note: @angelinazzz1809 expressed interest in this issue in June and I want to be explicit that this PR is not meant to step on that work — if they have a draft, I'm happy to fold theirs in or withdraw this one. The thread had been quiet for three months, so I built a working reference implementation (currently published as the external module tlyyxjz/authztest) to give the proposal something concrete to review.

What this adds

A new authztest sub-package with a deliberately small API surface, per the "thin wrappers around Enforce" design note:

authztest.AssertAllow(t, e, "alice", "data1", "read")
authztest.AssertDeny(t, e, "bob", "data1", "write")

The value is in failure context. When an assertion fails, instead of a bare false, you get a position-by-position near-miss attribution of the closest policies:

authztest: expected ALLOW, got DENY for (alice, data2, read)
no policy matched; closest candidates:
  p, admin, data2, write
    sub: ok ('alice' inherits 'admin' via g)
    obj: ok ('data2')
    act: 'read' does not match 'write' (not a wildcard)

There are also two exported escape hatches for people who want the diagnosis without testing.TB coupling: Diagnose(e, rvals...) (human-readable string, shown above) and Explain(e, rvals...) (structured []NearMiss).

Design decisions

  • Structural attribution, not matcher re-evaluation. The analysis walks policy lines and g (role) rules directly. A subject matched through role inheritance is credited, not falsely reported as a "sub mismatch" — the blame lands on the position that actually failed.
  • Domain-aware. For g = _, _, _ models, role links are checked in the policy line's own domain, so multi-tenant failures are unambiguous.
  • Wildcard-aware. The common p.obj == "*" idiom is recognized; arity mismatches between policy lines and the request definition are surfaced.
  • Enforce errors are errors. Wrong-arity requests report the error instead of being misattributed as denies.
  • No new dependencies, no core changes. The package only reads the model/policy/role-manager through existing public APIs; nothing in casbin core imports it.
  • T is a minimal interface (Helper + Errorf), so *testing.T, *testing.B, *testing.F and custom fakes all work.

Targeting the v3 line since that's where master development happens; the API has no v3-specific surface and can be backported to v2 if maintainers prefer. If the intended home for this is a separate repository under the casbin org rather than a sub-package (the open question from the issue thread), the code transfers as-is.

Test evidence

  • 10 unit tests + 1 runnable ExampleDiagnose in the package, covering: pass/fail paths for both assertions, RBAC role-inheritance attribution, domain-scoped attribution, wildcard idiom, near-miss ranking, arity mismatch, and Enforce error reporting.
  • go test ./... passes for the whole module; go vet and gofmt clean.

…re context

Implements the test helpers proposed in apache#1624 as a new sub-package:
AssertAllow/AssertDeny thin wrappers around Enforce, plus
Diagnose/Explain near-miss attribution that reports, position by
position, why a request was denied (or which policy line allowed an
unexpected allow). Role inheritance via g rules is credited rather
than falsely reported as a subject mismatch, domain-aware models
check links in the policy line's own domain, and wildcard idiom
(p.obj == '*') is recognized. Enforce errors are surfaced as errors,
not misattributed denies.
@Santoshkumarpuppala

Copy link
Copy Markdown

Ran AssertDeny against the repo's own keymatch2 fixtures at ee5784e, and the diagnostic contradicts itself on line 2:

authztest: expected DENY, got ALLOW for (alice, /alice_data/123, GET)
no policy matched; closest candidates:
  p, alice, /alice_data/:resource, GET
    sub: ok ('alice')
    act: ok ('GET')
    obj: '/alice_data/123' does not match '/alice_data/:resource' (not a wildcard)

The header says got ALLOW; the next line says no policy matched. Both come from the same call. And the per-position note blames obj — which is the position that actually did match. EnforceEx on the identical request returns [alice /alice_data/:resource GET]: exactly the rule the diagnostic says did not match, matched at exactly the position it says failed.

Why the two disagree. AssertDeny already has the real verdict at :81 (ok, err := e.Enforce(rvals...)) and drops it at :89 (d := Diagnose(e, rvals...)). Diagnose then recomputes the verdict structurally at :123:

allowed := misses[0].FullyMatched()

FullyMatched() is len(n.Failed) == 0 (:105), and the structural pass can't see keyMatch2, so it records a failure at obj and picks the "no policy matched" branch — while Enforce said allow. The header is authoritative in tone and computed from a weaker mechanism than the decision it describes.

The documented safeguard for this isn't implemented. :110-115 says matchers using built-in functions "may legitimately match in ways the structural analysis cannot see; the output is labelled accordingly." keyMatch, regexMatch and "structural" appear nowhere in the package outside that comment — the file has 14 output strings and none carries such a label. So the one thing that would tell a reader not to trust the header is the thing that's missing, which is why the contradiction reads as authoritative rather than as a caveat.

The fix I'd suggest keeps the signature. Diagnose already takes the IEnforcer, so it can ask rather than infer:

allowed, explain, err := e.EnforceEx(rvals...)

EnforceEx is on the IEnforcer interface (enforcer_interface.go:63) and returns the genuinely matched rule (enforcer.go:912-914). Two things fall out: the header always agrees with the enforcer because it is the enforcer's answer, and on the allow path you can print the real matched rule instead of near-misses — which is the more useful output anyway when someone is debugging an unexpected ALLOW.

I'd avoid threading the verdict in as a parameter (Diagnose(e, allowed bool, rvals...)). That lets a caller pass a verdict that disagrees with the enforcer, which is the same failure this report is about, just relocated to the call site. Asking the enforcer inside Diagnose can't drift.

Test. examples/keymatch2_model.conf and examples/keymatch2_policy.csv already ship, so it's a fixture load plus one assertion, and it would be the package's first test to reach the documented limitation. Worth noting go test ./authztest/ is green at ee5784e with this present — no current test exercises a matcher function, so nothing fails today.

Method note: measured by executing the package at ee5784e (output above is verbatim, EnforceEx result likewise). I haven't tried to judge whether the near-miss ranking is right in general — only that the header and the enforcer can disagree, and that here they do.

@tlyyxjz tlyyxjz closed this Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature] Add lightweight authorization test helpers

2 participants