Run a Swift Testing file: freestanding macros (#expect/#require) and host-declared attributes (@Test/@Suite) #14

Following #13 — leading-dot arguments and catchable bridge errors both work, and the XCUITest-shaped surface now drives real Mac apps and web pages. The remaining goal is the modern half of it: run a Swift Testing file as a script, unmodified.

Today neither of its two load-bearing pieces parses.

import Testing

@Test func loginWorks() async throws {          // error: unsupported attribute '@Test'
    let app = XCUIApplication(bundleIdentifier: "com.example.App")
    app.launch()
    #expect(app.buttons["Sign In"].exists)      // error: unsupported expression MacroExpansionExprSyntax
    let row = try #require(app.cells.firstMatch)
    row.tap()
}

1. Freestanding macro expressions

MacroExpansionExprSyntax is unsupported, so #expect / #require cannot be written at all.

They cannot be modelled as ordinary functions either, and the reason is the whole point of them being macros: #expect(a == b) reports the source text of the expression and the operand values — "Expectation failed: a == b (1 == 2)". A plain function receives false and can say nothing useful.

Suggested shape. Dispatch #name(args) to a registered handler, passing evaluated arguments and the source text of each argument expression:

case macro((_ args: [Value], _ sourceText: [String]) async throws -> Value)
// keyed "macro #expect" / "macro #require"

sourceText is what the handler needs to build Swift Testing's message; args is what it needs for the verdict. A richer version could hand over the decomposed sub-expressions and their values (which is how the real macro reports both sides of a binary operator), but source text alone already covers the common case, and an unregistered #name should stay a hard error.

2. Attached attributes the host declares

@Test and @Suite are refused by the attribute preflight, which is right by default — silently ignoring an attribute would diverge from stock Swift. But a host that knows what an attribute means should be able to say so.

Suggested shape. Let a host register attribute names it understands, and expose the declarations carrying them:

interpreter.registerAttribute("Test")       // no longer a hard error
interpreter.registerAttribute("Suite")
interpreter.declarations(withAttribute: "Test")   // -> name, arguments, invocable

That is what turns "a script" into "a test file the runner enumerates and runs". Without it a pasted test file cannot even load, however good the expectation support is.

Argument-carrying forms matter too, since they are how real suites are written: @Test("display name"), @Test(.disabled("flaky")), @Test(arguments: [1, 2, 3]). Exposing the attribute's arguments as Values lets the host decide what they mean — the same "defer to the callee" principle #13 established for leading-dot members.

Why this shape suits scripting specifically

Swift Testing's failure model is a better fit for UI automation than XCTest's, and it is worth preserving exactly:

• #expect records an issue and continues — one run surfaces every problem, with a screenshot per failure, instead of dying at the first.
• #require records and throws — for the preconditions where continuing is pointless.

So a host needs both an issue sink and a throwing path, which the handler shape above supports: #expect returns .void after recording, #require throws. A script that ran to completion having recorded issues is a failed run — the runner reports them together at the end.

Reference: https://github.com/swiftlang/swift-testing, Sources/Testing/Expectations/Expectation+Macro.swift and Sources/Testing/Test+Macro.swift.

Not blocking

The XCUI surface works today with function-shaped assertions; this is about being able to paste a real Swift Testing file in and have it run, which is the actual promise.