Idea: helper for exporting serverless job results as a portable EvalPort result set?

[adhabnr-ux]

Hi! I maintain EvalPort, an open spec (JSON Schema + Python/TS SDKs) for portable LLM eval datasets — test suites, test cases, and result sets that can move between eval tools instead of each one inventing its own file format.

The serverless worker pattern here (job["input"] in, handler output out, run via endpoint.run_sync() or checked with run_request.output()/.status()) maps pretty naturally onto an eval "test case in, actual output out, pass/fail" record. Something like:

from evalport import ResultSet, TestResult

def job_results_to_evalport(jobs: list[dict]) -> ResultSet:
    return ResultSet(results=[
        TestResult(
            test_case_id=job["id"],
            input=job["input"],
            actual_output=job.get("output"),
            passed=job.get("status") == "COMPLETED",
            error=job.get("error"),
        )
        for job in jobs
    ])

This seems like it could be a natural fit for the local test worker flow (python my_worker.py --rp_serve_api) — someone iterating on a handler could run a fixed batch of inputs through it and save the results as a portable regression-test fixture, or compare outputs across endpoint/model versions using runpod.Endpoint(...).run_sync().

Spec, for context: https://github.com/adhabnr-ux/evalport/blob/main/SPEC.md

This is an early interest-check, not a PR — happy to sketch it out further (maybe as a docs/ example) if it's useful, and no worries at all if it isn't a good fit here. Thanks for reading!