Search results

[Bug]: VLM (qwen3-235B) scoring fails: inference_endpoint/evaluation/scoring.py msgspec.DecodeError: JSON is malformed: invalid character (byte 145)

status: needs-triageNew issue, awaiting reviewNew issue, awaiting review

type: bugSomething isn't workingSomething isn't working

Status: Open.

#427 In mlcommons/endpoints;

· ai-nikolai opened on Jul 23, 2026

[Bug]: VLM Benchmark / Language Benchmark - OSError: [Errno 28] No space left on device

status: needs-triageNew issue, awaiting reviewNew issue, awaiting review

type: bugSomething isn't workingSomething isn't working

Status: Open.

#426 In mlcommons/endpoints;

· ai-nikolai opened on Jul 23, 2026

[Bug]: output config.yaml should respect --mode flag

status: needs-triageNew issue, awaiting reviewNew issue, awaiting review

type: bugSomething isn't workingSomething isn't working

Status: Open.

#420 In mlcommons/endpoints;

· arekay-nv opened on Jul 16, 2026

HTTP client: num_workers must be a multiple of the endpoint count for even load — auto-scale workers for multiple endpoints

Status: Open.

#419 In mlcommons/endpoints;

· viraatc opened on Jul 16, 2026

Connection pool: treat EADDRNOTAVAIL as backpressure so the full-budget auto ceiling is safe under max_throughput bursts

Status: Open.

#418 In mlcommons/endpoints;

· viraatc opened on Jul 16, 2026

Clarify BFCL / tool-call output token count (OSL is approximate for structured outputs)

Status: Open.

#410 In mlcommons/endpoints;

· arekay-nv opened on Jul 14, 2026

compliance: scope edge-agentic checker (rename to edge_agentic.py, required ruleset/model args)

Status: Open.

#397 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026

BFCL/openai: harden msgspec adapter tests + hoist imports + gc docstring for tool_choice

Status: Open.

#396 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026

BFCL/edge: make single-stream a first-class HTTPClientConfig option (drop accuracy-only override)

Status: Open.

#395 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026

BFCL test coverage: BFCLv4MultiTurnScorer.score() and bfcl_v4 generate() selection logic

Status: Open.

#394 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026

BFCL scoring/reporting & robustness cleanup (live_relevance, no-metric PASS, drain cap, plot guard, dead code)

Status: Open.

#393 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026

BFCL multi-turn: endpoint-controlled tool-call names reach bfcl-eval eval() (RCE on evaluator host)

Status: Open.

#391 In mlcommons/endpoints;

· Palanivelg opened on Jul 6, 2026