feat(benchmarks): Added dummy benchmark for testing

[christian-pinto]

No description provided.

[DRL-NextGen]

Checks Summary

Last run: 2026-06-24T15:29:19.017Z

Mend Unified Agent vulnerability scan found 15 vulnerabilities:

Severity	Identifier	Package	Details	Fix
❗ Critical	CVE-2025-69872	diskcache-5.6.3-py3-none-any.whl	DiskCache (python-diskcache) through 5.6.3 uses Python pickle for serialization by default. An attac... DiskCache (python-diskcache) through 5.6.3 uses Python pickle for serialization by default. An attacker with write access to the cache directory can achieve arbitrary code execution when a victim application reads from the cache.	Not Available
❗ Critical	CVE-2026-48746	vllm-0.21.0-1-cp38-abi3-manylinux_2_24_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). From 0.3.0 until 0.22.0, a... vLLM is an inference and serving engine for large language models (LLMs). From 0.3.0 until 0.22.0, a vulnerability in ASGI web servers and starlette's trust on those web servers enables an authentication bypass of the OpenAI API AuthenticationMiddleware. It allows to use the API without providing the configured VLLM_API_KEY or --api-key. This vulnerability is fixed in 0.22.0.	Upgrade to version https://github.com/vllm-project/vllm.git - v0.22.0
🔺 High	CVE-2026-41523	vllm-0.21.0-1-cp38-abi3-manylinux_2_24_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.22.0, an assert... vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.22.0, an assert-based security check in vLLM's activation function loading allows any unauthenticated attacker to achieve arbitrary code execution on the server by publishing a malicious HuggingFace model, when vLLM runs in Python optimized mode (python -O or PYTHONOPTIMIZE=1). This vulnerability is fixed in 0.22.0.	Upgrade to version https://github.com/vllm-project/vllm.git - v0.22.0
🔺 High	CVE-2026-5241	transformers-4.57.6-py3-none-any.whl	A vulnerability in the LightGlue model loading path of huggingface/transformers version 5.2.0 allows... A vulnerability in the LightGlue model loading path of huggingface/transformers version 5.2.0 allows an attacker-controlled model repository to execute arbitrary code during model initialization. The issue arises because the "trust_remote_code" parameter, intended to prevent remote code execution, is overridden by untrusted serialized configuration data in a nested code path. Specifically, when loading a LightGlue model using "AutoModel.from_pretrained()" with "trust_remote_code=False", the "LightGlueConfig" reads the "trust_remote_code" value from the untrusted "config.json" file and propagates it into nested "AutoConfig.from_pretrained()" calls. This results in the execution of attacker-provided Python modules, even when the victim explicitly disables remote code execution. The vulnerability poses a high risk for environments such as API inference servers, research notebooks, CI/CD pipelines, and model evaluation workers, potentially leading to credential theft, lateral movement, or persistence/backdoor deployment.	Upgrade to version transformers - 5.5.0,transformers - 5.5.0,https://github.com/huggingface/transformers.git - v5.5.0
🔺 High	CVE-2026-4372	transformers-4.57.6-py3-none-any.whl	A critical remote code execution vulnerability exists in all versions of the HuggingFace transformer... A critical remote code execution vulnerability exists in all versions of the HuggingFace transformers library prior to version 5.3.0. The vulnerability allows an attacker to craft a malicious "config.json" file containing the "_attn_implementation_internal" field set to an attacker-controlled HuggingFace Hub repository ID. When a victim loads this model using the standard "AutoModelForCausalLM.from_pretrained()" API, the library downloads and executes arbitrary Python code from the attacker's repository with the victim's full OS privileges. This issue arises due to unfiltered deserialization of configuration attributes, insufficient sanitization of internal fields, and unsandboxed execution of downloaded kernels. The vulnerability bypasses the "trust_remote_code" security mechanism, is invisible to the victim, and exploits the standard documented usage pattern, making it particularly severe. Users are advised to upgrade to version 5.3.0 or later to mitigate this issue.	Upgrade to version transformers - 5.3.0,transformers - 5.3.0,https://github.com/huggingface/transformers.git - v5.3.0
🔺 High	CVE-2025-14920	transformers-4.57.6-py3-none-any.whl	Hugging Face Transformers Perceiver Model Deserialization of Untrusted Data Remote Code Execution Vu... Hugging Face Transformers Perceiver Model Deserialization of Untrusted Data Remote Code Execution Vulnerability. This vulnerability allows remote attackers to execute arbitrary code on affected installations of Hugging Face Transformers. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file. The specific flaw exists within the parsing of model files. The issue results from the lack of proper validation of user-supplied data, which can result in deserialization of untrusted data. An attacker can leverage this vulnerability to execute code in the context of the current user. Was ZDI-CAN-25423.	Not Available
🔷 Medium	CVE-2026-53923	vllm-0.23.0-2-cp38-abi3-manylinux_2_28_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). From 0.5.5 until 0.23.1rc0... vLLM is an inference and serving engine for large language models (LLMs). From 0.5.5 until 0.23.1rc0, integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/gguf_kernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure. This vulnerability is fixed in 0.23.1rc0.	Not Available
🔷 Medium	CVE-2026-54233	vllm-0.23.0-2-cp38-abi3-manylinux_2_28_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, vLLM's... vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, vLLM's /v1/audio/transcriptions endpoint limits compressed upload size but not decoded PCM output. A 25MB OPUS file expands to ~14.9GB of float32 PCM at decode time. This vulnerability is fixed in 0.23.1rc0.	Not Available
🔷 Medium	CVE-2026-12491	vllm-0.23.0-2-cp38-abi3-manylinux_2_28_x86_64.whl	A flaw was found in vLLM, an open-source library for large language model inference. This vulnerabil... A flaw was found in vLLM, an open-source library for large language model inference. This vulnerability arises from improper handling of image metadata, specifically EXIF orientation and PNG transparency (tRNS) data, during image processing. When images are converted to RGB, transparency information may be implicitly discarded or remapped, leading to unexpected rendering of transparent pixels and distortion of input content. This can result in the model misinterpreting image content, potentially affecting the integrity of processed data.	Not Available
🔷 Medium	CVE-2026-54236	vllm-0.23.0-2-cp38-abi3-manylinux_2_28_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, the fi... vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, the fix for CVE-2026-22778, which introduced a sanitize_message helper that strips object-repr memory addresses from error messages before they reach the client, is incomplete: several response paths echo str(exc) directly to clients without calling sanitize_message. The unsanitized sites include the Anthropic API router in vllm/entrypoints/anthropic/api_router.py (the POST /v1/messages and POST /v1/messages/count_tokens handlers), the Server-Sent Events streaming converter in vllm/entrypoints/anthropic/serving.py, and the realtime speech-to-text WebSocket in vllm/entrypoints/speech_to_text/realtime/connection.py. These paths catch the exception inside the route coroutine and construct the JSONResponse themselves, bypassing the sanitizing global FastAPI exception handler, and WebSocket frames do not traverse that handler chain at all. Using the same primitive as the parent issue, an unauthenticated attacker can send malformed image bytes through the Anthropic Messages API image content parts so that PIL.Image.open raises an UnidentifiedImageError whose message contains the BytesIO object repr, leaking the heap memory address verbatim in the error.message field of the response body. This vulnerability is fixed in 0.23.1rc0.	Not Available
🔷 Medium	CVE-2026-54235	vllm-0.23.0-2-cp38-abi3-manylinux_2_28_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, ll tem... vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.23.1rc0, ll temperature validation gates use comparison operators (<, >), which silently evaluate to False for NaN and for positive Infinity in Python's IEEE 754 float semantics. Both values pass every guard and propagate to GPU sampling kernels, where they produce undefined behavior or CUDA errors that can crash the inference worker. This vulnerability is fixed in 0.23.1rc0.	Not Available
🔷 Medium	CVE-2026-47155	vllm-0.21.0-1-cp38-abi3-manylinux_2_24_x86_64.whl	vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.22.0, vLLM's re... vLLM is an inference and serving engine for large language models (LLMs). Prior to 0.22.0, vLLM's revision pinning controls do not consistently apply to all artifacts loaded for a model. A deployment that supplies --revision or --code-revision can still load dynamic code, GGUF files, image processors, retrieval side weights, or same-repository subfolder weights/config from an unpinned/default revision. This is a supply-chain integrity issue for pinned vLLM deployments. Operators can believe they are serving a reviewed model revision while vLLM resolves behavior-affecting nested or sibling artifacts outside that reviewed revision. This vulnerability is fixed in 0.22.0.	Upgrade to version https://github.com/vllm-project/vllm.git - v0.22.0
🔷 Medium	CVE-2025-3000	torch-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl	A vulnerability classified as critical has been found in PyTorch 2.6.0. This affects the function to... A vulnerability classified as critical has been found in PyTorch 2.6.0. This affects the function torch.jit.script. The manipulation leads to memory corruption. It is possible to launch the attack on the local host. The exploit has been disclosed to the public and may be used.	Not Available
🔷 Medium	CVE-2026-1839	transformers-4.57.6-py3-none-any.whl	A vulnerability in the HuggingFace Transformers library, specifically in the "Trainer" class, allows... A vulnerability in the HuggingFace Transformers library, specifically in the "Trainer" class, allows for arbitrary code execution. The "_load_rng_state()" method in "src/transformers/trainer.py" at line 3059 calls "torch.load()" without the "weights_only=True" parameter. This issue affects all versions of the library supporting "torch>=2.2" when used with PyTorch versions below 2.6, as the "safe_globals()" context manager provides no protection in these versions. An attacker can exploit this vulnerability by supplying a malicious checkpoint file, such as "rng_state.pth", which can execute arbitrary code when loaded. The issue is resolved in version v5.0.0rc3.	Upgrade to version https://github.com/huggingface/transformers.git - v5.0.0rc3,transformers - 5.0.0rc3
🔸 Low	CVE-2025-63396	torch-2.11.0-cp312-cp312-manylinux_2_28_x86_64.whl	An issue was discovered in PyTorch v2.5 and v2.7.1. Omission of profiler.stop() can cause torch.prof... An issue was discovered in PyTorch v2.5 and v2.7.1. Omission of profiler.stop() can cause torch.profiler.profile (PythonTracer) to crash or hang during finalization, leading to a Denial of Service (DoS).	Not Available

[christian-pinto]

test comment

[christian-pinto]

test comment 2

[christian-pinto]

meme

[florenzi002]

/run_benchmarks

[christian-pinto]

/run_benchmarks

[christian-pinto]

/run_benchmarks

[christian-pinto]

test

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks with more words

[DRL-NextGen]

Benchmarks submission summary

Status	Space ID	Ray Job ID	Instance
Successful	space-7aacee-default	raysubmit_9Af9wYXA2WtmD2d7	packages/terratorch/benchmark_instances/test_benchmark_base
Successful	space-dd8f7b-default	raysubmit_dSJiwZ6T3m9nAEJx	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
Successful	space-1bad39-default	raysubmit_8HbwyVTzUTWsegHC	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[christian-pinto]

add even more words

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

🔁 Benchmark Run (2026-06-19 09:05 UTC)

	Space ID	Ray Job ID	Instance
✅	space-f00f2e-default	raysubmit_Kwgbr68X1i32thX5	packages/terratorch/benchmark_instances/test_benchmark_base
✅	space-b917cd-default	raysubmit_mN8UfNc46FtQhkNJ	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	space-cdf994-default	raysubmit_Fj8SchEs6wedVqyk	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

🔁 Benchmark Run (2026-06-19 09:15 UTC)

Status	Space ID	Ray Job ID	Instance
✅	space-f5cbfe-default	raysubmit_JnfMqfAW6fy6KxBW	packages/terratorch/benchmark_instances/test_benchmark_base
✅	space-e36261-default	raysubmit_6wyWSXdk8X28QqBL	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	space-319f42-default	raysubmit_zEZjUuHvW9gWNnGJ	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

✅ Benchmark Run (2026-06-19 09:41 UTC)

Status	Space ID	Ray Job ID	Instance
✅	space-53b790-default	raysubmit_sB7xgxu5b7Kxb3Cm	packages/terratorch/benchmark_instances/test_benchmark_base
✅	space-885c49-default	raysubmit_WRnph8qRTX51gPnN	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	space-25fab0-default	raysubmit_VrtaWPALb4uffzrZ	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

Pipeline run

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

✅ Benchmark Run (2026-06-19 09:59 UTC)

Status	Space ID	Ray Job ID	Instance
✅	space-52206a-default	raysubmit_LH6nLQKumfnM7Ag2	packages/terratorch/benchmark_instances/test_benchmark_base
✅	space-99e4ab-default	raysubmit_GBn241asMEpjTArc	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	space-c0a627-default	raysubmit_8fzn2eK89KGju3t6	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

✅ Benchmark Run (2026-06-19 10:16 UTC)

Status	Space ID	Ray Job ID	Instance
✅	space-43756d-default	raysubmit_uHdDMyBmBMJzbSKJ	packages/terratorch/benchmark_instances/test_benchmark_base
✅	space-658dc1-default	raysubmit_pbZ1YCnbEYnWegHj	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	space-8a35e2-default	raysubmit_iuLvqz4eXXPVfk3c	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here

[christian-pinto]

/run_benchmarks

[DRL-NextGen]

✅ Benchmark Run (2026-06-19 11:10 UTC)

Status	Duration	Ray Job ID	Instance
✅	31s	raysubmit_SbcFqW2zMK4Tk27q	packages/terratorch/benchmark_instances/test_benchmark_base
✅	31s	raysubmit_BNQAJCs8finYNHrB	packages/terratorch/models/prithvi/benchmark_instances/test_benchmark
✅	31s	raysubmit_NBX9hX1HpZVjGpgT	packages/terratorch/models/terramind_base_fire/benchmark_instances/test_benchmark

[DRL-NextGen]

@christian-pinto @IBM/algorithm-nexus-maintainers All benchmark runs have completed, see summary here