Issues

• go install fails: Symflower binary download returns 404 when using latest clone

  bug
  symflower/eval-dev-quality#461 · Filter by author DaisyPo opened on Sep 13, 2025
  3

• Add Qwen3, GLM4.5, and DeepSeek series as an update

  symflower/eval-dev-quality#460 · Filter by author BradKML opened on Aug 30, 2025

• Models can gain additional points for Rust transpilation

  symflower/eval-dev-quality#459 · Filter by author bauersimon opened on May 21, 2025

• Rethink language scoring of "transpilation"

  enhancement
  Feature

  symflower/eval-dev-quality#446 · Filter by author zimmski opened on Apr 8, 2025 · v1.1.0
  

• Assessments should have a type based on their data

  bug
  Bug

  symflower/eval-dev-quality#436 · Filter by author zimmski opened on Mar 24, 2025 · v1.1.0
  1

  

• Do not copy assessments of base tasks

  bug
  Bug

  symflower/eval-dev-quality#435 · Filter by author ahumenberger opened on Mar 21, 2025 · v1.1.0
  1

  

• Make Symflower template generation a no-disk operation

  enhancement
  symflower/eval-dev-quality#434 · Filter by author zimmski opened on Mar 19, 2025 · v1.1.0
  

• Rework the JUnit4->JUnit5 migration LLM prompt

  bug
  Bug

  symflower/eval-dev-quality#422 · Filter by author zimmski opened on Feb 21, 2025 · v1.1.0
  

• Smart unit test templates for Spring services don't have Spring test annotation, misleading models to write non-Spring tests

  bug
  Bug

  symflower/eval-dev-quality#420 · Filter by author bauersimon opened on Feb 14, 2025 · v1.1.0
  

• Migration task and new attribute CLI arguments are not explained in README

  documentation
  Task

  symflower/eval-dev-quality#414 · Filter by author bauersimon opened on Feb 10, 2025 · v1.1.0
  1

  

• Do "symflower-fix" task for "code-repair"

  enhancement
  Feature

  symflower/eval-dev-quality#411 · Filter by author zimmski opened on Feb 4, 2025 · v1.1.0
  

• Should we remove *-symflower-template tasks? e.g. write-tests-symflower-template

  enhancement
  Feature

  symflower/eval-dev-quality#410 · Filter by author zimmski opened on Feb 3, 2025 · v1.1.0
  

• Query model list of "OpenAI API" provider

  enhancement
  Feature

  symflower/eval-dev-quality#409 · Filter by author zimmski opened on Feb 3, 2025 · v1.1.0
  

• Rework how we allow to set (query) attributes of models

  enhancement
  Feature

  symflower/eval-dev-quality#407 · Filter by author zimmski opened on Feb 1, 2025 · v1.1.0
  

• Add an indicator in the results directory that an evaluation has completed

  enhancement
  Feature

  symflower/eval-dev-quality#406 · Filter by author zimmski opened on Feb 1, 2025 · v1.1.0
  

• Redo options tokens and urls as it looks like we cannot separate multiple values with ","

  bug
  Bug

  symflower/eval-dev-quality#405 · Filter by author zimmski opened on Feb 1, 2025 · v1.1.0
  

• Extend the evalatuion.csv with the current maximum values, and keep them updated in the repositories

  enhancement
  Feature

  symflower/eval-dev-quality#404 · Filter by author zimmski opened on Jan 30, 2025 · v1.1.0
  

• Update README to align with latest changes

  enhancement
  Feature

  symflower/eval-dev-quality#398 · Filter by author zimmski opened on Jan 22, 2025 · v1.1.0
  

• Features, bugs, research, ... that we are not actively working on

  enhancementgood first issuehelp wantedroadmap
  Task

  symflower/eval-dev-quality#387 · Filter by author zimmski opened on Jan 15, 2025 · v1.1.0
  

• Roadmap for v1.1.0

  roadmap
  Task

  symflower/eval-dev-quality#382 · Filter by author bauersimon opened on Jan 15, 2025 · v1.1.0
  

• Add new subcommand verify that just does repository verification and run that in CI

  bug
  Feature

  symflower/eval-dev-quality#330 · Filter by author bauersimon opened on Aug 27, 2024 · v1.1.0
  

• Kubernetes job watching does not work when erroring

  bug
  Bug

  symflower/eval-dev-quality#329 · Filter by author Munsio opened on Aug 27, 2024 · v1.1.0
  

• Allow to store and load all configuration items with Docker / Kubernetes runners

  enhancement
  Feature

  symflower/eval-dev-quality#307 · Filter by author ruiAzevedo19 opened on Aug 1, 2024 · v1.1.0
  

• Multiple model parameter with same value result in multiple evaluations

  bugquestion
  Bug

  symflower/eval-dev-quality#220 · Filter by author Munsio opened on Jun 26, 2024 · v1.1.0