allow user to change llm - #23
Open
AlexNikic wants to merge 3 commits into
Open
Conversation
Contributor
Author
|
just quickly did this with claude opus 5 and it did pretty well |
Contributor
Author
|
i also added a yaml that does an R CMD check on merge requests |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Lets the user choose which LLM Harmony uses for matching from R, in the same way they can from Python.
match_instruments()already accepted amodelname through..., but it always sent it to the API asframework = "huggingface", so the cloud-hosted models the Harmony API supports (OpenAI, Google, Azure OpenAI) were unreachablefrom the R library. This PR promotes
modelto a documented argument and adds aframeworkargument alongside it, sothe full
parametersobject the API accepts can be set from R:The framework is inferred from the model name, so switching LLM normally only needs
model. An explicitframeworkalways wins, which keeps models the R library has not been taught about usable.
parametersis now always sent,populated with the API's own defaults, so the default behaviour is unchanged.
Also adds
list_models(), which callsGET /info/list-modelsand returns a data frame offramework,modelandavailable. Availability is a property of the API deployment rather than of this package: the public API reports onlythe three Hugging Face models as available, because the cloud models need API keys that the API holds.
list_models()is how a user finds out what the endpoint they are pointed at can actually serve.Finally, failed API requests now raise an R error carrying the
detailmessage returned by the API. Previously arejected model came back as a 422 whose body flowed on into the response handling and failed later on
conten$matches[[1]], which gave no clue that the model was the problem.This is backwards compatible.
modelmoves from...to a named argument, so existing calls passingmodel =behaveidentically;
...is retained. No new third party dependencies —httr::GET()andhttr::status_code()come fromthe
httrpackage the library already imports.Documentation: README gains a "Choosing which LLM to use" section covering
list_models()and the Docker route to thecloud models (
docker run -p 8000:80 -e OPENAI_API_KEY=... harmonydata/harmonyapiplusset_url()), and?match_instrumentsgains a "Choosing a model" section.Fixes # 22
Type of change
Testing
New test file
tests/testthat/test-model-selection.R. The framework-resolution tests run offline; the rest runagainst the live API and are wrapped in
skip_on_cran().frameworks
frameworkoverrides the inferred one, including for a model name the library does not knowframeworkwarns and falls back tohuggingface, i.e. the previous behaviourmodelandframeworkarguments are rejected with a clear errorlist_models()returns the expected data frame, and reports the default model ofmatch_instruments()asavailable
paraphrase-multilingual-mpnet-base-v2) returns a similarity matrix of theright shape and a diagonal of 1
shape
To reproduce:
and a full
R CMD checkreturnsStatus: OK— no errors, warnings or notes.The Harmony API and Python library are untouched by this PR, so the API unit tests are unaffected — the change only
alters what the R library puts in the request body, using fields the API already accepts (
MatchParameters).Test Configuration
https://api.harmonydata.ac.uk
Checklist
Harmony API or Python library
?match_instrumentsand?list_models, and in NEWS.md. No script added to the script examplesrepository yet; happy to add one if that would be useful.