Skip to content

LCORE-4710: Add inference registry startup integration tests - #3138

Open
asimurka wants to merge 1 commit into
lightspeed-core:mainfrom
asimurka:initialize-providers-registry-integration
Open

asimurka wants to merge 1 commit into
lightspeed-core:mainfrom
asimurka:initialize-providers-registry-integration

Conversation

@asimurka

@asimurka asimurka commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds integration tests for inference provider registry startup: skip-discovery wiring for every registered provider type, allow-list keep/drop, provider-specific discovery paths (OpenAI/vLLM/Bedrock/Vertex/sentence-transformers), and discovery failure abort behavior. Foreign APIs are mocked.

Type of change

  • Refactor
  • New feature
  • Bug fix
  • CVE fix
  • Optimization
  • Documentation Update
  • Configuration Update
  • Bump-up service version
  • Bump-up dependent library [pyproject.toml + uv.lock]
  • Bump-up dependent library [requirements.*.txt for Konflux]
  • Bump-up library or tool used for development (does not change the final image)
  • CI configuration change
  • Konflux configuration change
  • Unit tests improvement
  • Integration tests improvement
  • End to end tests improvement
  • Benchmarks improvement

Tools used to create PR

  • Assisted-by: Cursor (Composer)
  • Generated by: N/A

Related Tickets & Documents

Checklist before requesting a review

  • I have performed a self-review of my code.
  • PR has passed all pre-merge test jobs.
  • If it is a core feature, I have added thorough tests.

Testing

  1. uv run pytest tests/integration/inference_registry/ -q

Summary by CodeRabbit

  • Tests
    • Expanded integration coverage for inference provider startup, including provider configuration, model discovery, allow-list filtering, and provider-specific model handling.
    • Added checks for discovery failures and registry behavior when setup cannot complete. This improves confidence that provider initialization and model selection behave consistently across supported integrations.

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

This pull request adds integration tests for inference provider registry startup. The tests cover skipped discovery, provider-specific model catalogs, filtering, and OpenAI and Bedrock discovery failures.

Changes

Inference registry startup coverage

Layer / File(s) Summary
Startup test setup
tests/integration/inference_registry/*
Shared fixtures write and load inference configuration, reset the ProviderRegistry singleton around tests, and support checks for skipped discovery and empty provider configuration.
Provider discovery catalogs
tests/integration/inference_registry/*
Provider mocks and tests cover discovered model catalogs and filtering for OpenAI, vLLM, Bedrock, Vertex AI, and Sentence Transformers.
Discovery failure handling
tests/integration/inference_registry/*
Tests verify that OpenAI transport errors and Bedrock access-denied errors raise ModelDiscoveryError and leave the registry empty.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Possibly related PRs

Suggested reviewers: tisnik

Merge Risk: Minimal · up to 3b997

The change adds startup tests without changing runtime behavior. The tests and scanner still need a working CI environment.

Pre-merge checks: Passed 7
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "LCORE-4710: Add inference registry startup integration tests" matches the changeset. The PR adds integration tests under tests/integration/inference_registry/ covering registry startup with…
Docstring Coverage Passed Docstring coverage is 93.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Performance And Algorithmic Complexity Passed PASS. The pull request only adds integration-test modules, fixtures, and mocks. It does not change production handlers, Kubernetes or database operations, caches, watchers, or API endpoints. The added…
Security And Secret Handling Passed PASSED. The pull request adds only test files under tests/integration/inference_registry/. The credential-like values in _helpers.py:52-65,99 are explicit dummy fixtures such as test-key, `test-…
Full details: Title check

Explanation

The title "LCORE-4710: Add inference registry startup integration tests" matches the changeset. The PR adds integration tests under tests/integration/inference_registry/ covering registry startup with discovery skipped, allow-list filtering, provider-specific discovery for OpenAI, Bedrock, Vertex and vLLM, and startup failure on discovery errors. The title is concise, specific, and includes the ticket key without noise.

Full details: Docstring Coverage

Explanation

Docstring coverage is 93.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 15 functions across 4 files.

Full details: Performance And Algorithmic Complexity

Explanation

PASS. The pull request only adds integration-test modules, fixtures, and mocks. It does not change production handlers, Kubernetes or database operations, caches, watchers, or API endpoints. The added loops and list conversions process small in-memory test fixtures; no O(n^2) algorithm, N+1 calls, unbounded growth, or missing pagination is introduced.

Full details: Security And Secret Handling

Explanation

PASSED. The pull request adds only test files under tests/integration/inference_registry/. The credential-like values in _helpers.py:52-65,99 are explicit dummy fixtures such as test-key, test-bearer-token, unused, and xyzzy; no real secrets are present. The changes add no logging, API endpoints, response handling, command execution, SQL, or Kubernetes Secret resources. No plaintext secret leakage, missing endpoint authorization, injection flaw, or missing OwnerReference is introduced.

Finishing Touches
Generate unit tests (beta)
  • Create a new PR
Simplify code
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 668a10a and 3b99753.

Files selected for processing: 4
  • tests/integration/inference_registry/__init__.py
  • tests/integration/inference_registry/_helpers.py
  • tests/integration/inference_registry/conftest.py
  • tests/integration/inference_registry/test_startup.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Recent review details
Context from checks skipped due to timeout: 25
  • GitHub Check: Red Hat Konflux / lightspeed-core-0-8-enterprise-contract / lightspeed-stack-0-8
  • GitHub Check: Red Hat Konflux / lightspeed-stack-0-8-e2e-tests / lightspeed-stack-0-8
  • GitHub Check: integration_tests (3.12)
  • GitHub Check: integration_tests (3.13)
  • GitHub Check: list_outdated_dependencies
  • GitHub Check: mypy
  • GitHub Check: integration_tests (3.14)
  • GitHub Check: radon
  • GitHub Check: Pylinter
  • GitHub Check: check_dependencies
  • GitHub Check: Red Hat Konflux / rag-content-0-8-e2e-tests / lightspeed-stack-0-8
  • GitHub Check: pydocstyle
  • GitHub Check: build-pr
  • GitHub Check: spectral
  • GitHub Check: unit_tests (3.13)
  • GitHub Check: unit_tests (3.14)
  • GitHub Check: ruff
  • GitHub Check: black
  • GitHub Check: unit_tests (3.12)
  • GitHub Check: Pyright
  • GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-8-on-pull-request
  • GitHub Check: dispatch / Harness dispatch
  • GitHub Check: dispatch / Route
  • GitHub Check: authorize / Check repository owner or member
  • GitHub Check: check
CI failures not shown inline: 2

GitHub Actions: Bandit security scanner / 0_bandit.txt: LCORE-4710: Add inference registry startup integration tests

Conclusion: failure

View job details

##[group]Installed versions
 Version 3.8 was not found in the local cache
 (node:2287) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
 (Use `node --trace-deprecation ...` to show where the warning was created)
 ##[error]The version '3.8' with architecture 'x64' was not found for Ubuntu 26.04.

GitHub Actions: Bandit security scanner / bandit: LCORE-4710: Add inference registry startup integration tests

Conclusion: failure

View job details

##[group]Installed versions
 Version 3.8 was not found in the local cache
 (node:2287) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
 (Use `node --trace-deprecation ...` to show where the warning was created)
 ##[error]The version '3.8' with architecture 'x64' was not found for Ubuntu 26.04.
Additional context used
Code guidelines: 2
CLAUDE.md — auto-discovered
AGENTS.md — auto-discovered
Path-based instructions: 2
Source excerpt: **Required**: Use pytest for all unit and integration tests **Forbidden**: Do not use unittest - pytest is the standard for this project Source excerpt: **Coverage**: Unit tests require 60% coverage, integration 10%

CodeRabbit inference engine (CLAUDE.md)

Files:

  • tests/integration/inference_registry/__init__.py
  • tests/integration/inference_registry/conftest.py
  • tests/integration/inference_registry/test_startup.py
  • tests/integration/inference_registry/_helpers.py
Source excerpt: **Required**: Use pytest for all unit and integration tests Source excerpt: **Forbidden**: Do not use unittest - pytest is the standard for this project Source excerpt: **Coverage**: Unit tests require 60% coverage, integrat...

CodeRabbit inference engine (AGENTS.md)

Files:

  • tests/integration/inference_registry/__init__.py
  • tests/integration/inference_registry/conftest.py
  • tests/integration/inference_registry/test_startup.py
  • tests/integration/inference_registry/_helpers.py
Learnings: 1
Common learnings
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-10-31T11:32:30.158Z
Learning: Applies to tests/{unit,integration}/** : Source excerpt:
# Lightspeed Core Stack Development Guide

## Testing Framework

### Testing Framework Requirements

- **Required**: Use pytest for all unit and integration tests
- **Forbidden**: Do not use unittest - pytest is the standard for this project
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-10-31T11:32:13.138Z
Learning: Applies to tests/{unit,integration}/** : Source excerpt:
# Lightspeed Core Stack Development Guide

## Testing Framework

### Testing Framework Requirements

- **Forbidden**: Do not use unittest - pytest is the standard for this project
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-10-31T11:32:13.138Z
Learning: Applies to tests/{unit,integration}/** : Source excerpt:
# Lightspeed Core Stack Development Guide

## Testing Framework

### Testing Framework Requirements

- **Required**: Use pytest for all unit and integration tests

@fullsend-ai-review

fullsend-ai-review Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:10 PM UTC · Completed 3:24 PM UTC

Commit: 3b99753 · View workflow run →

Runtime: pi · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.49

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Nov 3, 2026
@fullsend-ai-review

Copy link
Copy Markdown

<!-- fullsend:risk-assessment -->
Risk Assessment: moderate (2/5)

Details

Test-only PR adding 4 new integration test files for inference registry startup. Zero production code changes, no protected paths, no security-sensitive files, no dependency changes. All files are brand new with no churn or contention history. Composite rounds to 2 (moderate) driven primarily by line count.

@fullsend-ai-review

Copy link
Copy Markdown

Looks good to me


Labels: PR adds Python integration tests for inference registry startup.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge python Pull requests that update python code labels Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests that update python code ready-for-merge All reviewers approved — ready to merge risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant