Find, download and serve OpenVINO models on your own machine, from one window.
A desktop front end for OpenVINO Model Server:
browse the OpenVINO models on Hugging Face, download them, and serve them through an
OpenAI-compatible API on your Intel CPU, GPU or NPU, for example to chat in Open WebUI.
A Monad app.
Monad · Install guide · Lectern · All Monad apps
Note
OVMS Manager is part of Monad, a family of desktop apps that run AI models locally
with Intel OpenVINO: no cloud, no accounts, no telemetry. It is the first app to install: it
downloads the models into ~/models, the folder every Monad app shares, and serves them to other apps.
The install guide covers the system packages and what comes next.
Install · Screenshots · The four pages · Open WebUI · Container runtime · Privacy · Troubleshooting
OVMS runs in a container (Podman or Docker), so you don't have to install it or the GPU drivers on your system.
From the latest release, download
OVMS-Manager-<version>-x86_64.AppImage, then:
chmod +x OVMS-Manager-*-x86_64.AppImage
./OVMS-Manager-*-x86_64.AppImageTo also install the Converter (optimum-intel and a CPU-only PyTorch, a few GB), use
./install.sh --with-optimum. The AppImage does not include the Converter.
The AppImage carries its own Python, Qt and OpenVINO: it installs nothing, needs no root and runs on any
x86-64 distribution with glibc 2.34 or newer. To verify it, compare it with SHA256SUMS from the same
release. To add it to the application menu, use Gear Lever
or AppImageLauncher, or install from source.
From the same release, download OVMS-Manager-<version>.tar.gz, then:
tar -xzf OVMS-Manager-*.tar.gz
cd ovms-manager-*/
./install.shThe installer works as described below. Once the app is installed you can delete the folder.
git clone https://github.com/VarCode-ai/ovms-manager.git
cd ovms-manager
./install.shThe installer copies the app to ~/.local/share/ovms-manager/app, creates a Python environment next to
it, adds OVMS Manager to the application menu and the ovms-manager command to ~/.local/bin.
Run ./install.sh --help for every option, and ./install.sh --uninstall to remove it.
Requirements: Python 3.10+, and Podman or Docker. See System packages in the Monad README for the packages of each distribution.
The interface is in English or Italian and follows your system language. To change it, use the 🌐 menu at the bottom of the sidebar, then restart the app.
Browse and filter the catalog of OpenVINO models ready to download.
Manage downloaded models and select them for serving.
Export and quantize Hugging Face models to OpenVINO IR.
Serve several local models at once through the same API on port 8000.
-
Under Models to serve, select a local model and its accelerator. Use Add model for each additional model. For example, select an embeddings model on the GPU for semantic search and a Whisper model on the CPU for audio transcription. The manager remembers each row.
-
Hardware accelerator:
Mode OpenVINO device Notes Intel GPU GPUFastest generation. Passes /dev/drito the container.Intel NPU NPULow power, ideal on battery. The image carries the NPU driver and compiler: the host only needs the intel_vpukernel module and the NPU firmware. PreferINT4 CWmodels.Hybrid GPU + CPU MULTI:GPU,CPUSplits requests between GPU and CPU. CPU only CPUWorks on any x86-64 machine. Each accelerator choice shows whether the hardware was detected. not Intel · unsupported means the machine has a GPU or NPU from another vendor, which OpenVINO cannot use. present · /dev not accessible means the app runs in a sandbox (Flatpak, toolbox…) that hides the device.
-
Start server applies all rows together. It prepares a deployment, replaces the
ovms-servercontainer and reports when each model is ready. After changing a row, start the server again to apply the change. Stop stops all the models in this deployment. -
Use the model ID printed in the console in the
modelfield of each request. For example, send queries to/v3/embeddingswith the embeddings model ID and recordings to/v3/audio/transcriptionswith the Whisper ID. Folders with the same API ID receive numeric suffixes in row order. -
Verbose log starts OVMS with
--log_level DEBUG. -
Localhost only publishes port 8000 on
127.0.0.1only, so other computers on your network cannot reach the API. Leave it off if Open WebUI runs in a container (see below). -
Test API checks each configured model and sends a short chat request when a chat model is ready.
-
Container log shows the last 80 lines of the server log.
-
Hover over the model count next to the server status to see the configured API IDs.
-
The console at the bottom stays folded to a single line with the latest message; click the arrow to open it. It opens by itself when an error arrives.
The catalog of models published by the OpenVINO organization, already converted and quantized. It is cached for 6 hours.
- Filter by type (LLM, vision-language, embeddings, rerank, image generation, speech-to-text,
text-to-speech) and by quantization (
INT4,INT8,FP16, other). The What is quantization? button explains the differences. - Community adds repositories from other authors tagged
openvino. - The details panel tells you at a glance whether the model fits in your RAM (green), is tight (amber) or is larger than your memory (red), from the size of the repository.
- Download with OVMS runs
ovms --pullin a container, which downloads the model into~/modelsand prepares it for serving. Custom repo accepts anyauthor/namerepository id. - The Speech-to-Text models land in the same folder Lectern reads.
For Lectern on the NPU prefer the
INT8Whisper models (whisper-large-v3-turbo-int8-ovfor live recording): they compile for the NPU in a few minutes the first time, theINT4ones take much longer.
The models in ~/models, with their size on disk. Use in server adds the selected model to the Server page.
Delete removes a model folder. If the files were created by the container with a different user ID,
OVMS Manager deletes them through podman unshare or a short-lived container.
Exports a PyTorch model from Hugging Face to OpenVINO IR with
optimum-intel, compressing the weights to INT4,
INT8 or FP16. It needs optimum-cli, installed by ./install.sh --with-optimum. OVMS Manager also
finds optimum-cli if it is installed in the app's own environment or on your PATH.
Some models with new architectures need Allow custom code (--trust-remote-code), which runs Python
code from the repository on your computer. Enable it only for repositories you trust.
- Ctrl+K, or Search… at the top of the sidebar, opens the command palette. Go to a page, start or stop the server, test the API, use any local model, or type a name and choose Search the catalog.
- The two menus at the bottom of the sidebar set the interface language and the theme (System, Light, Dark). The theme changes immediately; System follows the desktop.
- When a new version is released, a banner at the top links to its notes. In the command palette, Check for updates checks right away and Turn automatic update checks off disables the daily check.
In Open WebUI open Admin Settings → Connections → OpenAI API, press + and enter:
- URL:
http://127.0.0.1:8000/v3if Open WebUI runs directly on your system. If it runs in a container, usehttp://host.containers.internal:8000/v3with Podman orhttp://host.docker.internal:8000/v3with Docker. - API key: any text, for example
none.
The model shows up with the name of its folder without the -ov suffix (for example
Qwen3-8B-int4). If it doesn't, add that name to the Model IDs field of the connection.
To run Open WebUI in a container:
# Podman
podman run -d --name open-webui -p 3000:8080 -v open-webui:/app/backend/data \
ghcr.io/open-webui/open-webui:main
# Docker: host.docker.internal must be mapped explicitly on Linux
docker run -d --name open-webui -p 3000:8080 -v open-webui:/app/backend/data \
--add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:mainOVMS Manager uses Podman if it is installed, otherwise Docker. To force one, set
OVMS_CONTAINER_RUNTIME=docker (or podman). OVMS_EXTRA_RUN_ARGS adds options to every
podman run/docker run, for example OVMS_EXTRA_RUN_ARGS="--group-add keep-groups".
- Podman (rootless) is the recommended runtime. Files written by the container in
~/modelsbelong to a sub-UID of your user. - Docker runs the containers with your user and group IDs (
--user) and adds the groups that own/dev/driand/dev/accel(--group-add), so the files it writes belong to you. Your user must be allowed to use Docker: either it is in thedockergroup or you use rootless Docker.
Volumes are mounted with the shared SELinux label (:z), which on systems without SELinux has no effect.
The image is docker.io/openvino/model_server:latest-gpu, about 2 GB. It is downloaded the first time
you download a model or start the server.
| Path | Content |
|---|---|
~/models/ (or $OPENVINO_MODELS_DIR) |
Local model library shared with other apps. |
~/.cache/ovms_manager/hf_catalog.json |
Cached Hugging Face catalog. |
~/.cache/ovms_manager/state.json |
Selected models and accelerators, last deployed model IDs, Localhost only and interface language. |
~/.local/share/ovms-manager/deployment/ |
Server configuration and per-model runtime settings generated from the selected rows. |
~/.local/share/ovms-manager/ |
Installed app and Python environments. |
| Problem | Fix |
|---|---|
| The status pill shows Podman/Docker missing | Install Podman or Docker (see System packages). |
| The status pill shows Docker error | Your user cannot use Docker. Add it to the docker group (sudo usermod -aG docker $USER, then log out and back in) or use Podman. |
| The container stops right away | Press Container log. Common causes: not enough memory for the model, or a device the model doesn't support (try the CPU mode). |
GPU mode fails with rootless Podman (Context was not initialized, permission denied on /dev/dri) |
On some distributions the GPU is only accessible through your render/video group, which rootless containers drop by default. Start OVMS Manager with OVMS_EXTRA_RUN_ARGS="--group-add keep-groups" ovms-manager (requires the crun runtime, the default on recent distributions). |
| Still unavailable after 10 minutes | Large models can take more than 10 minutes on the first load. Check the log, then press Start server again: later loads are faster. |
| Converter: optimum-cli not found | Run ./install.sh --with-optimum and restart the app. |
More fixes are in docs/troubleshooting.md, and the problems common to every Monad app in the Monad troubleshooting guide.
The models answer from your own machine: prompts and answers never leave it. OVMS Manager connects to the internet only when you open Explore Hugging Face (the catalog is cached for 6 hours), download or convert a model, pull the OpenVINO Model Server image once, and to check at most once a day for a new release, which you can turn off from the command palette. The full list of connections, for every Monad app, is in Private by design.
The OVMS API has no authentication: on untrusted networks, enable Localhost only on the Server page.
Bug reports, translations and pull requests are welcome: see CONTRIBUTING.md.
Copyright © 2026 VarCode-ai. OVMS Manager is free software, released under the
GNU General Public License v3.0 or later. Third-party components keep their own license:
OpenVINO and OpenVINO Model Server (Apache 2.0), Qt for Python / PySide6 (LGPL v3), the Inter and
Newsreader fonts (SIL Open Font License 1.1, see common/fonts/), and the models you download, each
covered by the license on its Hugging Face page.
OVMS Manager is a Monad app · local AI for OpenVINO



