Skip to content

About

Find, download and serve OpenVINO models on your own machine, from one window. A Monad app.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Monad

OVMS Manager

Find, download and serve OpenVINO models on your own machine, from one window.

A desktop front end for OpenVINO Model Server: browse the OpenVINO models on Hugging Face, download them, and serve them through an OpenAI-compatible API on your Intel CPU, GPU or NPU, for example to chat in Open WebUI.
A Monad app.

Checks Latest release License: GPL-3.0-or-later Runs on Intel CPU, GPU and NPU A Monad app

Monad · Install guide · Lectern · All Monad apps


OVMS Manager: serve multiple local models on Intel CPU, GPU or NPU

Note

OVMS Manager is part of Monad, a family of desktop apps that run AI models locally with Intel OpenVINO: no cloud, no accounts, no telemetry. It is the first app to install: it downloads the models into ~/models, the folder every Monad app shares, and serves them to other apps. The install guide covers the system packages and what comes next.

Install · Screenshots · The four pages · Open WebUI · Container runtime · Privacy · Troubleshooting

OVMS runs in a container (Podman or Docker), so you don't have to install it or the GPU drivers on your system.

Install

Download the AppImage

From the latest release, download OVMS-Manager-<version>-x86_64.AppImage, then:

chmod +x OVMS-Manager-*-x86_64.AppImage
./OVMS-Manager-*-x86_64.AppImage

To also install the Converter (optimum-intel and a CPU-only PyTorch, a few GB), use ./install.sh --with-optimum. The AppImage does not include the Converter.

The AppImage carries its own Python, Qt and OpenVINO: it installs nothing, needs no root and runs on any x86-64 distribution with glibc 2.34 or newer. To verify it, compare it with SHA256SUMS from the same release. To add it to the application menu, use Gear Lever or AppImageLauncher, or install from source.

Download the archive and install it

From the same release, download OVMS-Manager-<version>.tar.gz, then:

tar -xzf OVMS-Manager-*.tar.gz
cd ovms-manager-*/
./install.sh

The installer works as described below. Once the app is installed you can delete the folder.

Install from source

git clone https://github.com/VarCode-ai/ovms-manager.git
cd ovms-manager
./install.sh

The installer copies the app to ~/.local/share/ovms-manager/app, creates a Python environment next to it, adds OVMS Manager to the application menu and the ovms-manager command to ~/.local/bin. Run ./install.sh --help for every option, and ./install.sh --uninstall to remove it.

Requirements: Python 3.10+, and Podman or Docker. See System packages in the Monad README for the packages of each distribution.

The interface is in English or Italian and follows your system language. To change it, use the 🌐 menu at the bottom of the sidebar, then restart the app.

Screenshots

Explore Hugging Face

Browse and filter the catalog of OpenVINO models ready to download.

OVMS Manager: Hugging Face model catalog with quantization filters

Local library

Manage downloaded models and select them for serving.

OVMS Manager: local model library with model sizes and quantization

Optimum converter

Export and quantize Hugging Face models to OpenVINO IR.

OVMS Manager: Optimum converter with precision and destination options

The four pages

Server

Serve several local models at once through the same API on port 8000.

  • Under Models to serve, select a local model and its accelerator. Use Add model for each additional model. For example, select an embeddings model on the GPU for semantic search and a Whisper model on the CPU for audio transcription. The manager remembers each row.

  • Hardware accelerator:

    Mode OpenVINO device Notes
    Intel GPU GPU Fastest generation. Passes /dev/dri to the container.
    Intel NPU NPU Low power, ideal on battery. The image carries the NPU driver and compiler: the host only needs the intel_vpu kernel module and the NPU firmware. Prefer INT4 CW models.
    Hybrid GPU + CPU MULTI:GPU,CPU Splits requests between GPU and CPU.
    CPU only CPU Works on any x86-64 machine.

    Each accelerator choice shows whether the hardware was detected. not Intel · unsupported means the machine has a GPU or NPU from another vendor, which OpenVINO cannot use. present · /dev not accessible means the app runs in a sandbox (Flatpak, toolbox…) that hides the device.

  • Start server applies all rows together. It prepares a deployment, replaces the ovms-server container and reports when each model is ready. After changing a row, start the server again to apply the change. Stop stops all the models in this deployment.

  • Use the model ID printed in the console in the model field of each request. For example, send queries to /v3/embeddings with the embeddings model ID and recordings to /v3/audio/transcriptions with the Whisper ID. Folders with the same API ID receive numeric suffixes in row order.

  • Verbose log starts OVMS with --log_level DEBUG.

  • Localhost only publishes port 8000 on 127.0.0.1 only, so other computers on your network cannot reach the API. Leave it off if Open WebUI runs in a container (see below).

  • Test API checks each configured model and sends a short chat request when a chat model is ready.

  • Container log shows the last 80 lines of the server log.

  • Hover over the model count next to the server status to see the configured API IDs.

  • The console at the bottom stays folded to a single line with the latest message; click the arrow to open it. It opens by itself when an error arrives.

Explore Hugging Face

The catalog of models published by the OpenVINO organization, already converted and quantized. It is cached for 6 hours.

  • Filter by type (LLM, vision-language, embeddings, rerank, image generation, speech-to-text, text-to-speech) and by quantization (INT4, INT8, FP16, other). The What is quantization? button explains the differences.
  • Community adds repositories from other authors tagged openvino.
  • The details panel tells you at a glance whether the model fits in your RAM (green), is tight (amber) or is larger than your memory (red), from the size of the repository.
  • Download with OVMS runs ovms --pull in a container, which downloads the model into ~/models and prepares it for serving. Custom repo accepts any author/name repository id.
  • The Speech-to-Text models land in the same folder Lectern reads. For Lectern on the NPU prefer the INT8 Whisper models (whisper-large-v3-turbo-int8-ov for live recording): they compile for the NPU in a few minutes the first time, the INT4 ones take much longer.

Local library

The models in ~/models, with their size on disk. Use in server adds the selected model to the Server page. Delete removes a model folder. If the files were created by the container with a different user ID, OVMS Manager deletes them through podman unshare or a short-lived container.

Converter

Exports a PyTorch model from Hugging Face to OpenVINO IR with optimum-intel, compressing the weights to INT4, INT8 or FP16. It needs optimum-cli, installed by ./install.sh --with-optimum. OVMS Manager also finds optimum-cli if it is installed in the app's own environment or on your PATH.

Some models with new architectures need Allow custom code (--trust-remote-code), which runs Python code from the repository on your computer. Enable it only for repositories you trust.

Command palette and appearance

  • Ctrl+K, or Search… at the top of the sidebar, opens the command palette. Go to a page, start or stop the server, test the API, use any local model, or type a name and choose Search the catalog.
  • The two menus at the bottom of the sidebar set the interface language and the theme (System, Light, Dark). The theme changes immediately; System follows the desktop.
  • When a new version is released, a banner at the top links to its notes. In the command palette, Check for updates checks right away and Turn automatic update checks off disables the daily check.

Connect Open WebUI

In Open WebUI open Admin Settings → Connections → OpenAI API, press + and enter:

  • URL: http://127.0.0.1:8000/v3 if Open WebUI runs directly on your system. If it runs in a container, use http://host.containers.internal:8000/v3 with Podman or http://host.docker.internal:8000/v3 with Docker.
  • API key: any text, for example none.

The model shows up with the name of its folder without the -ov suffix (for example Qwen3-8B-int4). If it doesn't, add that name to the Model IDs field of the connection.

To run Open WebUI in a container:

# Podman
podman run -d --name open-webui -p 3000:8080 -v open-webui:/app/backend/data \
    ghcr.io/open-webui/open-webui:main

# Docker: host.docker.internal must be mapped explicitly on Linux
docker run -d --name open-webui -p 3000:8080 -v open-webui:/app/backend/data \
    --add-host=host.docker.internal:host-gateway ghcr.io/open-webui/open-webui:main

Container runtime

OVMS Manager uses Podman if it is installed, otherwise Docker. To force one, set OVMS_CONTAINER_RUNTIME=docker (or podman). OVMS_EXTRA_RUN_ARGS adds options to every podman run/docker run, for example OVMS_EXTRA_RUN_ARGS="--group-add keep-groups".

  • Podman (rootless) is the recommended runtime. Files written by the container in ~/models belong to a sub-UID of your user.
  • Docker runs the containers with your user and group IDs (--user) and adds the groups that own /dev/dri and /dev/accel (--group-add), so the files it writes belong to you. Your user must be allowed to use Docker: either it is in the docker group or you use rootless Docker.

Volumes are mounted with the shared SELinux label (:z), which on systems without SELinux has no effect.

The image is docker.io/openvino/model_server:latest-gpu, about 2 GB. It is downloaded the first time you download a model or start the server.

Files

Path Content
~/models/ (or $OPENVINO_MODELS_DIR) Local model library shared with other apps.
~/.cache/ovms_manager/hf_catalog.json Cached Hugging Face catalog.
~/.cache/ovms_manager/state.json Selected models and accelerators, last deployed model IDs, Localhost only and interface language.
~/.local/share/ovms-manager/deployment/ Server configuration and per-model runtime settings generated from the selected rows.
~/.local/share/ovms-manager/ Installed app and Python environments.

Troubleshooting

Problem Fix
The status pill shows Podman/Docker missing Install Podman or Docker (see System packages).
The status pill shows Docker error Your user cannot use Docker. Add it to the docker group (sudo usermod -aG docker $USER, then log out and back in) or use Podman.
The container stops right away Press Container log. Common causes: not enough memory for the model, or a device the model doesn't support (try the CPU mode).
GPU mode fails with rootless Podman (Context was not initialized, permission denied on /dev/dri) On some distributions the GPU is only accessible through your render/video group, which rootless containers drop by default. Start OVMS Manager with OVMS_EXTRA_RUN_ARGS="--group-add keep-groups" ovms-manager (requires the crun runtime, the default on recent distributions).
Still unavailable after 10 minutes Large models can take more than 10 minutes on the first load. Check the log, then press Start server again: later loads are faster.
Converter: optimum-cli not found Run ./install.sh --with-optimum and restart the app.

More fixes are in docs/troubleshooting.md, and the problems common to every Monad app in the Monad troubleshooting guide.

Privacy

The models answer from your own machine: prompts and answers never leave it. OVMS Manager connects to the internet only when you open Explore Hugging Face (the catalog is cached for 6 hours), download or convert a model, pull the OpenVINO Model Server image once, and to check at most once a day for a new release, which you can turn off from the command palette. The full list of connections, for every Monad app, is in Private by design.

The OVMS API has no authentication: on untrusted networks, enable Localhost only on the Server page.

Contributing

Bug reports, translations and pull requests are welcome: see CONTRIBUTING.md.

License

Copyright © 2026 VarCode-ai. OVMS Manager is free software, released under the GNU General Public License v3.0 or later. Third-party components keep their own license: OpenVINO and OpenVINO Model Server (Apache 2.0), Qt for Python / PySide6 (LGPL v3), the Inter and Newsreader fonts (SIL Open Font License 1.1, see common/fonts/), and the models you download, each covered by the license on its Hugging Face page.


Monad
OVMS Manager is a Monad app · local AI for OpenVINO

About

Find, download and serve OpenVINO models on your own machine, from one window. A Monad app.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages