llm-server

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.

HostAI - Android LLM API Server

Please see the newer: https://github.com/Quazmoz/openvino-windows-llm

single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp

A lightweight LiteLLM server boilerplate pre-configured with uv and Docker for hosting your own OpenAI- and Anthropic-compatible endpoints. Includes LibreChat as an optional web UI.

macOS GUI for managing pure mlx_lm.server on Apple Silicon in Direct Mode.

Function-calling API for LLM from multiple providers

A complete, menu-driven AI model interface for Windows that simplifies running local GGUF language models with llama.cpp. This tool automatically manages dependencies, provides multiple interaction modes, and prioritizes user privacy through fully offline operation.

LLM / AI inference server for Windows + NVIDIA (EXL3/ExLlamaV3). OpenAI-compatible API, multi-GPU, Blazor admin. Ollama-like, no Docker.

Turn a fresh Apple Silicon Mac into a tuned local-LLM dev server in an evening (MIT)

A GUI launcher for llama.cpp local LLMs.

API server for `llm` CLI tool

A lightweight, zero-cost LLM server for deploying GGUF models via llama.cpp, featuring streaming support and an OpenAI-compatible API endpoint.

PHP Frontend for Hosting local LLM's (run via VSCode or basic php execution methods/ add to project)

A flexible FastAPI-based framework for handling AI tasks using Large Language Models (LLMs). Supports multiple providers, extensible tasks and routers, Redis caching, and OpenAI integration. Easily scalable for various LLM-based applications.

OpenAI-compatible local inference server for Apple Silicon using MLX. FastAPI server with Chat Completions and Responses APIs, multi-turn conversations, and streaming support.

Headless CLI for managing local MLX language-model HTTP servers on Apple Silicon Macs. Supports model discovery, server lifecycle management, performance benchmarking, and provider integration with OpenCode, Claude Code, and LiteLLM.

Sovereign AI API Gateway & Local Server — Zero-dependency, OpenAI-compatible drop-in endpoint for developers.