I build self-hosted, private LLM/RAG platforms and the tooling around them: local inference, document ingestion, model fine-tuning, serving benchmarks, and source-grounded agent tooling. No cloud, no vendor lock-in.
- LLM serving: vLLM, llama.cpp, Ollama — on NVIDIA GB10 (DGX Spark) and AMD Strix Halo
- RAG: vector DBs (Qdrant/Weaviate/Milvus), document parsing (Docling), chunking, reranking, evaluation
- AI platform: Docker Compose, k3s/Ansible, monitoring (Prometheus/Grafana/Loki), security hardening, Day-2 ops
- ML lifecycle: distillation, PEFT/LoRA, GGUF quantization, local deployment, evaluation
- Backend: Python, FastAPI, PostgreSQL, Redis, Celery
- Habr technical series on self-hosted LLM inference on NVIDIA DGX Spark (GB10) and AMD Strix Halo: serving benchmarks under parallel/multi-user load, speculative decoding, vLLM at 256K context, RU RAG-model fine-tuning, tokenization cost, and unified-memory observability. https://habr.com/ru/users/AGmind/
I use AI as a force multiplier. I own architecture, security, and deployment decisions, and I verify everything against real hardware and real data — including the parts AI can't reason about (driver regressions, hardware memory behavior, network topology).