vllm-mlx
by waybarrios
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code.
435stars
53forks
Python
Added 2/9/2026
MCP Serversanthropicapple-siliconaudio-processingclaude-codecomputer-vision
Installation
# Add to your Claude Code skills
git clone https://github.com/waybarrios/vllm-mlxvLLM-MLX
vLLM-like inference for Apple Silicon - GPU-accelerated Text, Image, Video & Audio on Mac
Overview
vllm-mlx brings native Apple Silicon GPU acceleration to vLLM by integrating:
- MLX: Apple's ML framework with unified memory and Metal kernels
- mlx-lm: Optimized LLM inference with KV cache and quantization
- mlx-vlm: Vision-language models for multimodal inference
- mlx-audio: Speech-to-Text and Text-to-Speech with native voices
- mlx-embeddings: Text embeddings for semantic search and RAG
Features
- Multimodal - Text, Image, Video & Audio in one platform
- Native GPU acceleration on Apple Silicon (M1, M2, M3, M4)
- Native TTS voices - Spanish, French, Chinese, Japanese + 5 more languages
- OpenAI API compatible - drop-in replacement for OpenAI client
Comments (0)
to leave a comment.
No comments yet. Be the first to share your thoughts!