LLM Infrastructure, ranked by momentum

What’s trending in LLM Infrastructure: the projects accelerating fastest right now — ranked by how quickly their growth is speeding up, not by how many stars they already have.

67 tracked projects · updated daily · 3 Nov 2026

#ScoreProjectAcceleratingStars

193

unslothai/unslothUnsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

+8.679.0k 293

mw00/project-mayaProject Maya - run GLM-5.3-Flash (321B MoE) and DeepSeek-V4.1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

+48.12.4k 388

vllm-project/semantic-routerA programmable Mixture-of-Models router for heterogeneous LLM inference

+8.96.4k 487

incoai/splashA local inference engine for Apple silicon, built around the model.

+27.02.0k 585

syv-ai/qwen38-27b-rtx3090Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks

+15.42.5k 681

FareedKhan-dev/kimi-k3-in-cA 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

+33.39.5k 778

ggml-org/llama.cppLLM inference in C/C++

+0.9133.2k 877

gpustack/gpustackA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

+2.76.0k 976

AtomicBot-ai/Atomic-ChatLocal AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

+4.31.9k 1074

ray-project/rayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

+1.444.2k 1173

vllm-project/vllm-ascendCommunity maintained hardware plugin for vLLM on Ascend

+0.93.0k 1268

raullenchai/Rapid-MLXRapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server and Mac app for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Release-gated with Claude Code, Codex CLI, Aider, Hermes and DeepSeek Harness.

+2.34.1k 1367

kubeflow/trainerDistributed AI Model Training and LLM Fine-Tuning on Kubernetes

+0.72.3k 1463

spiceai/spiceaiAdd a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

+1.43.1k 1562

beam-cloud/beta9Ultrafast serverless GPU inference, sandboxes, and background jobs

+1.31.8k 1660

ModelTC/LightLLMLightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

+0.64.4k 1760

skypilot-org/skypilotThe AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

+0.010.8k 1859

bentoml/BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

+0.38.9k 1959

xLLM-AI/xllmA high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

+0.71.6k 2058

openlake-project/openlakeOpenLake is a high performance storage engine for efficient LLM inference and GPU Training

+1.42.6k 2155

alibaba/rtp-llmRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

-0.11.4k 2252

linkedin/Liger-KernelEfficient Triton Kernels for LLM Training

+0.06.7k 2351

thu-pacman/chituHigh-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

+0.63.0k 2449

Tiiny-AI/PowerInferHigh-speed Large Language Model Serving for Local Deployment

+0.49.9k 2549

AIDotNet/OpenDeepWikiOpenDeepWiki is the open-source version of the DeepWiki project, aiming to provide a powerful knowledge management and collaboration platform. The project is mainly developed using C# and TypeScript, supporting modular design, and is easy to expand and customize.

+0.63.7k

View all 67 on the board →

What counts as LLM Infrastructure here

Inference servers, fine-tuning, and runtimes for large language models. This page lists only projects whose primary category is LLM Infrastructure; projects that merely connect to it are listed under Related projects below. Rankings cover the 1167 repositories Repoclimb tracks, not all of GitHub.

Related projects (24)

These belong to other categories first, but show up when people explore LLM Infrastructure.

What counts as LLM Infrastructure here

Related projects (24)

Other categories