Data / Repositories / LLM inference
LLM inference repositories on GitHub: 9 ranked by stars
9 public repositories tagged LLM inference are on GitHub, with 29,780 stars between them; 6 had a push in the last 30 days. Most are written in Rust (4), C++ (1), Python (1). The most starred: Michael-A-Kuykendall/shimmy (5,976), lemonade-sdk/lemonade (5,863), ruvnet/RuVector (4,565).
SpecStream is the live web data engine for teams and agents. It reads open-source repositories on GitHub and keeps what it reads, and this page groups that reading by topic. Ask for the repositories you track by topic, language, license or activity, and new ones and changes arrive in your sheet, CRM, Slack or agent.
Every repository, most starred first
| Repository | What it is | Language | Stars | Forks | License | Last push | Read |
|---|---|---|---|---|---|---|---|
| Michael-A-Kuykendall/shimmy | Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary. | Rust | 5,976 | 585 | Apache-2.0 | Aug 30 | Oct 14 |
| lemonade-sdk/lemonade | Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. | C++ | 5,863 | 526 | Apache-2.0 | Oct 11 | Oct 11 |
| ruvnet/RuVector | RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust. | Rust | 4,565 | 611 | MIT | Oct 14 | Oct 14 |
| algorithmicsuperintelligence/optillm | Optimizing inference proxy for LLMs | Python | 4,317 | 394 | Apache-2.0 | Oct 11 | Oct 14 |
| spiceai/spiceai | Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and… | Rust | 3,099 | 236 | Apache-2.0 | Oct 14 | Oct 14 |
| microsoft/aici | AICI: Prompts as (Wasm) Programs | Rust | 2,077 | 86 | MIT | Jan 22, 2025 | Oct 11 |
| pallavi-shekhar/ai-engineering-interview-questions-company-wise | Your Cheat Sheet For AI Engineering Interviews at Top AI Companies - Questions and Answers. | Markdown | 1,748 | 175 | Apache-2.0 | Oct 12 | Oct 15 |
| taielab/awesome-hacking-lists | A curated collection of top-tier penetration testing tools and productivity utilities across multiple domains. Join us to explore, contribute, and enhance your hacking toolkit! | 1,393 | 268 | Dec 4, 2025 | Oct 11 | ||
| v-modal/awesome-jev-tools | A curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions. | 742 | 51 | Oct 9 | Oct 15 |
By language
- Rust 4
Related topics
- AI 5
- LLM 4
- Rust 3
- Llama 2
- Machine learning 2
- Genai 2
- MCP 2
- MCP server 2
- OpenAI API 2
- WASM 2
- Agents 2
- Awesome list 2
Read from GitHub by SpecStream between October 11, 2026 and October 15, 2026; each row shows the day its repository was read. Stars and pushes change every day, and the GitHub page is the current one. Forks and archived repositories are left out.