Data / Repositories / LLM inference / Rust
Rust LLM inference repositories on GitHub: 4 ranked by stars
4 public Rust repositories tagged LLM inference are on GitHub, with 15,717 stars between them; 2 had a push in the last 30 days. The most starred: Michael-A-Kuykendall/shimmy (5,976), ruvnet/RuVector (4,565), spiceai/spiceai (3,099).
SpecStream is the live web data engine for teams and agents. It reads open-source repositories on GitHub and keeps what it reads, and this page groups that reading by topic. Ask for the repositories you track by topic, language, license or activity, and new ones and changes arrive in your sheet, CRM, Slack or agent.
Every repository, most starred first
| Repository | What it is | Stars | Forks | License | Last push | Read |
|---|---|---|---|---|---|---|
| Michael-A-Kuykendall/shimmy | Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary. | 5,976 | 585 | Apache-2.0 | Aug 30 | Oct 14 |
| ruvnet/RuVector | RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust. | 4,565 | 611 | MIT | Oct 14 | Oct 14 |
| spiceai/spiceai | Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and… | 3,099 | 236 | Apache-2.0 | Oct 14 | Oct 14 |
| microsoft/aici | AICI: Prompts as (Wasm) Programs | 2,077 | 86 | MIT | Jan 22, 2025 | Oct 11 |
Other languages
Related topics
- AI 5
- LLM 4
- Rust 3
- Llama 2
- Machine learning 2
- Genai 2
- MCP 2
- MCP server 2
- OpenAI API 2
- WASM 2
- Agents 2
- Awesome list 2
Every LLM inference repository
Read from GitHub by SpecStream between October 11, 2026 and October 14, 2026; each row shows the day its repository was read. Stars and pushes change every day, and the GitHub page is the current one. Forks and archived repositories are left out.