Back to directory
AI & Machine Learning · Machine Learning Infrastructure
llama.cpp
Reference C++ inference engine for running GGUF models on diverse hardware.
Before you deploy
- Confirm license terms for your use case (commercial, SaaS, internal only).
- Plan upgrades: who merges security patches and major version bumps.
- Map data residency if the tool stores user content or embeddings.
Related in Machine Learning Infrastructure
- AgentaCollaborative prompt and eval workspace for teams shipping LLM features.
- Arize PhoenixOpen tracing and evaluation toolkit for debugging retrieval and generation.
- DeepnoteCollaborative notebooks mixing Python, SQL, and AI cells for data teams.
- dstackGPU scheduling across cloud accounts and on-prem clusters.
- GPT4AllDesktop runner for open weights on CPU-first hardware.
- GreptureSDK that wraps LLM calls with tracing, eval hooks, and safety checks.