ORBIT – One gateway for every model and data source.

Open Retrieval-Based Inference Toolkit

Stop rewriting your app every time you switch LLMs. ORBIT unifies 20+ AI providers with your databases, vector stores, and APIs—all through one self-hosted gateway.

Ship faster. Stay portable. Keep your data private.

⭐ Help ORBIT grow: Star + watch the repo so other builders can find it and we can keep funding new adapters, tutorials, and sandbox credits.

orbit-demo.mp4

See ORBIT's unified LLM workspace in action.

---

🎯 Why ORBIT?

The Problem: Building AI apps means choosing between vendor lock-in, complex integrations, and privacy concerns.

• Vendor lock-in: Switching from OpenAI to Anthropic means rewriting your code
• Complex RAG: Connecting databases, vector stores, and APIs requires custom glue code
• Privacy concerns: Sending sensitive data to third-party APIs
• Fragmented tools: Different SDKs, auth methods, and APIs for each provider

The Solution: ORBIT is a self-hosted, unified gateway that gives you:

✅ Portability — Switch LLM providers with a config change, not code changes
✅ Built-in RAG — Connect SQL databases, vector stores, and APIs through natural language
✅ Full-Duplex Voice — Real-time speech-to-speech conversations with natural dynamics
✅ Data sovereignty — Keep your data on your infrastructure
✅ One API — OpenAI-compatible interface that works with any provider
✅ Production-ready — Rate limiting, security, monitoring, and fault tolerance built-in

---

Who it's for: Developers building AI applications who want flexibility, privacy, and control without the complexity of managing multiple integrations.

• Questions? Open an issue
• Updates: Check the changelog
• Commercial Support: OrbitPods.io
• Maintained by: Schmitech.ai

At a Glance

• ORBIT – One gateway for every model and data source.
  • 🎯 Why ORBIT?
  • At a Glance
  • ⭐ Help ORBIT Grow
  • 🚀 Highlights
  • 🧩 Supported Integrations
  • 🛠️ Common Use Cases
  • ⚡ Getting Started
    • 1. Explore the sandbox
    • 2. Run the Docker demo
    • 3. Install the latest release
    • 4. Build from source
  • 💬 Clients & SDKs
  • 🗃️ Chat with Your Data
  • 📐 Math & Scientific Rendering
  • 📚 Resources
  • 🧠 Need help?
  • 📄 License

---

⭐ Help ORBIT Grow

Open-source traction is the fuel that keeps ORBIT's sandbox, docs, adapters, and demos free for everyone.

• ⭐ Star + watch the repo to surface ORBIT in GitHub search and get notified when a new release drops.
• ✉️ Share the sandbox or demo video with the teammate who's still wiring SDKs by hand.
• 📣 Tell us what to build next in Discussions; upvotes there help us prioritize the roadmap you care about.

🚀 Highlights

• Unified API for 20+ AI providers – Swap between OpenAI, Anthropic, Gemini, Groq, DeepSeek, Cohere, Mistral, AWS Bedrock, Azure OpenAI, Together, xAI/Grok, and any local stack (Ollama, llama.cpp, vLLM, Hugging Face) via config only.
• Full-duplex speech-to-speech – Build voice assistants with real-time, bidirectional audio using NVIDIA's PersonaPlex integration. Natural conversations with interruptions and backchannels—no STT→LLM→TTS cascade needed.
• Adapters for every data shape – Intent + QA retrievers cover Postgres/MySQL/DuckDB/SQLite, MongoDB, Elasticsearch, HTTP APIs, and the major vector stores (Chroma, Qdrant, Pinecone, Milvus, Weaviate).
• Batteries included – Per-key RBAC, rate limits/quotas, audit logging (SQLite or MongoDB + optional Elasticsearch), content safety providers, retries/circuit breakers, and observability dashboards ship by default.
• Multimodal + widgets – Ship chat UIs via orbitchat CLI, React widget, or OpenAI-compatible API; render charts, math, and audio/video streams out of the box.
• Zero lock-in deployments – Try it in the hosted sandbox, run a Docker preset in under 60 seconds, or install the full server with TLS + GPU detection when you're ready for production.

🧩 Supported Integrations

No more glue code—connect everything through adapters and declarative config.

Inference & embeddings

• LLM providers: OpenAI, Anthropic, Google Gemini/Vertex, Groq, DeepSeek, Mistral, Cohere, AWS Bedrock, Azure OpenAI, Together, xAI, OpenRouter, IBM watsonx, plus local engines (Ollama, vLLM, llama.cpp, Hugging Face).
• Embeddings & rerankers: OpenAI, Jina, Cohere, Mistral, Ollama, llama.cpp, and any custom GGUF via the embeddings service.
• Vision/speech: Gemini, OpenAI, Anthropic, plus audio adapters documented in docs/audio/.
• Speech-to-speech: Full-duplex voice with PersonaPlex—real-time bidirectional audio, interruption handling, and backchannels.

Data & knowledge sources

• SQL + analytics: PostgreSQL, MySQL, SQL Server, DuckDB, SQLite, and intent-based adapters for customer schemas.
• NoSQL & search: MongoDB aggregation pipelines, Elasticsearch, Redis JSON/hashes.
• Vector stores: Chroma, Qdrant, Pinecone, Milvus, Weaviate (with QA + intent specializations).
• Files & APIs: PDF/Markdown/CSV ingestion, HTTP/REST connectors, and composite adapters that orchestrate multiple sources with guardrails.
• Infra glue: Redis caching, MongoDB or SQLite backends, optional Elasticsearch auditing.

🛠️ Common Use Cases

• Voice assistants & IVR – Deploy full-duplex voice agents for customer service, city hotlines, or internal helpdesks powered by PersonaPlex full-duplex speech-to-speech.
• Analytics copilots – Turn "How many signups came from APAC last week?" into SQL, MongoDB, or DuckDB queries with safety filters.
• Enterprise knowledge chat – Blend product docs, support tickets, and vector embeddings into a single chat endpoint with automatic citations.
• Provider gateways – Route traffic between OpenAI/Anthropic/local Ollama with fallbacks, budgets, and tenant-specific presets.
• Agentic workflows – Chain adapters that hit REST APIs, trigger automations, or fan out to multiple retrievers before summarizing responses.

⚡ Getting Started

1. Explore the sandbox

Browse interactive examples showcasing ORBIT's adapters, prompts, and capabilities—no install required.

2. Run the Docker demo

docker pull schmitech/orbit:basic
docker run -d --name orbit-basic -p 5173:5173 -p 3000:3000 schmitech/orbit:basic

• Auto-detects CPU/GPU via ORBIT_PRESET=auto (override with smollm2-1.7b-gpu or -cpu).
• Includes ORBIT server, orbitchat UI, and an Ollama preset so you can chat instantly.
• More options live in docker/README.md.

3. Install the latest release

• Prereqs: Python 3.12+, Node 18+, npm, and any AI-provider keys (OpenAI, Anthropic, Cohere, Gemini, Mistral, etc.).
• Optional deps: MongoDB, Redis, and a vector DB (Chroma, Qdrant, Pinecone, etc.).

curl -L https://github.com/schmitech/orbit/releases/download/v2.4.0/orbit-2.4.0.tar.gz -o orbit-2.4.0.tar.gz
tar -xzf orbit-2.4.0.tar.gz && cd orbit-2.4.0

cp env.example .env && ./install/setup.sh
source venv/bin/activate

./bin/orbit.sh start && cat ./logs/orbit.log

• To use local models with Ollama, first install it: curl -fsSL https://ollama.com/install.sh | sh, then pull a model such as ollama pull granite4:1b or another of your choice.
• Default adapters live in config/adapters/passthrough.yaml and config/adapters/multimodal.yaml; update config/ollama.yaml for model changes.
• Visit http://localhost:3000/dashboard to monitor the ORBIT server.

dashboard.mp4

The ORBIT dashboard for adapter management and monitoring.

4. Build from source

git clone https://github.com/schmitech/orbit.git
cd orbit && cp env.example .env
./install/setup.sh && source venv/bin/activate
./bin/orbit.sh start && ./bin/orbit.sh key create

• Bring your own API keys (OpenAI, Anthropic, Cohere, Gemini, Mistral, etc.).
• Works great with Ollama, vLLM, llama.cpp, Redis, MongoDB, and vector DBs.
• Check logs via tail -f ./logs/orbit.log and open http://localhost:3000/dashboard.

---

💬 Clients & SDKs

• orbit-chat CLI: pip install schmitech-orbit-client && orbit-chat --api-key <KEY>
• React web app: npm install -g orbitchat && orbitchat --api-url http://localhost:3000 --api-key <KEY> --open
• Embeddable widget: Drop @schmitech/chatbot-widget into any site for floating or inline chat.
• Node SDK: npm install @schmitech/chatbot-api and stream responses in TypeScript/JavaScript apps.
• OpenAI-compatible API: Point the official openai Python client at base_url="http://localhost:3000/v1" to reuse existing code.

orbit-cli-chat.mp4

Using the orbit-chat CLI. Run orbit-chat -h for options.

---

🗃️ Chat with Your Data

See the tutorial for step-by-step instructions on connecting your data, performing retrieval, and watching as ORBIT converts natural language into SQL queries and API calls.

ev-analytics-example.mp4

Electric Vehicle Population Data through natural language (Data Source: data.wa.gov).

---

📐 Math & Scientific Rendering

ORBIT's built-in adapters and clients support advanced rendering for mathematical notation (KaTeX) and interactive charts, perfect for educational and scientific applications.

quadratic.mp4

Demo of the Math Tutor adapter showcasing KaTeX and interactive graphing.

---

📚 Resources

• Installation guide
• Configuration reference
• Authentication & API keys
• RAG adapters
• Voice & audio adapters
• PersonaPlex speech-to-speech
• Roadmap
• Contributing & Code of Conduct

🧠 Need help?

• Open an issue or discussion.
• Commercial support + managed hosting: OrbitPods.io.
• Say hi on Schmitech.ai.

📄 License

Apache 2.0 – see LICENSE.