Skip to main content
LlamaIndex’s built-in OpenAI LLM validates model names against OpenAI’s catalog, so for gateway models (grok-4-fast, claude-sonnet-4-6, auto) use OpenAILike — it skips that validation and speaks the same OpenAI API.

​
Install

pip install -U orbitrage llama-index-core llama-index-llms-openai-like

​
Setup

import os, orbitrage
orbitrage.init(os.environ["ORBITRAGE_API_KEY"], user_id="customer_42")

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="grok-4-fast",                             # direct model — or "claude-sonnet-4-6"; "auto" to route
    api_base="https://api.orbitrage.ai/v1",
    api_key=os.environ["ORBITRAGE_API_KEY"],
    default_headers={"x-orbitrage-end-user-id": "customer_42"},
    is_chat_model=True,
    context_window=8000,
    max_tokens=512,
)

print(llm.complete("Summarize what an LLM router does, in one sentence."))
Use it anywhere LlamaIndex takes an LLM — Settings.llm = llm, query engines, chat engines, agents — and every call is routed and traced.

​
Managed tools (server-side, no keys)

Pass managed tools by name as a tools kwarg to chat() — Orbitrage runs them server-side and returns the final answer. 
from llama_index.core.llms import ChatMessage

resp = llm.chat(
    [ChatMessage(role="user", content="Use the calculator to compute 1234*5678. Only the number.")],
    tools=["calculator_orbitrage", "tavily_orbitrage"],
)
print(resp)   # assistant: 7006652
If you use the standard llama_index.llms.openai.OpenAI class, it only accepts OpenAI model ids (e.g. gpt-4o-mini) — those still route through the gateway, but for grok/claude/auto prefer OpenAILike as shown above.