Python
Connect the OpenAI Python client to Router and handle a complete response or stream.
Install the client
python -m pip install openaiSet ROUTER_BASE_URL to the Router endpoint ending in /v1, and store your API key in ROUTER_API_KEY.
Create a client
import osfrom openai import OpenAI
client = OpenAI( base_url=os.environ["ROUTER_BASE_URL"], api_key=os.environ["ROUTER_API_KEY"], timeout=45.0, max_retries=0,)This example disables automatic SDK retries so your application can make retry policy explicit. Add bounded retry behavior only for the temporary failures your product can safely repeat.
List available editions
models = client.models.list()
for model in models.data: print(model.id)The raw Router model payload also contains edition limits and provider counts. If your client version hides extension fields, inspect model.model_dump().
Create a response
response = client.chat.completions.create( model="gemma3:1b", messages=[ {"role": "system", "content": "Answer clearly and briefly."}, {"role": "user", "content": "What is decentralized inference?"}, ], max_tokens=160, temperature=0.4,)
print(response.choices[0].message.content)print(response.usage)Stream a response
stream = client.chat.completions.create( model="gemma3:1b", messages=[{"role": "user", "content": "Write a two-line welcome."}], stream=True,)
completed = False
for chunk in stream: if not chunk.choices: continue choice = chunk.choices[0] if choice.delta.content: print(choice.delta.content, end="", flush=True) if choice.finish_reason is not None: completed = True
print()if not completed: raise RuntimeError("The response stream ended before completion")See retries and cancellation for application-level recovery.