Design chat messages
Build clear text conversations within Router's supported message and context limits.
Use roles deliberately
| Role | Use it for |
|---|---|
system | Stable instructions, tone, boundaries, and response format guidance. |
user | The current request and user-provided context. |
assistant | Earlier model replies needed to continue a conversation. |
Router accepts between 1 and 16 messages. Message content must be text, and the combined UTF-8 message content cannot exceed 3,072 bytes. A model edition may enforce a smaller context than its upstream model family.
Keep durable instructions stable
Put product-wide behavior in one concise system message. Keep changing user context in user messages. This makes a conversation easier to inspect and reduces duplicated prompt text.
{ "model": "gemma3:1b", "messages": [ { "role": "system", "content": "You are a concise product support assistant. Say when information is missing." }, { "role": "user", "content": "Explain the difference between online providers and free capacity." } ]}Manage conversation history
Send only the history the next answer needs. For longer conversations, summarize older turns in your own application before the request reaches Router. Preserve facts, decisions, and unresolved questions in the summary.
Control output length
Set max_tokens to the smallest useful ceiling. Router’s platform ceiling is 512 output tokens, and the selected model edition can be lower. A response ending with finish_reason: "length" reached that limit and may be incomplete.
Choose a temperature
temperature accepts values from 0 through 2. Lower values usually suit extraction, support, and repeatable formatting. Higher values permit more variation. The exact effect depends on the provider runtime and model.
Next, learn how to stream the answer or inspect the chat-completions request schema.