Model publishing runbook
Publishing a model can change user-visible behavior, provider spend, feature eligibility, and routing. Treat model publication as a controlled platform change.
1. Validate provider mapping
Confirm:
- provider
- provider model identifier
- authentication/configuration health
- route endpoint/configuration
- expected region/deployment when relevant
Set only capabilities the route actually supports:
- context/output limits
- streaming
- reasoning
- vision
- tool use
- structured output
- embeddings
- Research/FORGE eligibility
Do not infer capability from model branding alone.
Decide which plan/rollout categories can see the model. Publishing globally is not required for a new model test.
Review provider-cost metadata and Starfire usage/credit configuration before exposing the model broadly.
Define primary route, timeout/retry behavior, fallback eligibility, and health controls.
Fallback routes must preserve required capabilities.
6. Test
Run representative checks for:
- ordinary generation
- streaming
- large context near configured limits
- tools when supported
- vision when supported
- structured output when supported
- Research/FORGE eligibility when enabled
- error handling and fallback
7. Stage rollout
Prefer a narrow internal/experimental cohort before broad plan availability when the model is new or provider behavior is not well understood.
8. Monitor after publish
Watch latency, success/error rate, provider throttling, fallback frequency, usage, and cost.
Never publish provider secrets or internal credentials in model descriptions, public docs, or user-visible metadata.
