Inference
All runtimes
CiphraX OS · Runtime
Inference
Any model, routed and governed.
A model-agnostic intelligence layer that routes, streams and governs every inference — across any provider or your own models.
Streaming-firstAutomatic failoverMulti-tenantAudit-ready by default
Capabilities.
Models
- GPT
- Claude
- Gemini
- DeepSeek
- Local models
Inference
- Streaming
- Structured outputs
- Tool calling
- Embeddings & reranking
Runtime
- Model routing
- Automatic failover
- Cost optimisation
Engineering
- Prompt versioning
- Model A/B testing
How it works.
Request→Policy→Model router→Selected model→Tool calls→Memory→Response
Streaming, full-duplex and policy-checked end to end — under 1.5 seconds.
Providers.
Azure OpenAI
OpenAI
AnthropicGemini
Mistral
Groq
DeepSeek+ 20 more
Deploy anywhere.
Azure
AWS
Google CloudPrivate cloudOn-premAir-gappedIn your region
AWSBuild with it.
PythonTypeScriptRESTWebSocketOpenAPI
Deploy the Inference module on CiphraX.
One module inside the CiphraX runtime.