Providers and Model Routing

What it is: TramAI supports multiple AI model providers with a unified API. Provider resolution follows a deterministic chain: operation override, model mapping, then default provider.

When to use it: Every TramAI application needs at least one provider configured. Use multiple providers when different operations need different models, you want fallback routing, or you need to switch between local and cloud models.

Provider Decision Guide

Not sure which provider to choose? Match your use case:

Need	Recommended Provider	Why
Fast prototyping	`OpenAiProvider`	Broadcast model support, easiest setup, best ecosystem
Complex reasoning	`AnthropicProvider`	Claude excels at structured reasoning, long context, and nuanced tasks
Production grade	`AnthropicProvider` or `OpenAiProvider`	Both are production-tested with high reliability
Privacy / offline	`OllamaProvider`	Runs 100% locally. Zero data leaves your machine
Budget-friendly	`DeepSeekProvider`	Competitive quality at lower cost via DeepSeek API
Google ecosystem	`GeminiProvider`	Native GCP integration, strong multimodal capabilities
AWS native	`BedrockProvider`	Managed access through AWS IAM, no API key management
Azure AD / enterprise	`AzureOpenAiProvider`	Azure OpenAI with managed identity and compliance controls
Custom gateway	`OpenAiCompatibleProvider`	Connect to internal AI gateways, self-hosted proxies, or any OpenAI-compatible endpoint

How Routing Works

TramAI resolves which provider to call in this exact order:

Operation override: If @Operation(provider = "anthropic") is set, use that provider directly (skips model mapping)
Model mapping: Look up the model name in the registry (e.g., "claude-sonnet-4-20250514" maps to "anthropic")
Default provider: Use the provider marked default = true

Resolution Example

val tramai = Tramai {
   provider(OpenAiProvider(key), name = "openai", default = true)
   provider(AnthropicProvider(key), name = "anthropic")
   model("gpt-4o", "openai")
   model("claude-sonnet-4-20250514", "anthropic")
}

// Operation A: @Operation(model = "gpt-4o") → resolves to openai
// Operation B: @Operation(model = "claude-sonnet-4-20250514") → resolves to anthropic
// Operation C: @Operation(model = "unknown-model") → falls back to default provider (openai)
// Operation D: @Operation(model = "gpt-4o", provider = "anthropic") → explicit override → anthropic

Minimum Setup

val tramai = Tramai {
   provider(
       OpenAiProvider(apiKey = System.getenv("OPENAI_API_KEY")),
       name = "openai",
       default = true,
   )
   model("gpt-4o", "openai")
}

That's one provider, one model mapping. Every operation defaults to gpt-4o on OpenAI.

All Supported Providers

OpenAI (`tramai-openai`)

The standard choice for most applications. Supports GPT-4o, GPT-4o-mini, o-series models, and DALL-E through image APIs.

// API key
provider(
   OpenAiProvider(apiKey = System.getenv("OPENAI_API_KEY")),
   name = "openai",
)

// Bearer token
OpenAiProvider.bearerToken(
   bearerToken = System.getenv("OPENAI_BEARER_TOKEN"),
)

// Codex auth (experimental)
OpenAiProvider.codexAuth()

Dependency: dev.tramai:tramai-openai:0.3.1

Anthropic (`tramai-anthropic`)

Claude models for complex reasoning, long context, and structured tasks.

provider(
   AnthropicProvider(apiKey = System.getenv("ANTHROPIC_API_KEY")),
   name = "anthropic",
)

Dependency: dev.tramai:tramai-anthropic:0.3.1

Gemini (`tramai-gemini`)

Google's Gemini models with native GCP integration and strong multimodal support.

provider(
   GeminiProvider(apiKey = System.getenv("GEMINI_API_KEY")),
   name = "gemini",
)

Dependency: dev.tramai:tramai-gemini:0.3.1

DeepSeek (`tramai-deepseek`)

DeepSeek models at competitive pricing. Built on OpenAiCompatibleProvider.

provider(
   DeepSeekProvider(apiKey = System.getenv("DEEPSEEK_API_KEY")),
   name = "deepseek",
)

Dependency: dev.tramai:tramai-deepseek:0.3.1

Ollama (`tramai-ollama`)

Run models locally with Ollama. Perfect for development, testing, and privacy-sensitive workloads.

provider(
   OllamaProvider(baseUrl = "http://localhost:11434"),
   name = "ollama",
)

Dependency: dev.tramai:tramai-ollama:0.3.1

Bedrock (`tramai-bedrock`)

AWS Bedrock with IAM-based access. No API key to manage — uses AWS credentials from the environment.

provider(
   BedrockProvider(region = "us-west-2"),
   name = "bedrock",
)

Dependency: dev.tramai:tramai-bedrock:0.3.1

Azure OpenAI (`tramai-azure-openai`)

Azure OpenAI Service with resource-based endpoints and managed identity support.

provider(
   AzureOpenAiProvider(
       resourceName = "my-resource",
       deploymentId = "gpt-4o",
       apiKey = System.getenv("AZURE_OPENAI_API_KEY"),
   ),
   name = "azure-openai",
)

Dependency: dev.tramai:tramai-azure-openai:0.3.1

OpenAiCompatibleProvider (`tramai-openai`)

Connect to any service implementing the OpenAI Chat Completions API format — internal AI gateways, self-hosted proxies, or specialized vendors.

// Bearer token auth
provider(
   OpenAiCompatibleProvider.bearerToken(
       bearerToken = System.getenv("GATEWAY_TOKEN"),
       baseUrl = "https://your-gateway.internal/v1",
       providerName = "internal-ai",
   ),
   name = "internal-ai",
)

// API key auth
OpenAiCompatibleProvider(
   baseUrl = "https://your-gateway.internal/v1",
   apiKey = System.getenv("GATEWAY_KEY"),
   providerName = "internal-ai",
)

Dependency: dev.tramai:tramai-openai:0.3.1

Fallback Routing

When the primary provider fails (network error, rate limit, model unavailable), TramAI can fall back to an alternative route:

val tramai = Tramai {
   provider(OpenAiProvider(key), name = "openai", default = true)
   provider(AnthropicProvider(key), name = "anthropic")

   model("gpt-4o", "openai")
   model("claude-sonnet-4-20250514", "anthropic")

   // Fallback: gpt-4o → claude-sonnet-4-20250514 on anthropic
   fallbackModel("gpt-4o", "claude-sonnet-4-20250514", "anthropic")

   // Alternative: keep the same model on a different provider
   fallbackProvider("gpt-4o", "openai-fallback")
}

Fallback happens only at operation startup (before the first token for streaming). Engine handles the retry scheduling, circuit breaker, and jitter automatically.

Kotlin + Java Examples

Kotlin

val tramai = Tramai {
   provider(OpenAiProvider(System.getenv("OPENAI_API_KEY")), name = "openai")
   provider(AnthropicProvider(System.getenv("ANTHROPIC_API_KEY")), name = "anthropic")
   provider(OllamaProvider("http://localhost:11434"), name = "ollama")

   model("gpt-4o", "openai")
   model("claude-sonnet-4-20250514", "anthropic")
   model("llama3.2", "ollama")

   fallbackModel("gpt-4o", "claude-sonnet-4-20250514", "anthropic")
   defaultProvider = "openai"
}

Java

Tramai tramai = Tramai.builder()
   .provider(new OpenAiProvider(System.getenv("OPENAI_API_KEY")), "openai", true)
   .provider(new AnthropicProvider(System.getenv("ANTHROPIC_API_KEY")), "anthropic", false)
   .model("gpt-4o", "openai")
   .model("claude-sonnet-4-20250514", "anthropic")
   .fallbackModel("gpt-4o", "claude-sonnet-4-20250514", "anthropic")
   .build();

Environmental Provider Selection

Swap providers based on environment:

val tramai = Tramai {
   val (provider, name) = if (isDevEnvironment()) {
       OllamaProvider("http://localhost:11434") to "ollama"
   } else {
       OpenAiProvider(System.getenv("OPENAI_API_KEY")) to "openai"
   }
   provider(provider, name = name, default = true)
}

Provider Configuration Cheat Sheet

Provider	Auth	Key Config
OpenAI	`apiKey` or `bearerToken`	`System.getenv("OPENAI_API_KEY")`
Anthropic	`apiKey`	`System.getenv("ANTHROPIC_API_KEY")`
Gemini	`apiKey`	`System.getenv("GEMINI_API_KEY")`
DeepSeek	`apiKey`	`System.getenv("DEEPSEEK_API_KEY")`
Ollama	`baseUrl`	`"http://localhost:11434"`
Bedrock	AWS credentials (env/instance profile)	`region = "us-west-2"`
Azure OpenAI	`apiKey` + `resourceName` + `deploymentId`	`System.getenv("AZURE_OPENAI_API_KEY")`
OpenAiCompatible	`bearerToken` or `apiKey` + `baseUrl`	Custom

Limitations

No dynamic provider selection: Provider resolution happens at call time based on the configured registry. There is no runtime provider selector that changes mid-operation.
No multi-provider aggregation: Each call goes to exactly one provider. No ensemble or voting across providers.
No provider-native streaming fallback: Fallback applies to the startup phase only. Once streaming starts, no failover occurs.
Provider-specific features: Provider-specific capabilities (thinking tokens, tool-use, image understanding) are available through the unified API but may have provider-specific behaviors.

Next Steps

Structured Output — Extract typed results from any provider
Tool Calling — Add typed tool execution to your operations
Spring Boot — Configure providers in YAML
Testing — Mock providers for deterministic testing