Custom Providers

TramAI is intentionally structured so new providers can be added without changing the public service model. The ModelProvider SPI lets you integrate any backend — proprietary APIs, internal gateways, community models, or custom inference servers.

What This Covers

• The ModelProvider SPI and its capability model
• Writing a full provider implementation
• Explicit registration and routing
• The provider transport boundary
• Provider implementation rules and module boundaries
• Proving conformance with ProviderTck

The ModelProvider SPI

All providers implement dev.tramai.core.provider.ModelProvider from tramai-core (tramai-core/src/main/kotlin/dev/tramai/core/provider/ModelProvider.kt:19):

interface ModelProvider {
    suspend fun complete(request: ModelRequest): ModelResponse

    fun providerId(): String = this::class.simpleName ?: "unknown"

    fun supportsCapability(capability: ProviderCapability): Boolean = false
}

enum class ProviderCapability {
    VISION,
    TOOL_CALLING,
    STRUCTURED_OUTPUT,
    STREAMING,
}

There is no ModelProviderAdapter interface, and there is no supportsStructuredOutput() method — structured-output support is declared through supportsCapability(ProviderCapability.STRUCTURED_OUTPUT).

Full Provider Example

package com.example.gateway

import dev.tramai.core.coroutines.rethrowIfCancellation
import dev.tramai.core.model.FinishReason
import dev.tramai.core.model.ModelRequest
import dev.tramai.core.model.ModelResponse
import dev.tramai.core.provider.ModelProvider
import dev.tramai.core.provider.ProviderCapability
import dev.tramai.core.provider.providerTransportFailure

class InternalGatewayProvider(
    private val baseUrl: String,
    private val apiKey: String,
) : ModelProvider {

    override fun providerId(): String = "internal-gateway"

    override fun supportsCapability(capability: ProviderCapability): Boolean = when (capability) {
        ProviderCapability.TOOL_CALLING,
        ProviderCapability.STRUCTURED_OUTPUT,
        -> true

        else -> false
    }

    override suspend fun complete(request: ModelRequest): ModelResponse {
        val payload = GatewayRequest(
            model = request.model,
            messages = request.messages.map { GatewayMessage(it.role.name, it.content) },
            temperature = request.temperature,
            maxTokens = request.maxTokens,
        )

        val response = try {
            post("$baseUrl/v1/chat", payload)
        } catch (e: Exception) {
            // Cancellation must propagate unchanged; everything else is sanitized
            // into a typed provider failure.
            e.rethrowIfCancellation()
            throw providerTransportFailure(providerId(), e)
        }

        return ModelResponse(
            content = response.choices.firstOrNull()?.text.orEmpty(),
            modelUsed = response.model,
            inputTokens = response.usage?.inputTokens,
            outputTokens = response.usage?.outputTokens,
            finishReason = mapFinishReason(response.choices.firstOrNull()?.finishReason),
        )
    }

    private fun mapFinishReason(raw: String?): FinishReason = when (raw) {
        "stop" -> FinishReason.STOP
        "length" -> FinishReason.LENGTH
        "content_filter" -> FinishReason.CONTENT_FILTER
        else -> FinishReason.OTHER
    }
}

Vendor DTOs (GatewayRequest, GatewayResponse, and so on) stay private to the provider module.

Failure Semantics

Never leak a raw vendor error to callers. Build failures with the factories in dev.tramai.core.provider (tramai-core/src/main/kotlin/dev/tramai/core/provider/ProviderFailures.kt):

Both transport factories call rethrowIfCancellation() first and return a sanitized ProviderException. Failure codes are the coarse taxonomy in ProviderFailureCode: HTTP_REJECTED, TIMEOUT, CONNECTION_FAILED, TRANSPORT_FAILED, UNEXPECTED_FAILURE. HTTP status, retryability, and retry timing travel on ProviderException.statusCode, .retryable, and .retryAfterMillis.

Error bodies are bounded: readErrorBodyPreview(input, limitBytes) caps the diagnostic preview at PROVIDER_ERROR_BODY_LIMIT_BYTES (8 KiB). Unbounded response-body reads are banned by verifyStaticSafetyGuards.

Registering the Provider

Providers are registered explicitly. They are not service-loaded, and resolution is registry-based: a model name maps to a provider through explicit configuration, never through a model-name prefix heuristic.

ProviderRegistry is a compatibility facade over a frozen ProviderRoutingPlan (tramai-core/.../provider/ProviderRegistry.kt:32) — every operation delegates to that exact plan instance.

val registry = ProviderRegistry.builder()
    .provider("gateway", InternalGatewayProvider(baseUrl, apiKey), default = true)
    .model("gpt-4o", "gateway")
    .build()

Standalone Builder

val tramai = Tramai {
    provider(
        InternalGatewayProvider(
            baseUrl = System.getenv("GATEWAY_URL"),
            apiKey = System.getenv("GATEWAY_API_KEY"),
        ),
        name = "gateway",
        default = true,
    )
    model("gpt-4o", "gateway")
}

Spring Boot

Expose the provider as a bean; the starter's auto-configuration picks it up.

@Bean
fun gatewayProvider(environment: Environment): ModelProvider = InternalGatewayProvider(
    baseUrl = environment.getProperty("gateway.base-url")!!,
    apiKey = environment.getProperty("gateway.api-key")!!,
)

The Transport Boundary

Transport helper APIs are annotated dev.tramai.core.provider.transport.ExperimentalProviderTransportApi and sit outside the stable compatibility promise. Existing adapters opt in at file level:

@file:OptIn(dev.tramai.core.provider.transport.ExperimentalProviderTransportApi::class)

If you use any dev.tramai.core.provider.transport.* helper, opt in the same way and treat it as experimental.

Provider Implementation Rules

A provider module is responsible for:

• Translating ModelRequest into provider-native HTTP payloads
• Mapping provider responses into ModelResponse
• Declaring capabilities accurately
• Surfacing transport and protocol failures as typed provider failures

Do not put in a provider:

• Retry policy or retry scheduling
• Fallback or failover routing
• Circuit breaking or admission control
• Token budget enforcement
• Structured-output parsing, validation, or repair logic
• Tracing policy or service proxy rules

tramai-engine's ProviderExecutionCoordinator owns retry, fallback, and admission. A provider that reimplements them double-applies policy and breaks the engine's accounting.

Proving Conformance With ProviderTck

Every provider in the repository passes ProviderTck, a deterministic offline behavioural contract: identity, capabilities, cancellation, safe-error redaction, HTTP wire behaviour, tools, vision, structured output, and streaming. All requests are answered by canned StubHttpClient responses — no network, no credentials.

class InternalGatewayProviderTckTest : ProviderTck() {
    override val harness: ProviderTckHarness = ProviderTckHarness(
        expectedProviderId = "internal-gateway",
        expectedCapabilities = setOf(ProviderCapability.TOOL_CALLING, ProviderCapability.STRUCTURED_OUTPUT),
        createProvider = { stub -> InternalGatewayProvider(stub) },
        // ... happy-path and error bodies
    )
}

ProviderTck lives in tramai-testing (tramai-testing/src/testFixtures/kotlin/dev/tramai/testing/provider/ProviderTck.kt:51), with the harness at ProviderTckHarness.kt:96-123. The harness pins expectedProviderId and expectedCapabilities, so a provider cannot make a contract test disappear by changing its own capability declaration.

If you contribute the adapter to the TramAI repository, the TCK is mandatory and architecture-enforced — see Adding a Provider.

Module Boundaries

Providers keep tramai-core as their contract dependency. config/quality/module-boundaries.yml in the TramAI repository enforces which layers a module with the provider-adapter dependency policy may reach, and config/quality/module-catalog.yml records the classification. ./gradlew verify060Architecture fails when a project is missing from the catalog or its declared dependencies violate the boundaries.

For your own out-of-tree provider module, the same principle applies: depend on the SPI, not on the runtime.

Provider Trust Zones (Sovereign Mode)

In sovereign mode each provider is assigned a trust zone, and routing is validated against it. The zone type is dev.tramai.security.ProviderTrustZone (tramai-security/src/main/kotlin/dev/tramai/security/ProviderRoutingConfiguration.kt:10-17):

Zones are configured declaratively through the sovereign profile, not through a builder property:

tramai:
  profile: sovereign
  sovereign:
    allowed-providers: [deterministic-local-provider]
    provider-zones:
      deterministic-local-provider: LOCAL

SovereignRoutingValidationPolicy rejects a route whose provider has no declared zone. For full setup, see Sovereign Mode.

Testing a Custom Provider

Use MockAiProvider and SimulatedFailureProvider from tramai-testing to test application code against your provider without network calls, and ProviderTck to prove the provider itself conforms. See Testing TramAI Code for the application-side patterns.

Before Adding Large Features

Check whether the feature belongs in a provider at all.

Good fit:

• A provider module for a public or internal API
• A TCK harness entry for a new transport shape
• Additional capability declarations

Higher-risk fit (discuss before building):

• Agent-style tool orchestration inside the provider
• Memory or session state management
• Generated code paths
• Provider-specific special cases leaking into the engine

Next Steps

• Custom Interceptors — DLP, redaction, and operation interceptors
• Custom Observers — the observation surfaces
• MCP Integration — the MCP server adapter
• Adding a Provider — the in-repository contribution path
• Providers & Routing — routing and fallback configuration
• Sovereign Mode — trust zones and the sovereign profile