TramAI - governed AI workflows for Java and Kotlin

Production Hardening & Security

TramAI is built for production environments where cost control, data privacy, and reliability matter. This guide covers the hardening mechanisms documented for the 0.6.0 line.

What This Covers

  • Runtime selection: tramai.profile and the sovereign profile
  • PII masking and data redaction with OperationInterceptor
  • DLP interceptors for model output and tool result scanning
  • Secret management with SecretValueResolver
  • Token budgets for cost control
  • Circuit breakers, retry pacing, and fallback routing
  • The 0.6.0 hardening guarantees: bounded reads, SSRF admission, sanitized storage failures, static-safety guards, cancellation propagation
  • Sovereign Mode for hardened, auditable deployments

Runtime Selection

One starter, two profiles. tramai-spring-boot-starter aggregates the standard and sovereign Spring integrations, and tramai.profile chooses which one runs:

tramai:
  profile: standard     # default; omit the property for the same result

tramai:
  profile: sovereign
  sovereign:
    allowed-models: [local-invoice-model]
    allowed-providers: [ollama]
    provider-zones:
      ollama: LOCAL
    models:
      local-invoice-model: ollama

Unsupported values fail at startup rather than falling back: Unsupported tramai.profile '<value>'. Supported values: standard, sovereign. The former standalone sovereign starter coordinate (tramai-spring-boot-starter-sovereign) is gone — there is no second dependency to add, and tramai.sovereign.enabled: false is rejected because the profile is the sole selector. See the Sovereign Runtime Quickstart.

0.6.0 Hardening Guarantees

These are enforced properties of the runtime, not opt-in features. They are the reason several failure modes now look different than they did in 0.4.x.

Bounded reads

Provider response bodies and downloaded payloads are read through bounded helpers (dev.tramai.core.provider.transport.readBoundedBodyBytes and friends) that fail loudly when the source exceeds the limit instead of silently truncating. The stream is always closed, so a hostile or broken endpoint cannot exhaust memory or leak a connection.

SSRF admission for image retrieval

Multimodal image retrieval (dev.tramai.core.util.ImageDownloader, backed by OutboundAddressSecurity) admits only what it can justify:

  • only http and https schemes — file:, jar:, data:, and friends are rejected
  • the host is canonicalized (IDN to ASCII, alternative IPv4 notations parsed) and resolved before connecting
  • loopback, link-local, RFC 1918 private, carrier-grade NAT, unique-local IPv6, multicast, and any-local addresses are rejected, as is localhost
  • URLs carrying embedded user info (http://user:pass@host/) are rejected
  • every redirect target is re-validated, with a redirect cap
  • the payload is capped at 20 MB per image, and the connection is disconnected on every path

Treat this as the reference implementation for any outbound call you add around TramAI: validate, then connect, then bound.

Sanitized storage failures

JDBC stores wrap database operations so raw driver diagnostics do not reach your application. A java.sql.SQLException becomes IllegalStateException with redacted details; coroutine CancellationException and TramAI domain exceptions (TramaiException, approval-store exceptions, IllegalArgumentException) propagate unwrapped.

Migration note: stop catching java.sql.SQLException around store calls. Catch IllegalStateException or a general RuntimeException instead — the underlying SQL text, table names, and connection details are no longer in the message.

Static safety guards

The build fails closed on four classes of production-code defect: raw coroutine-scope creation, unbounded response-body reads, logging of sensitive payloads, and System.out / System.err usage. If you are building an extension inside the TramAI repository, verifyStaticSafetyGuards is the gate you will hit; in your own application, the same rules are worth applying directly.

Cancellation propagation

CancellationException is preserved and propagated on every path — provider execution, tool coordination, workflow steps, storage. If you write catch (e: Exception) around TramAI calls, rethrow cancellation first:

import dev.tramai.core.coroutines.rethrowIfCancellation

try {
    service.analyse(input)
} catch (e: Exception) {
    e.rethrowIfCancellation()   // never swallow coroutine cancellation
    // handle the real failure
}

A cancelled call must not be converted into a retry, a fallback, a circuit-breaker count, or an approval failure.

Internal and preview boundaries

Symbols annotated @ExperimentalTramaiInternalApi or @ExperimentalProviderTransportApi are internal cross-module contracts, excluded from the stable compatibility guarantee. Reference them only if you are prepared to track them.

The same caution applies to policy presets: PolicyConfiguration.secure() is the deny-by-default baseline and the correct production posture. PolicyConfiguration.preview() is a 0.4.x migration/testing preset — it uses wildcard allowlists — and must not be shipped to production.

PII Masking with OperationInterceptor

When to use: You need to redact or transform sensitive data in request messages before they leave the JVM.

Minimum setup: Implement OperationInterceptor and register it on your TramAI builder or as a Spring bean.

The OperationInterceptor interface has two hooks:

  • interceptRequest(...) — inspect and modify messages before provider transport
  • interceptResponse(...) — inspect and modify provider responses before the engine processes them

Standalone

val tramai = Tramai {
    provider(OpenAiProvider(System.getenv("OPENAI_API_KEY")), name = "openai", default = true)
    model("gpt-4o", "openai")

    interceptor(object : OperationInterceptor {
        override fun interceptRequest(
            context: OperationCallContext,
            messages: List<Message>,
        ): List<Message> = messages.map { message ->
            message.copy(content = redactPii(message.content))
        }
    })
}

Spring Boot

Spring auto-configuration composes ordered OperationInterceptor beans into the generated TramAI instance.

@Configuration
class TramaiSecurityConfiguration {
    @Bean
    fun piiMaskingInterceptor(): OperationInterceptor = object : OperationInterceptor {
        override fun interceptRequest(
            context: OperationCallContext,
            messages: List<Message>,
        ): List<Message> = messages.map { message ->
            message.copy(content = maskSensitiveData(message.content))
        }
    }
}

Current limitations:

  • interceptors are engine-level request/response hooks
  • they are opt-in — no interceptor is applied by default
  • Spring auto-configuration composes registered interceptor beans in order via CompositeOperationInterceptor

DLP (Data Loss Prevention)

When to use: You need to scan model outputs and tool results for sensitive data before they reach the caller, structured parser, or cache.

Minimum setup: Register a DlpInterceptor on the TramAI builder or as a Spring bean.

The DLP system inspects model outputs and tool results after the provider returns but before the result reaches the caller or structured parser. The SPI is a single-function interface:

fun interface DlpInterceptor {
    fun inspect(context: DlpContext, text: String): DlpResult
}

Misconfiguration example — two DlpInterceptor beans will cause Spring auto-configuration to throw at startup. Define exactly one.

For complete DLP documentation including RuleBasedDlpInterceptor, DlpRedactionAuditEmitter, rule configuration, and cache bypass — see the dedicated DLP guide.

Secret Management

When to use: You need to resolve provider credentials without hard-coding them in application code.

Minimum setup: Use the built-in env: and file: resolvers, or implement SecretValueResolver for cloud secret stores.

Built-in Resolvers

  • env:NAME — resolves from environment variables
  • file:/path/to/secret.txt — resolves from a file on disk

Standalone

val secretResolver = CompositeSecretValueResolver(
    listOf(
        SecretValueResolver { secretRef ->
            if (!secretRef.startsWith("vault:")) null
            else vaultClient.read(secretRef.removePrefix("vault:"))
        },
        EnvironmentSecretValueResolver,
        FileSecretValueResolver,
    ),
)

val tramai = Tramai {
    provider(
        OpenAiProvider(
            apiKey = secretResolver.resolve("vault:providers/openai/api-key")
                ?: error("Missing OpenAI API key"),
        ),
        name = "openai",
        default = true,
    )
    model("gpt-4o", "openai")
}

Spring Boot

Spring Boot auto-configuration composes SecretValueResolver beans. Reference secrets with *-secret-ref properties:

tramai:
  providers:
    openai:
      api-key-secret-ref: vault:providers/openai/api-key

@Bean
fun vaultSecretValueResolver(): SecretValueResolver = SecretValueResolver { secretRef ->
    if (!secretRef.startsWith("vault:")) null
    else vaultClient.read(secretRef.removePrefix("vault:"))
}

Current limitations:

  • the built-in standalone resolvers cover env: and file:
  • cloud secret stores need a resolver implementation of your own, or the Spring starters below
  • in Spring Boot, tramai-spring-secrets-file, tramai-spring-secrets-vault, and tramai-spring-secrets-aws ship ready-made resolvers; each is a separate opt-in dependency

Token Budgets

When to use: You need to cap token consumption per attempt or per logical operation to control cost.

Minimum setup: Configure TokenBudgetSettings on the engine.

Available Controls

  • hardMaxTokensPerAttempt — hard cap per single provider call
  • hardMaxTokensPerOperation — hard cap across the entire logical operation (including retries and tool loops)
  • softMaxTokensPerOperation — emits a warning event when crossed, does not fail

Standalone

val tramai = Tramai {
    provider(OpenAiProvider(System.getenv("OPENAI_API_KEY")), name = "openai", default = true)
    model("gpt-4o", "openai")

    tokenBudget(
        TokenBudgetSettings(
            hardMaxTokensPerAttempt = 4_000,
            hardMaxTokensPerOperation = 20_000,
            softMaxTokensPerOperation = 10_000,
        ),
    )
}

Spring Boot

tramai:
  cost:
    token-budget:
      hard-max-tokens-per-attempt: 4000
      hard-max-tokens-per-operation: 20000
      soft-max-tokens-per-operation: 10000

Practical notes:

  • budgets apply across retries
  • structured-output retries count toward the operation total
  • tool-call loops count toward the operation total
  • soft limit crossing emits an engine event instead of failing the call

Resilience Controls

When to use: You need fallback models, circuit breakers, and retry pacing for reliable provider access.

Fallback Routing

Fallback routes are explicit ordered routes for a requested model.

Circuit Breaking

Prevents repeated calls into an unhealthy provider after a configurable failure threshold.

Retry Pacing

Exponential backoff with jitter, honoring provider Retry-After hints.

Standalone

val tramai = Tramai {
    provider(OpenAiProvider(System.getenv("OPENAI_API_KEY")), name = "openai", default = true)
    provider(AnthropicProvider(System.getenv("ANTHROPIC_API_KEY")), name = "anthropic")

    model("gpt-4o", "openai")
    model("claude-sonnet-4-20250514", "anthropic")

    fallbackModel("gpt-4o", "claude-sonnet-4-20250514", "anthropic")

    circuitBreaker(
        CircuitBreakerSettings(
            enabled = true,
            failureThreshold = 3,
            openDurationMillis = 30_000,
        ),
    )
    retryPolicy(
        RetryPolicySettings(
            maxRetryAfterMillis = 20_000,
            jitterRatio = 0.1,
        ),
    )
}

Spring Boot

tramai:
  models:
    gpt-4o: openai
    claude-sonnet-4-20250514: anthropic
  fallbacks:
    gpt-4o:
      - provider: anthropic
        model: claude-sonnet-4-20250514
  resilience:
    circuit-breaker:
      enabled: true
      failure-threshold: 3
      open-duration-millis: 30000
    retry:
      max-retry-after-millis: 20000
      jitter-ratio: 0.1

Current limitations:

  • fallback routing is explicit, not heuristic
  • streaming failover is only allowed before the first emitted token
  • once a stream has emitted user-visible output, TramAI returns a terminal error rather than stitching providers together

Sovereign Mode

When to use: You need a hardened, auditable deployment profile with deny-by-default policy, approved-model routing, classification-aware provider trust zones, and hash-chained audit.

In 0.6.0 Sovereign Mode is a profile of the same starter — tramai.profile: sovereign plus the tramai.sovereign.* allowlist block — and the embedded equivalent is the SovereignTramai builder. Either path enforces deny-by-default policy, fail-fast validation, and append-only hash-linked audit emission.

For the dependency list, the full property reference, and the startup error codes, see the Sovereign Runtime Quickstart; for the builder API and trust zones, see the Sovereign Mode guide.

Next Steps

Previous
Next