Design Deep Dive

Version: 0.4.0 Implementation language: Kotlin Framework policy: Framework-agnostic core, optional Spring adapter Serialization: Jackson (with kotlin-module)

Problem Statement

Situation

Backend engineers on the JVM are increasingly asked to integrate AI capabilities into existing applications. The tooling available is either Python-native (LangChain, LlamaIndex) or a direct port of Python idioms into Java (LangChain4j) — neither of which feels idiomatic to a Kotlin/Java engineer who thinks in types, interfaces, and coroutines.

Complication

LangChain4j exists but inherits LangChain's mental model: chains, agents, and prompt templates as first-class abstractions. This model has known production problems — hard to test, hard to observe, leaky abstractions that expose model internals to application code. Structured outputs are an afterthought. Observability requires manual instrumentation.

Question

Can we build a JVM-native AI library where typed interfaces are the only abstraction, structured outputs are the default contract, and every AI interaction is automatically observable — with zero boilerplate?

Answer

Yes. TramAI is that library.

Core Abstraction

The single unit of TramAI is the AI Operation — an annotated Kotlin interface method.

@AiService
interface InvoiceAnalyzer {

    @Operation(
        prompt = "Analyze this invoice and extract line items with cost categories",
        model = "claude-sonnet-4-20250514"
    )
    suspend fun analyze(invoice: Invoice): InvoiceAnalysis
}

Rules

• Interface must be annotated with @AiService
• Each method must be annotated with @Operation
• Method parameters are serialized as structured context
• Return type determines behavior:
  • String -> raw model output, no parsing
  • Unit -> fire-and-forget
  • Any data class or POJO -> structured output with schema validation and retry
  • suspend fun -> coroutine-native, non-blocking
  • Blocking methods -> supported for Java consumers

Structured Output Pipeline

This is the core intellectual work of the library.

1. SCHEMA GENERATION
   Java return type -> JSON Schema
   Uses Jackson SchemaGenerator + custom annotations
   Schema is cached per method at startup

2. PROMPT CONSTRUCTION
   User prompt + input serialization + schema injection
   Schema injected as system instruction: "Respond ONLY with valid JSON matching this schema"

3. MODEL CALL
   Request dispatched to ModelProvider
   Raw text response captured

4. RESPONSE PARSING
   Raw text -> JSON extraction (handles markdown fences, preamble)
   JSON -> Jackson deserialization into return type
   Bean Validation (JSR-380) applied if validator on classpath

5. VALIDATION FEEDBACK LOOP (if parsing fails)
   Validation error appended to conversation
   Re-sent to model, up to @Operation(maxRetries = N)
   Default: 2 retries

6. FAILURE
   After maxRetries exhausted: throw TramaiStructuredOutputException
   Exception contains: original prompt, last raw response, validation error, attempt count

Schema Annotations

data class SpendAnalysis(
    @AiDescription("Total spend in USD, always positive")
    val totalSpend: Double,

    @AiDescription("List of cost reduction recommendations, ordered by impact")
    @AiMinItems(1)
    val recommendations: List<Recommendation>,

    @AiDescription("Confidence score between 0.0 and 1.0")
    @AiRange(min = 0.0, max = 1.0)
    val confidence: Double
)

Provider Model

Provider Interface

interface ModelProvider {
    suspend fun complete(request: ModelRequest): ModelResponse
    fun supportsStructuredOutput(): Boolean = false
    fun providerId(): String = this::class.simpleName ?: "unknown"
}

Provider Resolution

• providers are registered explicitly
• models are registered explicitly against providers
• explicit override via @Operation(provider = "ollama") remains available
• unknown or unregistered models produce a deterministic resolution error

Native Structured Output

If the provider supports native structured output (OpenAI's response_format, Anthropic's tool-use trick), TramAI uses it. If not, TramAI falls back to the schema-in-prompt pipeline. supportsStructuredOutput() controls this routing.

Retry and Error Handling

Retry Hierarchy

Exception Hierarchy

TramaiException (base, unchecked)
-> TramaiStructuredOutputException   — exhausted retries on parsing
-> TramaiProviderException           — provider returned unrecoverable error
-> TramaiConfigurationException      — misconfiguration detected at startup
-> TramaiTimeoutException            — call exceeded configured timeout

Prompt Rendering

Template syntax maps to parameter names:

@Operation(prompt = "Analyze spend data for tenant {{tenantId}}")
suspend fun analyze(tenantId: String, data: SpendRecord): SpendAnalysis

For multi-turn prompts, @SystemPrompt at the interface level:

@AiService
@SystemPrompt("You are a cost optimization expert. Be concise and actionable.")
interface SpendAnalyzer

Framework Integration

TramAI has no framework dependency in its core. Framework adapters are thin modules that wire TramAI's standalone API into the framework's DI and configuration systems.

• Spring Boot: TramaiAutoConfiguration scans for @AiService interfaces, registers proxies as beans, reads application.yml config
• Quarkus/Micronaut: Not shipped in v1. Standalone API works in both — instantiate TramAI manually and register as a CDI bean or singleton
• Plain JVM/CLI: No framework, no classpath scanning, no magic. TramAI is a library, not a container.

Testing Support

The tramai-testing module provides:

• MockAiProvider — canned responses, no network calls
• SimulatedFailureProvider — retryable and non-retryable failures
• RecordingOperationObserver — verify call counts and behavior
• TramaiAssertions — fluent assertion API

Related Pages

• Architecture Overview — high-level architecture
• Module Design — module boundaries
• Why TramAI — comparison to alternatives