Architecture Overview

TramAI is a structured-first, observability-native AI workflow library for the JVM. The core product goal is to let application code interact with AI through typed interface methods instead of framework-specific chains, agents, or prompt orchestration objects.

This page describes the runtime shape. For the module-level picture — layers, ownership, and the machine-readable catalog — see Module Architecture.

Core Flow

Application code defines an @AiService interface.
TramAI generates a proxy for that interface at startup.
A method call is translated into an AI operation.
Inputs are rendered into model messages.
If the return type is structured, TramAI generates or loads a cached schema and validates the response.
Provider resolution selects the primary route and any configured fallbacks.
The provider call is wrapped in observability instrumentation when OpenTelemetry is available.
The provider returns raw model output plus metadata.
TramAI either returns raw text, streams chunks, or parses and validates structured output.
Parse failures enter a retry loop with validation feedback.

Ownership of each stage is stable and documented: the engine owns invocation, provider admission, tool lifecycle, and retry; tramai-structured owns schema generation, extraction, and structured failure analysis; provider adapters own transport and vendor translation only.

Layer Stack

In a Spring Boot application, TramAI adds AI as a 4th architectural layer:

Presentation Layer   @Controller / @RestController  — HTTP endpoints
Business Logic       @Service                        — Application logic
AI Layer             @AiService                      — Typed AI contracts  ← NEW
Persistence          @Repository / @Entity           — Data access

The @AiService annotation marks an interface as an AI service contract.

With the unified tramai-spring-boot-starter (tramai-spring-core + tramai-spring-sovereign), annotated interfaces become Spring beans through auto-configuration; @EnableTramai is not required for that. It remains a way to select a Spring model, not a runtime profile — the runtime profile is chosen with tramai.profile.

In standalone mode (no Spring), the same typed contract pattern applies via Tramai { ... } and Tramai.create<Service>() — the engine-owned infrastructure still exists, just without the DI container. TramaiEngine is AutoCloseable; close it so engine-owned coroutine scopes and resources are released.

Major Layers

Application Layer
    @AiService interfaces — any framework or plain JVM

Proxy Layer
    JDK dynamic proxy generated at startup
    TramaiInvocationHandler intercepts method calls, routes to the engine

Operation Engine
    Operation planning | Prompt rendering | Provider execution | Structured output | Retry

Governance Layer
    Policy enforcement points, DLP, approval suspension, audit, evidence

Observability Layer
    OTel span wraps every provider attempt
    GenAI semantic conventions, auto-detected

Provider Layer
    OpenAI | Anthropic | Ollama | Gemini | Bedrock | Azure OpenAI | DeepSeek

Application Layer

Consumer-defined @AiService interfaces. This is the only API surface most developers interact with.

Proxy Layer

Runtime-generated implementations that intercept method calls and transform them into AI operations. Uses JDK dynamic proxies through java.lang.reflect.Proxy.

Operation Engine

Operation planning, prompt rendering, provider dispatch, streaming, tool calling, parsing, retry, circuit breaking, caching, and token budgets all live in tramai-engine. Schema generation and structured response handling live in tramai-structured, whose contracts the engine consumes.

Governance Layer

Policy checks, DLP and redaction, approval gates, replay safety, audit, and evidence. Approval suspension stores a durable continuation and raises ApprovalSuspendedException; resume goes through the stored approval record rather than trusting a raw replay payload.

Observability Layer

OpenTelemetry spans, metrics, and semantic attributes, auto-detected when OTel is on the classpath. tramai-observability is optional and stays off the core happy path.

Provider Layer

Implementations for OpenAI (and OpenAI-compatible endpoints), Anthropic, Ollama, Azure OpenAI, Bedrock, Gemini, and DeepSeek. Each is a separate opt-in module. Providers are transport adapters: they must not implement retries, fallbacks, or circuit-breaker logic.

Orchestration Layer (Optional)

Typed workflow coordination above the engine, in tramai-orchestration: aiStep, localStep, gateStep, branchStep, parallelStep, external-step shapes, and checkpoint/resume. It composes the core runtime — it does not replace the engine as the owner of provider execution, retries, or tool loops.

Architectural Priorities

Typed contracts over prompt engineering — Application code never manipulates raw prompt strings
Framework-agnostic core with optional adapters
Automatic observability with OpenTelemetry semantic conventions
Kotlin-first APIs with Java-compatible entry points
Explicit module boundaries so higher-level runtime features do not leak into the core library contract

Design Philosophy

Typed contracts over prompt engineering — Inputs and outputs are typed data classes, not raw strings
Observable by default — Every AI call emits an OpenTelemetry span automatically
Fail loudly, recover gracefully — Structured output failures are retried with validation feedback; unsafe governance paths fail closed
Boring to use, interesting to build — Minimal consumer API surface
Kotlin-native, Java-friendly — Idiomatic Kotlin APIs with Java-callable entry points everywhere

Core Library Boundaries

The core @AiService runtime deliberately does not turn application code into an autonomous agent framework. In particular, the core does not handle:

autonomous agent loops
hidden cross-step orchestration
peer-to-peer agent chat
fine-tuning workflows

Where those concerns exist in this repository, they live in optional modules above the core library surface. The design goal is composability: application teams can use typed AI services alone, or add memory, RAG, orchestration, scheduling, server, MCP, or platform layers without changing the core execution model.

Module Structure

TramAI 0.6.0 ships 60 Gradle projects across 10 layers. The authoritative inventory is config/quality/module-catalog.yml, verified against settings.gradle.kts on every build — this page deliberately does not keep a second copy of it.

The 10 layers, in conceptual dependency order:

Layer	What lives there
`core-contracts`	`tramai-core` (annotations, request models, provider registry, exceptions), `tramai-bom`
`runtime-execution`	`tramai-engine`, `tramai-structured`, `tramai-orchestration`, `tramai-standalone`
`governance-security`	`tramai-security`, `tramai-sovereign`
`persistence`	`tramai-persistence-file`, `tramai-persistence-jdbc`
`provider-adapters`	the seven provider modules, behind the provider SPI
`framework-integrations`	`tramai-spring-core`, the starters, provider starters, secrets
`operations-observability`	`tramai-observability`, `tramai-platform`, and the server/MCP/dashboard surfaces
`higher-capabilities`	memory, RAG, embeddings, scheduler, vector stores
`applications-examples`	executable examples, excluded from the release
`testing-support`	TCKs, fakes, and consumer boundary tests

See Module Architecture for layer responsibilities, the exact boundary rules, and how ./gradlew verify060Architecture enforces them, or Module Design for the consumer-facing "which modules do I add" view.

Module Architecture — layers, catalog, and boundary enforcement
Module Design — module selection and dependency direction
Design Deep Dive — problem statement, design decisions, detailed architecture
API and Binary Compatibility — what the public surface guarantees
Quality Gates — how the architecture is kept honest
Why TramAI? — comparison to alternatives