Module: tramai-observability

One-liner: Optional, opt-in OpenTelemetry integration — traces, metrics, and events for operations and workflows. Module type: observabilityGroup: dev.tramai, Version: 0.6.0Source files: 4 (dev.tramai.observability), LOC: 775 — plus 8 test files Optional: nothing depends on this module; it is never pulled transitively. Add it explicitly and provide your own OpenTelemetry SDK/exporter.

L1: Quick Start (30-second read)

What

tramai-observability is an optional, opt-in module that bridges Tramai's observer SPIs (OperationObserver from tramai-core, WorkflowObserver and TramaiWorkerObserver from tramai-orchestration) to OpenTelemetry. It emits:

• Traces — spans for every provider call (OpenTelemetryOperationObserver), every workflow execution (OpenTelemetryWorkflowObserver), and every worker lifetime (OpenTelemetryTramaiWorkerObserver)
• Metrics — counters and histograms for attempt counts, duration, input/output token usage, parse failures, engine events, workflow runs, workflow events, and worker heartbeats/leases/shutdowns
• Events — structured span events for parse failures, engine resilience/routing events, step lifecycle, checkpoints, leases, delays, and the worker lifecycle

Every event name, attribute key, and metric name this module emits comes from the runtime event catalogue in tramai-core (dev.tramai.core.observation.event.RuntimeEventCatalogue); literals outside that package are architecture-guarded by a module test.

Why

Without this module, Tramai runs with no-op observers — operations and workflows execute silently. Add tramai-observability when you need:

• Production monitoring — track latency, token consumption, and error rates per service method and workflow
• Root-cause analysis — distributed traces connecting AI calls to the business operation that triggered them
• Cost attribution — token usage broken down by service interface, method, and model
• Workflow debugging — step-by-step execution traces with checkpoint and lease lifecycle events
• Structured parse failure visibility — see exactly which raw responses failed validation and why

When to use

• Production applications — always recommended when deploying to production
• Development — optional; the no-op observers keep test runs lightweight
• Never forced — tramai-observability is never pulled transitively. Add it explicitly when needed.

How to add

Gradle (Kotlin DSL):

dependencies {
    implementation(platform("dev.tramai:tramai-bom:0.6.0"))
    implementation("dev.tramai:tramai-observability")
    // You also need an OpenTelemetry SDK + exporter runtime dependency:
    runtimeOnly("io.opentelemetry:opentelemetry-exporter-otlp:...")
    runtimeOnly("io.opentelemetry:opentelemetry-sdk-extension-autoconfigure:...")
}

Maven:

<dependency>
    <groupId>dev.tramai</groupId>
    <artifactId>tramai-observability</artifactId>
    <version>0.6.0</version>
</dependency>

Wiring

Wire into tramai-engine (operations) and tramai-orchestration (workflows) at construction time:

val openTelemetry = OpenTelemetrySdk.builder()
    .setTracerProvider(tracerProvider)
    .setMeterProvider(meterProvider)
    .build()

// Operations
val engine = TramaiEngine(
    providerRegistry = registry,
    operationObserver = OpenTelemetryOperationObserver(openTelemetry),
)

// Workflows — the Workflow instance comes from the orchestration builder;
// run() accepts initialState, context, observer, and persistence
val result = workflow.run(
    initialState = MyState(),
    observer = OpenTelemetryWorkflowObserver(openTelemetry),
)

// Workers — TramaiWorker's observer parameter is named `observability`
val worker = TramaiWorker(
    config = workerConfig,
    leaseStore = leaseStore,
    checkpointStore = checkpointStore,
    workflowBindings = bindings,
    observability = OpenTelemetryTramaiWorkerObserver(openTelemetry),
)

The worker observer is wrapped in FailureIsolatingTramaiWorkerObserver when TramaiWorker is constructed, so a throwing observer cannot break the worker loop.

Where to go next

L2: Usage Guide (5-minute read)

OpenTelemetry setup

Both observers are constructable two ways:

The default instrumentation name is dev.tramai.observability. Override via the second constructor parameter.

Operation spans

Every call to an @AiService method that reaches a provider produces a span named ai.{methodName} (e.g., ai.getWeather).

Span attributes set at start:

Span attributes set during execution:

Span status: Set to ERROR with the exception message when onProviderFailure is called.

Workflow spans

Every workflow run produces a span named workflow.{workflowName} (e.g., workflow.invoice-analysis).

Span attributes:

Span status: Set to ERROR with exception message when workflow fails.

Operation metrics

All metrics are emitted via the Meter obtained from OpenTelemetry and carry dimension attributes for filtering.

Workflow metrics

Workflow span events

In addition to the root workflow span, the observer records structured span events for lifecycle transitions:

User-defined workflow events — emitted via observer.onWorkflowEvent(workflowName, name, attributes, context), or the catalogue-validated onWorkflowEvent(workflowName, event, context) overload — also appear as span events and contribute to tramai.workflow.events.

L3: Architecture & Mechanics (15-minute read)

Design philosophy

tramai-observability follows the observer pattern — it implements two SPI interfaces (OperationObserver, WorkflowObserver) and translates their callback methods into OpenTelemetry spans, metrics, and events. The module is intentionally thin:

• No shared state beyond the active workflow span bookkeeping map
• No configuration parsing — OpenTelemetry SDK configuration is left entirely to the application
• No exporter decisions — the module works with any OpenTelemetry instance; the application decides where and how to export
• Minimal surface — four source files totaling 775 LOC: OpenTelemetryOperationObserver.kt, OpenTelemetryWorkflowObserver.kt, OpenTelemetryTramaiWorkerObserver.kt, OpenTelemetryAttributes.kt

Dependency graph

tramai-observability
    ├── api: tramai-core           (OperationObserver, OperationCallContext, ModelResponse)
    ├── impl: tramai-orchestration (WorkflowObserver, WorkflowContext)
    └── impl: opentelemetry-api    (Tracer, Span, Meter, Attributes, etc.)

The module depends on tramai-core as an api dependency (exporting the implemented interfaces) and tramai-orchestration and opentelemetry-api as implementation (internal usage only). It has no runtime dependency on the OpenTelemetry SDK — only the API jar is required.

Source file breakdown

OpenTelemetryAttributes.kt

A single internal utility function that converts Map<String, Any?> to io.opentelemetry.api.common.Attributes with type-aware mapping:

This is used by both SpanBackedObservation and ActiveWorkflowSpan when adding event attributes.

OpenTelemetryOperationObserver

Implements dev.tramai.core.observation.OperationObserver. The lifecycle:

onCallStarted(context)
    │
    ├── Creates a span: "ai.{context.methodName}"
    ├── Sets span attributes: gen_ai.system, gen_ai.request.model,
    │   tramai.operation.interface, tramai.operation.method, tramai.retry.attempt
    └── Returns SpanBackedObservation
                 │
                 ├── onProviderResponse(response)
                 │   ├── Updates span: gen_ai.response.model, gen_ai.usage.*
                 │   └── Records metrics: input_tokens, output_tokens, *per_attempt
                 │
                 ├── onProviderFailure(error)
                 │   ├── span.recordException(error)
                 │   └── span.setStatus(StatusCode.ERROR)
                 │
                 ├── onStructuredParseFailure(rawResponse, errorSummary)
                 │   ├── span.addEvent("tramai.parse.failure", {raw, error})
                 │   └── metrics.parseFailures.add(1)
                 │
                 ├── onEngineEvent(name, attributes)
                 │   ├── span.addEvent(name, attributes)
                 │   └── metrics.engineEvents.add(1)
                 │
                 └── onCallCompleted(parseSuccess)
                     ├── span.setAttribute("tramai.structured.parse_success", ...)
                     ├── metrics.attempts.add(1)
                     ├── metrics.duration.record(...)
                     └── span.end()

Internal classes:

• SpanBackedObservation — implements OperationObservation, holds the active Span, OpenTelemetryMetrics, baseAttributes, and timing state (startedAtNanos)
• OpenTelemetryMetrics — holds all pre-built meter instrument instances (8 metrics total: 4 counters, 2 histograms, 2 LongHistograms)

tramai.outcome calculation:

Computed in SpanBackedObservation.currentOutcome(), in this precedence order:

• "failure" — if onProviderFailure was called
• "parse_failure" — if onStructuredParseFailure was called (the attempt may still be counted separately via tramai.operation.parse_failures)
• "success" — if onProviderResponse was called (and no failure)
• "unknown" — if none of the above
• "cancelled" — onCallCancelled() overrides the outcome with "cancelled" and still counts the attempt and records its duration before ending the span

OpenTelemetryWorkflowObserver

Implements dev.tramai.orchestration.WorkflowObserver. The lifecycle:

onWorkflowStarted(name, context)
    │
    ├── Creates span: "workflow.{name}"
    ├── Sets attributes: tramai.workflow.name, tramai.workflow.id,
    │   tramai.workflow.context.* (one per context.attributes entry)
    └── Registers ActiveWorkflowSpan in concurrent map keyed by (name, workflowId)
                 │
                 ├── onStepStarted / onStepCompleted / onStepFailed
                 │   └── Adds span event: tramai.workflow.step.{lifecycle}
                 │
                 ├── onWorkflowEvent(name, attributes, context)
                 │   ├── span.addEvent(name, attributes)
                 │   └── metrics.events.add(1)
                 │
                 └── onWorkflowCompleted / onWorkflowFailed
                     ├── Removes from active map
                     ├── (on failure): span.recordException, StatusCode.ERROR
                     ├── Sets tramai.workflow.outcome
                     ├── metrics.runs.add(1)
                     ├── metrics.duration.record(...)
                     └── span.end()

Internal classes:

• WorkflowRunKey — immutable data class (workflowName, workflowId) used as the key in ConcurrentHashMap for tracking active workflows
• ActiveWorkflowSpan — holds the Span, startedAtNanos, and baseAttributes; provides a recordEvent(name, attributes) helper
• OpenTelemetryWorkflowMetrics — holds meter instruments for runs (counter), duration (histogram), and events (counter)

Thread safety: The activeWorkflows map uses ConcurrentHashMap, so multiple workflow runs can be observed concurrently without data races. Each active workflow is tracked by its unique (name, workflowId) pair and removed on completion or failure.

OpenTelemetryTramaiWorkerObserver

Implements dev.tramai.orchestration.TramaiWorkerObserver and tracks one active worker span in an AtomicReference:

• onWorkerStarted(workerId) starts a span named worker.{workerId} and sets tramai.worker.id; onWorkerStopped(workerId) sets tramai.worker.outcome = "stopped" and ends it.
• Every other callback records a catalogue-defined span event carrying its declared attributes — heartbeat, lease acquired/released/expired/renewed/renewal-failed/release-failed, poll failed, work taken over, unknown attempt, step started/completed/failed, shutdown started/drain progress/shutdown complete, and workflow abandoned.
• Worker metrics: tramai.worker.heartbeats, tramai.worker.leases (dimensioned by tramai.lease.operation and tramai.worker.workflow_id), and tramai.worker.shutdowns.

Event mapping reference

What tramai-observability does NOT include

• No OpenTelemetry SDK or exporter — the module depends only on opentelemetry-api. SDK, exporters, and auto-configuration are the application's responsibility.
• No Spring Boot auto-configuration of these observers — they are plain constructor-injected collaborators, so wire them as beans yourself. (The sovereign ops starter tramai-spring-boot-starter-sovereign-ops-observability ships its own OpenTelemetry observer for the audit-outbox worker; it does not use this module.)
• No logging integration — spans and metrics are the delivery mechanism, not log statements.
• No workflow span nesting — operation spans (from the engine) and workflow spans (from orchestration) are independent. Workflow steps that make AI calls produce their own ai.* spans under the workflow step's context if propagation is configured.