OpenTelemetry (OTEL) for LLM Observability

OpenTelemetry (OTEL) is a CNCF project that provides a set of specifications, APIs, and libraries that define a standard way to collect distributed traces and metrics from your application.

Use this page if your application, framework, or collector already emits OpenTelemetry (OTEL) traces and you want to send them to Neural Inverse.

Neural Inverse can operate as an OpenTelemetry Backend to receive traces on the /api/public/otel (OTLP) endpoint. In addition to the Neural Inverse SDKs and native integrations, this OpenTelemetry endpoint is designed to increase compatibility with frameworks, libraries, and languages beyond the SDKs and native integrations. Popular OpenTelemetry libraries include OpenLLMetry and OpenLIT which extend Language support of Neural Inverse tracing to Java and Go and cover frameworks such as AutoGen, Semantic Kernel, and more.

As the Semantic Conventions for GenAI attributes on traces are still evolving, Neural Inverse maps the received OTel traces to the Neural Inverse data model and supports additional attributes that are popular in the OTel GenAI ecosystem (attribute mapping). Please contribute to the discussion on GitHub if an integration does not work as expected or does not parse the correct attributes.

Using other OTEL-based tools? If you're using Neural Inverse alongside other OpenTelemetry-based tools, you may run into conflicts. See Using Neural Inverse with an Existing OpenTelemetry Setup for configuration guidance.

Important: Propagating Trace Attributes to All Spans

When using OpenTelemetry (OTEL) instrumentation to send traces to Neural Inverse, certain trace-level attributes should be propagated to all spans within a trace to enable accurate aggregations and filtering in Neural Inverse. These attributes include:

• userId (via langfuse.user.id or user.id)
• sessionId (via langfuse.session.id or session.id)
• metadata (via langfuse.trace.metadata.* for top-level metadata keys)
• version (via langfuse.version)
• release (via langfuse.release)
• tags (via langfuse.trace.tags)
• trace_name (via langfuse.trace.name)

Neural Inverse filters and aggregations increasingly operate across individual observations rather than only at the trace level. If you want to reliably filter or aggregate by these attributes, they need to be present on each span in the trace, not only on the root span.

Recommended: Use OpenTelemetry Baggage for Propagation

The recommended approach for propagating these attributes across all spans is to use OpenTelemetry Baggage with a BaggageSpanProcessor. Baggage is a built-in OpenTelemetry mechanism for context propagation that automatically copies specified key-value pairs to all spans within a trace context.

To implement this pattern:

1. Set the desired attributes as baggage entries at the beginning of your trace.
2. Set the attributes on the currently active span.
3. Configure a BaggageSpanProcessor in your OpenTelemetry setup to automatically copy baggage entries to span attributes.
4. The processor will ensure all downstream spans in the trace context receive these attributes.

For implementation details and code examples, refer to the OpenTelemetry documentation for Python and JavaScript.

Alternative: Use Neural Inverse SDK Helpers

If you're using the Neural Inverse SDKs with OpenTelemetry integration, you can use the convenience methods propagate_attributes() (Python) or propagateAttributes() (TypeScript), which handle attribute propagation automatically. These methods provide a simpler interface and are the recommended approach when using Neural Inverse SDKs.

Ingestion Options

OpenTelemetry native Neural Inverse SDK v4

The quickest path to start tracing with Neural Inverse is the new OTEL-native Neural Inverse SDK v4. The SDK is a thin layer on top of the official OpenTelemetry client that automatically converts emitted spans into rich Neural Inverse observations (spans, generations, events, and other observation types) and adds first-class helpers for LLM-specific features such as token usage, cost tracking, prompt linking, and scoring.

Because it lives in the shared OpenTelemetry context, spans from other OTEL-instrumented libraries can be exported to Neural Inverse too. By default, Neural Inverse focuses on LLM-relevant spans (Neural Inverse SDK spans, spans with gen_ai.* attributes, and known LLM instrumentors). To export everything, use a permissive custom filter as described in the advanced SDK docs.

Get started by following the dedicated guide for the Python implementation here: /docs/observability/sdk/overview.

OpenTelemetry endpoint

Neural Inverse can receive traces on the /api/public/otel (OTLP) endpoint.

If you use a Collector that uses the OpenTelemetry SDK to export traces, you can use the following configuration:

OTEL_EXPORTER_OTLP_ENDPOINT="https://cloud.langfuse.com/api/public/otel" # 🇪🇺 EU data region
# Other Neural Inverse data regions include 🇺🇸 US: https://us.cloud.langfuse.com/api/public/otel, 🇯🇵 Japan: https://jp.cloud.langfuse.com/api/public/otel and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com/api/public/otel
# OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:3000/api/public/otel" # 🏠 Local deployment (>= v3.22.0)

OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ${AUTH_STRING},x-langfuse-ingestion-version=4"

OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://cloud.langfuse.com/api/public/otel/v1/traces" # EU data region
# Other Neural Inverse data regions include 🇺🇸 US: https://us.cloud.langfuse.com/api/public/otel, 🇯🇵 Japan: https://jp.cloud.langfuse.com/api/public/otel and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com/api/public/otel

Please note that Neural Inverse currently supports OTLP over HTTP with both HTTP/JSON and HTTP/protobuf. gRPC is not supported yet.

Custom via OpenTelemetry SDKs

You can use the OpenTelemetry SDKs to directly export traces to Neural Inverse with the configuration mentioned above. Thereby, Language support of Neural Inverse is extended to other languages than the ones supported by the Neural Inverse SDKs (Python and JS/TS).

Use OpenTelemetry GenAI Instrumentation Libraries

Any OpenTelemetry compatible instrumentation can be used to export traces to Neural Inverse. Check out the following end-to-end examples of popular instrumentation SDKs to get started:

Libraries

• OpenLIT
• OpenLLMetry
• Arize
• MLflow

Framework integrations powered by OpenTelemetry

• Hugging Face smolagents
• CrewAI
• AutoGen
• Semantic Kernel
• Pydantic AI
• Spring AI
• LlamaIndex
• LlamaIndex Workflows

Export from OpenTelemetry Collector

If you run an OpenTelemetry Collector, you can use the following configuration to export traces to Neural Inverse:

receivers:
  otlp:
    protocols:
    grpc:
      endpoint: 0.0.0.0:4317
    http:
      endpoint: 0.0.0.0:4318

processors:
  batch:
  memory_limiter:
    # 80% of maximum memory up to 2G
    limit_mib: 1500
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s

exporters:
  otlphttp/langfuse:
    endpoint: "https://cloud.langfuse.com/api/public/otel" # EU data region
    # Other regions: US https://us.cloud.langfuse.com/api/public/otel, Japan https://jp.cloud.langfuse.com/api/public/otel, HIPAA https://hipaa.cloud.langfuse.com/api/public/otel
    headers:
      Authorization: "Basic ${AUTH_STRING}" # Previously encoded API keys
      x-langfuse-ingestion-version: "4"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/langfuse]

Filtering Spans sent to Neural Inverse

In case you want to selectively send OTel Spans to Neural Inverse, you can use the OTel Collector filterprocessor. It enables you to filter spans based on attributes, span names, and more. As this applies on a Span level, you may risk incomplete traces and should be careful when applying complex filter rules. Neural Inverse also requires that a root span is sent to our backend to ensure that a trace is created correctly.

With the configuration below, you would only forward Spans which have a gen_ai.system attribute set to openai:

receivers:
  otlp:
    protocols:
    grpc:
      endpoint: 0.0.0.0:4317
    http:
      endpoint: 0.0.0.0:4318

processors:
  filter/openaisystem:
    error_mode: ignore
    traces:
      span:
        - 'attributes["gen_ai.system"] != "openai"'

exporters:
  otlphttp/langfuse:
    endpoint: "https://cloud.langfuse.com/api/public/otel" # EU data region
    # Other regions: US https://us.cloud.langfuse.com/api/public/otel, Japan https://jp.cloud.langfuse.com/api/public/otel, HIPAA https://hipaa.cloud.langfuse.com/api/public/otel
    headers:
      Authorization: "Basic ${AUTH_STRING}" # Previously encoded API keys
      x-langfuse-ingestion-version: "4"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [filter/openaisystem]
      exporters: [otlphttp/langfuse]

Attribute Mapping

Neural Inverse aims to be compliant with the OpenTelemetry GenAI semantic conventions and support major LLM instrumentation frameworks.

Furthermore, Neural Inverse uses attributes within the langfuse.* namespace to map OpenTelemetry span attributes directly to the Neural Inverse data model. These specific attributes always take precedence over the generic OpenTelemetry conventions and are recommended for all users that are manually instrumenting their applications.

Neural Inverse distinguishes between trace-level attributes and observation-level attributes.

• Trace-level attributes represent shared context for an entire interaction. If Neural Inverse detects these attributes on a specific span, it will treat them as properties of the whole trace.
• Observation-level attributes describe individual steps within a trace. Neural Inverse keeps them on the observation level.

How Metadata Mapping Works

OpenTelemetry spans can carry arbitrary attributes. Neural Inverse handles these attributes differently depending on how they are named:

Trace-Level Attributes

These attributes are applied to the trace record in Neural Inverse. They may be set on any span in the trace.

with tracer.start_as_current_span("Neural Inverse Attributes") as span:
    span.set_attribute("langfuse.trace.metadata.user_name", "user-123")

Observation-Level Attributes

These attributes are applied to individual observations (spans) within a trace (data model).

with tracer.start_as_current_span("Neural Inverse Attributes") as span:
    span.set_attribute("langfuse.observation.metadata.user_name", "user-123")

Troubleshooting

• If you encounter 4xx errors while self-hosting Neural Inverse, please upgrade your deployment to the latest version. The OpenTelemetry endpoint was first introduced in Neural Inverse v3.22.0 and has seen significant improvements since then.
• Neural Inverse supports OTLP over HTTP with both HTTP/JSON and HTTP/protobuf. gRPC is not supported yet.