open-telemetry/opentelemetry-collector

Dynamic headers to OTLP Exporter

Open

#7,800 opened on May 31, 2023

View on GitHub
 (26 comments) (32 reactions) (0 assignees)Go (2,146 forks)auto 404
enhancementexporter/otlphelp wanted

Repository metrics

Stars
 (7,237 stars)
PR merge metrics
 (PR metrics pending)

Description

Is your feature request related to a problem? Please describe. Today only static headers are allowed in OTLP exporter. I would like to have an option to pass dynamic headers to OTLP exporter.

Describe the solution you'd like Ability to set the header in OTLP exporter dynamically from attributes processor for example. receivers: otlp: protocols: grpc: include_metadata: true

attributes/get_tenant: actions: - key: "my-dynamic-value" from_attributes: app_tenant action: upsert otlp: endpoint: "http://some-endpoint:4317" headers: my-dynamic-header: <my-dynamic-value>

Describe alternatives you've considered If header_setter extension in OTEL contrib repo can support from_attributes then we will consider that as an alternatives.

Additional context

Today in our pipeline we have a OTLP receiver , resource attributes, attributes processor, OTLP exporter and auth extension to support us our multi-tenancy setup. The headers that auth extension requires from our client can be accessed as our pipeline is pretty simple.

As we are getting more applications getting onboarded to OTEL based implementation we want to implement tail sampling and introduce Kafka for temporary storage to allow scaling.

With the new setup we will have multiple pipelines 1 - OTEL Agent --> OTEL Receiver --> Kafka Exporter. 2 - Kafka Receiver --> Load Balancer --> Resource attributes & attributes processor --> Tail sampling processor --> Routing Processor --> OTLP Exporter

With this new pipeline (2) we can't access the headers (from_context) sent by the OTEL Agent hence we are not able to use the header_setter extension for our multi-tenancy use-case.

Upon checking we can use Routing processor which has the ability to get value from attributes (from_attributes) and based on match it can forward to specific exporter. But again, the headers we need to set is static and when new applications are onboarded we need to keep updating the pipeline configuration.

Contributor guide