Dynamic headers to OTLP Exporter
#7,800 opened on May 31, 2023
Repository metrics
- Stars
- (7,237 stars)
- PR merge metrics
- (PR metrics pending)
Description
Is your feature request related to a problem? Please describe. Today only static headers are allowed in OTLP exporter. I would like to have an option to pass dynamic headers to OTLP exporter.
Describe the solution you'd like
Ability to set the header in OTLP exporter dynamically from attributes processor for example.
receivers: otlp: protocols: grpc: include_metadata: true
attributes/get_tenant: actions: - key: "my-dynamic-value" from_attributes: app_tenant action: upsert
otlp: endpoint: "http://some-endpoint:4317" headers: my-dynamic-header: <my-dynamic-value>
Describe alternatives you've considered If header_setter extension in OTEL contrib repo can support from_attributes then we will consider that as an alternatives.
Additional context
Today in our pipeline we have a OTLP receiver , resource attributes, attributes processor, OTLP exporter and auth extension to support us our multi-tenancy setup. The headers that auth extension requires from our client can be accessed as our pipeline is pretty simple.
As we are getting more applications getting onboarded to OTEL based implementation we want to implement tail sampling and introduce Kafka for temporary storage to allow scaling.
With the new setup we will have multiple pipelines 1 - OTEL Agent --> OTEL Receiver --> Kafka Exporter. 2 - Kafka Receiver --> Load Balancer --> Resource attributes & attributes processor --> Tail sampling processor --> Routing Processor --> OTLP Exporter
With this new pipeline (2) we can't access the headers (from_context) sent by the OTEL Agent hence we are not able to use the header_setter extension for our multi-tenancy use-case.
Upon checking we can use Routing processor which has the ability to get value from attributes (from_attributes) and based on match it can forward to specific exporter. But again, the headers we need to set is static and when new applications are onboarded we need to keep updating the pipeline configuration.