grafana-loki lacks basic feature of extracting nested json labels
#6,994 opened on Aug 29, 2022
Repository metrics
- Stars
- (28,187 stars)
- PR merge metrics
- (Avg merge 5d 20h) (522 merged PRs in 30d)
Description
Is your feature request related to a problem? Please describe. I am running a java spring-boot application on AWS ECS and want to ship logs to loki/grafana. In order to have the java stack trace as a single log line in grafana I log as json to console using
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
The resulting json from AWS ECS looks like this:
{
"container_id": "ea7b47de37024af8a71629fc4c435e09-285964202",
"container_name": "backend",
"ecs_cluster": "dev-fargate",
"ecs_task_arn": "arn:aws:ecs:eu-central-1:xxx:task/dev-fargate/ea7b47de37024af8a71629fc4c435e09",
"ecs_task_definition": "dev-backend:72",
"log": {
"@timestamp": "2022-08-29T08:13:22.893Z",
"@version": "1",
"message": "Running with Spring Boot v2.7.3, Spring v5.3.22",
"logger_name": "com.example.Application",
"thread_name": "main",
"level": "DEBUG",
"level_value": 10000
}
}
Describe the solution you'd like I want to extract also lables from the log element, i.e. log level, logger name, ... and only keep the log.message as the log text.
So therefore I need configuration like this (cloudformation yaml config), which is currently not working/supported:
...
LogConfiguration:
LogDriver: awsfirelens
Options:
Name: grafana-loki
Url: https://loki:3000/loki/api/v1/push
Labels: "{source=\"console\"}"
LabelKeys: container_id,ecs_task_arn,ecs_task_definition,ecs_cluster,container_name,log.level,log.logger_name,log.thread_name
RemoveKeys: source,log.level_value,log.@version
LineFormat: key_value
insecure_skip_verify: true
SecretOptions:
- Name: TenantID
ValueFrom: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/config/loki/tenant-id
- Name: log_router
Image: grafana/fluent-bit-plugin-loki:2.6.1-amd64
Essential: false
Memory: 512
Cpu: 256
FirelensConfiguration:
Type: fluentbit
Options:
enable-ecs-log-metadata: true
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-stream-prefix: firelens
awslogs-group: !Ref LogGroup
awslogs-region: !Ref AWS::Region
...
Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. I have also tried using LabelMapPath, but this didn't work either. Also tried LineFormat=json but has same result.
Additional context This is really a basic feature I bet a lot of people need!
And maybe another option would be great to specify the final message target, i.e. "log.message" in my case... and other properties should be ignored. Would extremly ease the config avoiding the need of specifying tons of "RemoveKeys"...