higress-group/higress

ai-load-balancer 考虑支持 SGLang 吗？ || ai-load-balancer Consider supporting SGLang?

Open

#3,392 opened on Jan 26, 2026

View on GitHub
 (3 comments) (0 reactions) (0 assignees)Go (1,219 forks)github user discovery
help wanted

Repository metrics

Stars
 (9,036 stars)
PR merge metrics
 (Avg merge 7d 9h) (18 merged PRs in 30d)

Description

Why do you need it?

目前看支持了 vLLM，想问下后续有考虑支持 SGLang、Triton TensorRT-LLM 等其它推理引擎吗？ https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm

How could it be?

目前项目同时使用 vLLM 和 SGLang，若监控指标负载均衡能同时兼容两者，会更加便捷一些。

Why do you need it?

At present, vLLM is supported. I would like to ask if you will consider supporting other inference engines such as SGLang and Triton TensorRT-LLM in the future? https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm

How could it be?

The current project uses both vLLM and SGLang. It would be more convenient if the monitoring indicator load balancing could be compatible with both.

Contributor guide

Research direction
Study the existing vLLM backend metrics implementation in the ai load balancer plugin (path: plugins/wasm go/extensions/ai load balancer/endpoint metrics/backend/vllm) and understand how to expose similar metrics for SGLang. Investigate SGLang's monitoring APIs or metrics endpoints to design a compatible backend.
Tech stack
go
Domain
backend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Go
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70