GoogleCloudPlatform/kubectl-ai

Help needed: We need more evals

Open

#145 opened on May 6, 2025

View on GitHub
 (5 comments) (4 reactions) (0 assignees)Go (708 forks)auto 404
good first issuehelp wanted

Repository metrics

Stars
 (7,532 stars)
PR merge metrics
 (PR metrics pending)

Description

kubectl-ai has 10 eval tasks today covering different areas. We want to improve the evals coverage. The goal is to get the eval coverage closer to realistic scenario that users run into.

This is an area where we need input from the community. Share the scenarios that you or your team run into very regularly.

Adding an eval task to k8s-bench is very simple, you can see existing tasks.

Ask:

  • Share a scenario that we can add eval for
  • Better, send a PR with the eval task following an example under k8s-bench directory.

/cc @selimacerbas @rxinui @cwrau @hakman @mattn @DoctorLai @mschneider82 @tuannvm

Contributor guide

Research direction
Review existing eval tasks in k8s bench and propose realistic user scenarios.
Tech stack
go
Domain
devopstesting
Issue type
Research
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
1
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Under 1 hour
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Kuberneteskubectl ai
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70