containers/ramalama

ramalama chat should wait for llama.cpp to initiialize

Open

#2,343 opened on Jan 24, 2026

View on GitHub
 (4 comments) (0 reactions) (1 assignee)Python (341 forks)github user discovery
buggood first issue

Repository metrics

Stars
 (2,873 stars)
PR merge metrics
 (Avg merge 6d 7h) (32 merged PRs in 30d)

Description

Issue Description

llama-server /models api returns a 503 error while the model is loading. ramalama run handles this but ramalama chat currently does not. For interactive use this isn't a major issue but it would be good to fix this.

Worked around this for CI in https://github.com/containers/ramalama/pull/2342

Steps to reproduce the issue

Run ramalama chat soon after ramalama serve --detach

Describe the results you received

ramalama chat fails

Describe the results you expected

ramalama chat succeeds

ramalama info output

N/A

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response

Contributor guide

Research direction
Examine how `ramalama run` handles the 503 error from llama server when the model is loading. Look at the polling mechanism or retry logic used there and apply a similar approach to `ramalama chat`. Check if there is a simple way to wait for the `/models` endpoint to return a non 503 response before starting the chat session.
Tech stack
python
Domain
backendapi
Issue type
Bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonAPI testing
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65