CUDA inference failure on 5070 Ti – pp-doclayout-v3 crashes with CUBLAS_STATUS_INVALID_VALUE
#737 opened on May 28, 2026
Repository metrics
- Stars
- (4,568 stars)
- PR merge metrics
- (Avg merge 5d 11h) (15 merged PRs in 30d)
Description
Koharu launches successfully and partially uses GPU acceleration on an RTX 5070 Ti, but consistently fails during the pp-doclayout-v3 pipeline step with:
CublasError(CUBLAS_STATUS_INVALID_VALUE)
CPU fallback works correctly.
OS Linux (KDE Plasma Wayland session)
GPU NVIDIA GeForce RTX 5070 Ti
Driver / CUDA Driver Version: 595.71.05 CUDA Version: 13.2 GPU compute capability: 12.0
Koharu initially failed to launch UI under Wayland with: Gdk-Message: Error 71 (Protocol error) dispatching to Wayland display. ERROR: WebKit encountered an internal error.
This was worked around successfully by launching with: GDK_BACKEND=x11 WEBKIT_DISABLE_COMPOSITING_MODE=1 koharu
After this, the UI launches correctly, however processing an image consistently fails during pp-doclayout-v3. speech-bubble-segmentation appears to execute successfully on GPU before the failure occurs.
This suggests: CUDA initialization succeeds GPU detection succeeds at least some GPU inference paths work correctly
The crash appears isolated to the pp-doclayout-v3 model/runtime path.
This CPU fallback launch fully works: CUDA_VISIBLE_DEVICES="" GDK_BACKEND=x11 WEBKIT_DISABLE_COMPOSITING_MODE=1 koharu