[Bug]: Qwen2.5-VL Multimodal - Video

Your current environment

Model: Qwen2.5-VL-3B-Instruct

Launch parameter: --tensor-parallel-size=4 --enable-prefix-caching --enable-chunked-prefill --served-model-name=tpu --disable_chunked_mm_input --limit-mm-per-prompt='{"image":0, "video":1, "audio":0}' --max-model-len=64K --max-num-seqs=1 --max-num-batched-tokens=64K --mm-processor-kwargs='{"max_pixels": 589824, "nframes": 10, "fps": 1}'

Request payload from client: payload = { "stream": False, "chat_template_kwargs": {"enable_thinking": False}, "messages": [ { "role": "user", "content": [ {"type": "text", "text": "Describe the video content in detail."}, { "type": "video_url", "video_url": {"url": "https://content.pexels.com/videos/free-videos.mp4"} }, ] } ] }

ullm output: (APIServer pid=128) ERROR 10-27 23:25:34 [async_llm.py:516] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause. (EngineCore_DP0 pid=391) Traceback (most recent call last): (APIServer pid=128) INFO: 10.0.0.21:48750 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error (EngineCore_DP0 pid=391) File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap (EngineCore_DP0 pid=391) self.run() (EngineCore_DP0 pid=391) File "/usr/local/lib/python3.12/multiprocessing/process.py", line 108, in run (EngineCore_DP0 pid=391) self._target(*self._args, **self._kwargs) (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 786, in run_engine_core (EngineCore_DP0 pid=391) raise e (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 775, in run_engine_core (EngineCore_DP0 pid=391) engine_core.run_busy_loop() (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 802, in run_busy_loop (EngineCore_DP0 pid=391) self._process_engine_step() (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 831, in _process_engine_step (EngineCore_DP0 pid=391) outputs, model_executed = self.step_fn() (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 316, in step (EngineCore_DP0 pid=391) model_output = self.execute_model_with_error_logging( (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 302, in execute_model_with_error_logging (EngineCore_DP0 pid=391) raise err (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/engine/core.py", line 292, in execute_model_with_error_logging (EngineCore_DP0 pid=391) return model_fn(scheduler_output) (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/v1/executor/abstract.py", line 107, in execute_model (EngineCore_DP0 pid=391) output = self.collective_rpc( (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/executor/uniproc_executor.py", line 81, in collective_rpc (EngineCore_DP0 pid=391) return [run_method(self.driver_worker, method, args, kwargs)] (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/vertex_vllm/vllm/utils/init.py", line 2975, in run_method (EngineCore_DP0 pid=391) return func(*args, **kwargs) (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/tpu_inference/tpu_inference/worker/tpu_worker_jax.py", line 195, in execute_model (EngineCore_DP0 pid=391) output = self.model_runner.execute_model(vllm_scheduler_output) (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/tpu_inference/tpu_inference/runner/tpu_jax_runner.py", line 332, in execute_model (EngineCore_DP0 pid=391) return self._execute_model(scheduler_output)[1] (EngineCore_DP0 pid=391) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (EngineCore_DP0 pid=391) File "/workspace/tpu_inference/tpu_inference/runner/tpu_jax_runner.py", line 363, in _execute_model (EngineCore_DP0 pid=391) self.mm_manager.execute_mm_encoder(scheduler_output) (EngineCore_DP0 pid=391) File "/workspace/tpu_inference/tpu_inference/runner/multimodal_manager.py", line 128, in execute_mm_encoder (EngineCore_DP0 pid=391) batched_mm_inputs.pop('image_grid_thw') (EngineCore_DP0 pid=391) KeyError: 'image_grid_thw' (APIServer pid=128) INFO: Shutting down (APIServer pid=128) INFO: Waiting for application shutdown. (APIServer pid=128) INFO: Application shutdown complete. (APIServer pid=128) INFO: Finished server process [128]

🐛 Describe the bug

As mentioned here, https://github.com/vllm-project/tpu-inference/blob/main/examples/multi_modal_inference.py

video input to Qwen2.5-VL is not yet support.

Image process worked well, 3B(tp=4), 7B(tp=4), 32B(tp=8) on V6E

But input video gives crash during request process.

Before submitting a new issue...

• Make sure you already searched for relevant issues and checked the documentation page, which can answer lots of frequently asked questions.