HunYuanVLProcessor: AttributeError '_get_num_multimodal_tokens' with vLLM (tencent/HunyuanOCR + gpu_memory_utilization)
Hi everyone 👋
I’m running into an issue when trying to serve tencent/HunyuanOCR with vLLM.
When I use --gpu-memory-utilization (which normally works fine and is very helpful to control VRAM usage on smaller GPUs), I get this error:
AttributeError: 'HunYuanVLProcessor' object has no attribute '_get_num_multimodal_tokens'
It looks like the HunYuanVLProcessor used for tencent/HunyuanOCR doesn’t implement the private method _get_num_multimodal_tokens, which some multimodal tooling (vLLM in my case) expects for multimodal memory profiling / budgeting.
How do you install vLLM? Do you install nightly as suggested in the readme?
I followed "HunyuanOCR Usage Guide" in the docs and installed the nightly build as suggested
I'm experiencing the same issue. When using vLLM with --gpu-memory-utilization flag, I get the same AttributeError: 'HunYuanVLProcessor' object has no attribute '_get_num_multimodal_tokens' error.
I've also installed the nightly build of vLLM as suggested in the documentation.
Is there any workaround or fix for this issue?
make sure to use uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly , i.e. using uv . pip and uv have different behaviors here.
you can follow https://docs.vllm.ai/projects/recipes/en/latest/Tencent-Hunyuan/HunyuanOCR.html
Thank you, It worked with uv pip
i have apple silicon and uv pip is not download ing this properly...