The model says NVFP4 for H100s and A100s
#1
by
dnhkng
- opened
I thought that these older architectures didn't support NVFP4, and its only usable on Blackwell cards?
Hi ! You can find more context about it here: https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4#serve
vllm falls back to Marlin FP4 if the GPU doesnt support it !
Thanks! This is new to me, very interesting!