Instructions to use raxcore-dev/Rax-4.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use raxcore-dev/Rax-4.5 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="raxcore-dev/Rax-4.5")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("raxcore-dev/Rax-4.5")
model = AutoModelForImageTextToText.from_pretrained("raxcore-dev/Rax-4.5")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use raxcore-dev/Rax-4.5 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "raxcore-dev/Rax-4.5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raxcore-dev/Rax-4.5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/raxcore-dev/Rax-4.5

SGLang

How to use raxcore-dev/Rax-4.5 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "raxcore-dev/Rax-4.5" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raxcore-dev/Rax-4.5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "raxcore-dev/Rax-4.5" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "raxcore-dev/Rax-4.5",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use raxcore-dev/Rax-4.5 with Docker Model Runner:
```
docker model run hf.co/raxcore-dev/Rax-4.5
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Rax 4.5 - Efficient 2B Vision Language Model

Rax 4.5 is a state-of-the-art 2 billion parameter multimodal vision-language model optimized for production use. Process images and text together with up to 262K token context length.

Key Features

Fast & Efficient: Only 2B parameters for quick inference
Vision + Text: True multimodal understanding of images and language
Long Context: 262,144 token context window for complex tasks
Production Ready: Works with vLLM, SGLang, Transformers out of the box
Memory Efficient: Hybrid attention architecture reduces VRAM usage

Model Specifications

Feature	Details
Parameters	~2 Billion
Context Length	262,144 tokens
Input Types	Text + Images
Architecture	Hybrid Linear + Full Attention (24 layers)
Vision Encoder	24-layer ViT, 1024 hidden size
Text Hidden Size	2048
Precision	BFloat16
License	Apache 2.0

Capabilities

Image Understanding: Analyze, describe, and answer questions about images
Visual Question Answering: Extract information from screenshots, documents, charts
Multimodal Reasoning: Combine visual and textual information for complex tasks
Long Context Processing: Handle extensive documents with visual elements
Production Deployment: Optimized for real-world applications

Quick Start

Installation

```bash pip install transformers pillow torch accelerate ```

Basic Usage with Transformers

```python from transformers import AutoModelForVision2Seq, AutoProcessor from PIL import Image

Load model

model = AutoModelForVision2Seq.from_pretrained( "raxcore-dev/rax-3.5-chat", trust_remote_code=True ) processor = AutoProcessor.from_pretrained( "raxcore-dev/rax-3.5-chat", trust_remote_code=True )

Text generation

messages = [{"role": "user", "content": "Explain quantum computing"}] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = processor(text=text, return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=512) print(processor.decode(outputs[0], skip_special_tokens=True))

Image analysis

image = Image.open("photo.jpg") messages = [{ "role": "user", "content": [ {"type": "image"}, {"type": "text", "text": "What's in this image? Be detailed."} ] }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = processor(text=text, images=image, return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=512) print(processor.decode(outputs[0], skip_special_tokens=True)) ```

Deploy with vLLM

```bash vllm serve raxcore-dev/rax-3.5-chat --port 8000 --max-model-len 8192 ```

```python from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")

response = client.chat.completions.create( model="raxcore-dev/rax-3.5-chat", messages=[ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "Write a Python function to sort a list."} ], temperature=0.7, max_tokens=1024 )

print(response.choices[0].message.content) ```

Architecture Details

Hybrid Attention Mechanism: Alternates between linear and full attention for efficiency
Vision Transformer: 24-layer encoder with 16x16 patch size, 2x2 spatial merging
Optimized KV Cache: 2 key-value heads for 75% memory reduction
Multi-Resolution Position Embeddings: Handles various image sizes and long sequences
Cross-Modal Fusion: Advanced alignment between vision and language representations

Use Cases

Document Analysis: Extract data from invoices, receipts, forms
Visual QA Systems: Build AI that answers questions about images
Content Moderation: Analyze images with contextual understanding
Educational Tools: Explain diagrams, charts, and scientific images
Accessibility: Generate detailed image descriptions for visually impaired users
E-commerce: Product analysis and description generation
Medical Imaging: Assist with image interpretation (not diagnostic)

Performance Tips

Temperature: Use 0.6-0.8 for factual tasks, 0.8-1.0 for creative content
Context Window: For >32K tokens, ensure 24GB+ VRAM
Batch Processing: Process multiple images/texts together for efficiency
Quantization: Use 4-bit/8-bit quantization for lower memory footprint
GPU Requirements: Minimum 12GB VRAM (16GB recommended)

Limitations

2B parameters may struggle with highly complex reasoning vs larger models
Vision encoder optimized for natural images (not specialized medical/satellite imagery)
Long context (>100K tokens) requires significant GPU memory
Not fine-tuned for specific domains without additional training

Model Comparison

Model	Params	Context	Multimodal	Speed
Rax 4.5	2B	262K	Yes	Fast
LLaVA 1.5	7B	4K	Yes	Medium
GPT-4V	-	128K	Yes	Slow
Qwen-VL	7B	32K	Yes	Medium

Citation

```bibtex @misc{rax4.5, title={Rax 4.5: Efficient Multimodal Vision-Language Model}, author={Raxcore}, year={2026}, url={https://huggingface.co/raxcore-dev/rax-3.5-chat} } ```

License

Apache 2.0 - Free for commercial and research use

Keywords: vision language model, multimodal AI, image to text, VLM, computer vision, transformers, efficient LLM, 2B parameters, long context, production AI, visual question answering, image understanding, open source AI model

Downloads last month: 6,061

Safetensors

Model size

2B params

Tensor type

F32

BF16

Model tree for raxcore-dev/Rax-4.5

Quantizations

1 model