File size: 3,710 Bytes
a22e10f 2f86866 a22e10f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
---
license: mit
tags:
- llama-cpp
- llama-cpp-python
- gguf
- cuda
- windows
- prebuilt-wheels
- quantization
- local-llm
---
# llama-cpp-python Pre-built Windows Wheels
**Stop fighting with Visual Studio and CUDA Toolkit.** Just download and run.
Pre-compiled `llama-cpp-python` wheels for Windows across CUDA versions and GPU architectures.
## Quick Start
1. **Find your GPU** in the compatibility list below
2. **Download** the wheel for your GPU from [GitHub Releases](https://github.com/dougeeai/llama-cpp-python-wheels/releases) or [find your card on the README table](https://github.com/dougeeai/llama-cpp-python-wheels)
3. **Install**: `pip install <downloaded-wheel-file>.whl`
4. **Run** your GGUF models immediately
> **Platform Support:**
> ✅ Windows 10/11 64-bit (available now, biggest pain point)
> 🔜 Linux support coming soon
## Supported GPUs
### RTX 50 Series (Blackwell - sm_100)
RTX 5090, 5080, 5070 Ti, 5070, 5060 Ti, 5060, RTX PRO 6000 Blackwell, B100, B200, GB200
### RTX 40 Series (Ada Lovelace - sm_89)
RTX 4090, 4080, 4070 Ti, 4070, 4060 Ti, 4060, RTX 6000 Ada, RTX 5000 Ada, L40, L40S
### RTX 30 Series (Ampere - sm_86)
RTX 3090, 3090 Ti, 3080 Ti, 3080, 3070 Ti, 3070, 3060 Ti, 3060, RTX A6000, A5000, A4000
### RTX 20 Series & GTX 16 Series (Turing - sm_75)
RTX 2080 Ti, 2080 Super, 2070 Super, 2060, GTX 1660 Ti, 1660 Super, 1650, Quadro RTX 8000, Tesla T4
[View full compatibility table →](https://github.com/dougeeai/llama-cpp-python-wheels#available-wheels)
## Usage Example
```python
from llama_cpp import Llama
# Load your GGUF model with GPU acceleration
llm = Llama(
model_path="./models/llama-3-8b.Q4_K_M.gguf",
n_gpu_layers=-1, # Offload all layers to GPU
n_ctx=2048 # Context window
)
# Generate text
response = llm(
"Write a haiku about artificial intelligence:",
max_tokens=50,
temperature=0.7
)
print(response['choices'][0]['text'])
```
## Download Wheels
➡️ **[Download from GitHub Releases](https://github.com/dougeeai/llama-cpp-python-wheels/releases)**
### Available Configurations:
- **CUDA Versions**: 11.8, 12.1, 13.0
- **Python Versions**: 3.10, 3.11, 3.12, 3.13
- **Architectures**: sm_75 (Turing), sm_86 (Ampere), sm_89 (Ada), sm_100 (Blackwell)
## What This Solves
❌ No Visual Studio required
❌ No CUDA Toolkit installation needed
❌ No compilation errors
❌ No "No CUDA toolset found" issues
✅ Works immediately with GGUF models
✅ Full GPU acceleration out of the box
## Installation
Download the wheel matching your configuration and install:
```bash
# Example for RTX 4090 with Python 3.12 and CUDA 13.0
pip install llama_cpp_python-0.3.16+cuda13.0.sm89.ada-cp312-cp312-win_amd64.whl
```
## Build Details
All wheels are built with:
- Visual Studio 2019/2022 Build Tools
- Official NVIDIA CUDA Toolkits (11.8, 12.1, 13.0)
- Optimized CMAKE_CUDA_ARCHITECTURES for each GPU generation
- Built from official [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) source
## Contributing
**Need a different configuration?**
Open an [issue on GitHub](https://github.com/dougeeai/llama-cpp-python-wheels/issues) with:
- OS (Windows/Linux/macOS)
- Python version
- CUDA version
- GPU model
## Resources
- [GitHub Repository](https://github.com/dougeeai/llama-cpp-python-wheels)
- [Report Issues](https://github.com/dougeeai/llama-cpp-python-wheels/issues)
- [llama-cpp-python Documentation](https://github.com/abetlen/llama-cpp-python)
- [llama.cpp Project](https://github.com/ggerganov/llama.cpp)
## License
MIT License - Free to use for any purpose
Wheels are built from [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) (MIT License) |