Update Readme on usage with Infinity
#36
by
michaelfeil
- opened
README.md
CHANGED
|
@@ -246,6 +246,15 @@ with torch.no_grad():
|
|
| 246 |
print(all_scores)
|
| 247 |
```
|
| 248 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 249 |
## Fine-tune
|
| 250 |
|
| 251 |
### Data Format
|
|
|
|
| 246 |
print(all_scores)
|
| 247 |
```
|
| 248 |
|
| 249 |
+
### Using Infinity (normal reranker only (bge-reranker-base / bge-reranker-large / bge-reranker-v2-m3 )
|
| 250 |
+
|
| 251 |
+
For a self-hosted deployment with [Infinity](https://github.com/michaelfeil/infinity):
|
| 252 |
+
```bash
|
| 253 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
| 254 |
+
michaelf34/infinity:0.0.68 \
|
| 255 |
+
v2 --model-id BAAI/bge-reranker-v2-m3 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
|
| 256 |
+
```
|
| 257 |
+
|
| 258 |
## Fine-tune
|
| 259 |
|
| 260 |
### Data Format
|